Reference: Character Motion Systems

SIGGRAPH 94

The use of motion capture for computer character animation is relatively new, having begun in the late 1970’s, and only now beginning to become widespread.

Motion capture is the recording of human body movement (or other movement) for immediate or delayed analysis and playback. The information captured can be as general as the simple position of the body in space or as complex as the deformations of the face and muscle masses. Motion capture for computer character animation involves the mapping of human motion onto the motion of a computer character.

The mapping can be direct, such as human arm motion controlling a character’s arm motion, or indirect, such as human hand and finger patterns controlling a character’s skin color or emotional state.

The idea of copying human motion for animated characters is, of course, not new. To get convincing motion for the human characters in Snow White, Disney studios traced animation over film footage of live actors playing out the scenes. This method, called rotoscoping, has been successfully used for human characters ever since. In the late 1970’s, when it began to be feasible to animate characters by computer, animators adapted traditional techniques, including rotoscoping. At the New York Institute of Technology Computer Graphics Lab, Rebecca Allen used a half-silvered mirror to superimpose videotapes of real dancers onto the computer screen to pose a computer generated dancer for Twyla Tharp’s “The Catherine Wheel.”

The computer used these poses as keys for generating a smooth animation. Rotoscoping is by no means an automatic process, and the complexity of human motion required for “The Catherine Wheel,” necessitated the setting of keys every few frames. As such, rotoscoping can be thought of as a primitive form or precursor to motion capture, where the motion is “captured” painstakingly by hand.

Michele Bousquet says in her book 3ds max animation with biped,

“The walk cycle can have winning and loosing impact to the overall animation”

I have embedded an experimental biped walk cycle animation that I created to show the impact of a simple 10 step 20mm com step cycle.

See Below.

I composited the robot biped into a still plate to see if there was an increased emotional response from the viewer to the character, I added some lighting effects but didn’t take the time to add the shadow mapping as this was an experiment.

I found that compositing a simple biped animation into a live action plate does add impact to the character and its effect on its environment.

See Below

Tony Chance

http://www.imdb.com/name/nm0151243/

http://www.tonychance.net/

I Met Tony during a talk in the Arts Institute at Bournemouth on a cold Monday morning but soon after getting a brief introduction of his past and current projects I was quickly warmed up with excitement that I would get to pick at his brain.

Tony has been working in the industry for over 20 years and has a lot of big budget credits for some of the films that I have enjoyed and been inspired by. It wasn’t a sit there and listen talk we actually got to make some sketches and he went into detail about how the art departments process and create characters.

A point I picked up during his analysis of Sam Mendes “Road to Perdition” was the use of the narrative and the characters progression throughout the storyboarding process and than reapplying that to the characters and scenes to ensure that the story was progressing in each scene therefore ruling out any dead air or static shots that lead nowhere and just bore the viewer.

Although this is related to live action and not necessarily to the use of CGI or 3D animation I still think that this process should be carried through to all productions of story telling, therefore I created a flow chat that would show the process of creating a CG character taking into account the story, the environment, the modeling and all the composition needed.

See Below.

The building blocks of the character design process used by

www.catalyststudios.co.uk       See Below.

Creating a character

Research – Anthony Ward

After reading this book I got a great sense of the basic rules on creating a character.

1. Knowing Your Character.
2. Character Design Sheet
3. Making Friends

If there was one thing Anthony could pass on he reminds us over and over that when creating a character make sure that you know your character, your character should be your friend, and bringing that insight into the fray of my thesis how could anyone connect and get emotional involved if there is no basic friendship between my character and me.

Stan Winston is a self-described failed actor who began doing makeup effects and creature design 30 years ago as a way to pay the bills. After winning back-to-back Emmys for his first two productions, Winston decided that perhaps acting should take the back seat. Thirty years and four Academy Awards later, Winston is at the top of his game as the foremost creator of creatures in the world, a co-founder of Digital Domain, one of the leading companies in CGI special effects, and the go-to guy for a couple of filmmakers you may have heard of: Steven Spielberg and James Cameron. Unless you are a Luddite, or someone who doesn’t see films, or perhaps someone who sees only films with foreign titles, then you have seen Winston’s work.

Science Fiction Weekly 2002 (interviewer Hugo Perez)

You have worked extensively with “analog” visual effects, i.e., make-up, puppets. At the same time you are a founder of Digital Domain, which specializes in digital and computer effects. What do you think will be the relationship between “real” and “computer” effects as film becomes more digitally oriented? What is going to be the long-term impact of digital?

Winston: The long-term impact of digital is basically the long-term impact of any other great tool or device or technology that’s brought into the storytelling realm. Something to be embraced. It will continue to be used in storytelling to create certain realities or to create certain fantasies. It won’t replace anything. Technology is like that. You have to remember that the essence of filmmaking is storytelling, and the essence of art is communication, and stories and films will always be told in as many different ways as possible.

You asked me the importance of fantasy stories … basically to get your attention. The same stories will be told. The same stories continue to be told and people, audiences want to see them told in different ways, so you embrace the fact that we now have the ability to see a digitally animated film like Shrek or Toy Story. Ultimately, it was because it was a great movie with a great story that you enjoy it. It has great characters. The technology just allows us to see the movie in a different way.

You are not going to want to stop seeing movies that are actor-driven. An animated character is never going to replace Al Pacino. It will be another way of you looking at him. Digital animation will never replace cel animation. Cel animation will always be a form of storytelling that has its own life. Puppets have their own life. We do amazing animatronics and puppetry that are totally organic and real and we develop and create them with computers, we operate them with computers. The puppets that we’ve created, the animatronics characters that we have created at Stan Winston Studio, are superior to any animatronics that you have ever seen, but they are never going to replace the Muppets just because the technology is greater. There is an essence of character that you can get out of a sock puppet that you will get out of a sock puppet forever. In fact if you made the Muppets any more realistic, it would ruin the essence of what they are, and that won’t be replaced, and beautiful animation doesn’t mean that something like South Park which is the most crude animation of all time doesn’t have it’s place. It’s different methods of storytelling.

When movies came out, theatre actors thought that would be the end of theatre, because of the movies. There is more theatre today than there ever was. When television came out, moviemakers thought that was going to be the end of movies, because now no one would leave their house to go see a movie. There are more movies today than there have ever have been. Every time there is a new technology, there is this instant paranoia from an artistic community saying that I’m going to be replaced by this technology. It is totally absurd. When you can do it live, you will do it live. You will always want to go and see a play. There is something that is so visceral about storytelling when you are there, one-to-one live theatre that has it its own essence. No movie, no film, can ever replace that essence of live theatre. On the same line there is an essence of life on film, live actors that will never be replaced and can never be replaced by artificial actors or artificial life forms. You will want to see it, you will want to see the digital life form. You will want to see the digital animation, but you also want to be able to see a movie that is about life that you know is reality that you are touching.

One of the reasons that people love Jurassic Park, and these movies that have been created by filmmakers like James Cameron and Steven Spielberg is that you do not know whether it is all digital or real. You would not enjoy the movie nearly as much if you thought, “Oh, the dinosaurs are animated.” One of the things that you love about it is that people know that those dinosaurs are there. They are full size and they could eat you, and they could kill you. They are machines that are dangerous. You as an audience, you don’t want us ever to stop improving the robotic and live aspect of it, because it entertains you. Just the fact that I can give you a dinosaur that really is there is something that you will not give up as an audience and you would rebel if it all became digital, but you still want to see as much as you can digitally, you will want to see as much as you can robotically. You will want to see as much animatronics as you can. You want to see stuff that you’ve never seen before. You don’t care how you see it as long as it’s something that’s new, and there is a story to be told, and the story is told well. It’s not about the technology. Embrace the technology. Use it when it is a good storytelling tool but don’t think for a second that that’s what it’s about.

Taking all that I learned during my research into the uncanny valley and putting it to use in term of character development. The biggest choice when choosing a character is human or creature.. Looking at the success of all the CG films released over the last ten years you would be mistaken to thinking that creating a human character is the way to go, I don’t want to go to much into that right now as I will be covering that entire subject matter later on but for now, lets look at successful CG creatures.

Like most of research before I turned to the guys at Focal Press for an insight into all things computer generated. The book begins with a brief historical timeline and overview of creatures in the movies from the ice-monsters and lunar sprites of George Melies, to the earliest fantasies of mind-blowing CG creation.

So for my first piratical animation test I first wanted to try a proven technique that worked so I modeled a young girl and rigged her but only at the pivot points so she would move slightly but not human like only biped like. I choose to first try this technique and based my model on a very successful 2d cartoon kim possible. See video below.

As you can see Kim moves only at the main pivot point and you can clearly see that this is a 2d animation and conveys no connection to the real world physic, but does fulfill its brief of representing a young girl. If I was to place this within the uncanny valley it would fall lower than prosthetic somewhere in the plastic/lifeless area. My first task was to model Kim close to how she looks so I got some images and modeled her. See below.

The next stage was to rig Kim so that she would be able to complete a basic face morph in order for the charter to interact with the audience, I also decided to call my verson Clara instead of Kim now when it came to creating the morph I wanted to only use a maximum of 5 – 8 weights which would be similar to the Pixar Toy Story set up, or a stiff toy rig. See below

So here we have a version of Clara mouthing the basic vowels for speech, I only added a simply audio track i recorded in pro tools without syncing it as for this experiment it wouldn’t be needed but it will give the idea of a toy plastic movement nowhere near the realm of realism but yet like the 2d animation of Kim it does fulfill its purpose of interaction and animation. See below.

The next step just for kicks I added a bed room and some objects to see how the animation would interact and if the addition of physical object would add to the overall feel and emotional connection to the animation. See below

For experimental purposes i decided to edit the mesh and add some hair Segments. I converted the segments to strands and constrained them as an editable poly mesh. I wanted to see would more realistic hair strands add to the emotional response to the character… see below…

Consider the Uncanny Valley crossed… at least on some level.

That’s right, that picture is a CG version of Korean actress Song Hye Kyo.It was created by Indonesian CG artist Max Edwin Wahyudi using a 3ds max and zbrush.The next big jump will be animating the mesh without it looking all stiff and dead like…

There are a lot of people who would argue that just because these collections of polygons are a physical representation of the human form they do in fact lack the essence that is what makes humans human. So what makes us human humans.. is it our soul?

But our souls are more of a spiritual concoction than a physical object, right?

Do you have a soul?

Are you a soul?

How did that “soul” get into your body?

Was it imparted to you at the moment of conception?

Did it come into your body at some time during your fetal development?

Were you given an “soul” at the moment you took your first breath, shortly after birth?

If you have a “soul” can you touch it, taste it, feel it, hear it, or smell it? Can you take it out and look at it? Can you see the “souls” of others? Are You, you the real person, the innermost, private, human being with all of your thoughts, inclinations, perceptions, concepts and beliefs, hidden longings, doubts and fears, beliefs and convictions, sensory perceptions and feelings?

If that can be answered than maybe we can be one step closer to creating CG imagery that connects with the soul, but for now I think we will have to be happy with CG imagery that aesthetically connects to us.

The BBC tackles the photo realistic challenge facing video game developers dubbed “the uncanny valley”, that uncomfortable feeling one gets from looking at a digital recreation of a human being that looks… almost human.

Crossing the Valley in 2009

So where does that leave us… Does that mean that just because we can we should.. Does that mean that all animation or CGI work that doesn’t fall above the uncanny valley should be considered a failure.. Later on I will conduct an experiment to test that theory but for now i will leave you with the words of Thomas Kang. (USC) and Ralph Eggleston in response to the release of the big buget CGI Film “Beowulf” (Artist Director for the Incredibles) 

“In my opinion it’s always been a fallacy, the notion that human characters have to look photo-realistic in CG. You can do so much more with stylized human characters. Audiences innately know how humans move and gravity works, so if a human character doesn’t feel right, they’ll feel something’s wrong. But if the weight works for stylized characters, the audience doesn’t question it.”

And, the General Public writes:

*Okay, so maybe I’m just a geek cinephile but it totally looks like CGI to me – it’s the lack of emotion in the eyes, the stiff appendages and the rubberized body motions that give it away so easily.

However, that said, the technology has come a long way and it still looks pretty darn fantastico*

*I hate this type of lazy film making. It gets big stars on the cheap so they don’t have to work too many days. Then a computer does the rest. No real investment in actors at least trying to get into character (any wrong expression can be fixed by the computer guy) and no real sets. It’s lazy and an insult to ones intelligence*

*Animated Angelina HOT…Buuut. She’s also animated, which defeats the purpose of daydreaming in my book. Why go thru all the trouble for the animation technique when you can just shoot live-action. Obviously it’s cheaper, but it kinda takes the magic out of the movies*

Posted in Chapter 1.3 Film/Animation Tagged: jonathan joly, The Uncanny Valley ]]> http://jonathanjoly.wordpress.com/2008/10/28/crossing-the-uncanney-valley-in-2009/feed/ 0 jonathanjoly uncanny_valley1 http://jonathanjoly.wordpress.com/2008/10/27/chapter-3-character-gesture-and-the-brain/ http://jonathanjoly.wordpress.com/2008/10/27/chapter-3-character-gesture-and-the-brain/#comments Mon, 27 Oct 2008 22:17:31 +0000 Jonathan Joly http://jonathanjoly.wordpress.com/?p=63 ]]>

The Biped & Quadped

Character animation with 3d software is a relatively new art form. In 3d, animating a character is quite different from animation a bouncing ball or a car rolling down a street, a living creature has limbs that move in various ways. When it walks or runs, at least one body part (such as the left or right foot) reacts with the ground at any given time.

Throughout the 1990s three different approaches were developed to make 3d character animation possible: jointed characters, model control, and skeletons.

Jointed Characters

The first approach was to create jointed characters, similar to a marionette. The parts were linked together in a hierarchy; moving the torso would move the entire character in space, than the arms and legs could be rotated separately. This solution is still in use for many types of animation, but it has limitations. The joints are nearly always visible, or become visible as soon as the character starts to move.

Model Controls

To get around the problems of joints, another approach was developed for one-piece character models. An animator could place controls at key points on a character’s body and use the controls to push and pull parts of the mesh, Although this technique makes it possible to animate a character smoothly, it is also a tedious and difficult. A great nimber of controls is necessary to make the character deform correctly and when animated the character can seem to be made out of rubber or clay.

Skeletons and Skinning

A third approach was developed that mirrors real life more closely. A skeletal structure is placed inside the character model. When the skeleton is animated, its acts like bones inside a body and cause the character model to deform accordingly. This technique requires tools to make the skeleton affect the character model appropriately. For example, the skeletons upper arm bone must affect just the characters upper arm and no other body part.

This is an animation I did for my Higher National Diploma. The animation was created using a combination of biped motion capture and key framing. I put this animation in to show a basic walk cycle key-framed. You can see that the COM (Centre of Mass) doesn’t move from its pivot point and is stable through out the entire animation. Knowing what I know now after researching Biped Animation by Michele Bousquet and Michael McCarthy the COM should move with the hip joints and react with the gravity which should have a mass of -980. see below.

Form

Visual Appearance, Touch Sensation, Fragrance

Dynamics

Motion Quality, Speech Quality, Voice Quality

Contingency

Interacting, Timing

Havok/euphoria physics.

Here we have a video about the new Havok/euphoria physics engine to be used in the newest releases of video games. A question could be put forward that if the physical environment and the reactions from the character have its own AI and react to its environment does this bring about a sense of realism…

Judge for yourself… see below.

An editorial at The Wire claims the Nintendo Wii actually benefits from its technical inferiority. The Uncanny Valley is an issue we have discussed before: character models that are almost life like but lack a certain pizzazz that renders them creepy to humans

The E3 trailer for Heavy Rain showed a character that would fall within the Uncanny Valley. See below.

Author Shawn White claims that no Nintendo Wii title will be capable of life like characters (that fall within the Uncanny Valley), thus removing the possibility of protagonists that creep out the character and negating all possible emotional detachment — this is true (as far as we know). However, not all next-generation games will take full advantage of the technology, nor will they all try for life like animation.

Games like Metal Gear Solid 4 have amazing character animations without rendering its audience uneasy. See below.

Essentially, the decision is that of game design, not entirely of hardware — that is the biggest flaw of his argument. It is up to the developers to realize when a character design is lifeless and creepy. Some developers may try for naturalism with Wii games — is it really wise to say the Wii will probably fail?

During my investigate and research into the response from both live action and animation I strived to see if there could be a formula put in place as to force an emotional response to a 3d object as a similar response to human form. Using the Uncanny Valley as a drawing point I decided to create differnt types of charters that would draw from all the evidence I found.

My Dissertation will cover most of the theory and literary investigation and my Specialist and Final Major project will utilize my conclusions and research outcome to create a short animation film where the viewer will connect/interact on an emotional level with the 3d object characters as if they were human live action characters. A similar project was conducted by Brad Bird from Pixar when he was directing the Pixar animation “The Incredible” which was at the time the most successful animation to date that achieved the highest ranking and closest to achieve human like response from their audience within the graph of the uncanny valley. There has also been some failures for example The Polar Express and Beowulf where the characters form – visual appearance, touch sensation and motion quality was not human responsive and therefore there was no emotional, empathic or sorrow interaction with the audience.

The Science of YouTube

We take a ride down into the Uncanny Valley and explore that familiar-yet-different feeling. See below.

Chapter 1

THE UNCANNY VALLEY

Masahiro Mori (Mori Masahiro, born 1927) is a Japanese roboticist noted for his pioneering work on the emotional response of humans to non-human entities, as well as for his views on religion and robots.

In 1970, Mori published “Bukimi No Tani” (English title: The Uncanny Valley) in Energy. The article forwarded the hypothesis that as robots become more humanlike, they appear more familiar until a point is reached at which subtle imperfections of appearance make them look eerie. The observation leads Mori to the belief that robot builders should not attempt to make their creations overly life-like in appearance and motion. It states that the closer an object (a robot, an animated character) comes to resembling a human being in its motion and appearance, the more positive our emotional response to that object becomes until suddenly, at some point of very close (but not perfect) resemblance, our emotional response turns from empathy to revulsion. This revulsion or uneasiness, Mori says, is the result of a basic human tendency to look for anomalies in the appearance of other human beings. When an object such as a robot or an animated character is so anthropomorphic that it is nearly indistinguishable from a human being, we monitor the appearance of that object very closely and become extremely sensitive to any small anomalies that might identify the object as not fully human. For whatever reason, these anomalies create in many people a shudder of discomfort, similar in effect to the feeling we have when we watch a zombie movie or see an actual corpse. In both cases, what we see is both human and not fully human, and the contradiction produces a very negative reaction.

Karl F. Mac Dorman presents on the uncanny valley at the 2007 NMC Summer Conference (June 6-9, 2007, Indianapolis), hosted by the Indiana University School of Informatics. The talk introduces the uncanny valley in animation and robotics and morphing experiments. see below.

Thesis Abstract

CAN A POLYGON MAKE YOU CRY?

The Arts Institute at Bournemouth

www.arts-inst-bournemouth.ac.uk

2008

Introduction

The subject I have chosen to investigate is an extremely broad one, which requires months of extensive research. Therefore I have decided to investigate the following sub topics: The Uncanny Valley, Character Design and Character Animation. What I am investigating is the constant question about the validity of 3d mesh objects replacing real world objects in the entertainment industry. Is it possible for the user or viewer to get emotionally involved with their chosen platform of entertainment if CGI is present? I will be using a method of practical simulation and theory research to investigate the cognitive response to animation. In chapter one, I will be looking into The Uncanny Valley and how this chart has become a bench mark for animators. I will briefly explore the games industry and its use of physics engines to simulate emotional responses and look at how the industry is moving forward and reacting with the introduction of new technical achievements in terms of processer and graphic card abilities. I will also animate a practical experiment demonstrating this. In chapter two, I will look into the building blocks of the character design process and sit down with Tony Chance, storyboard artist and illustrator (10,000BC, 2008, Road to Perdition, 2002, American Beauty, 1999) to discuss the use of the narrative and the characters’ progression throughout the storyboarding process and than reapplying that to the characters and scenes to enhance the viewer’s emotional response. I also got to read an interview from Science Fiction Weekly with the late Stan Winston who is known for films such as The Terminator (James Cameron, 1984), and Jurassic Park (Steven Spielberg, 1993). In chapter three, I will look into character animation from the use of biped, quadped and motion capture to create characters that can cause a cognitive reaction or emotional response. I will also look at how character motion systems are applied. I will follow that up in chapter four, with a look at life-like animation and mage Metrics working side by side with animators to increase the emotional response in animation and film. Finally I will write up a conclusion to review my findings and set out some sort of basic formula for creating an emotional response from polygons.

Please use the index to navigate the thesis and please feel free to leave comments.

Chapter 1.2

The Uncanny Valley Chart

This version of the chart has been edited to show the most well known modern media animations and live actions to give you a better understanding of what the valley is and how characters are judged. At a first glance you will notice that the most “commercially” successful animations and live actions all hit the high graph and the ones that failed at the box office all fall well below the graph of plastic or lifeless.