Keynote. As the cost of hand animation and motion capture increase, more developers are turning to procedural techniques to argument their animation systems.
Ken Perlin (Professor at New York University, Chief Technology Advisor at ActorMachine).
Utterly fascinating prototypes and ideas – would be brilliant to get some games using this just because it would allow so much more movement.
Possibly easy to predict since you can make things up! Still going to predict.
Learning from linear narrative…can we tap deeper emotions or make more powerful interactive narratives? There should be some way of tapping into their emotions while still having all the advantages of interactive media – problem solving and actions.
Culture defines itself by the stories it tells. Where is the artist in interactive media? Are we asking the audience to be the artist? Are they stuck behind the creation?
Main obsession of people is other people – internal questions on what others are thinking and doing. Shooting at zombies doesn’t get close to this.
Narrative is generally driven by character. Catcher in the Rye/Gone with the Wind has agency with the characters. Psychological buy-in, even though the characters are not real and you can’t affect them.
“Plot is the drugged meat that you throw over the fence to put the dog to sleep, so you can rob the house.”. Plot is only there to support the characters.
In games you have the agency – and you don’t necessarily believe in the characters. Interesting development: the girl in ICO.
What is the “game mechanic” for novels/movies/plays? Questions: who are they? why are they doing that? what will they do next? eg: husband/wife/friend. In real life it is boring reasons but a film, you know someone is sleeping together if they give each other the cold shoulder.
Gives us a framing device when someone reads a novel!
Why are they not boring? because you are interested in what they are doing and why they are doing it.
Indiana Jones game; you’d never have the character be made angry thus destroying the character. If you do allow it, it will turn into the Sims, since it is more of a dollhouse. If a sub genre of games with characters that are interesting is created influencing directly should be impossible.
Paintings are not interactive – but sculpture is interactive, no two people see the same sculpture the same way. Same with music and architecture.
Jim (pretty, dumb) and Bobby (not as pretty, heart of gold) are in love with Lisa. Invested in Lisa’s character. Can’t directly get into Lisa’s head since she just becomes a Sim and not an interesting character. Jim goes to the bar because you open the bar early and Bobby and Lisa get together, hooray!
Who likes science fiction? Everyone! (More then who use Havok!). The improbability only comes globally as the setting or place – locally the things have to make sense and be believable.
Energy = Log(probability)
For coin flips; it might be possible to get 20 coin tosses in a row but it is practically impossible. Having Indiana Jones have a chicken suddenly appear in their head simply doesn’t work logically.
Layered Reaction Systems
To get exponential tree without exponential work means breaking down the parts of the story down into small bits which work together.
It only works if we have actors however!
Plan 9 from Outer Space – very, very bad acting. Without the acting you don’t have it. For good acting though, blending pre-created animation is a dead end – acting in games needs to be procedural, from the inside out.
Procedural stuff is very cool, and people with lots of money can do a lot with the groundwork.
Minimal face animation program, from around 15 years ago, is about as low as you can go to get enough detail. Got the low level vocabulary with the individual changes and can build up the descriptive words like “annoyed” or “happy” from the sliders.
Removing the animation – subtle movement – means it turns into a picture and loses the emotion.
Going onto animation testing of limbs – seems sine waves work well for big dramatic actions like Dracula/Fantasia animation.
Going simpler into a red shape, you see some real animations procedurally done simply but they are perceived and read into it by us.
Simple stylised duck bird things doing vaudeville Pride and Prejudice gives some testing and also inspiration for further work, and the looking at characters too.
Feet and the body basically supports the head and arms for expression, but the feet have limitations on where our feet will go. You need to determine where feet will go well because you cannot later correct it – you fall!
The feet are actors. big headed blinking (important ) thing with two feet. Feet are two bricks – one at the front and one at the back for the most basic level. Parameters for how fast to move feet, how far apart, how much goes on the ground, and so forth.
Divide and conquer idea – divide up the choice of where to put the footsteps different to the animation of how to put the feet down realistically.
With the yellow actor, can play around with looking at things, the foot size, leg size, and change to a human kind of character. Everything adjusts itself automatically. Now tying things together.
Architectally the body is made up of points. You need to have it separated since they all need to move well and be in the right places, the math cannot fix it since it removes the acting part.
Going to movement again, the feet need to follow a path, and the entire body bubbles up if a turn is made since the pelvis follows the feet, body follows the pelvis, head follows the body.
Even works for animals. Sheep – based on the mountain goat – the rhythm come in automatically working the front and back feet well together.
There is also pathfinding, there is no one way to do pathfinding. Procedural ways to do it, and things related to it like having body parts take care of themselves – hand knows all by its self how to do actions. Cooperation between the bricks in the body, connected by a spine allowing 6 degrees of freedom of movement.
Can’t get rid of the skill of the animations; need to allow them to do more. No blending because that wastes the work. Best example so far this week is Boxy Dog by Bruce because of no blending.
Getting the animator involved in a demo can lead to great procedural actions like Shy and Angry and so forth.
How does this affect the industry?
We are used to different people doing different parts of the aesthetic experience. Splitting it up with different kinds of talent. Should be able to split things up in the industry to use animators.
WSIWG editor is implemented with a lot of special language and interface which are done by different people (who may be dead!) with what you can choose for fonts or styles. Could be layers and layers of style experts.
Need to take the leap to get away from blending animation because it is killing us.
Has started a bit – 10 years ago now, OpenActor framework and ABL language from Facade. Not animators but looking at actions, but still is a start.
In interactive storytelling there is now Inform 7, semantically like Prolog, creating stories. Could hook into this and get writers putting forward great ideas without going through animators directly.
Easier path now to get new interactive content – the low end cheap games like Andriod, iPhone, means an area where people can actively use this quickly.
Current work: user create models in Unity engine. Trying to get it in at least one engine.
Direct and not just animate interactive actors! Have the games 16 year old Spielberg or Trauffaut who really knows what to do!
Q. Working on a game, with Max and hand animating, killing us, how do I add something to do this?
Talk to my coworker Jerry who did an actor machine. Ready to deliver to people at SIGGRAPH this year. Hopefully a lot of products which compete with each other.
Q. With companies like EA, who motion capture Tiger Woods and so forth, is it possible to use the model to reflect a specific person?
To a certain extent there is going to be a perceptual error. Gut feeling there is going to be some trade off, where if you’re making absolutely photo realistic characters it might fall into the uncanny valley. That level of fidelity isn’t the sweet spot – perhaps dial it down and do slightly cartoony or something.
It’s what Pixar does, and a crucial crucial thing.
Q. Trying to get the characterisation across with procedural animation is an issue. Animators might have a few choice words if left with sliders. What about getting their normal work into this tool?
Can set the style sliders after watching a walk cycle. They are funny, and a character sheet. It is going to be interesting to see whether animators will do things. They’re a valuable commodity since they know how people move. The instruments being made will not make music by themselves. Might figure out a way to reuse the media techniques to make expressive characters – are working on it.
Q. When will it be economically feasible to integrate procedural animation into triple A titles?
Computational costs are not high, there are no solvers in it.
(Bruce Blumberg comment) Great work. Going to find people use the techniques and do things which no one has done before. Equivalent to photoshop.
Q. Question earlier about animators, perhaps an intermediate tool? At the moment it is a bunch of sliders doing stuff.
I agree, need an embodied GUI, the intuitive space.
Q. What did you mean by contingent?
Instead of linear. Games have the player as an actor. All narrative is contingent in an interactive media.
Q. When dealing with more photorealistic characters, how do you propose to get animation from them?
Gradual change, like the procedural shaders – were new. Showing interest in it now, means teams with millions can start working. Has to start somewhere.
With uncanny valley – it’s the visual fidelity going beyond the animation. Quite a long time to get to Avatar and that doesn’t look real with different effects. Who knows what will be at SIGGRAPH in 5 years?