The Photoshop of AI: Debating the Structure vs Style Decomposition of Game AI

Chris Hecker, Steve Rabin, Damian Isla, Stuart Reynolds, Borut Pfeifer

Chris Hecker, Steve Rabin, Damian Isla, Stuart Reynolds, Borut Pfeifer get involved in a talk, mainly Chris versus everyone else, on the comments he made about the “Photoshop of AI” – a tool that AI designers can easily use to alter AI, as an artist alters textures. No easy answers, but some interesting discussion from it, if very civil πŸ™‚

Borut (mainly) chairs it, asking questions for the panel. Some pretty short, and some pretty long answers for the various parts.


Borut – The talk Chris Hecker did last year was AI should be a great improvement in gameplay, but it isn’t, while graphics – for instance triangle based rendering, is a great improvement. What would the photoshop of AI look like?

Borut – Chris, summarise in your own words your talk and thoughts in the previous year?

Chris – Computers are needed for it to be a videogame, the loop of user input and some output. Computers can’t generate the emotional depth that is wanted, and so a human is needed in the loop. The structure needs to be sorted ahead of time – the degrees of freedom are sorted beforehand:

  • Intuitive – floating point in a neural net is right out
  • Frugal – triangle works well for graphics because it is good enough
  • Blendable – a lot of degrees of freedom need to be blendable. Audio mixing, animation and so on.
  • You have code doing the behaviours, the style (data) doesn’t have any code in. Does behaviour require code?

Accessibility for the code – visual versus text – anecdote says some think text is bad, some say visual is bad, so no preference. Accessibility is maybe the wrong word – photoshop isn’t accessible. Accessibility is more subtle. Trying out Maya recently with it’s massive data structure which is hard to use – there were some good points. There is a tight feedback loop – the pixel you click on goes green. Long way away from this in code.

Chris – What is your gut thoughts on this everyone else?

Steve – I think your argument is incredibly seductive. We want it so bad. AI is about decision, decisions need if statements and if statements seem to need code.
Saying no code, the structure might be an improvisational actor. You give it the goals, background, motives, and be a ton of sliders if no code. Sport games have kind of done this – lots of somewhat individual players. Making an improvisational actor is not easy.

Damian – I’m really pissed at you, since I thought too that improvisational acting is the perfect photoshop of AI, like 3D sculpting inspires Maya and Max. An actor is given a script which is the most superficial level of action, so it has to work backwards to find out how to speak those lines. The actor needs to also stop when rehearsing to get feedback from the director – back and forth. There is no reason why the AI can’t do the same thing – have a meta-AI. Imagine a natural language interface where you speak to the AI cyborg to give it commands. Back and forth – saying the cyborg should be stopped and told extra things like β€œI forgot, you’re afraid of kittens”.

Stuart – With structure versus style we spend so much time crafting AI there must be a faster more efficient way of doing that. We think we have a solution with that possibly with machine learning. Having behaviour capture where you debug the AI after it has done something is used now while machine learning can just do the process in real time.

Borut – Considering something like Facade, there are a lot of hand written routines. Behaviour is procedural because of this so how can you get around this?

Chris – Analogy to motion capture to behaviour capture. If you have got behaviour capture you already have the degrees of freedom defined. I think it is just renaming the problem.

Stuart – You don’t have the continuity problems of animation and motion capture. You can connect learned behaviours. The training data is the specification is what I’m trying to get to.

Borut – How can you blend that data without problems?

Stuart – You essentially train a very aggressive AI and one who is very afraid of fire, you have the behaviour when there is fire, or else it will be aggressive.

Borut – You don’t blend just enable and disable behaviour.

Chris – You have a amount of floating points – it’s not learning structure. Your capturing something, what?

Stuart – You capture the gameplay. You give examples to the AI of things it should do in certain situations.

Chris – So you are saying if you have a Turing complete AI you can do it. Slider bars are the keyframe animation, versus motion capture. Picking what that slider bar adjusts is the hard part.

Stuart – You do need to do that, yes, but once you’ve done that anyone else can alter the AI.

Chris – When you have that good structure you can get the artists looking at it, but it is how to get that structure to start with.

Borut – How easy is to do, can we make generalisations by genre?

Chris – There are things that seem like they might be things that work with the slider bars for sports game for instance. Neural nets are just a bag of numbers, and fuzzy if/then rules that don’t go that far. Is it just solving the hard AI problem or what?

Stuart – You can capture the behaviour.

Damian – That is hard – getting all the information and making the decision from the players perspective.

Chris – How can you make two AI’s react differently without writing two AI’s?

Damian – You put a slider on it… πŸ™‚

Borut – Richard Evans is fond of saying AI doesn’t do adverbs well. An example like The Sims having the happy Sim does different things to the sad Sims. Those rules in Sims 3 selects a pallet of behaviours done by designers. Is that painting AI?

Damian – First time I heard the structure versus style I didn’t understand it exactly, but orthogonality from the Sims – how you do something, what you know and how you do it. One of the best things we can do is to look for as many of those orthogonalities, as possible especially ones which are across disciplines.

Chris – You don’t necessarily need hard AI actors to do it, but just separate the structure and the style.

Damian – That’s the point and it is if the structure and style can be done out of code.
Chris – It would be nice the AI changes like sound and other aspects of the game. The behavioural part of an action needs to be more open to editing (some more stuff he says fast here).

Damian – It’s going to be an if/then/else statement and so is always going to be code. Need to get people who are technical and writers.

Chris – We had this animation problem which was easier then AI.

Damian – That’s right. Laughter

Chris – The set of animators and scripters are so disjoint you need to then for this you need actors who can code.

Borut – It seems like it needs more of a standardised pipeline for getting all those elements to get it together.

Damian – It’s figuring out what talks to what and when is a big problem.

Borut – Some pros and cons are fine like with triangles (in graphics)

Stuart – AI is all about special cases.

Chris – People keep asking if it is time to standardise AI. To me it is like β€œNo” laughter because standardising too early is a huge cost. It should be custom now – it took a long time for the triangle to beat out the competition.

Damian – What works well with standardisation is well defined problems.

Q – Maybe one of the strengths of photoshop has pixel matrix and do lots of composable transform filters on top of that. What about that for AI?
Chris – The reason for the focus on tools is that is the way you manipulate that.
Steve – Attacking from the tools side is easier. From the structure side is much harder…(some more talking I didn’t note)

Q – So it is nice to say we need photoshop for AI but the panel hasn’t answered how to do this. We haven’t got one technique to cover everything and have something to cover the odd cases?
Damian – For the improvisational actor it is used for checking on the design side for behaviour conflicts and to take action – for me that is the only use of that technique.

Leave a Reply

Your email address will not be published. Required fields are marked *

Website and journal of Andrew Armstrong