Remco Straatman, Lead AI Programmer, Guerrilla Games and Alex J. Champandard, AiGameDev.com

This talk will discuss the implementation of bots in Killzone 2’s online and offline multi-player component. We will discuss the overall hierarchical AI framework, the way individual bot behaviours such as bad usage are implemented using our HTN planner, how individual behaviors mix with the overall objectives in multiplayer modes and how we use data acquired at run-time to influence the tactical decisions.

I really enjoyed seeing tactical squad-based combat done with bots. Influence maps, if nothing else, look ultra-cool! Some really good techniques used. Remco did the first bit (on the why’s and design decisions), Alex did the second bit (on the hows).

---

Why concentrate on bots in Killzone 2? The bots in Killzone 1 were well received, the greater focus on multiplayer too, as well as meaning it allowed the bots to teach the new game rules to players offline (with no abuse, and a chance to win).

The multiplayer allows multiple game modes in team based play – multiple game modes per map.

The scope also changed – bots added to a multiple human game (rather then just being offline only).

Architecture of the AI is based on levels of granularity – the strategy AI is a higher level abstract concept AI controller, which commands Squad AI – which also is abstract, which has individual AI’s in each squad. So these do:

Strategy AI to Squad AI: Sends strategic comamnds to squads, advance, defend, etc.
Squad AI Feedback to Strategy AI: Order failed, order completed etc.
Squad AI to individual AI: Sends orders on where to move
Individual AI feedback to Squad AI: Combat information

Sometimes special cases:
Individual AI: Different orders if assassination target.
Individual AI to Strategy AI: Request reassignment, eg: if placed a turret as an engineer.

(PIC) The individual AI’s thoughts on the world may be incompete since not all information is copied from the world – the information from stimuli about where things are might be only things they remember.

The individual AI works using a HTN, hierarchical task network planner (see Erol, Hendler and Nau paper: AAAI-94). Tasks have preconditions and actions, the example is not very designer-friendly.

On the monitoring of plans, there are easy fail cases, but also need to plan pre-emptively since a better plan might become available. Keep replanning on 5hz, but you need to prevent twitchy behaviour (“Attack nearest” – nearest one might change quickly), and all the same checks becomes expensive. So to solve you add “Continue current plan” at certain node branches and keep the old plan. Placing those carefully you get all the advantages to solve twitch and optimisations.

To choose individual actions, it depends on the class and action to be done.

However, you have the basic behaviours (using turrets, shooting), and opportunistic badge actions (healing as a medic, engineer repairing).

In addition there are orders (badge specific interpretation – depending on the class you do different things, like setting up turrets on defend orders). Mission specific also – such as being the flag carrier.

The How’s

Dealing with 1v1 bot, 8v8 bots up to 2 humans vs 14 bots. A fixed policy won’t work with bots entering and leaving all the time.

Looking at Halo 3 as a base for squad management, there are some differences – persistent bots, applied to general maps and for programmers. Halo 3 had disposable enemies, made for level designers and level based – so story driven, not strategic.

Both need decoupling of what to do and how to do it.

One part of the architecture creates and destroys objectives when the different game types happen. The objectives have static or dynamic offensive and defensive locations (for capturing/defending, and assassination respectively).

Objectives have different weights – advancing is more important then defending most of the time for instance. Influences the amount of bots assigned to each task.

For squad assignment, you do a step by step algorithm, which is rather complex. Start by calculating the ideal, then create new squads if needed, and remove old ones if there are extra ones left over from before. Then you pick an objective for each squad. You then unassign bots for special cases when objectives are completed.

Assigning a bot needs to look at the class combinations, but this is barely noticed anyway, and is very simple. The distance to a squad is based on their average location.

Assigning a squad to an objective goes on a first come, first serve.

Badge choices for bots are fixed for the first 6, 1 sniper, 1 saboteur then the rest semi-random.

Strategic methods:

Added annotations to the map – manually, no need to do it autoamatically. Key areas marked: regroup areas, hiding areas, (rough) sniper locations (which finds cover etc. in the low level), defend locations (for objectives, so entrances to a blow up thing objective), and retrieve location (for capturing flags), and finally assassination target areas – where the target runs to to hide.

Strategic graph – “Totally Awesome”. A high level graph is more strategic and easier to work with rather then low level waypoints. This is fine if you have a static map and don’t have things blocking areas – Killzone 2 doesn’t (Turrets you can walk through). Basically groups of waypoints together. Small amounts of connections gives you choke points.

To do area clustering you need the algorithm to have a small area size, low amount of connections (because the higher level graph should be really easy to use).

Influence mapping really helps determine who controls what, used to decide where to hide/regroup/defend. Calculated by the friendly/enemy bots/turrets locations and death (so lots of friendly dead means enemy controlled) – all added into a graph then smoothed. Bigger games the calculation is spread out in time.

Need to tweak the values of the influence maps – obviously friendly areas are low or minimum cost, and areas of death or enemy control are higher. Tweaking means not getting bots killed, eg: running at a turret.

Can plan one path for all bots through one set of nodes too. The way where to go is calculated is with a incremental single-source algorithm – scale with accuracy over distance, working on a cached high level graph to work on it over time.

When one squad moves out to a point to regroup then achieve an objective the squad marks that their path is being used, so the squads going after it go different ways.

Downside of the incremental way is having waypoints not being tested that they are not pathing near. Can always fall back onto the individual AI.

Final Bits

For future work there are several things that look interesting:

– Data mining rather then doing map annotations points by hand – using the multiplayer data saved from online games (not possibly so good for developing the game though).
– Could use it for automated tests of MP mode to discover bugs/balance problems – now planning to use bots a bit earlier in the gameplay testing phase to find these errors earlier
– Teaching the players via. Bots could be improved, by player observation of bots, and perhaps have the player added to a bot-controlled squad and given “commands” – really simple for doing this from the AI’s perspective (sending sound samples rather then orders via. The AI controllers), done in an actual level, in context – or they can ignore it.
– Reactions to friendly squads, human players – doesn’t take into account at all, just ignores them currently, might also want to do something about that to improve it.

Questions:

Q – Why is the squad level so important?
It’s a matter of decoupling stuff. Gluing the concepts of higher level strategic level and individual AI. One gameplay reason is snipers have their own squads, achieve objectives in a different way.

Q. How many people worked on the bots?
Feature of multiplayer bot AI was not high on the list, and so no resources, with an intern building some basics before convincing management to give them more resources.
Alex said there was always visual progress at that point, prototypes and so on.

Q. How about the complexity, when something goes wrong, how do you assign blame?
Can always be a problem with many systems, so there are clearly defined interfaces for what each system should do. Having split up and well defined is good, and no more a problem then normal procedures.

Q. Have you any performance numbers? How long did the pre-processing take?
Runtime was 30fps, updates of game logic was 1 frame, AI was part of the other frame (quarter of the CPU budget – 50% of a PPU, for perception/pathfinding/logic – not just planning), 15 bots, 6 turrets, 6-8 squads, updates for both sides. Offline part was 10-20 seconds, with some unoptimised loops.

Q. How different was the multiplayer bot compared to the singleplayer AI?
The singleplayer AI is basically the same but the higher level commands are a LUA script with various orders to squads and individuals. Squad AI is almost entirely the same, with only subtle differences. Badges only exist in multiplayer – but singleplayer has a few specific behaviours and lacks the badge behaviours.