Inside Your Player’s Mind with Playtesting

I need to get what the description is from the program! Anyway, it was a talk on how people perceive personalities and patterns, as well as difficulty and intelligence, in AI opponents.

Baylor Wetzel (Brown College). summary and slides.

My Thoughts

Very entertaining on how players perceive basically what comes down to patterns, psychology of opponents and so forth – very important for designing (or not designing) AI personalities, and knowing how to show what the personality is. Check the slides for more detail too.

Talk Content

At Brown College; get to experiment on students!

HOMM4 clone, turn based strategy; 22 types of AI tested and a questionnaire done.

Players taught by fighting against each other and listing the stats.

Can players tell the difference between simple, then more complex strategies?

Need to give them clear evidence else the players fill in the gaps (although they do a pretty good job).

Simple strategies also can give the same results as a complicated finely-tuned strategy, it perhaps means you don’t need the complicated strategy.

Factoring things in the results there is:

  • Complexity – complex vs. simple vs. stupid
  • Predictableness – If they are doing something the same each time
  • Obviousness of Strategy – Random is obvious but unpredictable
  • Direct vs. Indirect MEasures – Damage=direct, cost=indirect (and depends on designer).
  • Accounts for Damage Type
  • Revenge – Attacking back if you get attacked
  • Persistence – Swapping a lot or not
  • Hustling – In this game use weighted dice to actually make the players win, not the AI.

Rating the AI on scale of fun, realism and difficulty.

Of the AI’s, figuring out the AI’s; well, not good! (See slide!). Umm got some bits right – but most AI’s, didn’t know anything.

On the difficulty – mainly matched up.

Realism had some surprises – but did fit some of the complicated strategies in there.
Fun had for some reason RandomBot tied (with many others) for second!

Some good correlations between difficulty and realism, but none between difficulty-fun and realism-fun.

Audience Questions

Question: What about alphabetical versus numers in the list?

A. It ranks differently in past tests, it depends if it is made known by someone who figures it out.

Question: Longest Name seems flawed since it is linked to the stats.

A. Don’t recommend longest name used (and it has been used in a game before!). What was important that was there were patterns found. Can say something about the perceptions while playing not the difficulty of Longest Name AI.

Question: Players seem to like arbitrary things – like the look of an AI player like a demon lady!.

Before we had named AI’s like Alice, Bob, instead of numbers, which altered results since people had favourite names, so it is odd.

Rats and pattern matching – they’re better then humans by 16 points on a test finding a pattern when there are two buttons and it is a simple 20/80 split, since humans look for patterns. Another test was having the button go off any time and a reward always given and it still had humans thinking they had cracked the problem even though they could never fail.

Leave a Reply

Your email address will not be published. Required fields are marked *

Website and journal of Andrew Armstrong