Artificial Intelligence in Game Design

Representing NPCs as Finite State Machines

Finite State Machines

• Non-player character in one of several possible states– Must be some initial state

• That state controls actions taken by the NPC• Transitions to other states caused by

internal/external stimuli

Current StateActions in

current state

Another StateStimuli

Another StateStimuli

“Guardbot” Example

PatrolMove back and

forth in front of doorEnergy at 100%

ChaseMove towards playerEnergy = Energy - 1Player visible

ReturnMove towards doorEnergy = Energy - 1

At door

Player not visible

Energy below 50%

start

Pac-Man Example

Finite State Machines

• Each state can be reflex agentExample: “fight” state

Fight

Player swings Raise shieldPlayer waits Swing sword

Player moves Move towards player

RunHit points < 5

Designing FSMs

• What different things does NPC do?– Types of action taken

• Chase, wander, etc.– Steps in some process

• Get food from fridge Cook food eat food

– Can include “terminal” state• Object destroyed and deallocated

ReturnMove towards doorEnergy = Energy - 1Energy == 0

Dead

Implementing FSMs

• Can treat each state like a C++/Java class – Possibly derived from some base “State” class

• Typical methods in class:–Enter()

Executed once when state enteredExample: entering “Fight” state causes NPC to select weapon

–Exit() Executed before go to another state Example: exiting “Guard door” state causes NPC to lock door

Implementing FSMs

• Typical methods in class:–Update()

Executed by game engine each framewhile NPC in this stateExample: reflex actions taken by NPC in “Fight” state

–int CheckTransitions() • Executed by game engine each frame

while NPC in this state • Returns ID of the next state to enter based on

current stimuli (same as current state if no change)

Implementing FSMsclass Chase extends State { int stateNumber = 1; // Patrol = 0, Return = 2 public:

void Enter() {say(“intruder alert”);} void Exit() {say(“intruder has escaped”);} void Update() { moveTowards(player.getLocation); if (rand() < 0.3) say (“exterminate”); energyLevel--; } int checkTransitions() { if (energyLevel <= distance(location(door.getLocation)) || distance(location(door.getLocation)) > 10) return 2; else return 1; } }

Emotional FSMs

• States represent emotions for character• Actions express emotion• Stimuli change emotional state

Confident Angry

Frightened

Player HP < 10

Player hit > 5 HP

My HP < 10Heavy hit by me

Player hit > 10 HP

Emotional FSMs

• Can combine with action states appropriate to emotion– Looks more realistic if orc displays fear before running

Confident Angry

Frightened

Player HP < 10

Player hit > 5 HP

My HP < 10Heavy hit by me

Player hit > 10 HP

My HP < 5

Running

Emotional FSMs and Personalities

• Can “tweak” parameters for different NPCs• Differences must be large enough to be noticeable

Confident Angry

Frightened

Player HP < 5

Player hit > 1 HP

My HP < 5Heavy hit by me

Player hit > 20 HP

Orc with anger

management issues

Emotional FSMs

• NPC must clearly express emotional state– Facial expression (difficult in low resolution)– Body language

• Posture, motion, etc.– Sound (speakers must be on)

• Spoken phrases• Sounds (growl, etc.)

– Abilities• Strong emotion might make character less accurate!

Emotional FSMsConfident• Smiles• Shouts insult• Stands ground

Angry• Growls• Frowns• Faster attack• Less accurate

Fearful• Backs away• Grimaces• Slower attack

Timeouts in FSMs

• Problem: Abrupt transitions in FSMs

– As player approaches, NPC jumps back and forth between “Walk around” and “Run” states

Player < 5 feet away

Player >= 5 feet away

Timeouts in FSMs• Solution: State “timeouts”

– Continue high-emotion states for fixed amount of time after stimulus gone• Keep running for time even after at safe distance• Supported by evidence from biology

Stay in running state for at least 10 seconds even in player not close

Timeouts in FSMsclass Run extends State { int timeout;

void Update() { Flee(player.getLocation()); if (distance(location, player.getLocation()) < 5) timeout = 10; // run for 10 frames even after escape) }

int CheckTransitions() { if (timeout > 0) { timeout--; return 1; // stay in run state }

else return 0; // go to walk around state }

Extended “Guardbot” Example

PatrolMove back and

forth in front of doorEnergy at 100%

Player visible

EscapedMove towards doorEnergy = Energy - 1

At door

Player not visible

Energy below 50%

ReturnMove towards doorEnergy = Energy - 1

Player visible

Fire

Player within 2 units

Turning

Player in front

ForwardPlayer to side

Dodge

Obstacle in front

FSM Issues

Problems with Finite State Machines:• Complexity

– Potentially hundreds of states• Design difficult• Debugging impossible

• Duplication– Many blocks of states similar

• Example: Return state also needs “Turn”, “Move”, and “Dodge”– Many transitions similar

• Example: Need “low energy” transition from all states in “Chase”

Hierarchical State MachinesSingle high-level state contains entire low-level FSM

– Need initial state that high-level passes control to– Need final states that correspond to transitions from

high-level state

Chase

Turning

Forward

Player to side

Dodge

Obstacle in front

Start

Player to side

Player in front

Obstacle not in front

EscapedPlayer not within 10 units

FirePlayer within 2 units

Hierarchical State Machines

• Usually implemented as stack– Push low-level state on stack when enter– Pop and move to next state when finished

Start

Chasing

GuardingDoor

Chasing

GuardingDoor

Turning

Chasing

GuardingDoor

…

Escaped

Chasing

GuardingDoor

Escaped

GuardingDoor

Exiting Low-level States

• “Interrupts” may cause exit before task is completed– Example: Caution flag interrupts passing

• All states on stack may have interrupt conditions– Check all at each frame– Better than having same transition for all states

Turning

Chasing

GuardingDoor

ForwardPlayer in front

Return

Go to HQ

Energy < 50%

Emergency recall

Exiting Low-level States• Can store current low level state and return if

necessary once exception handled

Build Farm

Clear land Build barn Plant crops

Dam break

Fix Dam

Return to appropriate state when dam fixed

Weaknesses of FSMs

• Abrupt transitions between emotional states– Confident Terrified not a realistic transition– May need more intermediate states

Confident Worried Terrified

• Multiple next states may be indicated by stimuli– Must make sure all are mutually exclusive

ChasingAttackWithin 1 unit

ReturnEnergy < 10