Agent Architecture Considerations for Real-Time Planning in Games Jeff Orkin Monolith Productions.
-
Upload
russell-gibson -
Category
Documents
-
view
219 -
download
0
Transcript of Agent Architecture Considerations for Real-Time Planning in Games Jeff Orkin Monolith Productions.
Agent Architecture Considerations for Real-Time Planning in Games
Jeff Orkin
Monolith Productions
F.E.A.R.
Agenda
MotivationProblemsSolutionsWas it worth it?
Agenda
Motivation – Why plan?Problems – Performance!Solutions – Agent architectureWas it worth it?
Why plan in real-time?
Goal-Oriented Behavior
Goal-Oriented Behavior
Problems:Managing dependenciesSharing behaviors
Problem: Dependencies
AttackFromCover
Gotot Attack
Problem: Dependencies
AttackFromCover
Gotot AttackDraw t
Problem: Dependencies
AttackFromCover
Gotot AttackDraw t
Gotot Flipt
Problem: Dependencies
AttackFromCover
Gotot AttackDrawt
Gotot
Flipt
Lightt
Lightt
Reloadt
Problem: Dependencies
tt tt tttt
tttttt tt
Problem: Sharing
No One Lives Forever 2 TRON 2.0
Problem: Sharing
No One Lives Forever 2 TRON 2.0
Problem: Sharing
No One Lives Forever 2 TRON 2.0
P.D.D.L.
Planning Domain Definition Language Goals
– Desired state
Actions– Preconditions– Effects
P.D.D.L.
Goal:(define (problem get-paid)
(:domain briefcase-world)(:init (place home) (place office)
(object p) (object d) (object b)(at B home) (at P home) (at D home) (in
P))(:goal (and (at B office) (at D office) (at P home))))
P.D.D.L.
Action:(:action put-in
:parameters (?x - physob ?l - location):precondition (not (= ?x B)):effect (when (and (at ?x ?l) (at B ?l))
(in ?x)) )
Other actions: take-out, move
P.D.D.L.
Modular– Goals– Actions
Decoupled Modules Related by symbols– World State– Preconditions– Effects
Applied PDDL’s structure to C++ toolkit in game code.
Monolith Productions
Management
How to Plan in Real-Time and Keep Your Job
Jeff Orkin
Monolith Productions
The Plan
Goal: KillEnemy
Plan:
Goto (couch)
UseObject (couch)
Goto (coverNode)
AttackFromCover
AI Performance Guideline: 1.0ms / frame
1 .0 m s
AI Performance: Off the Chart!
1 .0m s
30 0m s
AI Performance: Off the Chart!
Preconditions: Visibility
1 .0m s
30 0m s
AI Performance: Off the Chart!
Preconditions: Visibility Pathfinding
1 .0m s
30 0m s
AI Performance: Off the Chart!
Preconditions: Visibility Pathfinding Tactical Position
Validity
1 .0m s
30 0m s
Solution
Re-consider Agent ArchitectureDistributed processingCaching
Solution
Re-consider Agent ArchitectureDistributed processingCaching
Inspiration:MIT Media Lab’s C4Robotics
Solution
SensorsWorkingMemory
Planner
BlackboardNav igation
Animation /Movement
Targeting
Weapons
World
World
What is a soldier?
What is a soldier?
What is a soldier?
What is a soldier?
Sensors: See
What is a soldier?
Sensors: See Hear
What is a soldier?
Sensors: See Hear Feel pain
What is a soldier?
Subsystems: Navigate / Move
What is a soldier?
Subsystems: Navigate / Move Attention Selection
(Targeting)
What is a soldier?
Subsystems: Navigate / Move Attention Selection
(Targeting) Weapons
Distributed Processing with Sensors
Pseudo-parallel Amortize precondition processing across
many frames Allow incremental processing
Distributed Processing with Sensors
More than Vision, Hearing, Touch Tactical analysis Internal desires
Distributed Processing with Sensors
Update:– Every frame – Periodic Polling – Event-driven
Limit total number of expensive sensor updates per frame.
Sensor Example: CoverNode Sensor
Update 3 times / second
E
A
Sensor Example: Cover Node Sensor
Update 3 times / second
E
A
Sensor Example: PassTarget Sensor
Incremental update
AE
Sensor Example: PassTarget Sensor
Incremental update
AE
Sensor Example: PassTarget Sensor
Incremental update
AE
Sensor Example: PassTarget Sensor
Incremental update
AE
Sensor Example: PassTarget Sensor
Incremental update
AE
Sensor Example: PassTarget Sensor
Incremental update
AE
Sensor Example: PassTarget Sensor
Incremental update
AE
Sensor Example: PassTarget Sensor
Incremental update
A
E
Caching
Sensor
Sensor
Sensor
WorkingMemory
Targeting
Planning
Caching: WorkingMemoryFacts
Facts are really beliefs Common representation of all knowledge:
– Character– Object– Node– Disturbance– Task– PathInfo– Desire
Caching: WorkingMemoryFacts
WorkingMemoryFact {
Attribute<Vector3D> PositionAttribute<Vector3D> DirectionAttribute<StimulusType> StimulusAttribute<HANDLE> ObjectAttribute<DesireType> Desire
// 16 attributes total...}
Caching: WorkingMemoryFacts
Attribute<Type>{
Type Valuefloat fConfidence
}
Caching: WorkingMemoryFacts
SeeEnemy Sensor: Stimulus Confidence
C = 0.0
C = 0.0
Caching: WorkingMemoryFacts
SeeEnemy Sensor: Stimulus Confidence
C = 1.0
C = 1.0
Caching: WorkingMemoryFacts
SeeEnemy Sensor: Stimulus Confidence
C = 1.0
C = 0.63
Caching: WorkingMemoryFacts
SeeEnemy Sensor: Stimulus Confidence
C = 1.0
C = 0.0
Caching: WorkingMemoryFacts
CoverNode Sensor: Position Confidence
E
A
C=0.2
C=0.1C=0.4 C=0.9
C=0.8
The Planning Process
The Planning Process
P
The Planning Process
P
The Planning Process
P
The Plan
Goal: KillEnemy
Plan:
Goto (couch)
UseObject (couch)
Goto (coverNode)
AttackFromCover
The Plan
Plan:
Goto (couch)
UseObject (couch)
Goto (coverNode)
AttackFromCover
Goal: KillEnemy
The Planning Process
Goal: KillEnemy
Satisfaction: ???
Planning Process: Symbols
Simple representation of World State: Array of symbols Fixed number of symbols Symbol = Key, Value pair Union of 4 byte values (int, float, HANDLE, etc)
Planning Process: Symbols
Simple representation of agent’s World State: Array of symbols Fixed number of symbols Symbol = Key, Value pair Union of 4 byte values (int, float, HANDLE, etc)
Examples: kWeaponArmed = TRUE kUsedObject = hSomeObject
Planning Process: Symbols
Goal: KillEnemy
Satisfaction: kTargetIsDead = TRUE
Planning Process: Symbols
Agent-centric representation
Character XStim Conf = 0.6
Character YStim Conf = 1.0
Character ZStim Conf = 0.0
Working Memory
Targeting
Blackboard
T arget =Character Y
Planning Process: Symbols
Agent-centric representation– kTargetIsDead– Subsystem handles target selection.– Select from Character Facts in WorkingMemory.– Less symbols to evaluate during plan formulation.
The Plan
Plan:
Goto (couch)
UseObject (couch)
Goto (coverNode)
AttackFromCover
Goal: KillEnemy
Planning Process: Actions
AttackUse
Object
DrawW eapon Look At
Goto
Reload
Idle
Planning Process: Actions
Action Preconditions & Effects: Represented with same array of symbols Value may be constant or variable
Planning Process: Actions
Attack
Attackfrom
Vehicle
Attackfrom
Cov er
E: kT argetIsDead = T RUE
Planning Process: Actions
Attack
Attackfrom
Vehicle
Attackfrom
Cov er
E: kT argetIsDead = T RUE
Cost = 5Cost = 3
Cost = 1
Planning Process: Context Preconditions
Context preconditions Filter function of arbitrary code
– Am I riding a vehicle?– Does a path exist?– Is object visible?– Is valid cover available?
Symbolic preconditions are only used when we want the planner to chain a preceding action.
Planning Process: Context Preconditions
A: AttackFromCover
bool ValidateContextPreconditions()
{
// Is target visible?
...
// Does a valid node
// exist?
}
Character X
Working Memory
Character Z
Character Y
CoverNode576
CoverNode34
CoverNode29
Planning Process: Context Preconditions
Query QQ.Type = kFact_NodeQ.Node = kNode_Cover
Fact* pFact = FindFactMax( Q, kAtt_Position )
Finds the WMFact with highest confidence in some attribute.
If returns NULL, no valid cover exists.
The Plan
Plan:
Goto (couch)
UseObject (couch)
Goto (coverNode)
AttackFromCover
Goal: KillEnemy
Planning Process: Variables
G: KillEnemyS: kTargetIsDead = TRUE
A: AttackFromCoverE: kTargetIsDead = TRUE
Planning Process: Variables
G: KillEnemyS: kTargetIsDead = TRUE
A: AttackFromCoverE: kTargetIsDead = TRUE
P: kWeaponArmed = TRUE
kWeaponLoaded = TRUE
kAtObject = ???
Planning Process: Variables
A: AttackFromCoverE: kTargetIsDead = TRUE
P: kWeaponArmed = TRUE
kWeaponLoaded = TRUE
kAtObject = FindFactMax( Q, kAtt_Position )
Planning Process: Variables
A: GotoE: kAtObject = Parent.kAtObject
Planning Process: Variables
A: AttackFromCoverP: kAtObject =
A: GotoE: kAtObject =
FindFactMax( Q, kAtt_Position )
Parent.kAtObject
Planning Process: Variables
KillEnemyAttackFromCover
Goto
The Plan
Plan:
Goto (couch)
UseObject (couch)
Goto (coverNode)
AttackFromCover
Goal: KillEnemy
Planning Process: Variables
A: Goto
E: kAtObject = Parent.kAtObject
Planning Process: Variables
A: GotoE: kAtObject =
P: kUsedObject =
Parent.kAtObject
GetDependency( Parent.kAtObject )
The Plan
Plan:
Goto (couch)
UseObject (couch)
Goto (coverNode)
AttackFromCover
Goal: KillEnemy
Was It Worth It?
Was It Worth It?
Did we handle? Managing dependencies Sharing behaviors
Additional benefit: Replanning
Replanning
PA
Replanning
PA
Replanning
P A
Replanning
P A
Replanning
P A
Parting Thoughts
Warcraft (1994)
Parting Thoughts
Star Wars (1987)
Parting Thoughts
F.E.A.R. (2005)
Parting Thoughts
2D 3D
Pre-rendered Real-time rendered
graphics graphics
Parting Thoughts
Scripts, FSMs Planning
Pre-rendered Real-time simulated
reasoning & behavior reasoning & behavior
What new types of gameplay are possible?!
Questions?
Jeff Orkin
Monolith Productions
http://www.jorkin.com