© HVR Consulting Services Ltd
AI for OOTWAI for OOTWRepresenting Representing
Plausible Behaviour in Plausible Behaviour in OOTW SimulatorsOOTW Simulators
© HVR Consulting Services Ltd
Steve CurramSteve Curram
Mirek GlinieckiMirek GlinieckiModelling and Analysis GroupModelling and Analysis Group
HVR Consulting Services LtdHVR Consulting Services Ltd
Selborne House, Mill LaneSelborne House, Mill Lane
Alton, HampshireAlton, Hampshire
GU34 2QJGU34 2QJ
[email protected]@hvr-csl.co.uk
Mark CusackMark Cusack
Nick ExonNick ExonDistributed Technologies GroupDistributed Technologies Group
QinetiQ LtdQinetiQ Ltd
St Andrews RoadSt Andrews Road
Malvern, WorcestershireMalvern, Worcestershire
WR14 3PSWR14 3PS
[email protected]@signal.qinetiq.com
© HVR Consulting Services Ltd
BackgroundBackground
Purpose of overall study is to create simulators Purpose of overall study is to create simulators that allow stress testing of Command Information that allow stress testing of Command Information Systems and the people who use them in an OOTW Systems and the people who use them in an OOTW situationsituation
Features:Features: Multiple factions and agenciesMultiple factions and agencies Highly political environmentHighly political environment Restrictive RoERestrictive RoE Media coverage and public opinionMedia coverage and public opinion Multiple information sourcesMultiple information sources
© HVR Consulting Services Ltd
Background (2)Background (2)
Purpose of simulators for use by commanders is to Purpose of simulators for use by commanders is to provide a challenging and thought provoking provide a challenging and thought provoking environmentenvironment
The simulators are The simulators are notnot designed to be analysis tools designed to be analysis tools
They will conform to the High Level Architecture (HLA) They will conform to the High Level Architecture (HLA) using QinetiQ’s BeanXS environmentusing QinetiQ’s BeanXS environment
Use Java and JavaBeans™ technologyUse Java and JavaBeans™ technology
AI is only one component of the studyAI is only one component of the study
Quick and cheap to set upQuick and cheap to set up
© HVR Consulting Services Ltd
Study ApproachStudy Approach
Identification of behaviour types and requirements Identification of behaviour types and requirements of AI for OOTW simulators of AI for OOTW simulators
Review of technology and case studies to generate Review of technology and case studies to generate shortlist of approaches for further investigationshortlist of approaches for further investigation
Develop stand-alone AI objects for analysisDevelop stand-alone AI objects for analysis
Test AI objects in an integrated environmentTest AI objects in an integrated environment
Prototype in a small simulation modelPrototype in a small simulation model
© HVR Consulting Services Ltd
Aims of the AIAims of the AI
Provide Provide plausibleplausible behaviour in simulation from a behaviour in simulation from a player’s perspectiveplayer’s perspective
– computer generated actorscomputer generated actors
– occurrence of eventsoccurrence of events
Give player something to think aboutGive player something to think about
Be responsive to player’s actionsBe responsive to player’s actions
Not to be too transparent - player should be Not to be too transparent - player should be immersed in environment rather than trying to immersed in environment rather than trying to beat the AIbeat the AI
© HVR Consulting Services Ltd
Soft AI It is more important that behaviour It is more important that behaviour appearsappears
plausible to a player rather than for the plausible to a player rather than for the mechanism to conform to any psychological mechanism to conform to any psychological theoriestheories
The AI should be reasonably quick and easy to The AI should be reasonably quick and easy to set upset up
Probably more akin to the requirements for Probably more akin to the requirements for commercial computer games than traditional commercial computer games than traditional military modelsmilitary models
It’s Artificial Intelligence, BUTIt’s Artificial Intelligence, BUT
© HVR Consulting Services Ltd
Approaches ConsideredApproaches Considered
Random SamplingRandom Sampling - does not take account of conditions, - does not take account of conditions, though not entirely predictable - could be combined with though not entirely predictable - could be combined with other approachesother approaches
Knowledge-Based Systems (& related)Knowledge-Based Systems (& related) - can be too - can be too predictable if rules are simple, while ease of setting up falls predictable if rules are simple, while ease of setting up falls rapidly as rules become more complexrapidly as rules become more complex
Fuzzy LogicFuzzy Logic - rules are generally simpler but more powerful - rules are generally simpler but more powerful than KBSs, needs some experience in choice of fuzzy than KBSs, needs some experience in choice of fuzzy regions, linguistic terms can help trainers in tweaking regions, linguistic terms can help trainers in tweaking scenariosscenarios
Case-Based ReasoningCase-Based Reasoning - full systems can be time consuming - full systems can be time consuming to set up and have overhead of a shell, simpler versions may to set up and have overhead of a shell, simpler versions may be useful for recognising when scripted events can occurbe useful for recognising when scripted events can occur
© HVR Consulting Services Ltd
Approaches Considered (2)Approaches Considered (2)
Finite State MachinesFinite State Machines - commonly used in computer games, - commonly used in computer games, simple to set up, can get predictable for player simple to set up, can get predictable for player
Fuzzy State MachinesFuzzy State Machines - adaptation of Finite State Machines - adaptation of Finite State Machines using fuzzy rules, becoming popular in computer games, using fuzzy rules, becoming popular in computer games, membership functions could be used for random samplingmembership functions could be used for random sampling
Bayesian Belief NetworksBayesian Belief Networks - powerful representation of factors - powerful representation of factors on behaviour, can be difficult to set up, difficulty of validationon behaviour, can be difficult to set up, difficulty of validation
Neural Networks (various types) Neural Networks (various types) - adaptive, powerful - adaptive, powerful representation but time consuming and difficult to set up and representation but time consuming and difficult to set up and validatevalidate
Genetic AlgorithmsGenetic Algorithms - adaptive, but too slow for interactive - adaptive, but too slow for interactive simulationsimulation
© HVR Consulting Services Ltd
Short-listed ApproachesShort-listed Approaches
Finite State Machine (FSM)Finite State Machine (FSM)– for simple behaviourfor simple behaviour– create generic FSM object to which states and create generic FSM object to which states and
transitions are addedtransitions are added– easy to set upeasy to set up
Patrol Alert
Retreat Com bat
START STATE TRANSITIONCONDITION
END STATE
Patrol Sound Heard AlertPatrol Armed Unit Spotted AlertPatrol Fired On CombatAlert Unit Recognised &
Attitude Hostile &Unit Outnumbered
Combat
Combat SignificantlyOutnumbered
Retreat
© HVR Consulting Services Ltd
Short-listed ApproachesShort-listed Approaches
Fuzzy State Machine (FuSM)Fuzzy State Machine (FuSM)– for more complex behaviourfor more complex behaviour– linguistic or numerical inputslinguistic or numerical inputs– smoother transitions between statessmoother transitions between states– create generic FuSM to which terms and fuzzy rules create generic FuSM to which terms and fuzzy rules
can be addedcan be added– state can be selected from centre of gravity or using state can be selected from centre of gravity or using
random sampling from fuzzy regionrandom sampling from fuzzy region
© HVR Consulting Services Ltd
Implementation ConsiderationsImplementation Considerations
Classes developed in Java and converted to Classes developed in Java and converted to JavaBeans™JavaBeans™
Will be used in simulations along with other Will be used in simulations along with other JavaBeans™JavaBeans™
They need to pick up messages and hold their They need to pick up messages and hold their own stateown state
Need to respond to a request or simulation timeNeed to respond to a request or simulation time
Behaviour should be able to be time-scaleableBehaviour should be able to be time-scaleable
© HVR Consulting Services Ltd
Fuzzy State MachineFuzzy State Machine
Fuzzy State Machine and test rig Fuzzy State Machine and test rig (shown) developed(shown) developed
Allows absolute rules -outcome is Allows absolute rules -outcome is state membership valuesstate membership values
Allows change rules - outcome is Allows change rules - outcome is amount of change to current state amount of change to current state valuevalue
States and rules can be read-in States and rules can be read-in using a scripting languageusing a scripting language
JavaBean version developedJavaBean version developed Wrapper/Interpreter allows Wrapper/Interpreter allows
FuSMBean to interact with rest of FuSMBean to interact with rest of simulation simulation
© HVR Consulting Services Ltd
Behaviour Type - AttitudeBehaviour Type - Attitude
Attitude of an actor towards other actorsAttitude of an actor towards other actors– may differ at different levels, e.g. faction leaders may differ at different levels, e.g. faction leaders
towards UN, or faction soldier towards UNtowards UN, or faction soldier towards UN– use Finite State Machine or Fuzzy State Machineuse Finite State Machine or Fuzzy State Machine
H o s tile F r ie n d lyS y m p a th e ticN e u tra lM is tru s tfu l
C u rren tA ttitu d e
F reed o m o fM o vem en t
U N S an c tio n sA ga in stF ac tio n
G ro u n d C ap tu reIn c id en ts o f
U N P ro tec tio n
In c id en ts o fU N C o u n te r-
ac tio n
NeutralMistrustful NeutralMistrustful
© HVR Consulting Services Ltd
Behaviour Type - NegotiationBehaviour Type - Negotiation
Negotiation between actors Negotiation between actors – manual:manual: player has role as a mediator, and suggests player has role as a mediator, and suggests
compromise solutionscompromise solutions– automatic:automatic: compromise solutions based on global compromise solutions based on global
approach to negotiation settingsapproach to negotiation settings– response of actors depends on the weighted distance response of actors depends on the weighted distance
from their ideal solution and their attitude towards the from their ideal solution and their attitude towards the negotiationsnegotiations
– makes use of Fuzzy State Machinesmakes use of Fuzzy State Machines– likely to be iterative process over time (may be short likely to be iterative process over time (may be short
or long periods)or long periods)
© HVR Consulting Services Ltd
Behaviour Type - NegotiationBehaviour Type - Negotiation
Distance On Issues
In tra n s ig e n t C o m p lia n tS u p p o r tiv eN e u tra lR e s is ta n t
Attitude to Negotiation
F ac tio nA ttitu d e to U N
P erce iv ed A ttitu d eo f C iv ilianP o p u la tio n
P erce iv edT h rea t fro m
E n em y F ac tio n
P erce iv ed U NC o n fro n ta tio n
L eve l
W eig h ted D iffe ren ceB e tw een D esired an d
O ffered P o sitio n s
O ffe red P o sitio n so n Issu es
R e la tiv e W eigh tin go f Issu es
C lo sen ess o f P ro p o sa lto D esired O u tco m e
(F u zzy S e t)
P o s it io n s o n I s su e s
L a n d A llo c a tio nG o v e rn m e n ta l P o w e rM ilita ry P o w e rE c o n o m ic F re e d o mF in a n c ia l/In f ra s tru c tu re A idU N R e ta in e d P re se n c eL a w A c tio n s A g a in s t P o l/M il L e a d e rs
Degree ofAgreement
© HVR Consulting Services Ltd
Vignette Development (WIP)Vignette Development (WIP)
Two warring factionsTwo warring factionsRefugees from both population groupsRefugees from both population groupsRefugee camps for both groupsRefugee camps for both groups– populations, food, medicine, death ratespopulations, food, medicine, death rates
UN convoys supplying campsUN convoys supplying campsRoad blocks run by militia factionsRoad blocks run by militia factionsMedia coverageMedia coverage Includes: crowd movement, terrain, path-Includes: crowd movement, terrain, path-
finding, attitude, negotiationfinding, attitude, negotiation
© HVR Consulting Services Ltd
Observations So FarObservations So Far
Generic and flexible FSM and FuSMs can be Generic and flexible FSM and FuSMs can be created and converted to JavaBeanscreated and converted to JavaBeans™™
Wrappers control interaction with other Wrappers control interaction with other JavaBeansJavaBeans™™ in the simulation in the simulation
Specific behaviour loaded in from XMLSpecific behaviour loaded in from XMLTypes of behaviour can be stored in librariesTypes of behaviour can be stored in librariesExperimenting with messagingExperimenting with messagingRefining rules - want sufficient power with Refining rules - want sufficient power with
simplest set possible simplest set possible
© HVR Consulting Services Ltd
Questions ?Questions ?
Top Related