© HVR Consulting Services Ltd AI for OOTW Representing Plausible Behaviour in OOTW Simulators.

© HVR Consulting Services Ltd

AI for OOTWAI for OOTWRepresenting Representing

Plausible Behaviour in Plausible Behaviour in OOTW SimulatorsOOTW Simulators


Steve CurramSteve Curram

Mirek GlinieckiMirek GlinieckiModelling and Analysis GroupModelling and Analysis Group

HVR Consulting Services LtdHVR Consulting Services Ltd

Selborne House, Mill LaneSelborne House, Mill Lane

Alton, HampshireAlton, Hampshire

GU34 2QJGU34 2QJ

[email protected]@hvr-csl.co.uk

Mark CusackMark Cusack

Nick ExonNick ExonDistributed Technologies GroupDistributed Technologies Group

QinetiQ LtdQinetiQ Ltd

St Andrews RoadSt Andrews Road

Malvern, WorcestershireMalvern, Worcestershire

WR14 3PSWR14 3PS

[email protected]@signal.qinetiq.com


BackgroundBackground

Purpose of overall study is to create simulators Purpose of overall study is to create simulators that allow stress testing of Command Information that allow stress testing of Command Information Systems and the people who use them in an OOTW Systems and the people who use them in an OOTW situationsituation

Features:Features: Multiple factions and agenciesMultiple factions and agencies Highly political environmentHighly political environment Restrictive RoERestrictive RoE Media coverage and public opinionMedia coverage and public opinion Multiple information sourcesMultiple information sources


Background (2)Background (2)

Purpose of simulators for use by commanders is to Purpose of simulators for use by commanders is to provide a challenging and thought provoking provide a challenging and thought provoking environmentenvironment

The simulators are The simulators are notnot designed to be analysis tools designed to be analysis tools

They will conform to the High Level Architecture (HLA) They will conform to the High Level Architecture (HLA) using QinetiQ’s BeanXS environmentusing QinetiQ’s BeanXS environment

Use Java and JavaBeans™ technologyUse Java and JavaBeans™ technology

AI is only one component of the studyAI is only one component of the study

Quick and cheap to set upQuick and cheap to set up


Study ApproachStudy Approach

Identification of behaviour types and requirements Identification of behaviour types and requirements of AI for OOTW simulators of AI for OOTW simulators

Review of technology and case studies to generate Review of technology and case studies to generate shortlist of approaches for further investigationshortlist of approaches for further investigation

Develop stand-alone AI objects for analysisDevelop stand-alone AI objects for analysis

Test AI objects in an integrated environmentTest AI objects in an integrated environment

Prototype in a small simulation modelPrototype in a small simulation model


Aims of the AIAims of the AI

Provide Provide plausibleplausible behaviour in simulation from a behaviour in simulation from a player’s perspectiveplayer’s perspective

– computer generated actorscomputer generated actors

– occurrence of eventsoccurrence of events

Give player something to think aboutGive player something to think about

Be responsive to player’s actionsBe responsive to player’s actions

Not to be too transparent - player should be Not to be too transparent - player should be immersed in environment rather than trying to immersed in environment rather than trying to beat the AIbeat the AI


Soft AI It is more important that behaviour It is more important that behaviour appearsappears

plausible to a player rather than for the plausible to a player rather than for the mechanism to conform to any psychological mechanism to conform to any psychological theoriestheories

The AI should be reasonably quick and easy to The AI should be reasonably quick and easy to set upset up

Probably more akin to the requirements for Probably more akin to the requirements for commercial computer games than traditional commercial computer games than traditional military modelsmilitary models

It’s Artificial Intelligence, BUTIt’s Artificial Intelligence, BUT


Approaches ConsideredApproaches Considered

Random SamplingRandom Sampling - does not take account of conditions, - does not take account of conditions, though not entirely predictable - could be combined with though not entirely predictable - could be combined with other approachesother approaches

Knowledge-Based Systems (& related)Knowledge-Based Systems (& related) - can be too - can be too predictable if rules are simple, while ease of setting up falls predictable if rules are simple, while ease of setting up falls rapidly as rules become more complexrapidly as rules become more complex

Fuzzy LogicFuzzy Logic - rules are generally simpler but more powerful - rules are generally simpler but more powerful than KBSs, needs some experience in choice of fuzzy than KBSs, needs some experience in choice of fuzzy regions, linguistic terms can help trainers in tweaking regions, linguistic terms can help trainers in tweaking scenariosscenarios

Case-Based ReasoningCase-Based Reasoning - full systems can be time consuming - full systems can be time consuming to set up and have overhead of a shell, simpler versions may to set up and have overhead of a shell, simpler versions may be useful for recognising when scripted events can occurbe useful for recognising when scripted events can occur


Approaches Considered (2)Approaches Considered (2)

Finite State MachinesFinite State Machines - commonly used in computer games, - commonly used in computer games, simple to set up, can get predictable for player simple to set up, can get predictable for player

Fuzzy State MachinesFuzzy State Machines - adaptation of Finite State Machines - adaptation of Finite State Machines using fuzzy rules, becoming popular in computer games, using fuzzy rules, becoming popular in computer games, membership functions could be used for random samplingmembership functions could be used for random sampling

Bayesian Belief NetworksBayesian Belief Networks - powerful representation of factors - powerful representation of factors on behaviour, can be difficult to set up, difficulty of validationon behaviour, can be difficult to set up, difficulty of validation

Neural Networks (various types) Neural Networks (various types) - adaptive, powerful - adaptive, powerful representation but time consuming and difficult to set up and representation but time consuming and difficult to set up and validatevalidate

Genetic AlgorithmsGenetic Algorithms - adaptive, but too slow for interactive - adaptive, but too slow for interactive simulationsimulation


Short-listed ApproachesShort-listed Approaches

Finite State Machine (FSM)Finite State Machine (FSM)– for simple behaviourfor simple behaviour– create generic FSM object to which states and create generic FSM object to which states and

transitions are addedtransitions are added– easy to set upeasy to set up

Patrol Alert

Retreat Com bat

START STATE TRANSITIONCONDITION

END STATE

Patrol Sound Heard AlertPatrol Armed Unit Spotted AlertPatrol Fired On CombatAlert Unit Recognised &

Attitude Hostile &Unit Outnumbered

Combat

Combat SignificantlyOutnumbered

Retreat


Short-listed ApproachesShort-listed Approaches

Fuzzy State Machine (FuSM)Fuzzy State Machine (FuSM)– for more complex behaviourfor more complex behaviour– linguistic or numerical inputslinguistic or numerical inputs– smoother transitions between statessmoother transitions between states– create generic FuSM to which terms and fuzzy rules create generic FuSM to which terms and fuzzy rules

can be addedcan be added– state can be selected from centre of gravity or using state can be selected from centre of gravity or using

random sampling from fuzzy regionrandom sampling from fuzzy region


Implementation ConsiderationsImplementation Considerations

Classes developed in Java and converted to Classes developed in Java and converted to JavaBeans™JavaBeans™

Will be used in simulations along with other Will be used in simulations along with other JavaBeans™JavaBeans™

They need to pick up messages and hold their They need to pick up messages and hold their own stateown state

Need to respond to a request or simulation timeNeed to respond to a request or simulation time

Behaviour should be able to be time-scaleableBehaviour should be able to be time-scaleable


Fuzzy State MachineFuzzy State Machine

Fuzzy State Machine and test rig Fuzzy State Machine and test rig (shown) developed(shown) developed

Allows absolute rules -outcome is Allows absolute rules -outcome is state membership valuesstate membership values

Allows change rules - outcome is Allows change rules - outcome is amount of change to current state amount of change to current state valuevalue

States and rules can be read-in States and rules can be read-in using a scripting languageusing a scripting language

JavaBean version developedJavaBean version developed Wrapper/Interpreter allows Wrapper/Interpreter allows

FuSMBean to interact with rest of FuSMBean to interact with rest of simulation simulation


Behaviour Type - AttitudeBehaviour Type - Attitude

Attitude of an actor towards other actorsAttitude of an actor towards other actors– may differ at different levels, e.g. faction leaders may differ at different levels, e.g. faction leaders

towards UN, or faction soldier towards UNtowards UN, or faction soldier towards UN– use Finite State Machine or Fuzzy State Machineuse Finite State Machine or Fuzzy State Machine

H o s tile F r ie n d lyS y m p a th e ticN e u tra lM is tru s tfu l

C u rren tA ttitu d e

F reed o m o fM o vem en t

U N S an c tio n sA ga in stF ac tio n

G ro u n d C ap tu reIn c id en ts o f

U N P ro tec tio n

In c id en ts o fU N C o u n te r-

ac tio n

NeutralMistrustful NeutralMistrustful


Behaviour Type - NegotiationBehaviour Type - Negotiation

Negotiation between actors Negotiation between actors – manual:manual: player has role as a mediator, and suggests player has role as a mediator, and suggests

compromise solutionscompromise solutions– automatic:automatic: compromise solutions based on global compromise solutions based on global

approach to negotiation settingsapproach to negotiation settings– response of actors depends on the weighted distance response of actors depends on the weighted distance

from their ideal solution and their attitude towards the from their ideal solution and their attitude towards the negotiationsnegotiations

– makes use of Fuzzy State Machinesmakes use of Fuzzy State Machines– likely to be iterative process over time (may be short likely to be iterative process over time (may be short

or long periods)or long periods)


Behaviour Type - NegotiationBehaviour Type - Negotiation

Distance On Issues

In tra n s ig e n t C o m p lia n tS u p p o r tiv eN e u tra lR e s is ta n t

Attitude to Negotiation

F ac tio nA ttitu d e to U N

P erce iv ed A ttitu d eo f C iv ilianP o p u la tio n

P erce iv edT h rea t fro m

E n em y F ac tio n

P erce iv ed U NC o n fro n ta tio n

L eve l

W eig h ted D iffe ren ceB e tw een D esired an d

O ffered P o sitio n s

O ffe red P o sitio n so n Issu es

R e la tiv e W eigh tin go f Issu es

C lo sen ess o f P ro p o sa lto D esired O u tco m e

(F u zzy S e t)

P o s it io n s o n I s su e s

L a n d A llo c a tio nG o v e rn m e n ta l P o w e rM ilita ry P o w e rE c o n o m ic F re e d o mF in a n c ia l/In f ra s tru c tu re A idU N R e ta in e d P re se n c eL a w A c tio n s A g a in s t P o l/M il L e a d e rs

Degree ofAgreement


Vignette Development (WIP)Vignette Development (WIP)

Two warring factionsTwo warring factionsRefugees from both population groupsRefugees from both population groupsRefugee camps for both groupsRefugee camps for both groups– populations, food, medicine, death ratespopulations, food, medicine, death rates

UN convoys supplying campsUN convoys supplying campsRoad blocks run by militia factionsRoad blocks run by militia factionsMedia coverageMedia coverage Includes: crowd movement, terrain, path-Includes: crowd movement, terrain, path-

finding, attitude, negotiationfinding, attitude, negotiation


Observations So FarObservations So Far

Generic and flexible FSM and FuSMs can be Generic and flexible FSM and FuSMs can be created and converted to JavaBeanscreated and converted to JavaBeans™™

Wrappers control interaction with other Wrappers control interaction with other JavaBeansJavaBeans™™ in the simulation in the simulation

Specific behaviour loaded in from XMLSpecific behaviour loaded in from XMLTypes of behaviour can be stored in librariesTypes of behaviour can be stored in librariesExperimenting with messagingExperimenting with messagingRefining rules - want sufficient power with Refining rules - want sufficient power with

simplest set possible simplest set possible


Questions ?Questions ?

© HVR Consulting Services Ltd AI for OOTW Representing Plausible Behaviour in OOTW Simulators.

Documents

Transcript of © HVR Consulting Services Ltd AI for OOTW Representing Plausible Behaviour in OOTW Simulators.