Post on 14-Dec-2015
© HVR Consulting Services Ltd
What Influences a Decision?What Influences a Decision?
David Daniel David Daniel HVR-CSLHVR-CSL
John HoltJohn Holt HVR-CSLHVR-CSL
Graham MathiesonGraham Mathieson DstlDstl
© HVR Consulting Services Ltd
OverviewOverview
Background Background – Experimental work to determine the impact of Experimental work to determine the impact of
information on decision-makinginformation on decision-makingSingle decision game Single decision game – Overview of earlier work by DanielOverview of earlier work by Daniel
Study comparisonsStudy comparisons– Review similar work over the last 25 yearsReview similar work over the last 25 years
Recent gamingRecent gamingConclusionsConclusions
© HVR Consulting Services Ltd
BackgroundBackground
A simple, single decision game used to A simple, single decision game used to investigate the impact of level of information on investigate the impact of level of information on quality of decision (Daniel, 1977)quality of decision (Daniel, 1977)– Based on pioneering work of Sutcliffe (1971)Based on pioneering work of Sutcliffe (1971)– Found that individual player’s ability had greater Found that individual player’s ability had greater
impact than information levelimpact than information level– Result has been replicated a number of timesResult has been replicated a number of times– John Holt, a psychology student, joined the team, to John Holt, a psychology student, joined the team, to
help evaluate the behavioural factorshelp evaluate the behavioural factors– Graham Mathieson has been active in this research Graham Mathieson has been active in this research
over the last few yearsover the last few years– This presentation provides a re-appraisal, 25 years This presentation provides a re-appraisal, 25 years
on, bringing together past and present perspectiveson, bringing together past and present perspectives
© HVR Consulting Services Ltd
© HVR Consulting Services Ltd
Single Decision GameSingle Decision Game
E. Quality of player’s decisions used to E. Quality of player’s decisions used to assess impact of informationassess impact of information
A. Use single player as advisor/ decision makerA. Use single player as advisor/ decision maker
B. Player given varying data levels asB. Player given varying data levels asdetermined by experimental designdetermined by experimental design
C. Player takes single decisionC. Player takes single decision
D. No learning effect between games allowedD. No learning effect between games allowed
A1 QuestionnairesA1 QuestionnairesA2 Thinking aloud/ recording of pairs dialogue A2 Thinking aloud/ recording of pairs dialogue
Cognitive Data GatheringCognitive Data Gathering
© HVR Consulting Services Ltd
Single Decision GameSingle Decision Game
1.1. Differences between players far greater impact than Differences between players far greater impact than differences in informationdifferences in information
2.2. Effect of prior information twice as great as other Effect of prior information twice as great as other informationinformation
3.3. Pairs effectPairs effect
4.4. Tighten game control – contol of game pace; provide Tighten game control – contol of game pace; provide secondary taskssecondary tasks
5.5. Players perform differently if their role is changedPlayers perform differently if their role is changed
ResultsResults
© HVR Consulting Services Ltd
Single Decision GameSingle Decision Game
to the Red advanceto the Red advance
© HVR Consulting Services Ltd
Single Decision GameSingle Decision Game
to the Red advanceto the Red advance
© HVR Consulting Services Ltd
Single Decision GameSingle Decision Game
to the Red advanceto the Red advance
© HVR Consulting Services Ltd
Study ComparisonsStudy Comparisons
Rose and Sutcliffe (1971) - Submarine Rose and Sutcliffe (1971) - Submarine CommanderCommander
Pickburn and Davis (1990) - ASW assessmentPickburn and Davis (1990) - ASW assessment Davis and Reimann (1991) - Task Force CdrDavis and Reimann (1991) - Task Force Cdr Perry and Moffat (1994) - Task Force CdrPerry and Moffat (1994) - Task Force Cdr McCarthy, Mathieson, Osborn, Rugg-Gunn and McCarthy, Mathieson, Osborn, Rugg-Gunn and
Payne (2001) - RN Principle Warfare Officer Payne (2001) - RN Principle Warfare Officer (PWO)(PWO)
Mathieson and Malish (2002) - Extend PWO Mathieson and Malish (2002) - Extend PWO resultsresults
© HVR Consulting Services Ltd
Study ComparisonsStudy Comparisons
A. Use single player as A. Use single player as advisor/ decision makeradvisor/ decision maker
B. Player given varying data levels asB. Player given varying data levels asdetermined by experimental designdetermined by experimental design
C. Player takes single decisionC. Player takes single decision
D. No learning effect between games allowedD. No learning effect between games allowed
E. Quality of player’s decisions used to E. Quality of player’s decisions used to assess impact of informationassess impact of information
A1 QuestionnairesA1 Questionnaires
A2 Thinking aloud/ recording of pairs dialogue A2 Thinking aloud/ recording of pairs dialogue
Cognitive Data GatheringCognitive Data Gathering
A.A. Used in all gamesUsed in all games
B.B. Used in all games Used in all games
C.C. Used in all gamesUsed in all games
D.D. Used in all gamesUsed in all games
E.E. Used in all except Used in all except McCarthy – correlation McCarthy – correlation between info and sets of between info and sets of COAs & geospatialCOAs & geospatial
A1 Two of the games usedA1 Two of the games used
A2A2 Moffat and Perry tape Moffat and Perry tape recorded dialoguerecorded dialogue
© HVR Consulting Services Ltd
Study ComparisonsStudy Comparisons
1.1. Differences between players far greater Differences between players far greater impact than differences in informationimpact than differences in information
2.2. Effect of prior information twice as great as Effect of prior information twice as great as other informationother information
3.3. Pairs effectPairs effect
4.4. Tighten game control – contol of game pace; Tighten game control – contol of game pace; provide secondary tasksprovide secondary tasks
5.5. Players perform differently if their role is Players perform differently if their role is changedchanged
ResultsResults
1.1. Significant for all games; Significant for all games; McCarthy sig for COA but McCarthy sig for COA but not for geospatialnot for geospatial
2.2. Prior info was significant in Prior info was significant in two gamestwo games
3.3. Not looked at in any studyNot looked at in any study
4.4. McCarthy controlled time; McCarthy controlled time; secondary task not used secondary task not used againagain
5.5. Not looked at againNot looked at again
© HVR Consulting Services Ltd
Study ComparisonsStudy Comparisons
Behaviour at least as important as information Behaviour at least as important as information flowsflows
Players acting as advisorsPlayers acting as advisors– Need to be aware of in future experimentsNeed to be aware of in future experiments
Correlation measures a useful innovation, Correlation measures a useful innovation, – Does no evaluation of players’ performance reduce Does no evaluation of players’ performance reduce
the pay-off from the gaming?the pay-off from the gaming?
Effect of player prior experience important Effect of player prior experience important impact on use of informationimpact on use of information
© HVR Consulting Services Ltd
Study ComparisonsStudy Comparisons
Moffat and Perry (1997) provide extensive Moffat and Perry (1997) provide extensive gaming methodology to correlate information, gaming methodology to correlate information, decisions and outcome, based on players decisions and outcome, based on players asking for informationasking for information– Regardless of decisions, two information sets Regardless of decisions, two information sets
dominated the choice of questions askeddominated the choice of questions asked– Regardless of information requested, players decided Regardless of information requested, players decided
between two specific courses of actionbetween two specific courses of action Not able to establish relationship between information Not able to establish relationship between information
requested and decisions takenrequested and decisions taken
– Demonstrated importance of internal influencesDemonstrated importance of internal influences
© HVR Consulting Services Ltd
Recent GamingRecent Gaming
Mathieson and Malish (2002)Mathieson and Malish (2002)– Gary Klein’s Recognition-Primed Decision- Gary Klein’s Recognition-Primed Decision-
making (RPD) model of decision makingmaking (RPD) model of decision making– Provides psychological perspectiveProvides psychological perspective– Initial work based on field studies of fire Initial work based on field studies of fire
chiefs in USchiefs in US• Decision makers match the patterns of cues and Decision makers match the patterns of cues and
indicators from past experienceindicators from past experience• Recognition provides access to pre-learned Recognition provides access to pre-learned
response cuesresponse cues
© HVR Consulting Services Ltd
Recent GamingRecent Gaming
Mathieson used Klein ideas to help Mathieson used Klein ideas to help structure gamestructure game– Placed subjects under time pressurePlaced subjects under time pressure– Get first reaction responseGet first reaction response• Subject briefed, given information updates then Subject briefed, given information updates then
askedasked– What action?What action?– What is the situation?What is the situation?– What are the key indicators?What are the key indicators?– Having thought, do you wish to amend your action?Having thought, do you wish to amend your action?
Used Endsley measure of situation Used Endsley measure of situation awarenessawareness
© HVR Consulting Services Ltd
Recent GamingRecent Gaming
Results and conclusionsResults and conclusions– Players not just driven by informationPlayers not just driven by information
• Information appears to drive Information appears to drive wherewhere players act players act
• Personality appears to dominate Personality appears to dominate howhow they act they act
– No relationship between course of action and level of No relationship between course of action and level of situation awarenesssituation awareness
– Demonstrated feasibility of studying links between Demonstrated feasibility of studying links between command behaviour and aspects of situation command behaviour and aspects of situation awarenessawareness
Future workFuture work– Land basedLand based– Repertory grids (in-depth construct elicitation) Repertory grids (in-depth construct elicitation)
• Accesses hidden knowledge in a different way, similar to Accesses hidden knowledge in a different way, similar to Moffat and PerryMoffat and Perry
© HVR Consulting Services Ltd
ConclusionsConclusions
Single decision gaming is a well established Single decision gaming is a well established potential research tool, with large data set now potential research tool, with large data set now available available – Wider use of the approach would be desirableWider use of the approach would be desirable– More research required on information and decision-More research required on information and decision-
makingmaking
Research on behavioural aspects at least as Research on behavioural aspects at least as important as information levelimportant as information level
Need more methods to investigate information Need more methods to investigate information and decision-making (e.g. Continuing the and decision-making (e.g. Continuing the approach of Moffat & Perry)approach of Moffat & Perry)
© HVR Consulting Services Ltd
Unanswered questionsUnanswered questions Could additional selecting or training for Could additional selecting or training for
commanders be helpful?commanders be helpful? Does the apparent dominance of prior Does the apparent dominance of prior
information diminish the value of digitization, information diminish the value of digitization, and similar initiatives? and similar initiatives?
Should the C2 systems be reduced in scope to Should the C2 systems be reduced in scope to provide information that will actually be used provide information that will actually be used by commanders?by commanders?
Is there a need to introduce information Is there a need to introduce information systems personalised to individual systems personalised to individual commanders?commanders?
ConclusionsConclusions
© HVR Consulting Services Ltd
For further detailsFor further details
Dr John Holt Dr John Holt HVR Consulting ServicesHVR Consulting ServicesSelborne HouseSelborne HouseMill LaneMill LaneAltonAltonHampshireHampshireGU34 2QJGU34 2QJ
Tel: 01420 87977Tel: 01420 87977
Email: Email: john.holt@hvr-csl.co.uk
Website: www.hvrgroup.comWebsite: www.hvrgroup.com