Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon...
-
Upload
susan-randall -
Category
Documents
-
view
216 -
download
0
description
Transcript of Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon...
![Page 1: Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon Fincham Michelle Moon Ben Poole Yulin Qin.](https://reader035.fdocuments.net/reader035/viewer/2022062905/5a4d1ae37f8b9ab059978225/html5/thumbnails/1.jpg)
Modeling Space FortressAbraham Anderson
John AndersonShawn BettsDan Bothell
Jennifer FerrisJon Fincham
Michelle MoonBen PooleYulin Qin
![Page 2: Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon Fincham Michelle Moon Ben Poole Yulin Qin.](https://reader035.fdocuments.net/reader035/viewer/2022062905/5a4d1ae37f8b9ab059978225/html5/thumbnails/2.jpg)
The Decomposition Hypothesis•Performance on complex tasks can be decomposed into primitive sets of mental operations of the sort revealed in simple laboratory experiments.
•This assumption underlies much of the work on latency in basic neuroimaging analyses.
•Nonetheless there has been very mixed evidence on part-to-whole training.
•This is generally thought to reflect that fact that one cannot use “pure insertion” because the one task will change when you insert another.
•Most sophisticated analyses of reaction time and imaging rely on some sort of “additive factors” where you keep all the processes but just make them more or less difficult.
•Still within the context of a cognitive model one ought to be able to decide when pure insertion will work and when it will not.
![Page 3: Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon Fincham Michelle Moon Ben Poole Yulin Qin.](https://reader035.fdocuments.net/reader035/viewer/2022062905/5a4d1ae37f8b9ab059978225/html5/thumbnails/3.jpg)
Pure Insertion in Space Fortress•Insert Mines or Fortress into game and check brain pattern
•Naïve Insertion: Full Game = Fortress+Mines
•However, this ignores the fact that we are really getting a double count of navigation and visual processing on the right hand side.
-Mines +Mines-Fortress Orbit Mines+Fortress Fortress Both
•Sophisticated Insertion: Orbit+Both=Mines+Fortress
•However, this ignores that there are points in time when one cannot do both fortress & mines in the both condition.
•One needs a model that will perform all of the conditions and look at its predictions for these conditions.
![Page 4: Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon Fincham Michelle Moon Ben Poole Yulin Qin.](https://reader035.fdocuments.net/reader035/viewer/2022062905/5a4d1ae37f8b9ab059978225/html5/thumbnails/4.jpg)
Game structure
• Played 16 games per session – Four of each condition
• 10 sessions 2/day• Sessions 2 and 10 with fMRI– Analyzed the 20s “Maybe mine” segments
Minute 1 3 IFF letters Condition Game Starts Maybe Mine
Minute 2 Maybe Mine Maybe Mine Maybe Mine
Minute 3 Maybe Mine Maybe Mine Maybe Mine
Minute 4 Maybe Mine Game Ends Feedback Fixation
Fixation
![Page 5: Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon Fincham Michelle Moon Ben Poole Yulin Qin.](https://reader035.fdocuments.net/reader035/viewer/2022062905/5a4d1ae37f8b9ab059978225/html5/thumbnails/5.jpg)
Changes from “regular” SF
• Created the separate conditions• Use a keyboard instead of joystick• Fixed timing for mine onsets• No need to choose between points and shots• Fortress vulnerable while mine on screen
![Page 6: Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon Fincham Michelle Moon Ben Poole Yulin Qin.](https://reader035.fdocuments.net/reader035/viewer/2022062905/5a4d1ae37f8b9ab059978225/html5/thumbnails/6.jpg)
0
1000
2000
3000
4000
5000
6000
7000
1 2 3 4 5 6 7 8 9 10Sessions
Poin
ts
Orbit Fortress Mines Both Mines+Fortress Orbit+Both
Points and Point Additivity
•The major point growth is in the fortress conditions•There is a necessary sub-additivity in points
![Page 7: Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon Fincham Michelle Moon Ben Poole Yulin Qin.](https://reader035.fdocuments.net/reader035/viewer/2022062905/5a4d1ae37f8b9ab059978225/html5/thumbnails/7.jpg)
Modeling Plan• Model the well practiced performance
• Consider learning later• Use participants fortress condition for sessions 7-9 as
the basis for the model’s navigating• Run model through all conditions• Compare it to session 9 behavioral data and session
10 fMRI data
![Page 8: Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon Fincham Michelle Moon Ben Poole Yulin Qin.](https://reader035.fdocuments.net/reader035/viewer/2022062905/5a4d1ae37f8b9ab059978225/html5/thumbnails/8.jpg)
![Page 9: Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon Fincham Michelle Moon Ben Poole Yulin Qin.](https://reader035.fdocuments.net/reader035/viewer/2022062905/5a4d1ae37f8b9ab059978225/html5/thumbnails/9.jpg)
![Page 10: Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon Fincham Michelle Moon Ben Poole Yulin Qin.](https://reader035.fdocuments.net/reader035/viewer/2022062905/5a4d1ae37f8b9ab059978225/html5/thumbnails/10.jpg)
Flight data
![Page 11: Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon Fincham Michelle Moon Ben Poole Yulin Qin.](https://reader035.fdocuments.net/reader035/viewer/2022062905/5a4d1ae37f8b9ab059978225/html5/thumbnails/11.jpg)
Keypresses per game
![Page 12: Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon Fincham Michelle Moon Ben Poole Yulin Qin.](https://reader035.fdocuments.net/reader035/viewer/2022062905/5a4d1ae37f8b9ab059978225/html5/thumbnails/12.jpg)
Keypress duration percentages
![Page 13: Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon Fincham Michelle Moon Ben Poole Yulin Qin.](https://reader035.fdocuments.net/reader035/viewer/2022062905/5a4d1ae37f8b9ab059978225/html5/thumbnails/13.jpg)
Scores
• Mine + Fortress = 7373.425 > Orbit + Full = 6959.95
![Page 14: Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon Fincham Michelle Moon Ben Poole Yulin Qin.](https://reader035.fdocuments.net/reader035/viewer/2022062905/5a4d1ae37f8b9ab059978225/html5/thumbnails/14.jpg)
Model total scores
![Page 15: Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon Fincham Michelle Moon Ben Poole Yulin Qin.](https://reader035.fdocuments.net/reader035/viewer/2022062905/5a4d1ae37f8b9ab059978225/html5/thumbnails/15.jpg)
Changes to ACT-R
• Special device for interfacing with the game• Using the Temporal module• Modified – Motor module– Visual module– Imaginal module
![Page 16: Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon Fincham Michelle Moon Ben Poole Yulin Qin.](https://reader035.fdocuments.net/reader035/viewer/2022062905/5a4d1ae37f8b9ab059978225/html5/thumbnails/16.jpg)
Motor module changes• Set motor feature prep time to 0ms– Suggested by Wayne based on Kieras’s 2009 ICCM paper
• Added mostly independent hands– Two execution paths
• Added individual finger queries?finger-check>
left-index busy
• Added new motor actions– Hold-key– Release-key– Release-all-keys– Delayed-punch
![Page 17: Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon Fincham Michelle Moon Ben Poole Yulin Qin.](https://reader035.fdocuments.net/reader035/viewer/2022062905/5a4d1ae37f8b9ab059978225/html5/thumbnails/17.jpg)
Delayed-punch• Holds a finger down for a prespecified time
+manual> isa delayed-punch hand left-or-right finger finger-name {delay time}
• Substitute for a strong visual-motor interface• For the model (mostly) only a few times used– Orbiting 90ms, 125ms, 160ms– Firing 70ms, 90ms, 125ms– IFF 125ms, 160ms– Mine aiming special
![Page 18: Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon Fincham Michelle Moon Ben Poole Yulin Qin.](https://reader035.fdocuments.net/reader035/viewer/2022062905/5a4d1ae37f8b9ab059978225/html5/thumbnails/18.jpg)
Visual features
• Standard text items – foes, conditions, bonus symbols, vlnr count, iff letter
• “Featureless” indicators – no information other than presence or absence – fortress, fortress hit, fortress explosion, ship
explosion, mine, mine explosion, trial over• Ship– Contains most of the critical information
![Page 19: Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon Fincham Michelle Moon Ben Poole Yulin Qin.](https://reader035.fdocuments.net/reader035/viewer/2022062905/5a4d1ae37f8b9ab059978225/html5/thumbnails/19.jpg)
Ship chunkMY-SHIP ISA SHIP SCREEN-POS, VALUE, STATUS, COLOR, HEIGHT, WIDTH, X, Y
ORIENTATION DIST VEL ANGLE VDIR HEX-HIT
MINE-DIST SHOOT-AT-MINE HIT-FORTRESS-FIRST
![Page 20: Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon Fincham Michelle Moon Ben Poole Yulin Qin.](https://reader035.fdocuments.net/reader035/viewer/2022062905/5a4d1ae37f8b9ab059978225/html5/thumbnails/20.jpg)
New visual request
• Attend-and-track– Combines the separate move-attention and start-
tracking requests into one action
![Page 21: Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon Fincham Michelle Moon Ben Poole Yulin Qin.](https://reader035.fdocuments.net/reader035/viewer/2022062905/5a4d1ae37f8b9ab059978225/html5/thumbnails/21.jpg)
New visual buffer
• Visual-search– A search buffer – Makes visicon chunks available for searching
in production matching • Visual-location information only
– Similar to the old LHS !find-location!• Less flexible than a visual-location request
– No :attended, :nearest, or current
• Two primary reasons for adding it– Buffer stuffing not available due to tracking– Allows for modeling control over “stuffing”
• Utility can decide which feature is more important when co-occurrences
(p start-playing =goal> isa goal state start-playing =visual-search> isa sf-visual-location kind ship ?visual> state free ==> +visual> isa attend-and-track screen-pos =visual-search …)
![Page 22: Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon Fincham Michelle Moon Ben Poole Yulin Qin.](https://reader035.fdocuments.net/reader035/viewer/2022062905/5a4d1ae37f8b9ab059978225/html5/thumbnails/22.jpg)
Using the visual-search buffer(p detect-bonus =goal> isa goal check-bonus nil =visual-search> isa sf-visual-location kind new-bonus =imaginal> isa game-state ==> =imaginal> +goal> check-bonus check)
(p detect-hit =visual-search> isa sf-visual-location kind fortress-hit ?imaginal> state free
=imaginal> isa game-state shot-count =count < shot-count 10 counted nil ==> !bind! =new-count (1+ =count) +imaginal> shot-count =new-count counted t)
(p see-mine-onset =goal> isa goal state play =visual-search> isa visual-location kind mine =visual> isa ship
?imaginal> state free … ==> +imaginal-action> isa generic-action action predict-mine-shooting …)
![Page 23: Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon Fincham Michelle Moon Ben Poole Yulin Qin.](https://reader035.fdocuments.net/reader035/viewer/2022062905/5a4d1ae37f8b9ab059978225/html5/thumbnails/23.jpg)
Special Imaginal request• Prediction for shooting the mine
– 250ms, slightly more than a standard imaginal request– Uses existing imaginal-action buffer +imaginal-action> isa generic-action action predict-mine-shooting
• Determines through simulation how to best shoot at the mine given current ship details
MINE-PREDICT ISA MINE-HEADING ACTION TURN FINGER RING DELAY 0.099 DIST 260.8793 HIT-DIST 180.8793
![Page 24: Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon Fincham Michelle Moon Ben Poole Yulin Qin.](https://reader035.fdocuments.net/reader035/viewer/2022062905/5a4d1ae37f8b9ab059978225/html5/thumbnails/24.jpg)
Mine aiming productions(p adjust-mine-heading-with-fortress =goal> isa goal cond2 "+fort“
=imaginal> isa mine-heading action turn finger =finger delay =delay dist =dist
=visual> isa ship <= mine-dist =dist =visual-search> isa visual-location kind mine… ==> +manual> isa delayed-punch hand left finger =finger delay =delay …)
(p reassess-mine =goal> isa goal =visual-search> isa visual-location kind mine
=imaginal> isa mine-heading action drift hit-dist =delay =visual> isa ship < mine-dist =delay … ==> +imaginal-action> Isa generic-action action predict-mine-shooting …)
![Page 25: Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon Fincham Michelle Moon Ben Poole Yulin Qin.](https://reader035.fdocuments.net/reader035/viewer/2022062905/5a4d1ae37f8b9ab059978225/html5/thumbnails/25.jpg)
Model overview
• 77 productions covering eight basic tasks– Encoding pre-trial info and rehearsal– Attend ship then initial thrust and turn to start orbiting– Orbiting the center hex– Shoot at fortress and count hits– Encode and respond to bonuses– Identify mine, aim, and shoot at it– Correct orbiting problems (hit hex or too far out) – Detect ship destruction or trial over
![Page 26: Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon Fincham Michelle Moon Ben Poole Yulin Qin.](https://reader035.fdocuments.net/reader035/viewer/2022062905/5a4d1ae37f8b9ab059978225/html5/thumbnails/26.jpg)
Orbiting productions
• Model tries to stay aimed at the center and orbit clockwise about 95 pixels out
• Broken into groups by ship speed< 1.0 tries to speed up1.0 – 1.7 maintains the normal orbiting > 1.7 tries to slow down
![Page 27: Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon Fincham Michelle Moon Ben Poole Yulin Qin.](https://reader035.fdocuments.net/reader035/viewer/2022062905/5a4d1ae37f8b9ab059978225/html5/thumbnails/27.jpg)
Normal speed (p right-norm-short =goal> isa goal state play dont-turn nil =visual> isa ship <= vel 1.7 >= vel 1.0 >= angle 5 <= angle 10 > vdir 92 ?manual> preparation free ?finger-check> left-index free ==> +manual> isa delayed-punch hand left finger index delay fast)
(p right-norm-default =goal> isa goal state play dont-turn nil =visual> isa ship <= vel 1.7 >= vel 1.0 >= angle 5 <= angle 20 > vdir 88 ?manual> preparation free ?finger-check> left-index free ==> +manual> isa delayed-punch hand left finger index)
(p right-norm-long =goal> isa goal state play dont-turn nil =visual> isa ship <= vel 1.7 >= vel 1.0 >= angle 15 <= angle 30 ?manual> preparation free ?finger-check> left-index free ==> +manual> isa delayed-punch hand left finger index delay slow)
(p thrust-norm-default =goal> isa goal state play =visual> isa ship <= vel 1.7 >= vel 1.0 > dist 93 >= vdir 92 <= angle 2 >= angle -4 ?manual> preparation free ?finger-check> left-middle free left-index free left-ring free ==> +manual> isa delayed-punch hand left finger middle)
(p left-norm-default =goal> isa goal state play dont-turn nil =visual> isa ship <= vel 1.7 >= vel 1.0 < angle -14 ?manual> preparation free ?finger-check> left-ring free left-index free left-middle free ==> +manual> isa delayed-punch hand left finger ring)
![Page 28: Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon Fincham Michelle Moon Ben Poole Yulin Qin.](https://reader035.fdocuments.net/reader035/viewer/2022062905/5a4d1ae37f8b9ab059978225/html5/thumbnails/28.jpg)
Consider the BOLD response• Use the BOLD prediction tools built into ACT-R to
determine activity in the modules• Few changes needed– Made all goal and imaginal modifications in the model
+’s so they’re counted– Added new buffers to track left and right hand actions
separately• Only tracks the execution
– Modified visual tracking so that it is only periodically busy instead of constantly• One attention shift per 2 seconds
![Page 29: Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon Fincham Michelle Moon Ben Poole Yulin Qin.](https://reader035.fdocuments.net/reader035/viewer/2022062905/5a4d1ae37f8b9ab059978225/html5/thumbnails/29.jpg)
![Page 30: Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon Fincham Michelle Moon Ben Poole Yulin Qin.](https://reader035.fdocuments.net/reader035/viewer/2022062905/5a4d1ae37f8b9ab059978225/html5/thumbnails/30.jpg)
-0.20%
0.00%
0.20%
0.40%
0.60%
0.80%
1.00%. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Left Motor Right Motor ACC Parietal Prefrontal Caudate Fusiform
% C
hang
e fr
om B
asel
ine
Orbit Mines Fortress Both
AnteriorPrefrontal
LIPFC
Fusiform
PosteriorParietal
ManualVocal
Auditory
![Page 31: Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon Fincham Michelle Moon Ben Poole Yulin Qin.](https://reader035.fdocuments.net/reader035/viewer/2022062905/5a4d1ae37f8b9ab059978225/html5/thumbnails/31.jpg)
![Page 32: Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon Fincham Michelle Moon Ben Poole Yulin Qin.](https://reader035.fdocuments.net/reader035/viewer/2022062905/5a4d1ae37f8b9ab059978225/html5/thumbnails/32.jpg)
Model’s BOLD motor mismatch
![Page 33: Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon Fincham Michelle Moon Ben Poole Yulin Qin.](https://reader035.fdocuments.net/reader035/viewer/2022062905/5a4d1ae37f8b9ab059978225/html5/thumbnails/33.jpg)
Curious…
![Page 34: Modeling Space Fortress Abraham Anderson John Anderson Shawn Betts Dan Bothell Jennifer Ferris Jon Fincham Michelle Moon Ben Poole Yulin Qin.](https://reader035.fdocuments.net/reader035/viewer/2022062905/5a4d1ae37f8b9ab059978225/html5/thumbnails/34.jpg)
Future work
• Address the issues with the model’s BOLD results in the motor system
• Model the performance on the early days and the learning that takes place over days