A Multisensory and Multimodal Game for Children with Cochlear Implants
description
Transcript of A Multisensory and Multimodal Game for Children with Cochlear Implants
![Page 1: A Multisensory and Multimodal Game for Children with Cochlear Implants](https://reader035.fdocuments.net/reader035/viewer/2022062310/5681633e550346895dd3cf37/html5/thumbnails/1.jpg)
A Multisensory and Multimodal Game for Children with Cochlear Implants
Maria Carmela SogonoSupervisor: Dr. Deborah RichardsITEC810 Information Technology ProjectDepartment of Computing, Macquarie University
![Page 2: A Multisensory and Multimodal Game for Children with Cochlear Implants](https://reader035.fdocuments.net/reader035/viewer/2022062310/5681633e550346895dd3cf37/html5/thumbnails/2.jpg)
2
Agenda• Problem• Related work• Methodology• Outcomes– Design principles– Instructional design– Game evaluation
• Conclusion
![Page 3: A Multisensory and Multimodal Game for Children with Cochlear Implants](https://reader035.fdocuments.net/reader035/viewer/2022062310/5681633e550346895dd3cf37/html5/thumbnails/3.jpg)
3
ProblemEven with hearing aids, quality is still not the same as with normal hearing
Can affect brain development in children
Problems in areas such as sound localisation
Can a multimodal, multisensory game be used to test, train, and improve sound localisation acuity?
Photo source: http://www.flickr.com/photos/adwriter/1305191984/
![Page 4: A Multisensory and Multimodal Game for Children with Cochlear Implants](https://reader035.fdocuments.net/reader035/viewer/2022062310/5681633e550346895dd3cf37/html5/thumbnails/4.jpg)
4
Why focus on children?
• 35:10000 children• Normal hearing is important for brain
development
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
![Page 5: A Multisensory and Multimodal Game for Children with Cochlear Implants](https://reader035.fdocuments.net/reader035/viewer/2022062310/5681633e550346895dd3cf37/html5/thumbnails/5.jpg)
5
Why focus on SSD?
There is less help available for children with SSD.
Photo source: http://www.firefliesandjellybeans.com/2012/05/can-you-hear-me-now.html
![Page 6: A Multisensory and Multimodal Game for Children with Cochlear Implants](https://reader035.fdocuments.net/reader035/viewer/2022062310/5681633e550346895dd3cf37/html5/thumbnails/6.jpg)
6
What is sound localisation?The ability to identify where sound has come from
Photo source: http://www.flickr.com/photos/reverses/5098858054/
![Page 7: A Multisensory and Multimodal Game for Children with Cochlear Implants](https://reader035.fdocuments.net/reader035/viewer/2022062310/5681633e550346895dd3cf37/html5/thumbnails/7.jpg)
7
Baha
Processes sound from the impaired side and sends it to the hearing ear
Boneanchored hearing aid
Photo source: http://products.cochlearamericas.com/diagram/baha-system
![Page 8: A Multisensory and Multimodal Game for Children with Cochlear Implants](https://reader035.fdocuments.net/reader035/viewer/2022062310/5681633e550346895dd3cf37/html5/thumbnails/8.jpg)
8
Would it be possible to improve sound localisation acuity for children with SSD and Bahas?
![Page 9: A Multisensory and Multimodal Game for Children with Cochlear Implants](https://reader035.fdocuments.net/reader035/viewer/2022062310/5681633e550346895dd3cf37/html5/thumbnails/9.jpg)
9
How?GamesMultisensoryMultimodal
• To appeal to the target audience• To make it easier to track, collect,
then analyse data
• Supported by perceptual motor training theory
• Supported by technological advancements
![Page 10: A Multisensory and Multimodal Game for Children with Cochlear Implants](https://reader035.fdocuments.net/reader035/viewer/2022062310/5681633e550346895dd3cf37/html5/thumbnails/10.jpg)
10
Can we design games to improve sound localisation acuity for children with SSD and Bahas?
Would it be possible to improve sound localisation acuity for children with SSD and Bahas?
![Page 11: A Multisensory and Multimodal Game for Children with Cochlear Implants](https://reader035.fdocuments.net/reader035/viewer/2022062310/5681633e550346895dd3cf37/html5/thumbnails/11.jpg)
11
Relevant work• Research studies testing sound
localisation acuity• Existing sound localisation games
developed at Macquarie University• Other games developed for
children with SSD
Photo source: http://www.flickr.com/photos/driek/5412231453/
![Page 12: A Multisensory and Multimodal Game for Children with Cochlear Implants](https://reader035.fdocuments.net/reader035/viewer/2022062310/5681633e550346895dd3cf37/html5/thumbnails/12.jpg)
12
Methodology1. Review of relatedliterature
![Page 13: A Multisensory and Multimodal Game for Children with Cochlear Implants](https://reader035.fdocuments.net/reader035/viewer/2022062310/5681633e550346895dd3cf37/html5/thumbnails/13.jpg)
13
Methodology2. Analysis of existing work
![Page 14: A Multisensory and Multimodal Game for Children with Cochlear Implants](https://reader035.fdocuments.net/reader035/viewer/2022062310/5681633e550346895dd3cf37/html5/thumbnails/14.jpg)
14
Methodology3. Development of design principles
![Page 15: A Multisensory and Multimodal Game for Children with Cochlear Implants](https://reader035.fdocuments.net/reader035/viewer/2022062310/5681633e550346895dd3cf37/html5/thumbnails/15.jpg)
15
Methodology4. Development of instructional design template
![Page 16: A Multisensory and Multimodal Game for Children with Cochlear Implants](https://reader035.fdocuments.net/reader035/viewer/2022062310/5681633e550346895dd3cf37/html5/thumbnails/16.jpg)
16
Methodology5. Development of game evaluation guidelines
![Page 17: A Multisensory and Multimodal Game for Children with Cochlear Implants](https://reader035.fdocuments.net/reader035/viewer/2022062310/5681633e550346895dd3cf37/html5/thumbnails/17.jpg)
17
Design principles1. The learning environment shall be
realistic.2. The stimulus shall be meaningful.3. The game shall start at an easy level.4. When the game level of difficulty
increases, the increase shall be incremental.
5. The game shall incorporate motivational elements.
6. The game shall provide feedback after each trial.
![Page 18: A Multisensory and Multimodal Game for Children with Cochlear Implants](https://reader035.fdocuments.net/reader035/viewer/2022062310/5681633e550346895dd3cf37/html5/thumbnails/18.jpg)
18
7. The game shall propose a challenge.8. The game shall make use of fantasy.9. The game shall incorporate sensory and
cognitive curiosity.10.The game shall adapt to the user’s progress.11.The game shall allow for the repetition of
tasks.12.The game shall track user data.13.The game shall use bright colours.
Design principles
![Page 19: A Multisensory and Multimodal Game for Children with Cochlear Implants](https://reader035.fdocuments.net/reader035/viewer/2022062310/5681633e550346895dd3cf37/html5/thumbnails/19.jpg)
19
14.The game shall be visually appealing to the target audience.
15.The game shall make use of high-quality audio.
16.The game shall allow for sound to originate from at least 2 distinct sources.
17.A non-accented, trained voice shall be used.
18.The game shall make use of multiple modalities.
Design principles
![Page 20: A Multisensory and Multimodal Game for Children with Cochlear Implants](https://reader035.fdocuments.net/reader035/viewer/2022062310/5681633e550346895dd3cf37/html5/thumbnails/20.jpg)
20
Interaction modalitiesModality Description TechnologyVision Visual perception Monitor
TouchscreenProjector & screen3D goggles
Audio Hearing SpeakersHaptic Deals with the sense of touch;
currently only used to provide pressure
JoystickWand
Action, manipulation
Directed by limbs and body KeyboardMouseGamepadJoystickWii controllerKinectTouchscreen
![Page 21: A Multisensory and Multimodal Game for Children with Cochlear Implants](https://reader035.fdocuments.net/reader035/viewer/2022062310/5681633e550346895dd3cf37/html5/thumbnails/21.jpg)
21
Instructional design templateTitle <Title of the game module>Description <Describes the game in one or two sentences>Objectives To improve skills in localization acuity through training
To track user progress whilst trainingGame Style <Describes what game elements are incorporated>
Game genres: Puzzle (Hidden object, reveal the picture) Action (Maze game, rhythm action game, shooter games) Strategy (Time management) Adventure
Background <Describes the game’s backstory>Initial Setup <Describes how the game is set up in the beginning. What a new player will see,
hear, or experience after opening the game for the first time? >Game Model <Describes game progression. What will the player have to do? What happens
when the player answers correctly or incorrectly?> Four general models of game flow: Sound identification Sound evasion Sound tracking Sound training
![Page 22: A Multisensory and Multimodal Game for Children with Cochlear Implants](https://reader035.fdocuments.net/reader035/viewer/2022062310/5681633e550346895dd3cf37/html5/thumbnails/22.jpg)
22
Instructional design templateSuccess Condition
<Describes how the game or a level within the game is won>
Fail Condition
<Describes how the game or a level within the game is lost>
Challenges <Describes how the game levels increase in difficulty> There are several ways to increase difficulty: Include increasing instances of the stimulus Decrease angular distance between the instances Include background noises Include additional elements onscreen to increase distraction Decrease stimulus visibility (for example, it is hiding behind
another object) Include a time limit Include a “number of times” limit
Additional Interaction
<Describes possible interaction styles that are available if the player has access to additional hardware>
![Page 23: A Multisensory and Multimodal Game for Children with Cochlear Implants](https://reader035.fdocuments.net/reader035/viewer/2022062310/5681633e550346895dd3cf37/html5/thumbnails/23.jpg)
23
Sample game modules
Bug KillerTreasure HuntWild Animal EvasionDance Beat
Shooter, soundidentification
Adventure, sound tracking
Maze, sound evasion
Rhythm action, sound training
![Page 24: A Multisensory and Multimodal Game for Children with Cochlear Implants](https://reader035.fdocuments.net/reader035/viewer/2022062310/5681633e550346895dd3cf37/html5/thumbnails/24.jpg)
Game evaluation• Game design
• Usability
• Content
24
“Does the game follow design principles and best practices?”“Is the game usable?”
“Does the game teach what it’s supposed to?”
![Page 25: A Multisensory and Multimodal Game for Children with Cochlear Implants](https://reader035.fdocuments.net/reader035/viewer/2022062310/5681633e550346895dd3cf37/html5/thumbnails/25.jpg)
25
Game design evaluationModule Name: <Module name>Does the module have… Rating…a realistic learning environment? <1-5>Explanation: <Describe how the game satisfies or does not satisfy this condition.>
…meaningful stimulus? <1-5>Explanation:<Describe how the game satisfies or does not satisfy this condition.>
…motivational elements? <1-5>Explanation: <Describe how the game satisfies or does not satisfy this condition.>
…feedback? <1-5>Explanation: <Describe how the game satisfies or does not satisfy this condition.>
![Page 26: A Multisensory and Multimodal Game for Children with Cochlear Implants](https://reader035.fdocuments.net/reader035/viewer/2022062310/5681633e550346895dd3cf37/html5/thumbnails/26.jpg)
26
Game design evaluation…a challenge? <1-5>
Explanation: <Describe how the game satisfies or does not satisfy this condition.>
…elements of fantasy? <1-5>Explanation: <Describe how the game satisfies or does not satisfy this condition.>
…adaptation to the user’s progress? <1-5>Explanation: <Describe how the game satisfies or does not satisfy this condition.>
…repetition of tasks? <1-5>Explanation: <Describe how the game satisfies or does not satisfy this condition.>
...multiple modalities? <1-5>Explanation: <Describe how the game satisfies or does not satisfy this condition.>
![Page 27: A Multisensory and Multimodal Game for Children with Cochlear Implants](https://reader035.fdocuments.net/reader035/viewer/2022062310/5681633e550346895dd3cf37/html5/thumbnails/27.jpg)
27
Game modules evaluated
Bug KillerTreasure HuntWild Animal EvasionDance Beat
4 3.22
3.89
3.11
![Page 28: A Multisensory and Multimodal Game for Children with Cochlear Implants](https://reader035.fdocuments.net/reader035/viewer/2022062310/5681633e550346895dd3cf37/html5/thumbnails/28.jpg)
28
Conclusion
![Page 29: A Multisensory and Multimodal Game for Children with Cochlear Implants](https://reader035.fdocuments.net/reader035/viewer/2022062310/5681633e550346895dd3cf37/html5/thumbnails/29.jpg)
29
• Augmented reality• Multiplayer games• Haptic, large scale mats• Games for adults
Recommendations