30 31 cognitive navigation and object detection system for blind people
Games with Sounds: the Blind Navigation and Target Acquisition
-
Upload
kato-sullivan -
Category
Documents
-
view
41 -
download
3
description
Transcript of Games with Sounds: the Blind Navigation and Target Acquisition
Games with Sounds: Games with Sounds:
the Blind Navigation the Blind Navigation
and Target Acquisition and Target Acquisition
Antti Nyman
Department of Computer Sciences
University of Tampere Finland
March, 2005
AAFG 2005
A. Nyman p 01_16 21.03.2005
Games with Sounds
Blind users do not prefer mouse (lack of feedback on movement &
position).
Making use of spatial sound has been suggested in [1, 4, 5].
Problems include the difficulty of recognizing simultaneously playing
sounds and mixing up of directions (forward <-> backward) [3].
Earlier studies on sound grids have shown that easily recognized
sound in the center of the grid reduces the navigation time and that a
“landmark” technique used by cane-travelers could be used to assist
blind persons in identifying different areas [5].
Commercial games could be modified and new game concepts
created to better support blind navigation within the game field [2].
The task is to optimize/minimize the number of sounds (3 or 6)
employed to mark 25 positions in the virtual sound grid (5 x 5) and to
estimate navigation performance with the use of conventional mouse
input in blind conditions.
A. Nyman p 02_16 21.03.2005
Demo 1
center – 1
angles – 4
middle1 – 4
middle2 – 4
middle3 – 6
middle4 – 6
= 25
Games with Sounds
Typographic Semantics of Web
Pages, P. Gorny, 2000 [3].
A. Nyman p 03_16 21.03.2005
Games with Sounds
Four subjects (two females, two males) with normal hearing and seeing
abilities participated in the tests.
During the test the subjects were blindfolded and used headphones to
minimize the effect of distractions.
Each subject played the game twice: with 3-sound grid and with 6-
sound grid. Each game consisted of 25 tasks presented in a random
order.
Application stored the track of mouse movement, the passed time, the
traveled distance and the number of the passed spots for each task.
The original application was modified to use animal sound .wav-files to
make sounds easier to recognize and to make game instructions
easier.
A. Nyman p 04_16 21.03.2005
Games with Sounds
Sound 1
Sound 1
Klikkaa eläintä vasemmassa ylänurkassa.
Klikkaa eläintä joka on ylärivissä toinen vasemmalta.
Klikkaa eläintä ylärivin keskellä.
Klikkaa eläintä joka on ylärivissä toinen oikealta.
Klikkaa eläintä oikeassa ylänurkassa.
Klikkaa eläintä joka on toiseksi ylimmän rivin vasemmanpuoleisin.
Klikkaa eläintä joka on toiseksi ylimmässä rivissä toinen vasemmalta
Klikkaa eläintä toisen rivin keskellä.
Klikkaa eläintä joka on toiseksi ylimmässä rivissä toinen oikealta.
Klikkaa eläintä joka on toiseksi ylimmän rivin oikeanpuoleisin.
Klikkaa eläintä joka on keskimmäisen rivin vasemmanpuoleisin.
Klikkaa eläintä joka on keskimmäisessä rivissä toinen vasemmalta.
Klikkaa eläintä keskimmäisen rivin keskellä.
Klikkaa eläintä joka on keskimmäisessä rivissä toinen oikealta.
Klikkaa eläintä joka on keskimmäisen rivin oikeanpuoleisin…..
3 1 2 1 3
1 2 1 2 1
2 1 3 1 2
1 2 1 2 1
3 1 2 1 3
1 2 3 2 1
4 5 4 5 4
3 2 6 2 3
4 5 4 5 4
1 2 3 2 1
Sound 2
Sound 2
Sound 3
Sound 3
Sound 4
Sound 5
Sound 6
A. Nyman p 05_16 21.03.2005
Games with Sounds
0
5
10
15
20
25
30
35
40
1 2 3 4 5 6 7 8 9 1011 121314 1516 171819 2021 222324 25
Task number
Time, s
0
5
10
15
20
25
Spots, nTime Passed spots
the target acquisition time and the number of spots passed at 3
sounds used for marking 25 positions
corr. (time to n spots) = 0.96
A. Nyman p 06_16 21.03.2005
Games with Sounds
the target acquisition time and the distance to target at 3 sounds
used for marking 25 positions
corr. (time to distance) = 0.347
0
5
10
15
20
25
30
35
40
1 2 3 4 5 6 7 8 9 10 111213 141516 171819 2021 222324 25
Task number
Time, s
0
100
200
300
400
500
600
700
800
Distance, pxlTime Distance
A. Nyman p 07_16 21.03.2005
Games with Sounds
the target acquisition time and the shortest paths to target at 3
sounds used for marking 25 positions
corr. (time to path) = 0.233
0
5
10
15
20
25
30
35
40
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
Task number
Time, s
0
2
4
6
8
10
Spots, nTime Shortest path
A. Nyman p 08_16 21.03.2005
Games with Sounds
Correlation between the target
acquisition time and the distance to
target for 6- and 3-sound matrices
in 4 subjects
t = - .271, df = 3, p < 0.01
Correlation between the target
acquisition time and the number of
passed spots (targets) for 6- and 3-
sound matrices in 4 subjects
t = - .611, df = 3, p < 0.01
0.8
0.85
0.9
0.95
1
1 2 3 4
Subjects
Corr. 3 sounds 6 sounds
0
0.2
0.4
0.6
0.8
1
1 2 3 4Subjects
Corr. 3 sounds 6 sounds
A. Nyman p 09_16 21.03.2005
Games with Sounds
the overall target acquisition time for
6- and 3-sound matrices in 4
subjects
t = 2.022, df = 3, p < 0.01
the overall averaged error rates in a
target acquisition task with the use of
6- and 3-sound matrices in 4 subjects
t = 1.578, df = 3, p < 0.01
010203040506070
1 2 3 4
Subject
Time, s 6 sounds 3 sounds
02468
10121416
1 2 3 4Subject
Errors, n 6 sounds 3 sounds
A. Nyman p 10_16 21.03.2005
Games with Sounds
overall data in a target acquisition task with the use of 6- and 3-sound matrices
A. Nyman p 11_16 21.03.2005
Games with Sounds
Behavioral patterns recorded when subjects inspected game field by
moving cursor down unintentionally while trying to move it vertically to
right.
A. Nyman p 12_16 21.03.2005
Games with Sounds
Behavioral patterns recorded during an inspection game field by using
the corners as navigational “landmarks” to make target acquisition
easier.
target
target
A. Nyman p 13_16 21.03.2005
Games with Sounds
The number of passed spots has more effect on target acquisition time
than the distance between starting and ending points Other factors
influence the difficulty of the task, they are e.g. the position of the
target in the grid and the distinctiveness of the sound in the target.
Average number of errors committed and average target acquisition
time both were notably better with 6-sound grid 6 different sounds
gives more support for navigation and target acquisition.
Least errors and shortest acquisition times were related with the tasks
to point out the target in corners of the grid Corners of the grid could
be used as “landmarks” to support navigation.
A. Nyman p 14_16 21.03.2005
Games with Sounds
The sense of position could be lost totally when lifting the mouse.
The different spots with the same sounds were easily mixed. This
happened especially after getting lost and was more evident in the
3-sound grid.
In future work we could further investigate the effect of different
sound combinations in the 6-sound grid. Also the effect of subjects
selecting the sounds themselves could be investigated since it has
been noted that sounds in audio interfaces should be adaptable to
personal preferences [3].
References
[1] Bölke, L., Gorny, P. Auditory Direct Manipulation of Acoustical Objects by Blind Computer Users.
Proc. International Conference on Auditory Display 1994, 287-288.
[2] Friberg, J., Gärdenfors, D. Audio Games: New perspectives on game audio. Proc. ACE
Conference 2004.
[3] Gorny, P. Typographic Semantics of Webpages Accessible for Visually Impaired Users: Mapping
Layout and Interaction Objects to an Auditory Interaction Space. Proc. Int’l Conf. on Computers
Helping People with Special Needs 2000, 251-257.
[4] Kamel, H.M., Roth, P., Sinha, R.R. Graphics and User’s Exploration via Simple Sonics (GUESS):
Providing Interrelational Representation of Objects in a Non-visual Environment. Proc. International
Conference on Auditory Display 2001, 261-266.
[5] Parente, P., Bishop, G. BATS: The Blind Audio Tactile Mapping System.
A. Nyman p 15_16 21.03.2005
Games with Sounds