Text Input to Handheld Devices for People with Physical Disabilities

Text Input to Handheld Devices for People with

Physical Disabilities

Text Input to Handheld Devices for People with

Physical Disabilities

Brad A. Myers and Jacob O. Wobbrock

Human Computer Interaction InstituteSchool of Computer ScienceCarnegie Mellon UniversityPittsburgh, PA 15213-3891

[email protected]://www.pebbles.hcii.cmu.edu/assistive/

Brad A. Myers and Jacob O. Wobbrock

Human Computer Interaction InstituteSchool of Computer ScienceCarnegie Mellon UniversityPittsburgh, PA 15213-3891

[email protected]://www.pebbles.hcii.cmu.edu/assistive/

Human Computer Interaction Institute Human Computer Interaction Institute

School of Computer ScienceSchool of Computer Science

Carnegie Mellon UniversityCarnegie Mellon University

http://www.hci-international.org/first.asp

Brad Myers Carnegie Mellon 2

Motivation

Provide text entry for people with physical disabilities

For handheld devices like Palm Pilots Also called Personal Digital Assistants (PDAs)

Also use Palm Pilots as interface to desktop computers

Provide text entry for people with physical disabilities

For handheld devices like Palm Pilots Also called Personal Digital Assistants (PDAs)

Also use Palm Pilots as interface to desktop computers


Part of the Pebbles Project

Investigate use of handheld devices like Palm Pilots and PocketPCs, and Smartphones at the same time as PCs and other devices

Investigate use of handheld devices like Palm Pilots and PocketPCs, and Smartphones at the same time as PCs and other devices

PEBBLES

PEBBLES

DAs forDAs forntry ofntry ofothothytes andytes andocations fromocations fromxternalxternalources.ources.

DAs forDAs forntry ofntry ofothothytes andytes andocations fromocations fromxternalxternalources.ources.

http://www.pebbles.hcii.cmu.edu/http://www.pebbles.hcii.cmu.edu/


First Steps

Support people with Muscular Dystrophy Low strength, limited motion, but high

accuracy Tiny keyboard on handheld screens are

good

Support people with Muscular Dystrophy Low strength, limited motion, but high

accuracy Tiny keyboard on handheld screens are

good


Use Our Remote Commander Palm Pilot application to control PC using the Palm Extend tiny keyboard with other keys

Word completion Other adaptations: no auto-repeat, no auto-off

Mouse control on same screen Wireless or wired

Palm Pilot application to control PC using the Palm Extend tiny keyboard with other keys

Word completion Other adaptations: no auto-repeat, no auto-off

Mouse control on same screen Wireless or wired


But what about other disabilities?

Cerebral Palsy, Parkinson’s, Spinal Injury, and many others

Loose accuracy as well as strength Tiny buttons are exactly wrong Still can’t do Graffiti, etc.

Cerebral Palsy, Parkinson’s, Spinal Injury, and many others

Loose accuracy as well as strength Tiny buttons are exactly wrong Still can’t do Graffiti, etc.


New Design: EdgeWrite™

Added custom template with a square hole to input area

Gain stability from edges Travel along edges and into corners Designed alphabet using

gestures inside of hole,along edges and intocorners

Added custom template with a square hole to input area

Gain stability from edges Travel along edges and into corners Designed alphabet using

gestures inside of hole,along edges and intocorners


Alphabet Properties Recognizes based on sequence of corners

Users can wiggle during diagonals Not speed dependent Easy to customize and adapt letter forms

Even user-driven

Letters “feel” normal Use multiple strokes along edge

Presentation (only) uses arcs No capitalization mode

Capitals end in upper left No special mode for numbers

I on left, 1 on right

Recognizes based on sequence of corners Users can wiggle during diagonals Not speed dependent Easy to customize and adapt letter forms

Even user-driven

Letters “feel” normal Use multiple strokes along edge

Presentation (only) uses arcs No capitalization mode

Capitals end in upper left No special mode for numbers

I on left, 1 on right


Actual Alphabet from User Studies Wanted high guessability

Correlates with initial usability Designed a technique to let users guess the

gestures Told users the rules Then resolve conflicts if same shape used twice Left multiple forms for each letter Result: guessability improved from 51% to 80%

Wanted high guessability Correlates with initial usability

Designed a technique to let users guess the gestures

Told users the rules Then resolve conflicts if same shape used twice Left multiple forms for each letter Result: guessability improved from 51% to 80%


Current Alphabet


EdgeWrite on Palm

Novice able-bodied users Same speed with Graffiti and EdgeWrite But EdgeWrite 18% more accurate

Handicapped users Too few for statistical significance, but dramatic E.g., accuracy: 31% with Graffiti vs. 94% EdgeWrite For example:

“The dog is going fast” vs:“The g i gbsiangu% fast”

Novice able-bodied users Same speed with Graffiti and EdgeWrite But EdgeWrite 18% more accurate

Handicapped users Too few for statistical significance, but dramatic E.g., accuracy: 31% with Graffiti vs. 94% EdgeWrite For example:

“The dog is going fast” vs:“The g i gbsiangu% fast”


EdgeWrite on Game Controllers

Some joysticks use a square hole Current techniques are very

awkward: On-screen keyboard “Date-stamp” cycling letters at each spot

Some joysticks use a square hole Current techniques are very

awkward: On-screen keyboard “Date-stamp” cycling letters at each spot

0

1

2

3

4

5

6

7

WPM

EdgeWrite

Date Stamp

SelectionKeyboard


EdgeWrite on Wheelchair

Look at people who use powerwheelchairs

Custom hardware and softwareto interface to commercial joystick

Informally evaluated with 7 disabled power-wheelchair users 6 with Cerebral Palsy, 1 with Multiple Sclerosis

Look at people who use powerwheelchairs

Custom hardware and softwareto interface to commercial joystick

Informally evaluated with 7 disabled power-wheelchair users 6 with Cerebral Palsy, 1 with Multiple Sclerosis


Touchpad EdgeWrite

Adaptation that uses elevated edges around a Synaptics touchpad May be easier for people with motor impairments

Adaptation that uses elevated edges around a Synaptics touchpad May be easier for people with motor impairments


Results

Touchpad worked better than wheelchair joystick

Still pretty slow

Touchpad worked better than wheelchair joystick

Still pretty slow


EdgeWrite on 4 Buttons Only need four corners Implementation for any 4 keys Four keys on the Logitech steering wheel

Not necessarily for text entry while driving Text entry especially for navigation system

Only need four corners Implementation for any 4 keys Four keys on the Logitech steering wheel

Not necessarily for text entry while driving Text entry especially for navigation system


EdgeWrite with Trackball & Pointing Stick

Newest version Many disabled users prefer trackball

for desktop pointing EdgeWrite for desktop

Optimized for trackball and pointing stick Also works with regular mouse

Highly parameterized Input window can be transparent when in use Has help, etc. Up to 20 wpm

Newest version Many disabled users prefer trackball

for desktop pointing EdgeWrite for desktop

Optimized for trackball and pointing stick Also works with regular mouse

Highly parameterized Input window can be transparent when in use Has help, etc. Up to 20 wpm


Initial Reaction to Trackball Version

One user in long-term trial Spinal injury, quadriplegic, with some mobility of one arm

Given up on-screen keyboard in favor of EdgeWrite Still uses speech entry when possible With practice, EdgeWrite is faster

“With an on-screen keyboard, there is just too much visual scanning and concentration. In EdgeWrite, if you know the letter, you just bang it out by feel, and you can keep your eyes on your document.”

One user in long-term trial Spinal injury, quadriplegic, with some mobility of one arm

Given up on-screen keyboard in favor of EdgeWrite Still uses speech entry when possible With practice, EdgeWrite is faster

“With an on-screen keyboard, there is just too much visual scanning and concentration. In EdgeWrite, if you know the letter, you just bang it out by feel, and you can keep your eyes on your document.”


Summary and Future Work EdgeWrite works across a wide range

of devices With the same alphabet

Learn once Use most comfortable device

Future work Continue to refine Add word completion Further tests and deployment New devices (phone?)

EdgeWrite works across a wide range of devices

With the same alphabet Learn once Use most comfortable device

Future work Continue to refine Add word completion Further tests and deployment New devices (phone?)


Acknowledgements

Funded by grants from NSF, General Motors and The NEC Foundation of America

Equipment grants from: Synaptics, Inc. A.T. Sciences Microsoft

Funded by grants from NSF, General Motors and The NEC Foundation of America

Equipment grants from: Synaptics, Inc. A.T. Sciences Microsoft

http://www.pebbles.hcii.cmu.edu/http://www.pebbles.hcii.cmu.edu/

http://www.edgewrite.com/http://www.edgewrite.com/

Text Input to Handheld Devices for People with Physical Disabilities

Documents

Transcript of Text Input to Handheld Devices for People with Physical Disabilities