Project ACCESS: Wayfinding for Persons with Cognitive Disabilities · 2018-05-02 · Project...
Transcript of Project ACCESS: Wayfinding for Persons with Cognitive Disabilities · 2018-05-02 · Project...
![Page 1: Project ACCESS: Wayfinding for Persons with Cognitive Disabilities · 2018-05-02 · Project ACCESS: Wayfinding for Persons with Cognitive Disabilities GaetanoBoriello, Dieter Fox,](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f05b5e77e708231d414511c/html5/thumbnails/1.jpg)
Project ACCESS: Wayfinding for Persons with Cognitive
Disabilities
Gaetano Boriello, Dieter Fox, Henry Kautz, Lin Liao, Don Patterson
Computer Science & EngineeringPat Brown, Mark Harniss, Kurt Johnson
Center for Technology and Disability Studies
University of Washington
![Page 2: Project ACCESS: Wayfinding for Persons with Cognitive Disabilities · 2018-05-02 · Project ACCESS: Wayfinding for Persons with Cognitive Disabilities GaetanoBoriello, Dieter Fox,](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f05b5e77e708231d414511c/html5/thumbnails/2.jpg)
The Need: Community Access for the Cognitively Disabled
![Page 3: Project ACCESS: Wayfinding for Persons with Cognitive Disabilities · 2018-05-02 · Project ACCESS: Wayfinding for Persons with Cognitive Disabilities GaetanoBoriello, Dieter Fox,](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f05b5e77e708231d414511c/html5/thumbnails/3.jpg)
The Problem
Using public transit is cognitively challengingo Learning bus routes and numberso Transferso Recovering from mistakeso Point to point shuttle service slow, expensive
Current GPS units hard to useo Require extensive user inputo Loss of signal near buildings, inside
vehicleso Little support for complex transit plans
![Page 4: Project ACCESS: Wayfinding for Persons with Cognitive Disabilities · 2018-05-02 · Project ACCESS: Wayfinding for Persons with Cognitive Disabilities GaetanoBoriello, Dieter Fox,](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f05b5e77e708231d414511c/html5/thumbnails/4.jpg)
Goal
A personal guidance system thato Requires no explicit programming by user
or caregiver• Learns the user’s routine
o Robust in the face of signal loss / noise• Leverage real-world constraints & user model
o Proactively assists user in completing transportation plans• Helps with changing mode of transportation• Recognizes user errors• Helps user recover
![Page 5: Project ACCESS: Wayfinding for Persons with Cognitive Disabilities · 2018-05-02 · Project ACCESS: Wayfinding for Persons with Cognitive Disabilities GaetanoBoriello, Dieter Fox,](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f05b5e77e708231d414511c/html5/thumbnails/5.jpg)
IdeaUser carries GPS cell phoneSystem infers user’s stateo Walking? Getting on a bus?System learns about usero Important places, routesBreaks from routine = user may be confused or losto Offer helpo Call caregiver
![Page 6: Project ACCESS: Wayfinding for Persons with Cognitive Disabilities · 2018-05-02 · Project ACCESS: Wayfinding for Persons with Cognitive Disabilities GaetanoBoriello, Dieter Fox,](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f05b5e77e708231d414511c/html5/thumbnails/6.jpg)
GPS readingzk-1 zk
Edge, velocity, positionxk-1 xk
θk-1 θk Data (edge) association
Time k-1 Time k
mk-1 mk Transportation mode
tk-1 tk Trip segment
gk-1 gk Goal
Cognitive mode{ routine, novel, error }
Approach: Learn Hierarchical DBN from Unlabeled Data
ck-1 ck
![Page 7: Project ACCESS: Wayfinding for Persons with Cognitive Disabilities · 2018-05-02 · Project ACCESS: Wayfinding for Persons with Cognitive Disabilities GaetanoBoriello, Dieter Fox,](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f05b5e77e708231d414511c/html5/thumbnails/7.jpg)
High Probability Trip SegmentsConditioned on Goal
Goal = Workplace Goal = Home
: bus : car : foot
![Page 8: Project ACCESS: Wayfinding for Persons with Cognitive Disabilities · 2018-05-02 · Project ACCESS: Wayfinding for Persons with Cognitive Disabilities GaetanoBoriello, Dieter Fox,](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f05b5e77e708231d414511c/html5/thumbnails/8.jpg)
Error Detection: Missed Bus Stop
blue = foot green = bus
![Page 9: Project ACCESS: Wayfinding for Persons with Cognitive Disabilities · 2018-05-02 · Project ACCESS: Wayfinding for Persons with Cognitive Disabilities GaetanoBoriello, Dieter Fox,](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f05b5e77e708231d414511c/html5/thumbnails/9.jpg)
ACCESS PrototypeCell phone with GPS, camera, high-speed internet accessPrompts when it infers that user is…
![Page 10: Project ACCESS: Wayfinding for Persons with Cognitive Disabilities · 2018-05-02 · Project ACCESS: Wayfinding for Persons with Cognitive Disabilities GaetanoBoriello, Dieter Fox,](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f05b5e77e708231d414511c/html5/thumbnails/10.jpg)
ACCESS PrototypeCell phone with GPS, camera, high-speed internet accessPrompts when it infers that user is…
About to begin a transportation plan
• Confirm destination?
• Here is yourroute!
![Page 11: Project ACCESS: Wayfinding for Persons with Cognitive Disabilities · 2018-05-02 · Project ACCESS: Wayfinding for Persons with Cognitive Disabilities GaetanoBoriello, Dieter Fox,](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f05b5e77e708231d414511c/html5/thumbnails/11.jpg)
ACCESS PrototypeCell phone with GPS, camera, high-speed internet accessPrompts when it infers that user is…
About to change mode
• This is your bus!
• Your stop is next!
![Page 12: Project ACCESS: Wayfinding for Persons with Cognitive Disabilities · 2018-05-02 · Project ACCESS: Wayfinding for Persons with Cognitive Disabilities GaetanoBoriello, Dieter Fox,](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f05b5e77e708231d414511c/html5/thumbnails/12.jpg)
ACCESS PrototypeCell phone with GPS, camera, high-speed internet accessPrompts when it infers that user is…
Making an error
• You missed your stop!
• Here is how to getback on track …
![Page 13: Project ACCESS: Wayfinding for Persons with Cognitive Disabilities · 2018-05-02 · Project ACCESS: Wayfinding for Persons with Cognitive Disabilities GaetanoBoriello, Dieter Fox,](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f05b5e77e708231d414511c/html5/thumbnails/13.jpg)
ACCESS PrototypeCell phone with GPS, camera, high-speed internet accessPrompts when it infers that user is…
Visiting a new destination
• Please take a picture!
![Page 14: Project ACCESS: Wayfinding for Persons with Cognitive Disabilities · 2018-05-02 · Project ACCESS: Wayfinding for Persons with Cognitive Disabilities GaetanoBoriello, Dieter Fox,](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f05b5e77e708231d414511c/html5/thumbnails/14.jpg)
StatusMedical partnershipso Funding by National Institute of Disability &
Rehabilitation Research (NIDRR)Extension to indoor navigationo Hospitals, nursing homes, assisted care communitieso Wi-Fi localization
Multi-modal interfaceo Speech, graphics, texto Guidance strategieso WOZ study with users with
cognitive disabilities
![Page 15: Project ACCESS: Wayfinding for Persons with Cognitive Disabilities · 2018-05-02 · Project ACCESS: Wayfinding for Persons with Cognitive Disabilities GaetanoBoriello, Dieter Fox,](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f05b5e77e708231d414511c/html5/thumbnails/15.jpg)
ObservationsBy and large, people lead regular lives – can learn useful movement patterns with surprisingly little data (2 weeks)o Many patterns hold across individuals
Modeling abstract behaviors (goals, trips) increases robustness & predictive powero 5 blocks into the future → 50 blocks
People with cognitive disabilities and their caregivers are eager for technological assistanceo Safety & independence trump privacyo Comfortable with technology – but may be too trustingo No one size fits all solution – seamless automatic
customization necessary
![Page 16: Project ACCESS: Wayfinding for Persons with Cognitive Disabilities · 2018-05-02 · Project ACCESS: Wayfinding for Persons with Cognitive Disabilities GaetanoBoriello, Dieter Fox,](https://reader034.fdocuments.net/reader034/viewer/2022052408/5f05b5e77e708231d414511c/html5/thumbnails/16.jpg)
PapersD. Patterson et al., Opportunity Knocks: a System to
Provide Cognitive Assistance with Transportation Services, UBICOMP-2004.
L. Liao, D. Fox, & H. Kautz, Learning and Inferring Transportation Routines, AAAI 2004.
A. Liu, H. Hile, H. Kautz, G. Borriello, P. Brown, M. Harniss, K. Johnson, Indoor Wayfinding: Developing a Functional Interface for Individuals with Cognitive Impairments, ASSETS 2006.