Using Augmented Reality to Create Empathic Experiences

download Using Augmented Reality to Create Empathic Experiences

of 71

  • date post

  • Category


  • view

  • download


Embed Size (px)


Keynote address by Mark Billinghurst at the IUI 2014 conference in Haifa Israel, February 27th, 2014.

Transcript of Using Augmented Reality to Create Empathic Experiences

  • 1.Using Augmented Reality to Create Empathic Experiences Mark Billinghurst The HIT Lab NZ, University of Canterbury February 27th 2014

2. Courtesy Matt Rettig, CMU 3. Processing PowerAdapt ExperienceOperate 4. Beyond the Desktop 5. Intelligent User Interfaces AI + HCI: User Interface involving some elements of Artificial Intelligence Computer having model of user/domain First IUI Conference in 1997 Readings in IUI (Wahster 1998) 6. Microsoft Clippy (1997) MS Office Intelligent User Interface 7. Intelligent User Interfaces 8. Multiple Intelligences Frames of Mind: The Theory of Multiple Intelligences Howard Gardner(1983) 9. Multiple Intelligences 10. Emotional Intelligence Emotional Intelligence - Why it can matter more than IQ Goldman (1995) Identify, assess, and control the emotions of oneself, of others, and of groups 11. Foundations of Emotional Intelligence 12. Empathy Empathy 13. Empathy vs. Intelligence Intelligence: the power of ones brain, divided into many categories and used in numerous ways. Empathy: the power of ones heart, expressing ones true emotions with themselves, those around them, and their own world. 14. IQ vs EQ? 15. Mirror Neurons Neuron that fires both when an animal acts and it observes the same action performed by another Giacomo Rizzolatti, Univ. of Palma (1980s/90s) 16. Empathic Computing 1. Computing systems that can understand your feelings and emotions 2. Computing systems that help you better understand the feelings of others 17. Affective Computing Ros Picard MIT Media Lab 18. Appliances That Make You Happy Jun Rekimoto Univ. Tokyo Smile detection + smart devices 19. Can we develop interfaces that allow us to be more empathetic to others? 20. adsf 21. Empathy Computing Requirements Basic Requirements Making the technology transparentEmpathy Definition Seeing with the eyes of another Hearing with the ears of another Feeling with the heart of another 22. Augmented Reality19772008 23. Using AR for Empathy Augmented Reality can: Remove technology barriers Enhance communication Change perspective Share experiences Enhance interaction in real world 24. Communication Seams Communication SpaceTask Space Technology introduces artificial seams in the communication (eg separate real and virtual space) 25. Removing Barriers: Shared Space Face to Face interaction, Tangible AR metaphor - ~3,000 users (Siggraph 1999) Easy collaboration with strangers Users acted same as if handling real objects Billinghurst, M., Poupyrev, I., Kato, H., & May, R. (2000). Mixing realities in shared space: An augmented reality interface for collaborative computing. In Multimedia and Expo, 2000. ICME 2000. 2000 IEEE International Conference on (Vol. 3, pp. 1641-1644). 26. Enhancing Face to Face Communication AR Pad Handheld AR device AR shows viewpoints Users collaborate easier Show communication cues Virtual Viewpoint VisualizationMogilev, D., Kiyokawa, K., Billinghurst, M., & Pair, J. (2002, April). AR Pad: An interface for face-to-face AR collaboration. In CHI'02 extended abstracts on Human factors in computing systems (pp. 654-655). 27. Changing Perspective CamNet (1992) British Telecom Wearable Teleconferencing audio, video Remote collaboration Sends task space video Similar CMU study (1996) cut performance time in half 28. WACL: Remote Expert Collaboration Wearable Camera/Laser Pointer Independent pointer control Remote panorama view 29. WACL: Remote Expert Collaboration Remote Expert View Panorama viewing, annotation, image capture Kurata, T., Sakata, N., Kourogi, M., Kuzuoka, H., & Billinghurst, M. (2004, October). Remote collaboration using a shoulder-worn active camera/laser. In Wearable Computers, 2004. ISWC 2004. Eighth International Symposium on (Vol. 1, pp. 62-69). 30. View Through Google GlassAlways available peripheral information display Combining computing, communications and content capture 31. Ego-Vision Collaboration Google Glass camera + processing + display + connectivity 32. Current Collaboration on Glass First person remote conferencing/hangouts Limitations Single POV, no spatial cues, no annotations, etc 33. Sharing Space: Social Panoramas Capture and share social spaces in real time Enable remote people to feel like theyre with you 34. Key Technology Google Glass Capture live image panorama (compass + camera) Capture spatial audio, live video Remote device (desktop, tablet) Immersive viewing, live annotation 35. Capturing Space: Real World Capture Hands free AR Portable scene capture (color + depth) Projector/Kinect combo, Remote controlled pan/tilt Remote expert annotation interface 36. Remote Expert View 37. Capturing Behaviours 3 Gear Systems Kinect/Primesense Sensor Two hand tracking 38. Skeleton Interaction + AR HMD AR View Viewpoint tracking Two hand input Skeleton interaction, occlusion 39. Ghostman Use AR to capture and overlay your actions into remote persons space Eg remote therapy 40. Looking to the FutureWhats Next? 41. Brain to Brain Control Rajesh Rao, University of Washington First Brain to Brain control 42. System Architecture 43. Scaling Up Seeing actions of millions of users in the world Augmentation on city/country level 44. AR + Smart Sensors + Social Networks Track population at city scale (mobile networks) Match population data to external sensor data medical, environmental, etc Mine data to improve social services 45. Research Challenges How to convey emotion? How to measure empathy? Interface/interaction models? How to communicate emotion? Scaling up to city/country scale? 46. Conclusion 47. Harvard Grant Study $20 million, 75 years study 268 Harvard graduates 456 disadvantaged people Led by George Valliant What makes us happy? warmth of relationships throughout life have the greatest positive impact on "life satisfaction". 48. The seventy-five years and twenty million dollars expended on the Grant Study points to a straightforward five-word conclusion: Happiness is love. Full stop. George Valliant 49. Conclusions Empathic Computing Sharing what you see, hear and feel AR Enables Empathic Experiences Removing technology Changing perspective Sharing space/experience Many directions for future research 50. More Information Mark Billinghurst Email: Twitter: @marknb00 Website