Designing Augmented Reality Experiences

download Designing Augmented Reality Experiences

of 80

  • date post

  • Category


  • view

  • download


Embed Size (px)


Talk on Designing Augmented Reality Experiences given by Mark Billinghurst at the AWE 2013 conference on June 5th 2013

Transcript of Designing Augmented Reality Experiences

  • Designing AugmentedReality ExperiencesMark Billinghurstmark.billinghurst@hitlabnz.orgThe HIT Lab NZ, University of CanterburyJune 5th 2013
  • How Would You Design This? Put nice AR Picture here and video
  • Or This?
  • DARE 1011. Know the Technology2. Design for User Experience All aspects of user experience3. Follow good Interaction Design principles Discover, Design, Evaluate4. Consider all the Design Elements Physical, Virtual and Metaphorical5. Know Future Research Directions
  • Know the Technology
  • What is Augmented Reality?Defining Characteristics (Azuma 97) Combines Real and Virtual ImagesBoth can be seen at the same time Interactive in real-timeThe virtual content can be interacted with Registered in 3DVirtual objects appear fixed in spaceAzuma, R., A Survey of Augmented Reality, Presence, Vol. 6, No. 4, August 1997, pp. 355-385.
  • AR From Science Fiction to Fact1977 Star Wars2008 CNN
  • AR Part of MR ContinuumMixed RealityReality - Virtuality (RV) ContinuumRealEnvironmentAugmentedReality (AR)AugmentedVirtuality (AV)VirtualEnvironment"...anywhere between the extrema of the virtuality continuum."P. Milgram and A. F. Kishino, Taxonomy of Mixed Reality Visual DisplaysIEICE Transactions on Information and Systems, E77-D(12), pp. 1321-1329, 1994.
  • Core TechnologiesCombining Real and Virtual Images Display technologiesInteractive in Real-Time Input and interactive technologiesRegistered in 3D Viewpoint tracking technologiesDisplayProcessingInput Tracking
  • Display TechnologiesTypes (Bimber/Raskar 2003)Head attached Head mounted display/projectorBody attached Handheld display/projectorSpatial Spatially aligned projector/monitorHMD Optical vs. Video see-through Optical: Direct view of real world -> safer, simpler Video: Video overlay -> more image registration options
  • Display Taxonomy
  • AR Input TechnologiesTangible objects Tracked itemsTouch (HHD) Glove, touchGesture Glove, free-handSpeech/MultimodalDevice motion HHD + sensors
  • Tracking TechnologiesActive Mechanical, Magnetic, Ultrasonic GPS, Wifi, cell locationPassive Inertial sensors (compass, accelerometer, gyro) Computer Vision Marker based, Natural feature tracking, model basedHybrid Tracking Combined sensors (eg Vision + Inertial)
  • Design for User Experience
  • The product is no longerthe basis of value. Theexperience is.Venkat RamaswamyThe Future of Competition.Interaction Design
  • experiencesservicesproductscomponentsValueGilmore + Pine: Experience EconomyFunctionEmotion
  • experiencesapplicationstoolscomponentsDesigning AR ExperiencesTracking, Display, InputAuthoringInteractionUsability
  • The Value of Good User ExperienceKenya: 20cMy house: 50cStarbucks: $3.50
  • Good Experience Design Reactrix Top down projection Camera based input Reactive Graphics No instructions No training
  • Would You Wear This?
  • User Experience is All About You Designing good userexperience involvesmany aspects Consider all theneeds of the user Especially context ofuse
  • Web Based AR Flash, HTML 5 based AR Marketing, education Outdoor Mobile AR GPS, compass tracking Viewing Points of Interest in real world Handheld AR Vision based tracking Marketing, gaming Location Based Experiences HMD, fixed screens Museums, point of sale, advertisingTypical AR Experiences
  • What Makes a Good AR Experience? Compelling Engaging, Magic moment Intuitive, ease of use Uses existing skills Anchored in physical world Seamless combination of real and digital
  • Demo: colAR Turn colouring books pages into AR scenes Markerless tracking, use your own colours.. Try it yourself:
  • Follow Good InteractionDesign Principles
  • Interaction Design Answering three questions: What do you do? - How do you affect the world? What do you feel? What do you sense of the world? What do you know? What do you learn?The Design of UserExperience with TechnologyDesigning interactive products tosupport people in their everydayand working livesPreece, J., (2002). Interaction Design
  • Interaction Design ProcessInteraction Design
  • AR UI Design Consider your user Follow good HCI principles Adapt HCI guidelines for AR Design to device constraints Using Design Patterns to Inform Design Design for you interface metaphor Design for evaluation
  • Consider Your User Consider context of user Physical, social, emotional, cognitive, etc Mobile Phone AR User Probably Mobile One hand interaction Short application use Need to be able to multitask Use in outdoor or indoor environment Want to enhance interaction with real world
  • AR vs. Non AR Design Design Guidelines Design for 3D graphics + Interaction Consider elements of physical world Support implicit interactionCharacteristics Non-AR Interfaces AR InterfacesObject Graphics Mainly 2D Mainly 3DObject Types Mainly virtual objects Both virtual and physical objectsObject behaviors Mainly passive objects Both passive and active objectsCommunication Mainly simple Mainly complexHCI methods Mainly explicit Both explicit and implicit
  • Maps vs. Junaio Google Maps 2D, mouse driven, text/image heavy, exocentric Junaio 3D, location driven, simple graphics, egocentric
  • Design to Device Constraints Understand the platform and design for limitations Hardware, software platforms Eg Handheld AR game with visual tracking Use large screen icons Consider screen reflectivity Support one-hand interaction Consider the natural viewing angle Do not tire users out physically Do not encourage fast actions Keep at least one tracking surface in view32Art of Defense Game
  • Design PatternsEach pattern describes a problem which occursover and over again in our environment, and thendescribes the core of the solution to that problem insuch a way that you can use this solution a milliontimes over, without ever doing it the same way twice. Christopher Alexander et al.Use Design Patterns to Address Reoccurring ProblemsC.A. Alexander, A Pattern Language, Oxford Univ. Press, New York, 1977.
  • Handheld AR PatternsTitle Meaning Embodied SkillsDevice Metaphors Using metaphor to suggest available playeractionsBody A&S Nave physicsControl Mapping Intuitive mapping between physical anddigital objectsBody A&S Nave physicsSeamful Design Making sense of and integrating thetechnological seams through game designBody A&SWorld Consistency Whether the laws and rules inphysical world hold in digital worldNave physicsEnvironmental A&SLandmarks Reinforcing the connection between digital-physical space through landmarksEnvironmental A&SPersonal Presence The way that a player is represented in thegame decides how much they feel like livingin the digital game worldEnvironmental A&SNave physicsLiving Creatures Game characters that are responsive tophysical, social events that mimic behavioursof living beingsSocial A&S Body A&SBody constraints Movement of ones body positionconstrains another players actionBody A&S Social A&SHidden information The information that can be hidden andrevealed can foster emergent social playSocial A&S Body A&S
  • Example: Seamless Design Design to reduce seams in the user experience Eg: AR tracking failure, change in interaction mode Paparazzi Game Change between AR tracking to accelerometer inputYan Xu , , Pre-patterns for designing embodied interactions in handheld augmented realitygames, Proceedings of the 2011 IEEE International Symposium on Mixed and Augmented Reality--Arts, Media, and Humanities, p.19-28, October 26-29, 2011
  • Example: Living Creatures Virtual creatures respond to real world events eg. Player motion, wind, light, etc Creates illusion creatures are alive in the real world Sony EyePet Responds to player blowing on