1

Augmented Reality

Günter Alce, Ph.D. candidate

: gunterAlce

2

Günter Alce

• Industrial Ph.D. - AR Visor interaction

• Sony Mobile Communications

• Department of Design sciences

• What should the AR Visor offer the user in different

contexts?

• How to combine different interaction techniques, such

as gestures, gaze and speech?

3

Overview

• Introduction of AR

• Evolution of AR

• Some applications areas

• AR technologies

• My research

• Summary

• HITLabNZ

4

What is augmented reality?

5

6

7

8

Definition

Augmented reality is a technology which:

1. Combines real and virtual

2. Interactive in real time

3. Registered in 3D

Azuma (1997)

9

The evolution of

Augmented Reality hardware

10

Steve Mann

11

The evolution of AR hardware

• Generation “Kit bag”

12

The evolution of AR hardware

• Generation “Hand bag”

13

14

Keyhole AR

• Limited view

• Awkward interaction

• Relying on a camera sensor

• Limited use

• Ergonomic issues

15

The evolution of AR hardware

• Generation “No bag”

“…uses a holographic waveguide… we have developed in partnership with Nokia”

“They are scheduled to be delivered in 2013… between 350 to 600 dollars” “We want these to be a consumer product” “It connects to any mobile device through a HDMI cable”

Vuzix at CES 2012

17

18

Upcoming glasses

•Olympus MEG4.0

•Project Glass

•Vuzix M100

19

Project glass (2012)

• Video clip

20

What could it be used for?

21

Some application areas

• Marketing

• Entertainment

• Education

• Assistance

• Medical Applications

22

Marketing

23

Marketing

24

Marketing (Kinect fitting room)

25

Entertainment

Nerdherder ARDefender

Danger copter AR Blitz

26

Entertainment

27

Entertainment (Experience for tourists)

28

Entertainment (Experience for tourists)

HistoriskaSkyltar P l a t s b a s e r a d h i s t o r i a i ö g o n b l i c k e t

29

Education

Iulian Radu

30

Assistance

31

Assistance

ionRoad

32

Medicine

M. Billinghurst

33

Medicine

34

Diminished Reality

Diminished reality

35

AR Technologies

36

AR Technologies

• AR Devices

• AR Interaction

37

AR Devices

• Displays

• Input devices

• Tracking

38

Displays

AR Devices

39

Displays

Azuma 1997

Video see-through Optical see-through

40

Video see-through

• Better consistency between real and virtual views • Better handling of occlusion problems

41

Optical see-through

• Natural, instantaneous view • Zero parallax

42

Optical see-through

43

Projection-based displays

44

OmniTouch

45

AR Devices

• Displays

• Input devices

• Tracking

46

Input devices

• Gloves

• Wristband (ReachMedia)

• Phone as pointing device

• Gestures

• Speech

• Gaze interaction

47

AR Devices

• Displays

• Input devices

• Tracking

48

Tracking

• The act of aligning the virtual content with the real

world view

49

Tracking

• Sensor-based

• Vision-based

• Hybrid

50

Sensor-based tracking

Technology Range (m) Precision (m)

• Accelerometers 1 0,1

• Inertial 1 0,01

• Magnetic 1 0,01

• WiFi 100 1

• RFID: active 20-100 0,5

• RFID: passive 0,05-5 0,5

• GPS ∞ 5

Table from Handbook of AR - An Overview

51

Micro Electro-Mechanical Systems (MEMS)

52

Tracking

• Sensor-based

• Vision-based

• Hybrid

53

Vision-based tracking

• Marker-based

• Markerless

• Feature-based

• SLAM

• PTAM

54

Marker-based tracking

55

Vision-based tracking

• Marker-based

• Markerless

• Feature-based

• SLAM

• PTAM

56

Feature-based tracking

Detecting feature points

57

SLAM and PTAM

Simultaneous Localization and Mapping (SLAM)

Parallel Tracking and Mapping (PTAM)

Video clip

58

Tracking

• Sensor-based

• Vision-based

• Hybrid

59

Hybrid

Camera

Accelerometer, gyroscope & compass

60

AR Technologies

• AR Devices

• AR Interaction

61

Interaction

ISMAR 2011 papers and posters User interfaces, interaction, human

factors, ergonomics, perception: ≈14%

ISMAR 2011

62

Interaction styles

• Tangible AR

• Collaborative AR

• Hybrid AR

• Multimodal AR

63

Tangible AR

• The user can manipulate the virtual information

through physical objects

64

Collaborative AR

65

Hybrid AR

66

Multimodal AR

• Gestures

• Speech

• Touch

• Gaze

67

Where are we today?

68

Perceptually correct augmentation remains a crucial challenge. ”

Ernst,, Kruijff, Swan, J. E., & Steve,, Feiner (2010). Perceptual Issues in Augmented Reality Revisited. IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2010). Seoul, Korea.

69

Vergence-accomodation conflict

Eyes focusing at a distance that differs from the

point of visual convergence

70

Definition

Augmented reality is a technology which:

1. Combines real and virtual

2. Interactive in real time

3. Registered in 3D

Azuma (1997)

71

Research

• What should the AR Visor offer the user in

different contexts?

• How to combine different interaction techniques,

such as gestures, gaze and speech?

72

Attention resources

73

What about SONY?

74

AR Visor concept (2011)

• Video clip

75

Wizard of Oz

76

Simulate navigation

Navigation

77

Simulate face recognition

Face recognition

78

VENTURI – a three year EU project about AR

• A consolidated prototype at the end of

each year

• Evaluated through use cases

• Gaming

• Personal assistant

• Tourism

79

Tools - AR SDK

1. Metaio SDK, http://www.metaio.com/

2. Vuforia, https://developer.qualcomm.com/mobile-

development/mobiletechnologies/augmented-reality

3. HITLabNZ, http://hitlabnz.org/index.php/products/mobile-ar-framework

4. AndAR - Android Augmented Reality, http://code.google.com/p/andar/

80

Summary of AR

• Mixing real and virtual

• Interaction

• Display

• Tracking

81

Master thesis

HITLabNZ, http://hitlabnz.org/

82

Thanks for listening!

Questions?