Augmented Reality

Augmented Reality

CHAPTER 1

INTRODUCTION

Video games have been entertaining us for nearly 30 years, ever since Pong was

introduced to arcades in the early 1970s. Computer graphics have become much more

sophisticated since then, and game graphics are pushing the barriers of photorealism.

Now, researchers and engineers are pulling graphics out of your television screen or

computer display and integrating them into real-world environments. This new

technology, called augmented reality, blurs the line between what's real and what's

computer-generated by enhancing what we see, hear, feel and smell.

On the spectrum between virtual reality, which creates immersive, computer-

generated environments, and the real world, augmented reality is closer to the real world.

Augmented reality adds graphics, sounds, haptic feedback and smell to the natural world

as it exists. Both video games and cell phones are driving the development of augmented

reality.

Augmented reality is changing the way we view the world - or at least the way its

users see the world. Picture yourself walking or driving down the street. With augmented-

reality displays, which will eventually look much like a normal pair of glasses,

informative graphics will appear in your field of view, and audio will coincide with

whatever you see.

Figure 1.1 The Sixth Sense augmented reality system lets you project a phone pad onto your hand

and phone a friend -- without removing the phone from your pocket.

Department of ECE, BGSIT

Augmented Reality

These enhancements will be refreshed continually to reflect the movements of your head.

Similar devices and applications already exist, particularly on smartphones like the

iPhone. Picture shows The SixthSense augmented reality system lets you project a phone

pad onto your hand and phone a friend -- without removing the phone from your pocket.

1.1 What is Mobile Augmented Reality for?

As computers increase in power and decrease in size, new mobile, wearable, and

pervasive computing applications are rapidly becoming feasible, providing people access

to online resources always and everywhere. This new flexibility makes possible new kind

of applications that exploit the person's surrounding context. Augmented reality (AR)

presents a particularly powerful user interface (UI) to context-aware computing

environments. AR systems integrate virtual information into a person's physical

environment so that he or she will perceive that information as existing in their

surroundings. Mobile augmented reality systems (MARS) provide this service without

constraining the individual’s whereabouts to a specially equipped area. Ideally, they work

virtually anywhere, adding a palpable layer of information to any environment whenever

desired. By doing so, they hold the potential to revolutionize the way in which

information is presented to people.

Computer-presented material is directly integrated with the real world surrounding

the freely roaming person, who can interact with it to display related information, to pose

and resolve queries, and to collaborate with other people. The world becomes the user

interface


Augmented Reality

CHAPTER 2

MOBILE AUGMENTED REALITY

Augmented reality (AR) is a live, direct or indirect, view of a physical, real-world

environment whose elements are augmented by computer-generated sensory input such as

sound, video, graphics or GPS data. It is related to a more general concept called

mediated reality, in which a view of reality is modified (possibly even diminished rather

than augmented) by a computer. As a result, the technology functions by enhancing one’s

current perception of reality. By contrast, virtual reality replaces the real world with a

simulated one.

Augmentation is conventionally in real-time and in semantic context with

environmental elements, such as sports scores on TV during a match. With the help of

advanced AR technology (e.g. adding computer vision and object recognition) the

information about the surrounding real world of the user becomes interactive and digitally

manipulable. Artificial information about the environment and its objects can be overlaid

on the real world. The term augmented reality is believed to have been coined in 1990 by

Thomas Caudell, working at Boeing.

Research explores the application of computer-generated imagery in live-video

streams as a way to enhance the perception of the real world. AR technology includes

head-mounted displays and virtual retinal displays for visualization purposes, and

construction of controlled environments containing sensors and actuators.

Augmented Reality is considered as an extension of Virtual Reality. Virtual Reality

(VR) is a virtual space where the player immerses themselves into that exceed the bounds

of physical reality. In the VR, time, physical laws and material properties no longer hold

in contrast to real-world environment. Instead of considering AR and VR as exact

opposite concepts.

AR is about augmenting the real world environment with virtual information

by improving people’s senses and skills. AR mixes virtual characters with the actual

world. There are three common characteristics of AR scenes: the combination of real

world environment with computer characters, interactive scenes, and scenes in 3D.


Augmented Reality

2.1 Augmented Reality in Cell Phones

Figure 2.1 iPhone applications using Augmented Reality

While it may be some time before you buy a device like SixthSense, more

primitive versions of augmented reality are already here on some cell phones, particularly

in applications for the iPhone and phones with the Android operating system. In the

Netherlands, cell phone owners can download an application called Layar that uses the

phone's camera and GPS capabilities to gather information about the surrounding area.

Layar then shows information about restaurants or other sites in the area, overlaying this

information on the phone's screen. You can even point the phone at a building, and Layar

will tell you if any companies in that building are hiring, or it might be able to find photos

of the building on Flickr or to locate its history on Wikipedia.

Layar isn't the only application of its type. In August 2009, some iPhone users were

surprised to find an augmented-reality "easter egg" hidden within the Yelp application.

Yelp is known for its user reviews of restaurants and other businesses, but its hidden

augmented-reality component, called Monocle, takes things one step further. Just start up

the Yelp app, shake your iPhone 3GS three times and Monocle activates. Using your

phone's GPS and compass, Monocle will display information about local restaurants,

including ratings and reviews, on your cell phone screen. You can touch one of the

listings to find out more about a particular restaurant.


Augmented Reality

There are other augmented reality apps out there for the iPhone and other similar

phones -- and many more in development. Urbanspoon has much of the same

functionality as Yelp's Monocle. Then there's Wikitude, which finds information from

Wikipedia about sites in the area. Underlying most of these applications are a phone's

GPS and compass; by knowing where you are, these applications can make sure to offer

information relevant to you. We're still not quite at the stage of full-on image recognition,

but trust us, people are working on it. We've looked at some of the existing forms of

augmented reality. On the next page, we'll examine some of the other applications of the

technology, such as in video games and military hardware.

2.2 Augmented Reality in Video Games and the Military

Video game companies are quickly hopping aboard the augmented-reality

locomotive. A company called Total Immersion makes software that applies augmented

reality to baseball cards. Simply go online, download the Total Immersion software and

then hold up your baseball card to a webcam. The software recognizes the card (and the

player on it) and then displays related video on your computer screen. Move the card in

your hands -- make sure to keep it in view of the camera - and the 3-D figure on your

screen will perform actions, such as throwing a ball at a target.

Total Immersion's efforts are just the beginning. In the next couple of years, we'll

see games that take augmented reality out into the streets. Consider a scavenger-hunt

game that uses virtual objects. You could use your phone to "place" tokens around town,

and participants would then use their phones (or augmented-reality enabled goggles) to

find these invisible objects.

Demos of many games of this order already exist. There's a "human Pac-Man"

game that allows users to chase after each other in real life while wearing goggles that

make them look like characters in Pac-Man.

Arcane Technologies, a Canadian company, has sold augmented-reality devices to

the U.S. military. The company produces a head-mounted display -- the sort of device

that was supposed to bring us virtual reality -- that superimposes information on your

world. Consider a squad of soldiers in Afghanistan, performing reconnaissance on an

opposition hideout. An AR-enabled head-mounted display could overlay blueprints or a

view from a satellite or overheard drone directly onto the soldiers' field of vision.


Augmented Reality

2.3 Augmenting Our World

The basic idea of augmented reality is to superimpose graphics, audio and other

sensory enhancements over a real-world environment in real time. Sounds pretty simple.

Besides, haven't television networks been doing that with graphics for decades? However,

augmented reality is more advanced than any technology you've seen in television

broadcasts, although some new TV effects come close, such as RACEf/x and the super-

imposed first down line on televised U.S. football games, both created by Sportvision.

But these systems display graphics for only one point of view. Next-generation

augmented-reality systems will display graphics for each viewer's perspective.

Some of the most exciting augmented-reality work is taking place in research labs

at universities around the world. In February 2009, at the TED conference, Pattie Maes

and Pranav Mistry presented their augmented-reality system, which they developed as

part of MIT Media Lab's Fluid Interfaces Group. They call it SixthSense, and it relies on

some basic components that are found in many augmented reality systems:

Camera

Small projector

Smartphone

Mirror

These components are strung together in a lanyard like apparatus that the user

wears around his neck. The user also wears four colored caps on the fingers, and these

caps are used to manipulate the images that the projector emits. SixthSense is remarkable

because it uses these simple, off-the-shelf components that cost around $350. It is also

notable because the projector essentially turns any surface into an interactive screen.

Essentially, the device works by using the camera and mirror to examine the surrounding

world, feeding that image to the phone (which processes the image, gathers GPS

coordinates and pulls data from the Internet), and then projecting information from the

projector onto the surface in front of the user, whether it's a wrist, a wall, or even a

person. Because the user is wearing the camera on his chest, SixthSense will augment

whatever he looks at; for example, if he picks up a can of soup in a grocery store,

SixthSense can find and project onto the soup information about its ingredients, price,

nutritional value -- even customer reviews.


Augmented Reality

CHAPTER 3

WORKING OF AUGMENTED REALITY AND ITS

TECHNOLOGY INTRODUCED INTO MOBILE

On the spectrum between virtual reality, which creates immersive, computer-

generated environments, and the real world, augmented reality is closer to the real world.

Augmented reality adds graphics, sounds, haptic feedback and smell to the natural world

as it exists. Both video games and cell phones are driving the development of augmented

reality. Everyone from tourists, to soldiers, to someone looking for the closest subway

stop can now benefit from the ability to place computer-generated graphics in their field

of vision.

Augmented reality is changing the way we view the world -- or at least the way its

users see the world. Picture yourself walking or driving down the street. With augmented-

reality displays, which will eventually look much like a normal pair of glasses,

informative graphics will appear in your field of view, and audio will coincide with

whatever you see. These enhancements will be refreshed continually to reflect the

movements of your head.

3.1 Hardware

The main hardware components for augmented reality are: processor, display,

sensors and input devices. These elements, specifically CPU, display, camera and MEMS

sensors such as accelerometer, GPS, solid state compass are often present in modern

smartphones, which make them prospective AR platforms.

3.1.1 Computer

The computer analyzes the sensed visual and other data to synthesize and position

augmentations. Camera based systems require powerful CPU and considerable amount of

RAM for processing camera images. Wearable computing systems employ a laptop in a


Augmented Reality

backpack configuration. For stationary systems a traditional workstation with a powerful

graphics card. Sound processing hardware could be included in augmented reality

systems.

3.1.2 Tracking System

Modern mobile augmented reality systems use one or more of the following

tracking technologies: digital cameras and/or other optical sensors, accelerometers, GPS,

gyroscopes, solid state compasses, RFID and wireless sensors. These technologies offer

varying levels of accuracy and precision. Most important is the position and orientation of

the user's head. Tracking the user's hand(s) or a handheld input device can provide a

6DOF interaction technique.

3.1.3 Display

There are three major display techniques for Augmented Reality: head-mounted

displays, handheld displays and spatial displays. Some examples of spatial augmented

reality displays include shader lamps, mobile projectors, virtual tables, and smart

projectors, described by O. Bimber and R. Raskar in 2005. Shader lamps, developed by

Raskar et al. in 1999, mimic and augment reality by projecting imagery onto neutral

objects, providing the opportunity to enhance the object’s appearance. This can be

accomplished with materials of a simple unit- a projector, camera, and sensor. Handheld

projectors further this goal by enabling cluster configurations of environment sensing,

reducing the need for additional environmental sensing.

Other tangible applications include table and wall projections. One such innovation,

the Extended Virtual Table, separates the virtual from the real by including beam-splitter

mirrors attached to the ceiling at an adjustable angle. Virtual showcases, which employ

beam-splitter mirrors together with multiple graphics displays, provide an interactive

means of simultaneously engaging with the virtual and the real. Altogether, augmented

reality display technology can be applied to improve design and visualization, or function

as scientific simulations, and tools for education or entertainment.

3.1.4 Input Devices

Some systems such as the tinsmith system, employ pinch glove techniques.

Another common techniques is a wand with a button on it. In case of smartphone, phone


Augmented Reality

itself could be used as 3D pointing device, with 3D position of the phone restored from

the camera images.

3.2 Software and Algorithms

A key measure of AR systems is how realistically they integrate augmentations

with the real world. The software must derive real world coordinates, independent from

the camera, from camera images. That process is called image registration and is part of

Azuma's definition of Augmented Reality.

Image registration uses different methods of computer vision, mostly related to

video tracking. Many computer vision methods of augmented reality are inherited from

visual odometry. Usually those methods consist of two parts. First detect interest points,

or fiduciary markers, or optical flow in the camera images. First stage can use feature

detection methods like corner detection, blob detection, edge detection or thresholding

and/or other image processing methods.

The second stage restores a real world coordinate system from the data obtained in

the first stage. Some methods assume objects with known geometry (or fiduciary

markers) present in the scene. In some of those cases the scene 3D structure should be

precalculated beforehand. If part of the scene is unknown simultaneous localization and

mapping (SLAM) can map relative positions. If no information about scene geometry is

available, structure from motion methods like bundle adjustment are used. Mathematical

methods used in the second stage include projective (epipolar) geometry, geometric

algebra, and rotation representation with exponential map, kalman and particle filters,

nonlinear optimization, and robust statistics.


Augmented Reality

CHAPTER 4

TRACKING AND ORIENTATION

The biggest challenge facing developers of augmented reality is the need to know where

the user is located in reference to his or her surroundings. There's also the additional

problem of tracking the movement of users' eyes and heads. A tracking system has to

recognize these movements and project the graphics related to the real world environment

the user is seeing at any given moment. Currently, both video see through and optical see-

through displays typically have lag in the overlaid material due to the tracking

technologies currently available.

4.1 Indoor Tracking

Tracking is easier in small spaces than in large spaces. Trackers typically have

two parts: one worn by the tracked person or object and the other built into the

surrounding environment, usually within the same room. In optical trackers, the targets--

LEDs or reflectors, for instance--can be attached to the tracked person or object, and an

array of optical sensors can be embedded in the room's ceiling. Alternatively, the tracked

users can wear the sensors, and the targets can be fixed to the ceiling. By calculating the

distance to each visible target, the sensors can determine the user's position and

orientation.

Researchers at the University of North Carolina-Chapel Hill have developed a

very precise system that works within 500 square feet. The HiBall Tracking System is an

optoelectronic tracking system made of two parts:

Figure 4.1 LEDs and Reflectors used for tracking in Indoor


Augmented Reality

External view of the columbian printer maintainence application. Note that all the

components must be tracked. External view of the columbian printer maintainence

application. Note that all the components must be tracked. The system uses the known

location of the LEDs, the known geometry of the usermounted optical sensors and a

special algorithm to compute and report the user's position and orientation. The system

resolves linear motion of less than .2 millimeters, and angular motions less than .03

degrees. It has an update rate of more than 1500 Hz, and latency is kept at about one

millisecond.

In everyday life, people rely on several senses--including what they see, cues from

their inner ears and gravity's pull on their bodies--to maintain their bearings. In a

similarfashion, "hybrid trackers" draw on several sources of sensory information. For

example, thewearer of an AR display can be equipped with inertial sensors (gyroscopes

and accelerometers) to record changes in head orientation. Combining this information

with data from the optical, video or ultrasonic devices greatly improves the accuracy of

the tracking.

4.2 Outdoor Tracking

Head orientation is determined with a commercially available hybrid tracker that

combines gyroscopes and accelerometers with a magnetometer that measures the earth's

magnetic field. For position tracking, we take advantage of a high-precision version of the

increasingly popular Global Positioning System receiver.

A GPS receiver determines its position by monitoring radio signals from

navigation satellites. GPS receivers have an accuracy of about 10 to 30 meters. An

augmented-realitysystem would be worthless if the graphics projected were of something

10 to 30 meters away from what you were actually looking at. Users can get better results

with a technique known as differential GPS. In this method, the mobile GPS receiver also

monitors signals from another GPS receiver and a radio transmitter at a fixed location on

the earth. This transmitter broadcasts corrections based on the difference between the

stationary GPS antenna's known and computed positions. By using these signals to

correct the satellite signals, differential GPS can reduce the margin of error to less than

one meter. Our system is able to achieve centimeter-level accuracy by employing real-


Augmented Reality

time kinematic GPS, a more sophisticated form of differential GPS that also compares the

phases of the signals at the fixed and mobile receivers.

Unfortunately, GPS is not the ultimate answer to position tracking. The satellite

signals are relatively weak and easily blocked by buildings or even foliage. This rules out

useful tracking indoors or in places like midtown Manhattan, where rows of tall buildings

block most of the sky. GPS tracking works well in wide open spaces and relatively low

buildings. GPS provides far too few updates per second and is too inaccurate to support

the precise overlaying of graphics on nearby objects.

Augmented-reality systems place extraordinarily high demands on the accuracy,

resolution, repeatability and speed of tracking technologies. Hardware and software

delays introduce a lag between the user's movement and the update of the display. As a

result, virtual objects will not remain in their proper positions as the user moves about or

turns his or her head. One technique for combating such errors is to equip AR systems

with software that makes short-term predictions about the user's future motions by

extrapolating from previous movements. And in the long run, hybrid trackers that include

computer vision technologies may be able to trigger appropriate graphics overlays when

the devices recognize certain objects in the user's view.


Augmented Reality

CHAPTER 5

CLASSIFICATION OF MOBILE AR

5.1 Properties of MOBILE AR User Interface

Mobile AR presents a way for people to interact with computers that is radically

different from the static desktop or mobile office. One of the key characteristics of MARS

is that both virtual and physical objects are part of the UI, and the dynamic context of the

user in the environment can influence what kind of information the computer needs to

present next. This raises several issues:

Control: Unlike a stand-alone desktop UI, where the only way the user can

interact with the presented environment is through a set of well defined

techniques,the MARS UI needs to take into account the unpredictability of the

real world. Forexample, a UI technique might rely on a certain object being in the

user's field of view and not occluded by other information. Neither of the

properties can be guaranteed: the user is free to look away, and other information

could easily get in the way, triggered by the user's own movement or an

unforeseen event (such as another user entering the field of view). Thus, to be

effective, the UI technique either has to relax the non-occlusion requirement, or

has to somehow guarantee on-occlusion in spite of possible contingencies.

Consistency: People have internalized many of the laws of the physical world.

When using a computer, a person can learn the logic of a new UI. As long as these

two worlds are decoupled (as they are in the desktop setting), inconsistencies

between them are often understandable. In the case of MARS, however, we need

to be very careful to design UIs in which the physical and virtual world are

consistent with each other. Need for embedded semantic information: In MARS,

virtual material is overlaid on top of the real world. Thus we need to establish

concrete semantic relationships between virtual and physical objects to

characterize UI behavior. In fact, since many virtual objects are designed to


Augmented Reality

annotate the real world, these virtual objects need to store information about the

physical objects to which they refer (or at least have to know how to access that

information). Display space: In terms of the available display space and its best

use, MARS UIs have to deal with a much more complicated task compared to

traditional 2D UIs. Instead of one area of focus (e.g., one desktop display), we

have to deal with a potentially unlimited display space surrounding the user, only

a portion of which is visible at any time. The representation of that portion of

augmented space depends on the user's position, head orientation, personal

preferences (e.g., filter settings) and ongoing interactions with the augmented

world, among other things. Management of virtual information in this space is

made even more difficult by constraints that other pieces of information may

impose. Certain virtual or physical objects may, for example, need to be visible

under all circumstances, and thus place restrictions on the display space that other

elements are allowed to obstruct. The display management problem is further

complicated by the possibility of taking into account multiple displays. MARS, as

a nonexclusive UI to the augmented world, may seamlessly make use of other

kinds of displays, ranging from wall-sized, to desk-top, to hand-held. If such

display devices are available and Telegeoinformatics: Location-Based Computing

and Services 33 accessible to the MARS, questions arise as to which display to

use for what kind of information and how to let the user know about that decision.

Scene dynamics: In a head-tracked UI, the scene will be much more dynamic than

in a stationary UI. In MARS, this is especially true, since in addition to all the

dynamics due to head motion, the system has to consider moving objects in the

real world that might interact visually or audibly with the UI presented on the

headworn display. Also, we have to contend with a potentially large variability in

tracking accuracy over time. Because of these unpredictable dynamics, the spatial

composition of the UI needs to be flexible and the arrangement of UI elements

may need to be changed. On the other hand, traditional UI design wisdom

suggests minimizing dynamic changes in the UI composition (Shneiderman,

1998). One possible solution to this dilemma lies in the careful application of

automated UI management techniques.


Augmented Reality

5.2 Classification of Augmented Reality

Four major classes of AR can be distinguished by their display type: Optical

sensor through, Virtual retinal system, Video see-through, Monitor based AR and

Projector Based AR.

The following sections show the corresponding devices and present their main

features.

5.2.1 Optical See through Head Mounted Display

Optical see-through AR uses a transparent head mounted display to show the virtual

environment directly over real world (Figures 5.1 & 5.2). It works by placing optical

combiners in front of the users eye. These combiners are partially transmissive, so that the

user can look directly through them to see the real world. The combiners are also partially

reflective, so that the user sees virtual images bounced off the combiners from head-

mounted monitors.

Figure 5.1 Optical See Through HMD


Augmented Reality

Fig 5.2 Optical See Through-Scheme

A simple approach to optical see-through display employs a mirror beam splitter,

a half-silvered mirror that both reflects and transmits light. If properly oriented in front of

the user's eye, the beam splitter can reflect the image of a computer display into the user's

line of sight yet still allow light from the surrounding world to pass through. Such beam

splitters, which are called combiners, have long been used in "head-up" displays for

fighter-jet pilots (and, more recently, for drivers of luxury cars).

Lenses can be placed between the beam splitter and the computer display to focus

the image so that it appears at comfortable viewing distance. If a display and optics are

provided for each eye, the view can be in stereo. Sony makes a see-through display that

some researchers use, called the Glasstron.

Prime examples of an optical see-through AR system are the various augmented medical

systems. The MIT image guided surgery has concentrated on brain surgery. UNC has

been working with an AR enhanced ultrasound system and other ways to superimpose

radiographic images on a patient. There are many other optical see-through systems, as it

seems to be main direction of AR.

Recent optical see through HMD’s are being built for well known companies like SONY

and OLYMPUS and have support for occlusion, varying accommodations. There are very

small prototypes that can be attached to conventional eyeglasses.


Augmented Reality

Figure 5.3 Eyeglass display with Virtual reality

5.2.2 Virtual Retinal Display

The VRD (VIRTUAL RETINAL DISPLAY) was invented at the university of

Washington in the Human Interface Technology Lab (HIT) in 1991. The aim was to

produce a full colour, wide field-of-view, high resolution, high brightness, low cost

virtual display. Microvision inc. Has the exclusive licence to commercialize the VRD

technology (Figure 5.4)

Figure 5.4 Virtual Retinal System HMD


Augmented Reality

This technology has many potential applications, from head-mounted displays

(HMD’s) for military/aerospace applications to medical purposes. The VRD projects a

modulate beam of light directly onto retina of the eye producing a rasterized image (fig

5.5). the viewer has the illusion of seeing image as if he/she stands two feet away in front

of a 14-inch monitor. In reality, the image is on the retina of its eye and not on a screen.

The quality of the image he/she sees is excellent with stereo view, full colour, wide field

of view and no flickering characteristics.

Figure 5.5 Virtual Retinal System Scheme

5.2.3 Video See-Through HMD

In contrast, a video see-through display uses video mixing technology, originally

developed for television special effects, to combine the image from a headworn camera

with synthesized graphics. The merged image is typically presented on an opaque head-

worn display. With careful design, the camera can be positioned so that its optical path is

close to that of the user's eye; the video image thus approximates what the user would

normally see. As with optical see-through displays, a separate system can be provided for

each eye to support stereo vision.

Video composition can be done in more than one way. A simple way is to use

chroma-keying: a technique used in many video special effects. The background of the

computer graphic images is set to a specific color, say green, which none of the virtual

objects use. Then the combining step replaces all green areas with the corresponding parts


Augmented Reality

from the video of the real world. This has the effect of superimposing the virtual objects

over real world. A more sophisticated composition would use depth information. If the

system had depth information at each pixel for the real world images, it could combine

the real and virtual images by a pixel-by-pixel depth comparison. This would allow real

objects to cover virtual objects and vice-versa.

Figure 5.6 Video See-Through HMD

5.2.4 Monitor Based Augmented Reality

Monitor based AR also uses merged video streams but the display is a more

conventional desktop monitor or a hand held display. It is perhaps the least difficult AR

setup, as it eliminates HMD issues. Princeton Video Image , inc. has developed a

technique for merging graphics into real time video streams. Their work is regularly seen

as the first down line in American Football Games. It is also used for placing advertising

logos into various broadcasts.


Augmented Reality

Figure 5.7 Monitor Based Scheme.

Figure 5.8 Monitor Based Example.


Augmented Reality

CHAPTER 6

APPLICATIONS OF MOBILE AUGMENTED REALITY

6.1 Mobile Augmented Reality Contact Lenses - Create New

World of Visions

Figure 6.1 Mobile AR Contact Lens

Mobile augmented realitycontact lenses may do more that improve your sight.

Someday they could replace your mobile phone and let you communicate visually

anywhere in the world, improve your health and make virtual reality real. Perhaps your

ophthalmologist could perform Lasik surgery, burning a wireless circuit into your cornea?

6.1.1 Mobile Augmented Reality Hits Contact Lens Technology

Babak Parviz at the University of Washington in Seattle, is working on a contact

lens technology that could revolutionize wireless health monitoring and mobile

applications for your iPhone. But don’t stop there…

Babak Parviz’ lenses become biosensors that monitor internal body functions. While the

prototype version of the lens is powered by radio waves beaming electricity to a loop

antenna embedded in the contact lens, Parviz thinks a mobile phone or solar cells

(wireless electricity) could generate power for the lenses. Mobile augmented reality could

be just around the corner.


Augmented Reality

6.2 iPHONE & Augmented Reality Apps: The Future of Gaming

Figure 6.2 iPHONE Mobile AR applications.

The starting point for this article came from an unconventional place – a furniture

app from Harveys that allows you to place the sofa of your dreams (possibly the most

unnecessary use of hyperbole ever there) into a room in your house without actually

having to buy it and lug it into your house. Very simply, the app allows users to see what

particular furniture will look like in their rooms, using an innovative augmented reality

camera, in a hypothetical try-before-you-buy approach to pre-shopping. A great app, by

the way, and one that scratches an itch you never thought you knew existed, which is

always key to anything like this being successful, but hardly one of the great

advancements of the 21st century.

6.3 See the Future: Augmented Reality Head-Up Displays

Beckon

Picture-in-picture may be coming to your next car courtesy of head-up displays that put

more snippets of information in your line of sight. By giving you controlled doses of data

projected onto a reflective rectangle just above the steering wheel, automakers say you’ll

be safer because your eyes don’t wander about the cockpit looking to the center stack and

instrument panel.


Augmented Reality

Fig 6.3 Shows object or gives information heading up front.

Imagine an exit or turn arrow that doesn’t just point to the right but has the same

angle as the turn you’re approaching and expands as you reach the turn. You’ll interact

with the displays by arm gestures, voice input, or traditional dashboard buttons and

knobs. Cost will be an issue since current HUDs run more than $1,000.

Fig 6.4 Implementation of head up display in Mercedes.

The Mercedes demo on the show floor took the form of a virtual drive in a

simulator through San Francisco with points of interest circled, the idea being you could

gesture to get more information. The gesture you’d use in a busy urban area might be a

raised middle finger because of the potential information overload: restaurant, bar,

jewelry shops, tour bus stops, bridge and tunnel congestion. And that’s even before you

wonder who’s providing the POI information and is it there because it’s the best, or

because it pays the automaker the best. This is an issue for the whole industry, not just

Mercedes, whose motto is The Best or Nothing.


Augmented Reality

6.4 Wire up your Eyes: 2012 is the Year of Augmented Reality

The term ‘augmented reality,’ or AR, is used to describe technology that enables

normal sight to be modified or enhanced by a computer, allowing data on an object in the

field of view to be displayed alongside it, or to add objects or menus that aren’t really

there to ‘augment’ your normal senses. If this all sounds a long way off to you, you need

look no further than your smartphone’s app store to find a host of smartphone apps

currently in use that use augmented reality to some extent, using the phone’s camera.

The Nintendo 3DS uses cards for its augmented reality content. If you place an

AR card on a table the 3DS’s camera recognises the card and its orientation and inserts

Nintendo characters onto your screen to do battle on the table top. This works just like

QR codes for your smartphone, as a kind of visual barcode for your device’s camera-

each card represents a different character or action. This kind of content will become a

standard feature of handheld gaming in 2012.

6.5 New Augmented Reality Apps Point toward Future Trends

6.5.1 New technologies are evolving to make smartphones really useful.

Augmented reality is a technology that combines the real world with digital

information. It gives users the impression that they are interacting with real and physical


Augmented Reality

objects. The technology itself is not exactly new and is being used in a varied of

applications from GPS systems to fitness apps.

Fig 6.5 Feature allowing tactile feedback in Smartphones.

Future

The future of augmented reality seems to be more inclined towards haptics and

tactile feedback (which uses sense of touch).

Earlier, Senseg, which claims mastery over such technology, had demonstrated at

the Consumer Electronics Show, haptic technology that allowed users to feel their apps

on the touchscreen by manipulating an electric field. With the technology, users will now

be able to feel bumps and ridges, and also figure out which areas are more rough than

others. The company believes if users are able to get the feel of anything other than glass,

it would be a better experience. The company demonstrated an Android tablet with a

touchscreen, which had different textures on it. Users get used to such a touchscreen

quickly and may not want to go back to a regular touchscreen. Senseg has deliberately

made the effect subtle so it doesn't distract the users while making its presence clearly

felt. The company is still working on different kind of sensations. Another app from

application developer CrowdOptic may point towards a new trend in augmented reality

apps. The new technology of CrowdOptic focuses on crowds, such as in concert or sports

events. When the camera of the smartphone is pointed at a player during a sporting event,

it displays real time information about the player and the game. The details and context


http://www.themobileindian.com/glossary/GPS

Augmented Reality

can also be shared through different social networks. So far, getting information on

moving objects through augmented reality apps was not possible.


Augmented Reality

CONCLUSION AND FUTURE OF MOBILE

AUGMENTED REALITY

In this chapter, we have presented an overview of the field of mobile AR,

Augmented reality still has some challenges to overcome. For example, GPS is only

accurate to within 30 feet (9 meters) and doesn't work as well indoors, although improved

image recognition technology may be able to help.

People may not want to rely on their cell phones, which have small screens on

which to superimpose information. For that reason, wearable devices like SixthSense or

augmented-reality capable contact lenses and glasses will provide users with more

convenient, expansive views of the world around them. Screen real estate will no longer

be an issue. In the near future, you may be able to play a real-time strategy game on your

computer, or you can invite a friend over, put on your AR glasses, and play on the

tabletop in front of you.

There is such a thing as too much information. Just as the "CrackBerry"

phenomenon and Internet addiction are concerns, an overreliance on augmented reality

could mean that people are missing out on what's right in front of them. Some people may

prefer to use their AR iPhone applications rather than an experienced tour guide, even

though a tour guide may be able to offer a level of interaction, an experience and a

personal touch unavailable in a computer program. And there are times when a real

plaque on a building is preferable to a virtual one, which would be accessible only by

people with certain technologies.

There are also privacy concerns. Image-recognition software coupled with AR

will, quite soon, allow us to point our phones at people, even strangers, and instantly see

information from their Facebook, Twitter, Amazon, LinkedIn or other online profiles.

With most of these services people willingly put information about themselves online, but

it may be an unwelcome shock to meet someone, only to have him instantly know so

much about your life and background.


Augmented Reality

Despite these concerns, imagine the possibilities: you may learn things about the

city you've lived in for years just by pointing your AR-enabled phone at a nearby park or

building. If you work in construction, you can save on materials by using virtual markers

to designate where a beam should go or which structural support to inspect.

Paleontologists working in shifts to assemble a dinosaur skeleton could leave virtual

"notes" to team members on the bones themselves, artists could produce virtual graffiti

and doctors could overlay a digital image of a patient's X-rays onto a mannequin for

added realism.

The future of augmented reality is clearly bright, even as it already has found its

way into our cell phones and video game systems.


Augmented Reality

REFERENCES

[01] M. Uenohara and T. Kanade. Vision-based object registration for real-time image

overlay. In Proc. of the IEEE Conference on Computer Vision, Virtual Reality and

Robotics in Medicine (CVRMed’95), pages 13–22, 1995.

[02] B. Walter, G. Alppay, E. Lafortune, S. Fernandez, and D. Greenberg. Fitting virtual

lights for non-diffuse walkthroughs. In Proceedings of SIGGRAPH 97, 1997.

[03] P. Wellner. Interacting with paper on the digital desk. Communications of the ACM,

36(7):87–96, 1993.

[04] J. Weng, P. Cohen, and N. Rebibo. Motion and structure estimation from stereo

image sequences. IEEE Transactions on Robotics and Automation, 8:362–382, June

1992.

[05] J. Weng, T. S. Huang, and N. Ahuja. Motion and structure from two perspective

views: Algorithms, error analysis, and error estimation. IEEE Transactions on Pattern

Analysis and Machine Intelligence, 11(5):451–476, 1989. This is a computer vision paper

that has geometry extraction from motion and stereo.

[06] R. Whitaker, C. Crampton, D. Breen, M. Tuceryan, and E. Rose. Object calibration

for augmented reality. In Proceedings of Eurographics’95, pages 15–27, Maastricht, The

Netherlands, August 1995.

[07] M. Wloka and B. Anderson. Resolving occlusion in augmented reality. In

Proceedings of the ACM Symposium on Interactive 3D Graphics, pages 5–12, 1995.

[08] Yizhou Yu and Jitendra Malik. Recovering photometric properties of architectural

scenes from photographs. In Proceedings of the SIGGRAPH 98 conference, Annual

Conference Series, pages 207–217, 1998.


Augmented Reality

Documents

Transcript of Augmented Reality