Bridging the Real World and the Virtual

WorldA look at Augmented Reality and

Ubiquitous Computing

By Karthik KasturyDate 15th February, 2010Monday 5 July 2010

The Past 20 Years of Computing

Monday 5 July 2010

For How Long have we been computing

this way?...Paradigm Shift in Computing is here and

its Augmented Reality

Monday 5 July 2010

Augmented RealityWhats that?

Monday 5 July 2010

Watch this clip...From Iron Man the Movie

Monday 5 July 2010

Monday 5 July 2010

Yes, that’s Augmented Reality

And don’t worry its not only in movies, but real life as well...

Monday 5 July 2010

Augmented   reality   (AR)   is   a   term   for   a  live   direct   or   indirect   view   of   a   physical  real-­‐world  environment  whose  elements  are   merged   with   (or   augmented   by)  virtual   computer-­‐generated   imagery   -­‐  crea<ng  a  mixed  reality.

Monday 5 July 2010

Augmented Reality Techniques

• Heads Mounted Display

• Handheld Devices

• Spatial Displays

• Modern Software Based AR

Monday 5 July 2010

Head Mounted Display• User can simultaneously view both

the real world and the virtual world.

• Optical See through or Video See Through.

• Can be a bit bulky in its current form.

Monday 5 July 2010

Handheld Devices• Most commonly used AR Devices

• Uses a variety of Sensors

• ARToolkit is usually used to map the real world.

Monday 5 July 2010

Spatial Devices

• User does not carry any device with him.

• The data is projected onto real world entities.

• Multiple Users can interact with the system.

• Can support large areas of displays.

Monday 5 July 2010

Software Based AR

• Use multiple sensors and tracking devices, like GPS, RFID, Compasses etc...

• Are leading the innovation in AR now.

• Two Stages, Detection of Interest Points, Mapping the two worlds.

Monday 5 July 2010

Working Principle

• First interest points, or fiduciary markers, or optical flow detected in the camera images [Corner Detection].

• Second, a real world coordinate system is restored from the data obtained in the first stage[SLAM].

Monday 5 July 2010

Corner Detection

• Used to identify the interest points.

• Can use Correlation Algorithms, but they tend to be very intensive computationally.

• So we use Harris and Stephens Corner Detection algorithm

Monday 5 July 2010

Harris and Stephens Corner Detection

AlgorithmAssume a grayscale 2-dimensional image is used. Let this image be given by I. Consider taking an image patch over the area (u,v) and shifting it by (x,y). The weighted sum of squared differences (SSD) between these two patches, denoted S, is given by:

I(u + x,v + y) can be approximated by a Taylor expansion. Let Ix and Iy be the partial derivatives of I, such that

This produces the approximation

Monday 5 July 2010

contd...which can be written in matrix form

where A is the structure tensor,

This matrix is a Harris matrix, and angle brackets denote averaging (i.e. summation over (u,v)). By analyzing the eigenvalues of A, this characterization can be expressed in the following way: If and then this pixel (x,y) has no features of interest.

If and λ2 has some large positive value, then an edge is found.If λ1 and λ2 have large positive values, then a corner is found.

Monday 5 July 2010

Simultaneous Localization and Mapping

• SLAM is a technique to build up a map within an unknown environment (without a priori knowledge) or to update a map within a known environment (with a priori knowledge from a given map) while at the same time keeping track of the current location.

• Maps are used to determine a location in an environment and to depict this environment for an overview.

• Simultaneous localization and mapping (SLAM) is a concept to bind the process of locating and mapping in a loop and therefore supports the continuity of both aspects in separated processes.

Monday 5 July 2010

So this how AR Systems work...

Now let us have a look at a Software Based AR Tool for the iPhone/Android

Monday 5 July 2010

Monday 5 July 2010

Other Applications of AR

• Support with complex tasks

• Navigation device

• Military and emergency services

• Sightseeing

• and  many  more...

Monday 5 July 2010

Ubiquitous Computing

• Ubiquitous computing (ubicomp) is a post desktop model of human-computer interaction in which information processing has been thoroughly integrated into everyday objects and activities.

• This paradigm is also described as pervasive computing, ambient intelligence. When primarily concerning the objects involved, it is also physical computing, the Internet of Things, haptic computing, and things that think.

Monday 5 July 2010

Challenges of Ubiquitous Computing

!"#$%!"%&'%#$%%

4. Ubiquitous Computing and its Challenges

%()*+,*-&,.% /&01,-*2#% 3,)*/&014% *.% "% 1&.-56$.7-&1% 0&6$8% &'% 9,0"25/&01,-$:%*2-$:"/-*&2%*2%;9*/9%*2'&:0"-*&2%1:&/$..*2#%9".%)$$2%-9&:&,#98<%*2-$#:"-$6%*2-&%$=$:<6"<% &)>$/-.% "26% "/-*=*-*$.?% @2% -9$% /&,:.$% &'% &:6*2":<% "/-*=*-*$.A% .&0$&2$%B,.*2#B%,)*+,*-&,.%/&01,-*2#%$2#"#$.%0"2<%/&01,-"-*&2"8%6$=*/$.%"26%.<.-$0.%.*0,8-"2$&,.8<A%"26%0"<%2&-%2$/$..":*8<%$=$2%)$%";":$%-9"-% -9$<%":$%6&*2#%.&?%C9*.%0&6$8%*.%,.,"88<%/&2.*6$:$6%"6="2/$0$2-%':&0%-9$%6$.7-&1%1":"6*#0?%%C9*.% 1":"6*#0% *.% "8.&% 6$./:*)$6% ".% 1$:=".*=$% /&01,-*2#A% "0)*$2-% *2-$88*#$2/$?%D9$2%1:*0":*8<%/&2/$:2*2#%-9$%&)>$/-.%*2=&8=$6A%*-%*.%"8.&%19<.*/"8%/&01,-*2#A%-9$%@2-$:2$-%&'%C9*2#.A%9"1-*/%/&01,-*2#A%"26%-9*2#.%-9"-%-9*27?%E"-9$:%-9"2%1:&1&.$%"% .*2#8$% 6$'*2*-*&2% '&:% ,)*+,*-&,.% /&01,-*2#% "26% '&:% -9$.$% :$8"-$6% -$:0.A%-"F&2&0<% &'% 1:&1$:-*$.% '&:% ,)*+,*-&,.% /&01,-*2#% 9".% )$$2% 1:&1&.$6A% ':&0%;9*/9% 6*''$:$2-% 7*26.% &:% '8"=&:.% &'% ,)*+,*-&,.% .<.-$0.% "26% "118*/"-*&2.% /"2% )$%6$./:*)$6?%%

%%G-% -9$*:% /&:$A% "88% 0&6$8.% &'% ,)*+,*-&,.% /&01,-*2#% 3"8.&% /"88$6% 1$:=".*=$%/&01,-*2#4% .9":$% "% =*.*&2% &'% .0"88A% *2$F1$2.*=$A% :&),.-% 2$-;&:7$6% 1:&/$..*2#%6$=*/$.A%6*.-:*),-$6%"-%"88%./"8$.%-9:&,#9&,-%$=$:<6"<%8*'$%"26%#$2$:"88<%-,:2$6%-&%6*.-*2/-8<% /&00&2518"/$% $26.?% H&:% $F"018$A% "% 6&0$.-*/% ,)*+,*-&,.% /&01,-*2#%$2=*:&20$2-% 0*#9-% *2-$:/&22$/-% 8*#9-*2#% "26% $2=*:&20$2-"8% /&2-:&8.% ;*-9%

Monday 5 July 2010

Redefining the Vision of Future Computing

!"#$%!"%&'%#$%%

5. Redefining the Vision of Future Computing

()&*%+,$%&)-#-."/%01.2,%2")34%".3%0)-.+&1+4%+&%*&.-+&)45%*&14$45%".3%6$78&")345%"//%+,$%9"7%+&%+,$%+)"26%0"35%:&-2$%)$2&#.-+-&.5%".3%-.+$)'"2$4%3$4-#.$3%+&%*"6$%-+%$"4-$)%'&)%+,$%3-4"8/$3%+&%14$%2&*01+$)45%2&*01+-.#%-.+$)'"2$4%,":$%0)&#)$44$3%)"0-3/7%9-+,-.%+,$%/"4+%'$9%3$2"3$4;%%<1+%+,-4%-4%.&+%+,$%$.3%&'%+,$%)&"35%+,$%(1+1)$%&'%=&*01+-.#%/-$4%-.%+,$%*$)#$)%&'%+$2,.&/&#-$4% /-6$%>1/+-?&12,5%@$4+1)$45%A&-2$%=&.+)&/5%B1#*$.+$3%C$"/-+75%<)"-.%=&*01+-.#%".3%,&4+%&'%&+,$)%)$4$")2,%")$"4%-.%.$:$)%8$'&)$%4$$.%9"74;%%B+% +,$%=DE%FGHG% I=&.41*$)%D/$2+)&.-24%E,&95%J"4%A$#"4K% 2&*0".-$4%1.:$-/$3%"%,&4+%&'% +$2,.&/&#-$4% +,"+%9-//% '&)$:$)%2,".#$%+,$% /".342"0$%&'%2&*01+-.#;%L$)$%")$%4&*$%&'%+,$%+$2,.&/&#-$4%+,"+%9-//%8$%2&**&.0/"2$%-.%&1)%/-:$4%M14+% /-6$%+,$%+)"3-+-&."/%3$46+&0%-.+$)'"2$4%,":$%8$$.%'&)%+,$%0"4+%FG%7$")4%&)%4&;%%%&'()*'&+(,-./01-12)*3'4-

%%5678+(,+9-:+'32,4-2(-'-;<(,'=,-/+()-

%

!"#$%!"%&'%#$%%

5. Redefining the Vision of Future Computing

()&*%+,$%&)-#-."/%01.2,%2")34%".3%0)-.+&1+4%+&%*&.-+&)45%*&14$45%".3%6$78&")345%"//%+,$%9"7%+&%+,$%+)"26%0"35%:&-2$%)$2&#.-+-&.5%".3%-.+$)'"2$4%3$4-#.$3%+&%*"6$%-+%$"4-$)%'&)%+,$%3-4"8/$3%+&%14$%2&*01+$)45%2&*01+-.#%-.+$)'"2$4%,":$%0)&#)$44$3%)"0-3/7%9-+,-.%+,$%/"4+%'$9%3$2"3$4;%%<1+%+,-4%-4%.&+%+,$%$.3%&'%+,$%)&"35%+,$%(1+1)$%&'%=&*01+-.#%/-$4%-.%+,$%*$)#$)%&'%+$2,.&/&#-$4% /-6$%>1/+-?&12,5%@$4+1)$45%A&-2$%=&.+)&/5%B1#*$.+$3%C$"/-+75%<)"-.%=&*01+-.#%".3%,&4+%&'%&+,$)%)$4$")2,%")$"4%-.%.$:$)%8$'&)$%4$$.%9"74;%%B+% +,$%=DE%FGHG% I=&.41*$)%D/$2+)&.-24%E,&95%J"4%A$#"4K% 2&*0".-$4%1.:$-/$3%"%,&4+%&'% +$2,.&/&#-$4% +,"+%9-//% '&)$:$)%2,".#$%+,$% /".342"0$%&'%2&*01+-.#;%L$)$%")$%4&*$%&'%+,$%+$2,.&/&#-$4%+,"+%9-//%8$%2&**&.0/"2$%-.%&1)%/-:$4%M14+% /-6$%+,$%+)"3-+-&."/%3$46+&0%-.+$)'"2$4%,":$%8$$.%'&)%+,$%0"4+%FG%7$")4%&)%4&;%%%&'()*'&+(,-./01-12)*3'4-

%%5678+(,+9-:+'32,4-2(-'-;<(,'=,-/+()-

%

!"#$%!"%&'%#$%%

%&'()'*+,-&&,

,

./+0,12+3)4(,%5(56,

,

7&42+8+9(,:;2954),<+=>;(&'?,

,

!"#$%!"%&'%#$%%

%&'()'*+,-&&,

,

./+0,12+3)4(,%5(56,

,

7&42+8+9(,:;2954),<+=>;(&'?,

,

Transparent OLED AR In a Contact Lens

XBox Project NatalMicrosoft Surface Computing

Monday 5 July 2010

Office Labs Vision 2019

Monday 5 July 2010

Final Words...

• The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it.

• Augmented Reality and various principles of Ubiquitous Computing will drive innovation in the future and take computing to an extent that our future generations will no longer be able to relate themselves to the initial forms of computing.

Monday 5 July 2010

References

• Azuma, R. A Survey of Augmented Reality, Presence, 6, 4, pp.355-385, 1997

• "If You’re Not Seeing Data, You’re Not Seeing". Wired Magazine

• Hand of God - Wearable Computer Lab, South Australia

• Mark Weiser - Ubiquitous Computing, during his term at XEROX-SPARC

• Pervasive Computing – M Satyanarayan, School of Computer Science, Carnegie Mellon University

• Collaborative Augmented Reality - Mark Billinghurst, Hirokazu Kato

Monday 5 July 2010

Thank YouDo you have any questions?

Monday 5 July 2010