Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Learning With Augmented Reality

Thomas Vantroys, Benjamin Barbrythomas.vantroys@lifl.fr

Universite de Lille 1Laboratoire d’Informatique Fondamentale de Lille

Batiment M3 - Cite Scientifique59655 Villeneuve d’Ascq

7 july 2009

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

1 Introduction

2 Tools

3 Augmented Learning Environment

4 Conclusion

5 Bibliography

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Augmented Reality : first approach

Augmented Reality = real-world + computer generated data

Aims: bridging the gap between real-world and virtual world

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Augmented Reality: present or future ?

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

A practical example of augmented reality

The animals of the futurehttp://www.lesanimauxdufutur.com/

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Augmented Reality or Mixed Reality ?

From [MK94]

Augmented Reality

Adding information and meaning to a real object or place

Virtual Reality

Immersing a user inside a virtual world

Mixed Reality

Merging the real and virtual worlds to produce new environments

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Spectrum of Augmented Reality

”Classical” Augmented Reality

Tangible interface

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Spectrum of Augmented Reality

”Disappeared” interface

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Has Augmented Reality an interest for learning ?

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Has Augmented Reality an interest for learning ?

Is it only a toy ?

NO, it allows ”realistic” simulation or experimentation

http://digitable.imag.fr

à Raise the participation of the studentsà Facilitate the experimentations

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Has Augmented Reality an interest for learning ?

Is it only a toy ?

NO, it allows ”realistic” simulation or experimentation

http://digitable.imag.fr

à Raise the participation of the studentsà Facilitate the experimentations

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Has Augmented Reality an interest for learning ?

It allows to link different representation of the sameinformation

It offers a game approach of learning

[WB03]

à game approach ”have the potential to keep the user focusing forlonger period of time”

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Has Augmented Reality an interest for learning ?

Every object or place as an history and a context. Thatcontent can be available to individuals for a richer experience.

The instructor can add content to a place to permit a moreeasy comprehension by the learners

Information can also come from learners.

à raising comprehension and implication of the learners

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Has Augmented Reality an interest for learning ?

From [Mar07]

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Tools for Augmented Reality

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Elements of a pervasive system

A context-aware pervasive system have three basic functionalities[Lok07] :

1 Sensing

2 Thinking

3 Acting

An augmented reality system act the same way

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Sensing

Computer vision

Localisation

GPS, WiFi, RFID, QR-CODE, . . .

Classical sensors

ligth, motion, touch, microphone, biological signal, ...

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Acting

graphic overlay

sound

motors

LED

”mechanical” feedback (force, vibration, . . . )

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Rapid development of Augmented Reality

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

ARToolKit

ARToolKit is a software library forbuilding Augmented Realityapplications

http://www.hitl.washington.edu/artoolkit/

Use fiducial markers recognition

Overlay 3D virtual image

Need only a webcam and a paper withthe marker

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Phidgets

Phidgets = Physical Widget

Designed by the University ofCalgary (Saul Greenberg andChester Fitchett)

Aims: Hardware as Software

http://www.phidgets.com

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Example: Treasure hunt

Two games: Mastermind andTangram

Two technics: augmentedreality, tangible interface

One aim: stimulatecollaboration

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Middleware for AugmentedReality

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Why do we need middleware ?

In a pervasive system, concurrent tasks can be rendered onseparate devices in a heterogeneous environment:

supporting data exchangesupporting different protocolsupporting context managementmanaging devices and services (can be as subpart of context)

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

OSGI

previously : Open Services Gateway Initiative

Dynamical services platform, founded in 1999

Main actors :

home/building automation, telematics(car automation)Mobile phones (Nokia, Motorola)Eclipse about plugins for IDEApache for his servers

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

architecture

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Motivations

a shell over Java

Download/Upload dynamicaly programs

Use Java

Dynamic deployment of applications without stoping theplatform

LyfeCycle management : install, uninstall, stop, update...

Resolution of dependancies about versionned programs

SOA dynamic

Adapted for devices with few memory capacity

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

OSGi

An application is seen as a set of components called BundlesBundle :

functional unit : provide services

deployment unit : jar file

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Bundle life cycle

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Example

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

UPnP

Universal Plug and Play

Network protocol, from 1999

In 1999, main actors :

MicrosoftIBMIntelSiemens...

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Motivations

Provide ”invisible” network infrastructure for SOHO (SmallOffice Home Office)

network

Dynamic and transientno administrator

nodes

Self configurationSelf description

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

UPnP nodes

Equipment

provides serviceshas state variablesnotifies his state changement through its variables

Control point

use equipmentreact to state cangements from equipments

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

How it works

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

How it works

Adressing : DHCP, AutoIP

Discovery : SSDP

Description : XML

Control : SOAP

Eventing : GENA

Presentation : HTML

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

UPnP : SOA dynamic and shared

OSGi : SOA dynamic and centralized

Motivations

gatewaways developped with OSGi for different micro-world(with specific technology), one of them with UPnP support

Control points developped with UPnP

needs for specifying remote interactions between OSGibundles and devices or control points UPnP

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

UPnP and OSGi

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

bibliographie

UPnP Forum : http://www.upnp.org

OSGi service gateway specification:http://www.osgi.org/resources/spec-download.asp

Didier Donsez, Mise en oeuvre dUPnP avec OSGI

Michael Jeronimo, Jack Weast, ”UPnP Design by Example: ASoftware Developer’s Guide to Universal Plug and Play”,Pub.Intel Press, ISBN: 0971786119, May 2003

http://www.intel.com/intelpress/excerpts/upnp1.htm

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Examples of middlewares

CORBA

ReMMoC

AMIGO

FRACTAL

Aura

...

2 types:

reflexive middleware

protocol translation

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

ReMMoC

Reflective Middleware for Mobile Computing

reflexive middlewaredynamically configurableprovides :

binding type unitservice discovery unit

http://www.comp.lancs.ac.uk/computing/research/mpg/reflection/remmoc.php

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

AURA

Towards distraction free pervasive computing

http://www.cs.cmu.edu/ aura/

Integrated distributed resource management to create anenvironment that adapts to the users context and need

Integrated distributed resource management for enablingsmart spaces [JS03]

Task-based applications for deployment in the changing smartspaces

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

AMIGO

Ambient Intelligence for the networked home environment

http://www.hitech-projects.com/euprojects/amigo/

protocol translation

dynamically configurable

based on OSGi and UPnP

purpose : Feeling at home everywhere

Enabling seamless use of technology through adequate userinterfaces and system transparencyBringing relevant applications to people

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

AMIGO

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

WCOMP

platform based on components

based on

UPnP and DPWS : Device Profile for Web Serviceseventsservices orchestration

goals :

facilitate development of multi-devices applicationstake into account their evolution within their environment

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Ubiquarium

Ubiquarium : experimental environment

Simulation with virtual and real objects

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Example

Controling objects from a 3d scene.

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

p-LearNet project

Aims

1 Explore the potential offered by the ubiquitous computing toextend learning until within the work situation.

2 Experimenting learning in the work situation by building acontinuity between information and learning

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

p-LearNet project

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Examples of Augmented Reality

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Wikitude

Wikitude is an AR guide for Android platform

Based on location-based Wikipedia and Qype content

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Augmented Reality Language Learning

Wikitude is not designed for learning activies but it offers a basisfor Language Learning. www.avatarlanguages.com/blog/arll.Different activities are conceivable:

AR Geocaching (hidden containers are virtual)

Projects and fieldtrips (exploration and collect data)

+ ”classical” m-learning activities (microblogging, Wikiassociated with location, . . . )

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Interactive room:when m-learning go back toschool

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Augmented Room, five years ago

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

TinkerTable

The project aims to integratetheoric concept with concreteexperiments. [ZJD08]

Used by logistics managerslearners

http://dualt.epfl.ch/page62923-fr.html

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

TinkerLamp

”Mobile” version of theTinkerTable

For small group collaboration

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

CarettaKids

CarettaKids supportsface-to-face collaborativelearning by children withpersonnal and sharedspaces [Sug09].

Used for learning aboutenvironmental issues

http://www.itl.k.u-tokyo.ac.jp/~sugi/CarettaKids/

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

CarettaKids

Shared space is composed of a”Sensing Board”

Each physical device has anRFID Tag

Private space is manipulatedthrough a PDA

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Navigating the Human Body

Tangible interface to learnhuman body [DlHCM+03]

Designed for school children

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Interactive Furniture: Reflect table

aims: provide feedback of the participation during face-to-facecollaborative learning. [BKD08]

http://craft.epfl.ch/page66272-fr.html

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Conclusion

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Conclusion

Augmented Reality is a way to enhance learning:

encourage situated learningnatural interaction: manipulation of concrete physical objectsshared space for collaborative transactions

But:

don’t forget the learning !needs for more empirical studies

Toward ”augmented” places

learning occurs everywhereLocation can have ”depth”, i.e., each person can enrich it withinformation (like standard m-learning) but retrieving theinformation and manipulate them can be easierthe challenge is to convert places in order to support learning”Designing Spaces for Effective Learning” and beyond(seamless integration)

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Bibliography

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Khaled Bachour, Frederic Kaplan, and Pierre Dillenbourg.Reflect: An interactive table for regulating face-to-facecollaborative learning.In Times of Convergence. Technologies Across LearningContexts, pages 39–48, Berlin/Heidelberg, 2008. Springer.

Edward Deguzman, F. Wai ling Ho-Ching, Tara Matthews, TyeRattenbury, Maribeth Back, and Steve Harrison.Eewww!: Tangible instruments for navigating into the humanbody.In Extended Abstracts of CHI 2003, pages 806–807, 2003.http://taramatthews.org/projects/body.html.

Glenn Judd and Peter Steenkiste.Providing contextual information to pervasive computingapplications.In IEEE International Conference on Pervasive Computing(PERCOM), Dallas, 23-25 march 2003.http://www.cs.cmu.edu/~aura/docdir/Percom03.pdf.

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

Seng Loke.Context-Aware Pervasive Systems.Auerbach Publications, 2007.

Paul Marshall.Do tangible interfaces enhance learning?In TEI ’07: Proceedings of the 1st international conference onTangible and embedded interaction, pages 163–170, NewYork, NY, USA, 2007. ACM.

Paul Milgram and Fumio Kishino.A taxonomy of mixed reality visual displays.IEICE Transactions on Information Systems, E77-D(12),december 1994.http://vered.rose.utoronto.ca/people/paul_dir/IEICE94/ieice.html.

Masanori Sugimoto.Design of systems for supporting collaborative learningaugmented with physical artefacts.

Plan Introduction Tools Augmented Learning Environment Conclusion Bibliography

In Pierre Dillenbourg, Jeffrey Huang, and Mauro Cherubini,editors, Interactive Artifacts and Furniture SupportingCollaborative Work and Learning, pages 15–34. Springer, 2009.

Daniel Wagner and Istvan Barakonyi.Augmented reality kanji learning.In ISMAR03, 2003.

Guillaume Zufferey, Patrick Jermann, and Pierre Dillenbourg.A tabletop learning environment for logistics assistants:Activating teachers.In Proceedings of the Third IASTED International ConferenceHuman-Computer Interaction, pages 37–42. ACTA Press,2008.