Augmented Reality With FlarToolkit and Papervision3D
-
Upload
roman-protsyk -
Category
Technology
-
view
132 -
download
1
description
Transcript of Augmented Reality With FlarToolkit and Papervision3D
![Page 1: Augmented Reality With FlarToolkit and Papervision3D](https://reader033.fdocuments.net/reader033/viewer/2022061300/54c8c8684a79591b0f8b45f9/html5/thumbnails/1.jpg)
Augmented Reality with Flex,
FlarToolkit and Papervision3D
Roman Protsyk20.12.2010
![Page 2: Augmented Reality With FlarToolkit and Papervision3D](https://reader033.fdocuments.net/reader033/viewer/2022061300/54c8c8684a79591b0f8b45f9/html5/thumbnails/2.jpg)
Agenda
• Augmented reality definition and examples
• Augmented Reality in web browser with FlarToolkit
• Augmented reality demo creation using Flex, FlarToolkit and Papervision3D demo
![Page 3: Augmented Reality With FlarToolkit and Papervision3D](https://reader033.fdocuments.net/reader033/viewer/2022061300/54c8c8684a79591b0f8b45f9/html5/thumbnails/3.jpg)
What is Augmented Reality? Augmented Reality (AR) term referring to all the projects to complement the reality of any virtual
elements
The term was proposed by Tom Caudell, a researcher of Boeing Corporation,in 1990 . Ronald Azuma in 1997 defined the augmented reality as a system that 1. Combines the virtual and real. 2. Interact in real-time3. Works in 3D.
![Page 4: Augmented Reality With FlarToolkit and Papervision3D](https://reader033.fdocuments.net/reader033/viewer/2022061300/54c8c8684a79591b0f8b45f9/html5/thumbnails/4.jpg)
Nintendo – Wii
Augmented Reality Examples : Game Consoles
Sony PlayStation 3 – Controller + Eye Cam
Sony PlayStation 3 – Eye Pet
XBox – Project Natal
![Page 5: Augmented Reality With FlarToolkit and Papervision3D](https://reader033.fdocuments.net/reader033/viewer/2022061300/54c8c8684a79591b0f8b45f9/html5/thumbnails/5.jpg)
1. Mobilizy – Wikitude2. Nokia – Conceptwith Image Comparison
3. Whiteapp – AR iPhone apps
Augmented Reality examples: Location Based Mobile Apps
![Page 6: Augmented Reality With FlarToolkit and Papervision3D](https://reader033.fdocuments.net/reader033/viewer/2022061300/54c8c8684a79591b0f8b45f9/html5/thumbnails/6.jpg)
Augmented Reality examplesThere are AR applications outside of console
games, including military and consumer products, too. Night-vision goggles and targeting-assistance technology help marksmen in battle, and children's stories come to life with AR-enhanced books. The uses are vast.
![Page 7: Augmented Reality With FlarToolkit and Papervision3D](https://reader033.fdocuments.net/reader033/viewer/2022061300/54c8c8684a79591b0f8b45f9/html5/thumbnails/7.jpg)
Augmented reality in web browser
• The world’s first Flash based augmented reality library ported from NyARToolkit (Java ported version of ARToolKit) by Tomohiko Koyama aka Saqoosha.
• First version was released in May 2008.• Porting was completed in about 1 week.• FLAR is a part of Spark project -
Flash/ActionScript open source community. http://www.libspark.org/wiki/saqoosha/FLARToolKit/en
![Page 8: Augmented Reality With FlarToolkit and Papervision3D](https://reader033.fdocuments.net/reader033/viewer/2022061300/54c8c8684a79591b0f8b45f9/html5/thumbnails/8.jpg)
FlarToolkit examplehttp://ge.ecomagination.com/smartgrid/?c_id=Matter#/augmented_reality
![Page 10: Augmented Reality With FlarToolkit and Papervision3D](https://reader033.fdocuments.net/reader033/viewer/2022061300/54c8c8684a79591b0f8b45f9/html5/thumbnails/10.jpg)
Basic usage• FLARToolKit doesn’t render the 3D objects.• Calculates only the orientation of the marker.• Use other 3D engines to render 3D objects.
- Papervision3D
- Away3D (both 2.x and 3.x)
- Sandy
- Alternativa3D• Start-up guide for FLARToolKit.
http://saqoosha.net/en/flartoolkit/start-up-guide/
![Page 11: Augmented Reality With FlarToolkit and Papervision3D](https://reader033.fdocuments.net/reader033/viewer/2022061300/54c8c8684a79591b0f8b45f9/html5/thumbnails/11.jpg)
Demo
Let’s create demo application and look inside FLARToolkit in parallel
![Page 12: Augmented Reality With FlarToolkit and Papervision3D](https://reader033.fdocuments.net/reader033/viewer/2022061300/54c8c8684a79591b0f8b45f9/html5/thumbnails/12.jpg)
Creation of sample application
• Create marker image “pattern16.pat”:• http://flash.tarotaro.org/ar/MarkerGeneratorOnli
ne.swf
• Open Flex Builder• Create sample ActionScript “FlarDemo” project• Download FlarToolkit:
http://www.libspark.org/svn/as3/FLARToolKit/ • Download Papervision3d:
http://code.google.com/p/papervision3d/downloads/list
![Page 13: Augmented Reality With FlarToolkit and Papervision3D](https://reader033.fdocuments.net/reader033/viewer/2022061300/54c8c8684a79591b0f8b45f9/html5/thumbnails/13.jpg)
Inside FLARToolKit
Marker pattern specification: Needs to be square. Only 50% of center area is used in matching process. Marker pattern is 16 x 16 dots bitmap by default. Size of the pattern can be lager, but it requires more recognition time.
![Page 14: Augmented Reality With FlarToolkit and Papervision3D](https://reader033.fdocuments.net/reader033/viewer/2022061300/54c8c8684a79591b0f8b45f9/html5/thumbnails/14.jpg)
Creation of sample application
• Get “camera_para.dat” file which contains default properties for camera from: http://www.libspark.org/svn/as3/ARToolKitMarkerGenerator/trunk/
• Camera calibration - is the problem of obtaining internal and external camera parameters from the available photographs or video filmed with it
![Page 15: Augmented Reality With FlarToolkit and Papervision3D](https://reader033.fdocuments.net/reader033/viewer/2022061300/54c8c8684a79591b0f8b45f9/html5/thumbnails/15.jpg)
Inside FLARToolKit• Capture the image from webcam. (Step 1 of 7)
– Simply use Camera and Video classes.– then draw video instance onto the BitmapData.
![Page 16: Augmented Reality With FlarToolkit and Papervision3D](https://reader033.fdocuments.net/reader033/viewer/2022061300/54c8c8684a79591b0f8b45f9/html5/thumbnails/16.jpg)
Inside FLARToolKit• Binarize input image. (Step 2 of 7)
![Page 17: Augmented Reality With FlarToolkit and Papervision3D](https://reader033.fdocuments.net/reader033/viewer/2022061300/54c8c8684a79591b0f8b45f9/html5/thumbnails/17.jpg)
Inside FLARToolKit
• Labeling. (Step 3 of 7)
![Page 18: Augmented Reality With FlarToolkit and Papervision3D](https://reader033.fdocuments.net/reader033/viewer/2022061300/54c8c8684a79591b0f8b45f9/html5/thumbnails/18.jpg)
Inside FLARToolKit• Find squares. (Step 4 of 7)
![Page 19: Augmented Reality With FlarToolkit and Papervision3D](https://reader033.fdocuments.net/reader033/viewer/2022061300/54c8c8684a79591b0f8b45f9/html5/thumbnails/19.jpg)
Inside FLARToolKit
• Matching with patterns.
(Step 5 of 7)Extract the images from detected rectangles using homography transform.
![Page 20: Augmented Reality With FlarToolkit and Papervision3D](https://reader033.fdocuments.net/reader033/viewer/2022061300/54c8c8684a79591b0f8b45f9/html5/thumbnails/20.jpg)
Inside FLARToolKit• Render the 3D objects. (Step 6 of 7)
- FLARToolKit includes support classes which converts FLAR’s transform matrix to each 3D engine internal matrix classes.
![Page 21: Augmented Reality With FlarToolkit and Papervision3D](https://reader033.fdocuments.net/reader033/viewer/2022061300/54c8c8684a79591b0f8b45f9/html5/thumbnails/21.jpg)
Inside FLARToolKit• Calculate transform matrix. (Step 7 of 7)
- It’s calculated by square’s vertices of the detected marker.
![Page 22: Augmented Reality With FlarToolkit and Papervision3D](https://reader033.fdocuments.net/reader033/viewer/2022061300/54c8c8684a79591b0f8b45f9/html5/thumbnails/22.jpg)
Inside FLARToolkit
![Page 23: Augmented Reality With FlarToolkit and Papervision3D](https://reader033.fdocuments.net/reader033/viewer/2022061300/54c8c8684a79591b0f8b45f9/html5/thumbnails/23.jpg)
Demo applicationpublic function FlarDemo(){setupFlar();setupCamera();setupBitmapData();setupPV3D();addEventListener(Event.ENTER_FRAME, loop);}
private function setupFlar():void{fparams=new FLARParam();fparams.loadARParam(new params() as ByteArray);mpattern=new FLARCode(16, 16);mpattern.loadARPatt(new pattern());}
private function setupCamera():void{vid=new Video(640, 480);cam=Camera.getCamera();cam.setMode(640, 480, 30);vid.attachCamera(cam);addChild(vid);}
private function setupBitmapData():void{bmd = new BitmapData(640, 480);bmd.draw(vid);raster = new FLARRgbRaster_BitmapData(bmd);detector = new FLARSingleMarkerDetector(fparams,mpattern, 80);}
![Page 24: Augmented Reality With FlarToolkit and Papervision3D](https://reader033.fdocuments.net/reader033/viewer/2022061300/54c8c8684a79591b0f8b45f9/html5/thumbnails/24.jpg)
Demo applicationprivate function setupPV3D():void{scene = new Scene3D();camera = new FLARCamera3D(fparams);container = new FLARBaseNode();scene.addChild(container);
var pl:PointLight3D = new PointLight3D();pl.x = 1000;pl.y = 1000;pl.z = -1000;
var ml:MaterialsList = new MaterialsList({all: new FlatShadeMaterial(pl)});
var cube1:Cube = new Cube(ml,30,30,30);var cube2:Cube = new Cube(ml,30,30,30);cube2.z = 50;var cube3:Cube = new Cube(ml,30,30,30);cube3.z = 100;
container.addChild(cube1);container.addChild(cube2);container.addChild(cube3);
bre = new BasicRenderEngine();trans = new FLARTransMatResult();
vp = new Viewport3D();addChild(vp);}private function loop(e:Event):void{bmd.draw(vid);try{if(detector.detectMarkerLite(raster, 80) && detector.getConfidence()>0.5){vp.visible = true;detector.getTransformMatrix(trans);container.setTransformMatrix(trans);bre.renderScene(scene, camera, vp);}else{vp.visible = false}}catch(e:Error){}}
http://protsyk.org.ua/flartoolkitdemo/FlarDemo.zip
![Page 25: Augmented Reality With FlarToolkit and Papervision3D](https://reader033.fdocuments.net/reader033/viewer/2022061300/54c8c8684a79591b0f8b45f9/html5/thumbnails/25.jpg)
Demo applicationLoading of different Papervision3D objects
http://protsyk.org.ua/flartoolkitdemo/Flar3DEarth.zip
![Page 26: Augmented Reality With FlarToolkit and Papervision3D](https://reader033.fdocuments.net/reader033/viewer/2022061300/54c8c8684a79591b0f8b45f9/html5/thumbnails/26.jpg)
Future of FLARToolKit
![Page 27: Augmented Reality With FlarToolkit and Papervision3D](https://reader033.fdocuments.net/reader033/viewer/2022061300/54c8c8684a79591b0f8b45f9/html5/thumbnails/27.jpg)
Alchemy version of FLARToolKit
• Done by Ryo Iizuka (aka nyatla)
- 2 times faster than AS3 version.
- You can use more polygons to create 3D models.
- It runs on more slower PCs.
- Committed at alchemy branch.
http://www.libspark.org/svn/as3/FLARToolKit/branches/alchemy/
![Page 28: Augmented Reality With FlarToolkit and Papervision3D](https://reader033.fdocuments.net/reader033/viewer/2022061300/54c8c8684a79591b0f8b45f9/html5/thumbnails/28.jpg)
Links• FLARToolkit project web page by Saqoosha
http://www.libspark.org/wiki/saqoosha/FLARToolKit/en
• Google group: FLARToolKit-userz.
http://groups.google.com/group/flartoolkit-userz • FlarToolkit/Flash Augmented Reality Getting
Started by Mikko Haapoja
http://www.mikkoh.com/blog/2008/12/flartoolkitflash-augmented-realitygetting-started/