Philipp Nagele (Wikitude) Wikitude SDK Tutorial
-
Upload
augmentedworldexpo -
Category
Technology
-
view
254 -
download
6
Transcript of Philipp Nagele (Wikitude) Wikitude SDK Tutorial
![Page 1: Philipp Nagele (Wikitude) Wikitude SDK Tutorial](https://reader033.fdocuments.net/reader033/viewer/2022042723/587fc4ea1a28ab3b158b5cd1/html5/thumbnails/1.jpg)
Wikitude SDK Advanced
![Page 2: Philipp Nagele (Wikitude) Wikitude SDK Tutorial](https://reader033.fdocuments.net/reader033/viewer/2022042723/587fc4ea1a28ab3b158b5cd1/html5/thumbnails/2.jpg)
Agenda for today
● Who is Wikitude
● What has the Wikitude SDK to offer
-Architecture and APIs
-Plugins API
● Latest Additions to the Wikitude SDK – 3D Tracking
![Page 3: Philipp Nagele (Wikitude) Wikitude SDK Tutorial](https://reader033.fdocuments.net/reader033/viewer/2022042723/587fc4ea1a28ab3b158b5cd1/html5/thumbnails/3.jpg)
3
![Page 4: Philipp Nagele (Wikitude) Wikitude SDK Tutorial](https://reader033.fdocuments.net/reader033/viewer/2022042723/587fc4ea1a28ab3b158b5cd1/html5/thumbnails/4.jpg)
v v v
Wikitude AR at a glance.
First AR app in the world in
2008
10,000 apps powered by
wikitude SDK
3D tracking technology
vREFERENCES
![Page 5: Philipp Nagele (Wikitude) Wikitude SDK Tutorial](https://reader033.fdocuments.net/reader033/viewer/2022042723/587fc4ea1a28ab3b158b5cd1/html5/thumbnails/5.jpg)
Wikitude’s path to success
20142013 2015
Business Model: B2CFocus: Wikitude AR App
2016201220112009
Founder stage in Salzburg garage:
First AR Browser launched.
2010
VCs brought in., Management hired:Martin Herdina, Andy Gstoll
Established Wikitude Inc. in USA
Business Model: B2BFocus: AR technology and tools
Achieved 25 million downloads and app installs gloabally
Launch of Wikitude SDK for 2D image recognition
Launch of Wikitude Studio, the AR CMS
Launch of Wikitude Cloud Recognition service
Launch of 3D tracking SLAM
![Page 6: Philipp Nagele (Wikitude) Wikitude SDK Tutorial](https://reader033.fdocuments.net/reader033/viewer/2022042723/587fc4ea1a28ab3b158b5cd1/html5/thumbnails/6.jpg)
Wikitude offices
wikitude GmbH (HQ) in Salzburg Austriawikitude Inc. in San Francisco, USA
wikitude Inc.
wikitude GmbH.
![Page 7: Philipp Nagele (Wikitude) Wikitude SDK Tutorial](https://reader033.fdocuments.net/reader033/viewer/2022042723/587fc4ea1a28ab3b158b5cd1/html5/thumbnails/7.jpg)
AR creator and CMSCloud Recognition
Wikitude AR Products & Tools
Connect your own CMS
![Page 8: Philipp Nagele (Wikitude) Wikitude SDK Tutorial](https://reader033.fdocuments.net/reader033/viewer/2022042723/587fc4ea1a28ab3b158b5cd1/html5/thumbnails/8.jpg)
Native API JavaScript API
Easy access to AR mobile development
Xamarin
PhoneGapCordova
titaniumunity
Android
iOS
![Page 9: Philipp Nagele (Wikitude) Wikitude SDK Tutorial](https://reader033.fdocuments.net/reader033/viewer/2022042723/587fc4ea1a28ab3b158b5cd1/html5/thumbnails/9.jpg)
GEO / sensor based AR
Pioneering products and technologies
![Page 10: Philipp Nagele (Wikitude) Wikitude SDK Tutorial](https://reader033.fdocuments.net/reader033/viewer/2022042723/587fc4ea1a28ab3b158b5cd1/html5/thumbnails/10.jpg)
Maturing products and technologies
2D Image Recognition & Tracking
![Page 11: Philipp Nagele (Wikitude) Wikitude SDK Tutorial](https://reader033.fdocuments.net/reader033/viewer/2022042723/587fc4ea1a28ab3b158b5cd1/html5/thumbnails/11.jpg)
Tailored SDKs for Smart Glasses
Partnership since 2014, optimized SDK for BT-200, full documentation →BT-300 in pipeline
Partnership since 2016 optimized SDK for R-7
Partnership since 2014, SDK for M100, full documentation →M300 in pipeline
![Page 12: Philipp Nagele (Wikitude) Wikitude SDK Tutorial](https://reader033.fdocuments.net/reader033/viewer/2022042723/587fc4ea1a28ab3b158b5cd1/html5/thumbnails/12.jpg)
Wikitude SDK Architecture
2D On-DeviceReco & Tracking
2D Cloud Reco3D Reco & Tracking
Unity3D
Unity Plugin
Plugins API
Custom CV plugin
Custom camera plugins
Your App
Your App Cordova
PluginTitanium Module
XamarinComponent
Your App
Cordova Titanium Xamarin
Your App
Your App
JavaScript APINative API
Your App
3D Rendering
Augmen-tations LBS
Plugin ManagerOpenGL ES2 Rendering
ARMv7 NEON optimizations
ARMv8 optimizationsCamera GPU
optimizations
OpenCL
RenderScript
Metal API
IMU
Hardware Abstraction Hardware Optimizations
C++ Layer
![Page 13: Philipp Nagele (Wikitude) Wikitude SDK Tutorial](https://reader033.fdocuments.net/reader033/viewer/2022042723/587fc4ea1a28ab3b158b5cd1/html5/thumbnails/13.jpg)
Plugins API - Modularization
13
Description
The Plugins APIs allows to extend the Wikitude SDK by 3rd party functionality. Plugins for the Wikitude SDK have access to the camera information, recognized images. This is perfect for additional functionality that also requires camera access. Plugins are written in C++ and can communicate both with the JavaScript API and the Native API.
![Page 14: Philipp Nagele (Wikitude) Wikitude SDK Tutorial](https://reader033.fdocuments.net/reader033/viewer/2022042723/587fc4ea1a28ab3b158b5cd1/html5/thumbnails/14.jpg)
Plugins API in detail
14
Custom Frame
Provider
Image Frame
Standard Frame
Provider
WikitudeCV EngineRendering
Custom CV Plugin
Plugins API
class Plugin { public:…void cameraFrameAvailable(const Frame&;
cameraFrame_); void update(const vector<RecognizedTarget>
recognizedTargets_); void startRender(); void endRender…
};
1234567
class RecognizedTarget {public:
const string& getIdentifier() const;const Mat4& getModelViewMatrix() const;const Mat4& getProjectionMatrix() const;const float getDistanceToCamera() const;
};
![Page 15: Philipp Nagele (Wikitude) Wikitude SDK Tutorial](https://reader033.fdocuments.net/reader033/viewer/2022042723/587fc4ea1a28ab3b158b5cd1/html5/thumbnails/15.jpg)
Use-Cases for Plugins API
● Run own computer-vision related code in parallel to Wikitude SDK
-OCR Engine
-Barcode Reader
-Face Recognition
●Own the camera stream and the camera rendering
-Custom effects and shaders
-Remote camera stream with local processing
![Page 16: Philipp Nagele (Wikitude) Wikitude SDK Tutorial](https://reader033.fdocuments.net/reader033/viewer/2022042723/587fc4ea1a28ab3b158b5cd1/html5/thumbnails/16.jpg)
JavaScript Augmented Reality Experience
16
Architect Worlds consist of:
• .html file(s)
• .js file(s)
• .css file(s)
• Defines your augmented reality experience
• Use the Wikitude JS API provided within the Architect view context<script src=“architect://architect.js"></script>
• Loaded by the Wikitude SDKs Architect view
![Page 17: Philipp Nagele (Wikitude) Wikitude SDK Tutorial](https://reader033.fdocuments.net/reader033/viewer/2022042723/587fc4ea1a28ab3b158b5cd1/html5/thumbnails/17.jpg)
Architect world
17
Javascript execution triggers actions in the underlying C++ layer
![Page 18: Philipp Nagele (Wikitude) Wikitude SDK Tutorial](https://reader033.fdocuments.net/reader033/viewer/2022042723/587fc4ea1a28ab3b158b5cd1/html5/thumbnails/18.jpg)
Architect view composition
18
OpenGL ES 2 view* Renders camera stream* Renders augmentations
Web view* Loads main .html file* Transparent background
![Page 19: Philipp Nagele (Wikitude) Wikitude SDK Tutorial](https://reader033.fdocuments.net/reader033/viewer/2022042723/587fc4ea1a28ab3b158b5cd1/html5/thumbnails/19.jpg)
GeoObjects & HTML content
Menu barjQuery support
Geo-located POIs
Multiple Drawables- 3 images- 3 text labels
Web View
Augmented Reality View
![Page 20: Philipp Nagele (Wikitude) Wikitude SDK Tutorial](https://reader033.fdocuments.net/reader033/viewer/2022042723/587fc4ea1a28ab3b158b5cd1/html5/thumbnails/20.jpg)
Support for jQuery and web APIs
![Page 21: Philipp Nagele (Wikitude) Wikitude SDK Tutorial](https://reader033.fdocuments.net/reader033/viewer/2022042723/587fc4ea1a28ab3b158b5cd1/html5/thumbnails/21.jpg)
Image Recognition
Augment target images using JavaScript API capabilities (Drawables, Sounds, Animations,Videos, 3D Models)
var kitchenTracker = new AR.Tracker("Kitchen.wtc");var overlay = new AR.VideoDrawable("howdens.mp4", 0.65);var trackable2DObject = new AR.Trackable2DObject(
kitchenTracker, "Kitchen",{ drawables:
{ cam: overlay}});
![Page 22: Philipp Nagele (Wikitude) Wikitude SDK Tutorial](https://reader033.fdocuments.net/reader033/viewer/2022042723/587fc4ea1a28ab3b158b5cd1/html5/thumbnails/22.jpg)
Types of augmentations
22
Augmentations are represented by AR.Drawable subclasses
• Images - AR.ImageResource / AR.ImageDrawable
• Videos (also with alpha channel) - AR.VideoDrawable
• 3D models - AR.Model
• Web views - AR.HTMLDrawable
• Labels - AR.Label
![Page 23: Philipp Nagele (Wikitude) Wikitude SDK Tutorial](https://reader033.fdocuments.net/reader033/viewer/2022042723/587fc4ea1a28ab3b158b5cd1/html5/thumbnails/23.jpg)
….architecture, real estate….. ….gaming, filming industry….. ….indoor navigation, retail…..
New algorithms and concepts
3D Tracking / SLAMObject Reco & Tracking
![Page 24: Philipp Nagele (Wikitude) Wikitude SDK Tutorial](https://reader033.fdocuments.net/reader033/viewer/2022042723/587fc4ea1a28ab3b158b5cd1/html5/thumbnails/24.jpg)
Recognition & tracking (core engine)
3D
One core engine with many “flavors”
Small Scale Scenes / Spaces / Objects
- SLAM- Point cloud- Smaller scenes- desktop, small rooms and
environments
Object Recognition Edge Detection / CAD models- pre-registered CAD
models and displayed silhouettes
- Focus: shapes & lines
Positional Tracking Indoor Navigation”
- Computer vision plus IMU / sensor fusion
- Malls, airports, buildings- Untethered indoor gaming in
apartment, house or anywhere
Large Scale Scenes Spaces
- SLAM based- Point cloud- buildings / spaces- indoors, outdoors
![Page 25: Philipp Nagele (Wikitude) Wikitude SDK Tutorial](https://reader033.fdocuments.net/reader033/viewer/2022042723/587fc4ea1a28ab3b158b5cd1/html5/thumbnails/25.jpg)
3D Tracking Flavors1. Object Recognition Edge Detection / CAD models
Object Recognition Edge Detection / CAD models- pre-registered CAD
models and displayed silhouettes
- Focus: shapes & lines
![Page 26: Philipp Nagele (Wikitude) Wikitude SDK Tutorial](https://reader033.fdocuments.net/reader033/viewer/2022042723/587fc4ea1a28ab3b158b5cd1/html5/thumbnails/26.jpg)
Wikitude Lab – RnD Sneak Preview
![Page 27: Philipp Nagele (Wikitude) Wikitude SDK Tutorial](https://reader033.fdocuments.net/reader033/viewer/2022042723/587fc4ea1a28ab3b158b5cd1/html5/thumbnails/27.jpg)
3D Tracking Flavors2. Large Scale Scenes / Spaces
Large Scale Scenes Spaces
- SLAM based- Point cloud- buildings / spaces- indoors, outdoors
Watch here: https://www.youtube.com/watch?v=Zqb6Fo4uRkM
![Page 28: Philipp Nagele (Wikitude) Wikitude SDK Tutorial](https://reader033.fdocuments.net/reader033/viewer/2022042723/587fc4ea1a28ab3b158b5cd1/html5/thumbnails/28.jpg)
Positional Tracking – Demo Video
![Page 29: Philipp Nagele (Wikitude) Wikitude SDK Tutorial](https://reader033.fdocuments.net/reader033/viewer/2022042723/587fc4ea1a28ab3b158b5cd1/html5/thumbnails/29.jpg)
Philipp Nagele, CTOWikitude
See more.