Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu...
-
date post
21-Dec-2015 -
Category
Documents
-
view
219 -
download
0
Transcript of Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu...
![Page 1: Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu LYU0604Lai Chung Sum (04523002)Siu Ho Tung (04559983)](https://reader030.fdocuments.net/reader030/viewer/2022032521/56649d5f5503460f94a3f079/html5/thumbnails/1.jpg)
Virtual Dart – An Augmented Reality Game on Virtual Dart – An Augmented Reality Game on Mobile DeviceMobile DeviceSupervised by Prof. Michael R. LyuSupervised by Prof. Michael R. Lyu
LYU0604LYU0604 Lai Chung Sum (04523002)Lai Chung Sum (04523002) Siu Ho Tung (04559983)Siu Ho Tung (04559983)
![Page 2: Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu LYU0604Lai Chung Sum (04523002)Siu Ho Tung (04559983)](https://reader030.fdocuments.net/reader030/viewer/2022032521/56649d5f5503460f94a3f079/html5/thumbnails/2.jpg)
2
OutlineOutline
• Background Information & Objective• Summary of Works Done in Semester 1• Works in Semester 2• Game Flow• Programming Tricks• Difficulties• Conclusion• Q & A
![Page 3: Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu LYU0604Lai Chung Sum (04523002)Siu Ho Tung (04559983)](https://reader030.fdocuments.net/reader030/viewer/2022032521/56649d5f5503460f94a3f079/html5/thumbnails/3.jpg)
3
ObjectivesObjectives
• Demonstrate how a game can Demonstrate how a game can “recognize” its external environment for “recognize” its external environment for Augmented RealityAugmented Reality
• Develop a mobile game, Virtual Dart, Develop a mobile game, Virtual Dart, which illustrates the proposed which illustrates the proposed methodologymethodology
![Page 4: Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu LYU0604Lai Chung Sum (04523002)Siu Ho Tung (04559983)](https://reader030.fdocuments.net/reader030/viewer/2022032521/56649d5f5503460f94a3f079/html5/thumbnails/4.jpg)
4
DefinitionDefinition
Augmented Reality is . . .Augmented Reality is . . .• a combination of real world and computer generated data• the use of video and which is augmented by addition of computer
graphics
![Page 5: Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu LYU0604Lai Chung Sum (04523002)Siu Ho Tung (04559983)](https://reader030.fdocuments.net/reader030/viewer/2022032521/56649d5f5503460f94a3f079/html5/thumbnails/5.jpg)
5
Background InformationBackground Information
Programming
Sybmian C++
C#
Target Mobile Phone
Nokia N90 (S60 2rd Edition FP3)
Nokia N80 (S60 3rd Edition)
(Nokia 6600)
![Page 6: Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu LYU0604Lai Chung Sum (04523002)Siu Ho Tung (04559983)](https://reader030.fdocuments.net/reader030/viewer/2022032521/56649d5f5503460f94a3f079/html5/thumbnails/6.jpg)
6
Summary of Works Done in Semester 1Summary of Works Done in Semester 1
Feature Selection
Feature Recognition
Motion Tracking on the featue
![Page 7: Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu LYU0604Lai Chung Sum (04523002)Siu Ho Tung (04559983)](https://reader030.fdocuments.net/reader030/viewer/2022032521/56649d5f5503460f94a3f079/html5/thumbnails/7.jpg)
7
Summary of Works Done in Semester 1Summary of Works Done in Semester 1
Program CoreProgram Core1. Implementation of Feature
Selection Algorithm2. Implementation of Initial Feature
Recognition Algorithm3. Implementation of Enhanced
Feature Recognition Algorithm4. Modification of mVOTE Motion
Tracking Engine5. Implementation of mobile game,
Virtual Dart, in prototype form
Virtual DartVirtual Dart
Studied MaterialsStudied Materials 1. Existing mVOTE Engine & its
related papers2. Symbian C++ &
Nokia S60 3rd edition programming guides
3. Corner Detection Algorithms such as FAST Corner Detector and Harris Corner Detector
4. Block Matching technique
![Page 8: Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu LYU0604Lai Chung Sum (04523002)Siu Ho Tung (04559983)](https://reader030.fdocuments.net/reader030/viewer/2022032521/56649d5f5503460f94a3f079/html5/thumbnails/8.jpg)
8
Problem Discovered in Semester 1Problem Discovered in Semester 1
Problem
Enhanced Feature Recognition Algorithm takes more than 30 seconds to recognize the saved features
Cause
1. Nokia N80 has only limited computational power
2. The algorithm maybe too general to recognize features
Solution
Design a new and specific algorithm (To be discussed later)
![Page 9: Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu LYU0604Lai Chung Sum (04523002)Siu Ho Tung (04559983)](https://reader030.fdocuments.net/reader030/viewer/2022032521/56649d5f5503460f94a3f079/html5/thumbnails/9.jpg)
9
What does the program need to What does the program need to recognize?recognize?
LED Recognition
Bar Code Recognition
Common in indoor environment
Can emit light+
Goal: Use the simplest method to build the recognizer
![Page 10: Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu LYU0604Lai Chung Sum (04523002)Siu Ho Tung (04559983)](https://reader030.fdocuments.net/reader030/viewer/2022032521/56649d5f5503460f94a3f079/html5/thumbnails/10.jpg)
10
How to locate LED?How to locate LED?
Filter out the surrounding environment
1. Locate a block which contains LED
2. If the block contains more than a threshold of non-black pixels
1. Filter for LED 2. Find LED
![Page 11: Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu LYU0604Lai Chung Sum (04523002)Siu Ho Tung (04559983)](https://reader030.fdocuments.net/reader030/viewer/2022032521/56649d5f5503460f94a3f079/html5/thumbnails/11.jpg)
11
LED Recognizer – FilterLED Recognizer – Filter
Use Green / Red LED as target object
Goal: Separate the target object from its surrounding environment
![Page 12: Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu LYU0604Lai Chung Sum (04523002)Siu Ho Tung (04559983)](https://reader030.fdocuments.net/reader030/viewer/2022032521/56649d5f5503460f94a3f079/html5/thumbnails/12.jpg)
12
Add constraint to other color channelsResult image of G > 230 and B < 204
LED Recognizer – FilterLED Recognizer – Filter
Set a lower bound for Green color channel of the pixel
![Page 13: Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu LYU0604Lai Chung Sum (04523002)Siu Ho Tung (04559983)](https://reader030.fdocuments.net/reader030/viewer/2022032521/56649d5f5503460f94a3f079/html5/thumbnails/13.jpg)
13
LED Recognizer – Filter ExtensionLED Recognizer – Filter Extension
• Use similar technique to build a filter for Fluorescent Light, Light Bulb
![Page 14: Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu LYU0604Lai Chung Sum (04523002)Siu Ho Tung (04559983)](https://reader030.fdocuments.net/reader030/viewer/2022032521/56649d5f5503460f94a3f079/html5/thumbnails/14.jpg)
14
LED Recognizer – Version 1LED Recognizer – Version 1
Check every Check every block in the block in the
screenscreen
Prevent block Prevent block overlappingoverlapping
![Page 15: Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu LYU0604Lai Chung Sum (04523002)Siu Ho Tung (04559983)](https://reader030.fdocuments.net/reader030/viewer/2022032521/56649d5f5503460f94a3f079/html5/thumbnails/15.jpg)
15
Limitations of LED Recognizer – Version 1Limitations of LED Recognizer – Version 1
Problem
1.Slow
2.Target object at the corner of the block
3.Select more than one blocks for same object
![Page 16: Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu LYU0604Lai Chung Sum (04523002)Siu Ho Tung (04559983)](https://reader030.fdocuments.net/reader030/viewer/2022032521/56649d5f5503460f94a3f079/html5/thumbnails/16.jpg)
16
LED Recognizer – Version 2LED Recognizer – Version 2
Search the Search the center region center region of the blockof the block
onlyonly
Distance Distance checkcheck
Stop after Stop after finding 2 finding 2 feature feature blocksblocks
![Page 17: Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu LYU0604Lai Chung Sum (04523002)Siu Ho Tung (04559983)](https://reader030.fdocuments.net/reader030/viewer/2022032521/56649d5f5503460f94a3f079/html5/thumbnails/17.jpg)
17
Comparison of 2 versions of Comparison of 2 versions of LED RecognizerLED Recognizer
Version 1 Version 2
![Page 18: Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu LYU0604Lai Chung Sum (04523002)Siu Ho Tung (04559983)](https://reader030.fdocuments.net/reader030/viewer/2022032521/56649d5f5503460f94a3f079/html5/thumbnails/18.jpg)
18
LED Recognizer – Final VersionLED Recognizer – Final Version
LED Recognizer – Version 2LED Recognizer – Version 2LED Recognizer – Version 2LED Recognizer – Version 2
RemoveRemoveDistanceDistance
CheckCheck
Remove Remove Block Block
Overlapping Overlapping CheckCheck
Stop After Stop After Finding Finding
First First FeatureFeature
Simplify by
Reduce the number of tracking point to 1Reduce the number of tracking point to 1Reduce the number of tracking point to 1Reduce the number of tracking point to 1
+
Final Version of LED RecognizerFinal Version of LED RecognizerFinal Version of LED RecognizerFinal Version of LED Recognizer
![Page 19: Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu LYU0604Lai Chung Sum (04523002)Siu Ho Tung (04559983)](https://reader030.fdocuments.net/reader030/viewer/2022032521/56649d5f5503460f94a3f079/html5/thumbnails/19.jpg)
19
Motion Tracking by mVOTE EngineMotion Tracking by mVOTE Engine
Problem
The motion tracking of the LED would be lost very easily
Cause Solution
1. Detect if the motion tracking of the LED is lost or not
2. Relocate the LED if the motion tracking is lost
Areas around the LED are flat regions. mVOTE Engine cannot work well in flat region
![Page 20: Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu LYU0604Lai Chung Sum (04523002)Siu Ho Tung (04559983)](https://reader030.fdocuments.net/reader030/viewer/2022032521/56649d5f5503460f94a3f079/html5/thumbnails/20.jpg)
20
Virtual Dart ModelingVirtual Dart Modeling
Dart Throwing Preview Scene
![Page 21: Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu LYU0604Lai Chung Sum (04523002)Siu Ho Tung (04559983)](https://reader030.fdocuments.net/reader030/viewer/2022032521/56649d5f5503460f94a3f079/html5/thumbnails/21.jpg)
21
Virtual Dart ModelingVirtual Dart Modeling
X39 pixels0.279 Meter
4 pixels0.0286 Meter
1.6 + 0.279 = 1.879 m
1.6 - 0.0286 = 1.5714 m
1.6 m
![Page 22: Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu LYU0604Lai Chung Sum (04523002)Siu Ho Tung (04559983)](https://reader030.fdocuments.net/reader030/viewer/2022032521/56649d5f5503460f94a3f079/html5/thumbnails/22.jpg)
22
Game FlowGame Flow
![Page 23: Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu LYU0604Lai Chung Sum (04523002)Siu Ho Tung (04559983)](https://reader030.fdocuments.net/reader030/viewer/2022032521/56649d5f5503460f94a3f079/html5/thumbnails/23.jpg)
23
Programming TricksProgramming Tricks
A lot of Square Root calculation in Projectile MotionA lot of Square Root calculation in Projectile MotionA lot of Square Root calculation in Projectile MotionA lot of Square Root calculation in Projectile Motion
Found a fast Square Root calculation by John CarmackFound a fast Square Root calculation by John CarmackFound a fast Square Root calculation by John CarmackFound a fast Square Root calculation by John Carmack
Newton Newton Approximation Approximation
MethodMethod
![Page 24: Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu LYU0604Lai Chung Sum (04523002)Siu Ho Tung (04559983)](https://reader030.fdocuments.net/reader030/viewer/2022032521/56649d5f5503460f94a3f079/html5/thumbnails/24.jpg)
24
Programming TricksProgramming Tricks
![Page 25: Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu LYU0604Lai Chung Sum (04523002)Siu Ho Tung (04559983)](https://reader030.fdocuments.net/reader030/viewer/2022032521/56649d5f5503460f94a3f079/html5/thumbnails/25.jpg)
25
DifficultiesDifficulties
Build 3D model of dart images
Insufficient support from emulator
Colors in images captured change in different platforms
Unclear explanations from Nokia S60 documentations
“Active Object” in Symbian platform
![Page 26: Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu LYU0604Lai Chung Sum (04523002)Siu Ho Tung (04559983)](https://reader030.fdocuments.net/reader030/viewer/2022032521/56649d5f5503460f94a3f079/html5/thumbnails/26.jpg)
26
ConclusionConclusion
• Successfully recognized Green / Red LED for Augmented Reality
• Developed a mobile game with Augmented Reality – Virtual Dart
![Page 27: Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu LYU0604Lai Chung Sum (04523002)Siu Ho Tung (04559983)](https://reader030.fdocuments.net/reader030/viewer/2022032521/56649d5f5503460f94a3f079/html5/thumbnails/27.jpg)
27
Virtual DartVirtual Dart
EndEnd
![Page 28: Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu LYU0604Lai Chung Sum (04523002)Siu Ho Tung (04559983)](https://reader030.fdocuments.net/reader030/viewer/2022032521/56649d5f5503460f94a3f079/html5/thumbnails/28.jpg)
28
LED Recognizer – FilterLED Recognizer – Filter
Before Filtering After Filtering
![Page 29: Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu LYU0604Lai Chung Sum (04523002)Siu Ho Tung (04559983)](https://reader030.fdocuments.net/reader030/viewer/2022032521/56649d5f5503460f94a3f079/html5/thumbnails/29.jpg)
29
LED Recognizer – FilterLED Recognizer – Filter
Before Filtering After Filtering
![Page 30: Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu LYU0604Lai Chung Sum (04523002)Siu Ho Tung (04559983)](https://reader030.fdocuments.net/reader030/viewer/2022032521/56649d5f5503460f94a3f079/html5/thumbnails/30.jpg)
30
LED Recognizer – FilterLED Recognizer – Filter
• Use similar technique to build filter for Red LED
• Set lower bound for Red color channel and upper bound for Blue color channel
![Page 31: Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu LYU0604Lai Chung Sum (04523002)Siu Ho Tung (04559983)](https://reader030.fdocuments.net/reader030/viewer/2022032521/56649d5f5503460f94a3f079/html5/thumbnails/31.jpg)
31
Projectile Motion with Init. Velocity = 16m/s
0
0.5
1
1.5
2
2.5
3
3.5
0 2 4 6 8 10
X Displacement (in Meter)
Y D
ispl
acem
ent
(in M
eter
)
Angle = 0
Angle = 5
Angle = 10
Angle = 15
Angle = 20
Angle = 25
Projectile MotionProjectile Motion
tVxx ixii ,1
tm
VVBVV ixi
ixix
,2,1,
tVyy iyii ,1
tm
VVBtgVV iyi
iyiy
,2,1,
![Page 32: Virtual Dart – An Augmented Reality Game on Mobile Device Supervised by Prof. Michael R. Lyu LYU0604Lai Chung Sum (04523002)Siu Ho Tung (04559983)](https://reader030.fdocuments.net/reader030/viewer/2022032521/56649d5f5503460f94a3f079/html5/thumbnails/32.jpg)
32
Programming TricksProgramming Tricks
float Cos(int aAngle){
switch(aAngle){
case(0): return 1.0; break;case(1): return 0.99984769516; break;case(2): return 0.99939082702; break;case(3): return 0.99862953475; break;...case(12): return 0.97814760073; break;case(13): return 0.97437006479; break;case(14): return 0.97029572628; break;case(15): return 0.96592582629; break;default: return 1.0; break;
}}