Virtual Image Peephole By Kyle Patience Supervisor: Reg Dodds Co Supervisor: Mehrdad Ghaziasgar.
Transcript of Virtual Image Peephole By Kyle Patience Supervisor: Reg Dodds Co Supervisor: Mehrdad Ghaziasgar.
Virtual Image PeepholeBy
Kyle PatienceSupervisor: Reg Dodds
Co Supervisor: Mehrdad Ghaziasgar
Quick Recap
• Development of an interface that does the following: • A very large virtual screen is
imagined to exist, • The mobile phone screen is used
as a "peephole" into that interface.
Overview
•Design Decisions and System Changes• Implementation •Finding and Tracking Features•User Interface •Tools
Design Decisions and System Changes
PREVIOSLY CURRENTLY
• No NDK • Uses NDK
• Use Android camera API • Uses device’s camera natively
• Didn’t use sensor readings • Uses both accelerometer and gyroscope
• Display was modifies camera frames • 3D interface
NDK
Implementation
Get frames Process frames
Get sensor readings
Output
Finding and Tracking Features• The Shi-Tomasi Corner Detector
• Based on the Harris Corner Detector. • Tries to find little patches of image that generate a large variation when moved around.• Ultimately finds small corners in a frame.
• Lucas & Kanade Method• Technique which can provide an estimate of the movement of certain features in successive images of a
scene.• Algorithm makes a "best guess" of the displacement of a neighbourhood by looking at changes in pixel
intensity.• These intensities are known through intensity gradients of the image in that neighbourhood.
User Interface using camera framesMoving the device without tilt will move the interface
User Interface using gyroscope
Tilting phone to the left Tilting phone to the right
User Interface using accelerometer
Tilting phone upwards
Tilting phone downwards
ToolsPlatform Windows 8.1 x64
Applications Android Studio
SDK Android SDKOpenCV Android SDKAndroid NDK
Libraries OpenCVjPCT
Languages JavaC++
Project Plan
Term 1 Learn OpenCV Learn Android
Term 2Learn NDKBuild Android PrototypeCapture Footage
Term 3Process FramesProcess Accelerometer and Gyroscope ReadingsCreate 3D InterfaceIntergrade all components
Term 4ImplementingTuningTesting
References
• Ali, S. I., Jain, S., Lal, B., & Sharma, N. (2012). A framework for modelling and designing of intelligent and adaptive interfaces for human computer interaction. International Journal of Applied Information Systems (IJAIS) Volume.
• Anuar, A., Saipullah, K. M., Ismail, N. A., & Soo, Y. (2011). OpenCV based real-time video processing using android smartphone. International Journal of Computer Technology and Electronics Engineering (IJCTEE), 1(3).
• Shi, J., & Tomasi, C. (1994, June). Good features to track. In Computer Vision and Pattern Recognition, 1994. Proceedings CVPR'94., 1994 IEEE Computer Society Conference on (pp. 593-600). IEEE.
• Tomasi, C., & Kanade, T. (1991). Detection and tracking of point features. Pittsburgh: School of Computer Science, Carnegie Mellon Univ..
Questions?