Touchscreen Everywhere: On Transferring a Normal Planar...
Transcript of Touchscreen Everywhere: On Transferring a Normal Planar...
![Page 1: Touchscreen Everywhere: On Transferring a Normal Planar …imi.ntu.edu.sg/NewsEvents/Events/PastSeminars/Documents/... · 2017-08-11 · Skinput (bio-acoustic sensing array) LightSpace,](https://reader036.fdocuments.net/reader036/viewer/2022070715/5ed822f80fa3e705ec0de6f6/html5/thumbnails/1.jpg)
Touchscreen Everywhere:
On Transferring a Normal Planar
Surface to a Touch-Sensitive Display
Jingwen Dai
Postdoctoral Research Associate
University of North Carolina at Chapel Hill
![Page 2: Touchscreen Everywhere: On Transferring a Normal Planar …imi.ntu.edu.sg/NewsEvents/Events/PastSeminars/Documents/... · 2017-08-11 · Skinput (bio-acoustic sensing array) LightSpace,](https://reader036.fdocuments.net/reader036/viewer/2022070715/5ed822f80fa3e705ec0de6f6/html5/thumbnails/2.jpg)
Motivation
04/11/2013
Bigger Display Portability
VS.
NTU IMI PhD Seminar 2/30
![Page 3: Touchscreen Everywhere: On Transferring a Normal Planar …imi.ntu.edu.sg/NewsEvents/Events/PastSeminars/Documents/... · 2017-08-11 · Skinput (bio-acoustic sensing array) LightSpace,](https://reader036.fdocuments.net/reader036/viewer/2022070715/5ed822f80fa3e705ec0de6f6/html5/thumbnails/3.jpg)
Motivation
04/11/2013 3/30
DLP Pico Projector
DC DV Mobile Phone
NTU IMI PhD Seminar
![Page 4: Touchscreen Everywhere: On Transferring a Normal Planar …imi.ntu.edu.sg/NewsEvents/Events/PastSeminars/Documents/... · 2017-08-11 · Skinput (bio-acoustic sensing array) LightSpace,](https://reader036.fdocuments.net/reader036/viewer/2022070715/5ed822f80fa3e705ec0de6f6/html5/thumbnails/4.jpg)
04/11/2013 NTU IMI PhD Seminar 4/30
~400mm Single Image 3D Action
![Page 5: Touchscreen Everywhere: On Transferring a Normal Planar …imi.ntu.edu.sg/NewsEvents/Events/PastSeminars/Documents/... · 2017-08-11 · Skinput (bio-acoustic sensing array) LightSpace,](https://reader036.fdocuments.net/reader036/viewer/2022070715/5ed822f80fa3e705ec0de6f6/html5/thumbnails/5.jpg)
Previews Works
Additional Sensors Light Touch (IR optical sensors)
Diamondtouch (capacitive sensor array)
Smartskin (mesh-shaped antenna)
Skinput (bio-acoustic sensing array)
LightSpace, Omnitouch (Kinect)
Computer Vision [Letessier2004] -- Fingertip tracking, not touching detection
[Kjeldsen2002, Hardenberg2001] -- Delay-based scheme
[Marshall2008] – Color change of the fingernail
[Song2007, PlayAnywhere2005] -- Shadow casted by finger
[Fitriani2007] -- Deformation on soft surface
04/11/2013 NTU IMI PhD Seminar 5/30
![Page 6: Touchscreen Everywhere: On Transferring a Normal Planar …imi.ntu.edu.sg/NewsEvents/Events/PastSeminars/Documents/... · 2017-08-11 · Skinput (bio-acoustic sensing array) LightSpace,](https://reader036.fdocuments.net/reader036/viewer/2022070715/5ed822f80fa3e705ec0de6f6/html5/thumbnails/6.jpg)
Overview
04/11/2013 NTU IMI PhD Seminar 6/30
![Page 7: Touchscreen Everywhere: On Transferring a Normal Planar …imi.ntu.edu.sg/NewsEvents/Events/PastSeminars/Documents/... · 2017-08-11 · Skinput (bio-acoustic sensing array) LightSpace,](https://reader036.fdocuments.net/reader036/viewer/2022070715/5ed822f80fa3e705ec0de6f6/html5/thumbnails/7.jpg)
Priors in Projector-Camera System
Geometric (Homography)
All these geometric priors can be derived through 2 projection-capture cycles in initialization stage.
04/11/2013 NTU IMI PhD Seminar 7/30
![Page 8: Touchscreen Everywhere: On Transferring a Normal Planar …imi.ntu.edu.sg/NewsEvents/Events/PastSeminars/Documents/... · 2017-08-11 · Skinput (bio-acoustic sensing array) LightSpace,](https://reader036.fdocuments.net/reader036/viewer/2022070715/5ed822f80fa3e705ec0de6f6/html5/thumbnails/8.jpg)
Embedding Codes into Video Projection
04/11/2013 8/30 NTU IMI PhD Seminar
R. Raskar, G. Welch, M. Cutts, A. Lake, L. Stesin, and H. Fuchs, “The office of the future”, SIGGRAPH98.
![Page 9: Touchscreen Everywhere: On Transferring a Normal Planar …imi.ntu.edu.sg/NewsEvents/Events/PastSeminars/Documents/... · 2017-08-11 · Skinput (bio-acoustic sensing array) LightSpace,](https://reader036.fdocuments.net/reader036/viewer/2022070715/5ed822f80fa3e705ec0de6f6/html5/thumbnails/9.jpg)
Embedded Pattern Design Strategy
Method Array Size Win. Size Alph. Length
[Morita 1988] 24 * 24 3 * 4 2
[Kiyasu 1995] 18 * 18 4 * 2 2
[Salvi 1998] 29 * 29 3 * 3 3
[Spoelder 2000] 65 * 63 2 * 3 2
[Albitar 2007] 27 * 29 3 * 3 3
[Desjardins 2007] 53 * 38 3 * 3 3
[Chen 2008] 82 * 82 3 * 3 7
04/11/2013 9/30
Summary of typical spatial coding methods
Constraints of Pattern Generation Code Uniqueness
Large Hamming Distance
NTU IMI PhD Seminar
![Page 10: Touchscreen Everywhere: On Transferring a Normal Planar …imi.ntu.edu.sg/NewsEvents/Events/PastSeminars/Documents/... · 2017-08-11 · Skinput (bio-acoustic sensing array) LightSpace,](https://reader036.fdocuments.net/reader036/viewer/2022070715/5ed822f80fa3e705ec0de6f6/html5/thumbnails/10.jpg)
Hand Segmentation
04/11/2013 10/30
A
A
B
C
C
NTU IMI PhD Seminar
Contrast Saliency
Region Discontinuity
![Page 11: Touchscreen Everywhere: On Transferring a Normal Planar …imi.ntu.edu.sg/NewsEvents/Events/PastSeminars/Documents/... · 2017-08-11 · Skinput (bio-acoustic sensing array) LightSpace,](https://reader036.fdocuments.net/reader036/viewer/2022070715/5ed822f80fa3e705ec0de6f6/html5/thumbnails/11.jpg)
Histogram-based Contrast Saliency
M. Cheng et al. Global contrast based salient region detection, CVPR2011.
04/11/2013 11/30 NTU IMI PhD Seminar
![Page 12: Touchscreen Everywhere: On Transferring a Normal Planar …imi.ntu.edu.sg/NewsEvents/Events/PastSeminars/Documents/... · 2017-08-11 · Skinput (bio-acoustic sensing array) LightSpace,](https://reader036.fdocuments.net/reader036/viewer/2022070715/5ed822f80fa3e705ec0de6f6/html5/thumbnails/12.jpg)
Mean-Shift Region Smoothing
04/11/2013 12/30 NTU IMI PhD Seminar
D. Comaniciu and P. Meer. Mean shift: A robust approach toward feature space analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(5):603–619, 2002.
![Page 13: Touchscreen Everywhere: On Transferring a Normal Planar …imi.ntu.edu.sg/NewsEvents/Events/PastSeminars/Documents/... · 2017-08-11 · Skinput (bio-acoustic sensing array) LightSpace,](https://reader036.fdocuments.net/reader036/viewer/2022070715/5ed822f80fa3e705ec0de6f6/html5/thumbnails/13.jpg)
Precise Segmentation by Fusing
04/11/2013 13/30
CF > Δ
NTU IMI PhD Seminar
![Page 14: Touchscreen Everywhere: On Transferring a Normal Planar …imi.ntu.edu.sg/NewsEvents/Events/PastSeminars/Documents/... · 2017-08-11 · Skinput (bio-acoustic sensing array) LightSpace,](https://reader036.fdocuments.net/reader036/viewer/2022070715/5ed822f80fa3e705ec0de6f6/html5/thumbnails/14.jpg)
Fingertip Detection
04/11/2013 NTU IMI PhD Seminar 14/30
Fingertip Candidates Detected Fingertips
Hand Contour Binary Hand Image
![Page 15: Touchscreen Everywhere: On Transferring a Normal Planar …imi.ntu.edu.sg/NewsEvents/Events/PastSeminars/Documents/... · 2017-08-11 · Skinput (bio-acoustic sensing array) LightSpace,](https://reader036.fdocuments.net/reader036/viewer/2022070715/5ed822f80fa3e705ec0de6f6/html5/thumbnails/15.jpg)
Touch Detection Through
Homography and Embedded Codes
04/11/2013 NTU IMI PhD Seminar 15/30
![Page 16: Touchscreen Everywhere: On Transferring a Normal Planar …imi.ntu.edu.sg/NewsEvents/Events/PastSeminars/Documents/... · 2017-08-11 · Skinput (bio-acoustic sensing array) LightSpace,](https://reader036.fdocuments.net/reader036/viewer/2022070715/5ed822f80fa3e705ec0de6f6/html5/thumbnails/16.jpg)
From Resistive Touch to Capacitive
Touch or Floating Touch
04/11/2013 16/30 NTU IMI PhD Seminar
Homology
![Page 17: Touchscreen Everywhere: On Transferring a Normal Planar …imi.ntu.edu.sg/NewsEvents/Events/PastSeminars/Documents/... · 2017-08-11 · Skinput (bio-acoustic sensing array) LightSpace,](https://reader036.fdocuments.net/reader036/viewer/2022070715/5ed822f80fa3e705ec0de6f6/html5/thumbnails/17.jpg)
Experiments – System Setup
04/11/2013 17/30 NTU IMI PhD Seminar
![Page 18: Touchscreen Everywhere: On Transferring a Normal Planar …imi.ntu.edu.sg/NewsEvents/Events/PastSeminars/Documents/... · 2017-08-11 · Skinput (bio-acoustic sensing array) LightSpace,](https://reader036.fdocuments.net/reader036/viewer/2022070715/5ed822f80fa3e705ec0de6f6/html5/thumbnails/18.jpg)
Experiments – System Initialization
Camera-Projector Homography Estimation
04/11/2013 18/30
Projected chessboard Captured image
NTU IMI PhD Seminar
![Page 19: Touchscreen Everywhere: On Transferring a Normal Planar …imi.ntu.edu.sg/NewsEvents/Events/PastSeminars/Documents/... · 2017-08-11 · Skinput (bio-acoustic sensing array) LightSpace,](https://reader036.fdocuments.net/reader036/viewer/2022070715/5ed822f80fa3e705ec0de6f6/html5/thumbnails/19.jpg)
Experiments – System Initialization
Camera-Table Homography Estimation
Four Correspondences: C1 ~ O, C2 ~ Px, C3 ~ Py, C4 ~ Pxy
04/11/2013 19/30
Credit Card Captured image
NTU IMI PhD Seminar
![Page 20: Touchscreen Everywhere: On Transferring a Normal Planar …imi.ntu.edu.sg/NewsEvents/Events/PastSeminars/Documents/... · 2017-08-11 · Skinput (bio-acoustic sensing array) LightSpace,](https://reader036.fdocuments.net/reader036/viewer/2022070715/5ed822f80fa3e705ec0de6f6/html5/thumbnails/20.jpg)
Experiments – Display Quality Evaluation
04/11/2013 20/30 NTU IMI PhD Seminar
![Page 21: Touchscreen Everywhere: On Transferring a Normal Planar …imi.ntu.edu.sg/NewsEvents/Events/PastSeminars/Documents/... · 2017-08-11 · Skinput (bio-acoustic sensing array) LightSpace,](https://reader036.fdocuments.net/reader036/viewer/2022070715/5ed822f80fa3e705ec0de6f6/html5/thumbnails/21.jpg)
Experiments – Hand Segmentation Accuracy
Evaluation
(a) Original Image
(b) Ground-truth
(c) Our Method
(d) SCM
(e) BkSub
(f) GB
04/11/2013 21/30 NTU IMI PhD Seminar
![Page 22: Touchscreen Everywhere: On Transferring a Normal Planar …imi.ntu.edu.sg/NewsEvents/Events/PastSeminars/Documents/... · 2017-08-11 · Skinput (bio-acoustic sensing array) LightSpace,](https://reader036.fdocuments.net/reader036/viewer/2022070715/5ed822f80fa3e705ec0de6f6/html5/thumbnails/22.jpg)
Experiments – Hand Segmentation Accuracy
Evaluation
04/11/2013 22/30 NTU IMI PhD Seminar
![Page 23: Touchscreen Everywhere: On Transferring a Normal Planar …imi.ntu.edu.sg/NewsEvents/Events/PastSeminars/Documents/... · 2017-08-11 · Skinput (bio-acoustic sensing array) LightSpace,](https://reader036.fdocuments.net/reader036/viewer/2022070715/5ed822f80fa3e705ec0de6f6/html5/thumbnails/23.jpg)
04/11/2013 NTU IMI PhD Seminar 23/30
Experiments – Touch Accuracy Evaluation
More HD videos http://cs.unc.edu/~dai/Research/TouchEverywhere/TouchEverywhere.htm
![Page 24: Touchscreen Everywhere: On Transferring a Normal Planar …imi.ntu.edu.sg/NewsEvents/Events/PastSeminars/Documents/... · 2017-08-11 · Skinput (bio-acoustic sensing array) LightSpace,](https://reader036.fdocuments.net/reader036/viewer/2022070715/5ed822f80fa3e705ec0de6f6/html5/thumbnails/24.jpg)
Experiments – Touch Accuracy Evaluation
04/11/2013
Comparison with recent depth-camera sensing based methods
In [Wilson 2010], the informal observed spatial error of finger detection on planar surface was between 3-6 pixels In [Omni-Touch 2011], the FRR and FAR of finger click detection on four different surfaces were 0.8% and 3.3%.
NTU IMI PhD Seminar 24/30
![Page 25: Touchscreen Everywhere: On Transferring a Normal Planar …imi.ntu.edu.sg/NewsEvents/Events/PastSeminars/Documents/... · 2017-08-11 · Skinput (bio-acoustic sensing array) LightSpace,](https://reader036.fdocuments.net/reader036/viewer/2022070715/5ed822f80fa3e705ec0de6f6/html5/thumbnails/25.jpg)
Experiments – Trajectory Tracking
Evaluation
04/11/2013 NTU IMI PhD Seminar 25/30
![Page 26: Touchscreen Everywhere: On Transferring a Normal Planar …imi.ntu.edu.sg/NewsEvents/Events/PastSeminars/Documents/... · 2017-08-11 · Skinput (bio-acoustic sensing array) LightSpace,](https://reader036.fdocuments.net/reader036/viewer/2022070715/5ed822f80fa3e705ec0de6f6/html5/thumbnails/26.jpg)
Experiments – Multi-Touch Evaluation
04/11/2013 NTU IMI PhD Seminar 26/30
![Page 27: Touchscreen Everywhere: On Transferring a Normal Planar …imi.ntu.edu.sg/NewsEvents/Events/PastSeminars/Documents/... · 2017-08-11 · Skinput (bio-acoustic sensing array) LightSpace,](https://reader036.fdocuments.net/reader036/viewer/2022070715/5ed822f80fa3e705ec0de6f6/html5/thumbnails/27.jpg)
Experiments – Efficiency Evaluation
04/11/2013 27/30
Subroutine Hand Seg.
FTip Loc.
Touch Det.
Total
Time (ms/frame) 14.63 1.32 1.74 17.69
NTU IMI PhD Seminar
![Page 28: Touchscreen Everywhere: On Transferring a Normal Planar …imi.ntu.edu.sg/NewsEvents/Events/PastSeminars/Documents/... · 2017-08-11 · Skinput (bio-acoustic sensing array) LightSpace,](https://reader036.fdocuments.net/reader036/viewer/2022070715/5ed822f80fa3e705ec0de6f6/html5/thumbnails/28.jpg)
Summary
Using only off-the-shelf devices
Precise hand segmentation in PROCAMS
Achieving 3D sensing without explicit 3D reconstruction
04/11/2013 NTU IMI PhD Seminar 28/30
![Page 29: Touchscreen Everywhere: On Transferring a Normal Planar …imi.ntu.edu.sg/NewsEvents/Events/PastSeminars/Documents/... · 2017-08-11 · Skinput (bio-acoustic sensing array) LightSpace,](https://reader036.fdocuments.net/reader036/viewer/2022070715/5ed822f80fa3e705ec0de6f6/html5/thumbnails/29.jpg)
04/11/2013 NTU IMI PhD Seminar
Related Publications Conference paper [1] J. Dai and R. Chung, Making Any Planar Surface into a Touch-sensitive Display by a Mere Projector and Camera, In Proc. of 9th IEEE International Workshop on Projector–Camera Systems (PROCAMS2012), pages 35-42, 2012.
[2] J. Dai and R. Chung, Combining Contrast Saliency and Region Discontinuity for Precise Hand Segmentation in Projector-Camera System, In Proc. of The 21st International Conference on Pattern Recognition (ICPR’12), pages 2161-2164, 2012.
Journal paper [3] J. Dai and R. Chung, Touchscreen Everywhere: On Transferring a Normal Planar Surface to a Touch-Sensitive Display, Submitted to IEEE Trans. on System, Man and Cybernetics (TSMC-B), November 2012. (under 2nd round review)
29/30
http://cs.unc.edu/~dai/Publications/Publications.htm
![Page 30: Touchscreen Everywhere: On Transferring a Normal Planar …imi.ntu.edu.sg/NewsEvents/Events/PastSeminars/Documents/... · 2017-08-11 · Skinput (bio-acoustic sensing array) LightSpace,](https://reader036.fdocuments.net/reader036/viewer/2022070715/5ed822f80fa3e705ec0de6f6/html5/thumbnails/30.jpg)
THANK YOU!!
04/11/2013 30/30 NTU IMI PhD Seminar