It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze:...
-
Upload
bryce-marvin-watkins -
Category
Documents
-
view
218 -
download
0
description
Transcript of It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze:...
![Page 1: It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses.](https://reader035.fdocuments.net/reader035/viewer/2022062523/5a4d1aee7f8b9ab05997c992/html5/thumbnails/1.jpg)
It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses
It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses Presentor: Aouaidjia Kamel 1/35
Lan Zhang, Xiang-Yang Li, Wenchao Huang, Kebin Liu, Shuwei Zong, Xuesi Jian, Puchun Feng, Taeho Jung,
Yunhao Liu ACM. New York, NY, USA ©2014
Advanced Topics on Internet Of Things
![Page 2: It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses.](https://reader035.fdocuments.net/reader035/viewer/2022062523/5a4d1aee7f8b9ab05997c992/html5/thumbnails/2.jpg)
Content
1. Background
2. iGaze Idea
3. Challenges and Solutions
4. System Design
5. Empirements and Evaluation
6. Future Work
It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses Presentor: Aouaidjia Kamel 2/35
![Page 3: It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses.](https://reader035.fdocuments.net/reader035/viewer/2022062523/5a4d1aee7f8b9ab05997c992/html5/thumbnails/3.jpg)
Background
Collect information sensors
Can run Mobile App
Control, or retrieve data from, other
Support wireless technologies Bluetooth, WiFi, and GPS
May also have all the features of a smartphone
It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses Presentor: Aouaidjia Kamel 3/35
Smart Glasses
![Page 4: It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses.](https://reader035.fdocuments.net/reader035/viewer/2022062523/5a4d1aee7f8b9ab05997c992/html5/thumbnails/4.jpg)
Scanning Distance < 3Scanning Angle <30°
High Computation Cost
Displacement > 20cm
QR code
Image Recognition
Hand gesture
It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses Presentor: Aouaidjia Kamel 4/35
Background Existing Technologies
![Page 5: It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses.](https://reader035.fdocuments.net/reader035/viewer/2022062523/5a4d1aee7f8b9ab05997c992/html5/thumbnails/5.jpg)
Voice Commands Head Gesture Control TouchPad or Buttons
Smart glasses are still looking for revolutionary applications
Background Human Computer Interface
It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses Presentor: Aouaidjia Kamel 5/35
![Page 6: It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses.](https://reader035.fdocuments.net/reader035/viewer/2022062523/5a4d1aee7f8b9ab05997c992/html5/thumbnails/6.jpg)
New Networking Mode
.
Can understand the user's visual attention
using an eye camera
Camera Can automatically connect to the
target of interest
Can run on top of existing networking
protocols, e.g., Wi-Fi.
iGaze Idea
It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses Presentor: Aouaidjia Kamel 6/35
![Page 7: It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses.](https://reader035.fdocuments.net/reader035/viewer/2022062523/5a4d1aee7f8b9ab05997c992/html5/thumbnails/7.jpg)
.
Museum
Obtain the detailed descriptions of art works by glazing at it instead.
iGaze Idea
It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses Presentor: Aouaidjia Kamel 7/35
![Page 8: It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses.](https://reader035.fdocuments.net/reader035/viewer/2022062523/5a4d1aee7f8b9ab05997c992/html5/thumbnails/8.jpg)
.
A house can turn on a smart appliance only by taking a glance at them
Device ControliGaze Idea
It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses Presentor: Aouaidjia Kamel 8/35
![Page 9: It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses.](https://reader035.fdocuments.net/reader035/viewer/2022062523/5a4d1aee7f8b9ab05997c992/html5/thumbnails/9.jpg)
.
When people look at a signboard or product, information can be sent to the potential customer immediately.
Advertising
iGaze Idea
It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses Presentor: Aouaidjia Kamel 9/35
![Page 10: It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses.](https://reader035.fdocuments.net/reader035/viewer/2022062523/5a4d1aee7f8b9ab05997c992/html5/thumbnails/10.jpg)
.
iGaze Idea
It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses Presentor: Aouaidjia Kamel 10/35
![Page 11: It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses.](https://reader035.fdocuments.net/reader035/viewer/2022062523/5a4d1aee7f8b9ab05997c992/html5/thumbnails/11.jpg)
iGaze Principales
Unidirectional
Given an observer and his/her gaze vector, only the device vector to the correct visual target’s device is consistent with the gaze vector.
iGaze Idea
It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses Presentor: Aouaidjia Kamel 11/35
![Page 12: It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses.](https://reader035.fdocuments.net/reader035/viewer/2022062523/5a4d1aee7f8b9ab05997c992/html5/thumbnails/12.jpg)
Bidirectional
Given a pair of users who are
looking at each other, their gaze
vectors have opposite directions.
iGaze Idea iGaze Principales
It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses Presentor: Aouaidjia Kamel 12/35
![Page 13: It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses.](https://reader035.fdocuments.net/reader035/viewer/2022062523/5a4d1aee7f8b9ab05997c992/html5/thumbnails/13.jpg)
iGaze Idea
It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses Presentor: Aouaidjia Kamel 13/35
![Page 14: It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses.](https://reader035.fdocuments.net/reader035/viewer/2022062523/5a4d1aee7f8b9ab05997c992/html5/thumbnails/14.jpg)
• Develop a low-cost glasses hardwarembedded
with attention capture software
• Capture the gaze direction using the orientation
sensor and eye camera.
Question 1 : How to accurately capture the vision plane attention of a user in real time ?
Answer:
Challenges and Solutions
It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses Presentor: Aouaidjia Kamel 14/35
![Page 15: It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses.](https://reader035.fdocuments.net/reader035/viewer/2022062523/5a4d1aee7f8b9ab05997c992/html5/thumbnails/15.jpg)
Visual Attention Detection
• Visual attention detection Fixation
• A fixation duration window threshold for visual attention
Challenges and Solutions
It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses Presentor: Aouaidjia Kamel 15/35
![Page 16: It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses.](https://reader035.fdocuments.net/reader035/viewer/2022062523/5a4d1aee7f8b9ab05997c992/html5/thumbnails/16.jpg)
Projection on the image plane is elliptical
Look ahead: the projection looks more like a circle
Look off to one side: it looks more close to an
ellipse
estimate the pose of the iris circle by back-
projecting the ellipse onto a circle in 3D space
Determine Gaze Direction Challenges and Solutions
It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses Presentor: Aouaidjia Kamel 16/35
![Page 17: It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses.](https://reader035.fdocuments.net/reader035/viewer/2022062523/5a4d1aee7f8b9ab05997c992/html5/thumbnails/17.jpg)
Capture the direction between glasses
Leveraging Doppler effect caused by arbitrary mild movement of head-
mounted speakers.
Question 2 : How to match the target on device plane according to user’s visual attention?
Answer :
Challenges and Solutions
It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses Presentor: Aouaidjia Kamel 17/35
![Page 18: It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses.](https://reader035.fdocuments.net/reader035/viewer/2022062523/5a4d1aee7f8b9ab05997c992/html5/thumbnails/18.jpg)
Challenges and Solutions
Head Node or Shake
It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses Presentor: Aouaidjia Kamel 18/35
Determine Device Direction
![Page 19: It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses.](https://reader035.fdocuments.net/reader035/viewer/2022062523/5a4d1aee7f8b9ab05997c992/html5/thumbnails/19.jpg)
System Design
It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses Presentor: Aouaidjia Kamel 19/35
![Page 20: It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses.](https://reader035.fdocuments.net/reader035/viewer/2022062523/5a4d1aee7f8b9ab05997c992/html5/thumbnails/20.jpg)
System Design Gaze Vector Acquisition
It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses Presentor: Aouaidjia Kamel 20/35
![Page 21: It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses.](https://reader035.fdocuments.net/reader035/viewer/2022062523/5a4d1aee7f8b9ab05997c992/html5/thumbnails/21.jpg)
System Design Device Vector Estimation
It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses Presentor: Aouaidjia Kamel 21/35
![Page 22: It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses.](https://reader035.fdocuments.net/reader035/viewer/2022062523/5a4d1aee7f8b9ab05997c992/html5/thumbnails/22.jpg)
System Design Matching and Networking
It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses Presentor: Aouaidjia Kamel 22/35
![Page 23: It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses.](https://reader035.fdocuments.net/reader035/viewer/2022062523/5a4d1aee7f8b9ab05997c992/html5/thumbnails/23.jpg)
System Design Application Layer
It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses Presentor: Aouaidjia Kamel 23/35
![Page 24: It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses.](https://reader035.fdocuments.net/reader035/viewer/2022062523/5a4d1aee7f8b9ab05997c992/html5/thumbnails/24.jpg)
System DesignVisual Attention Networking (VAN) Protocol
It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses Presentor: Aouaidjia Kamel 24/35
![Page 25: It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses.](https://reader035.fdocuments.net/reader035/viewer/2022062523/5a4d1aee7f8b9ab05997c992/html5/thumbnails/25.jpg)
System DesignVisual Attention Networking (VAN) Protocol
It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses Presentor: Aouaidjia Kamel 25/35
![Page 26: It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses.](https://reader035.fdocuments.net/reader035/viewer/2022062523/5a4d1aee7f8b9ab05997c992/html5/thumbnails/26.jpg)
System DesignVisual Attention Networking (VAN) Protocol
It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses Presentor: Aouaidjia Kamel 26/35
![Page 27: It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses.](https://reader035.fdocuments.net/reader035/viewer/2022062523/5a4d1aee7f8b9ab05997c992/html5/thumbnails/27.jpg)
System DesignVisual Attention Networking (VAN) Protocol
It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses Presentor: Aouaidjia Kamel 27/35
![Page 28: It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses.](https://reader035.fdocuments.net/reader035/viewer/2022062523/5a4d1aee7f8b9ab05997c992/html5/thumbnails/28.jpg)
Empirements and Evaluation
It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses Presentor: Aouaidjia Kamel 28/35
![Page 29: It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses.](https://reader035.fdocuments.net/reader035/viewer/2022062523/5a4d1aee7f8b9ab05997c992/html5/thumbnails/29.jpg)
Attention & Gaze Direction Accuracy
Attention capture accuracy: ~95%Gaze direction accuracy: < 5° (91 %) < 10° (99 %)
Empirements and Evaluation
It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses Presentor: Aouaidjia Kamel 29/35
![Page 30: It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses.](https://reader035.fdocuments.net/reader035/viewer/2022062523/5a4d1aee7f8b9ab05997c992/html5/thumbnails/30.jpg)
Direction estimation is robust against different head gestures and patterns.
Only one mild head gesture is sufficient for highly accurate estimation.
Empirements and Evaluation Device Direction Accuracy
It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses Presentor: Aouaidjia Kamel 30/35
![Page 31: It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses.](https://reader035.fdocuments.net/reader035/viewer/2022062523/5a4d1aee7f8b9ab05997c992/html5/thumbnails/31.jpg)
When the distance gets larger, the accuracy decreases
Device Direction AccuracyEmpirements and Evaluation
It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses Presentor: Aouaidjia Kamel 32/35
![Page 32: It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses.](https://reader035.fdocuments.net/reader035/viewer/2022062523/5a4d1aee7f8b9ab05997c992/html5/thumbnails/32.jpg)
Computation Cost Energy Cost
Computation Cost & Energy ConsumptionEmpirements and Evaluation
It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses Presentor: Aouaidjia Kamel 33/35
![Page 33: It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses.](https://reader035.fdocuments.net/reader035/viewer/2022062523/5a4d1aee7f8b9ab05997c992/html5/thumbnails/33.jpg)
Multiple objects in the gaze direction.
Multiple users may be interested in a same object.
User may be interested in a moving object.
How to avoid insecure connection.
Aesthetic design.
Future Work
It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses Presentor: Aouaidjia Kamel 34/35
![Page 34: It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses.](https://reader035.fdocuments.net/reader035/viewer/2022062523/5a4d1aee7f8b9ab05997c992/html5/thumbnails/34.jpg)
It Starts with iGaze: Visual Attention Driven Networking with Smart Glasses Presentor: Aouaidjia Kamel 35/35
Thank you for listening