NATURAL INTERACTIONSFULL BODY GESTURES AND GESTURES ON BODY OR SURFACE

NAME: YAJING WU (B00701148), NAZIM ALI (B00728400)

Introduction

´ Full body Gesture

1. Gesture

2. Body-gesture captured Model

3. How model work

4. User Data Storage

´ Gestures on surface

´ Gestures on Body

Gesture

´ Gesture is the use of motions of the limbs or body as ameans of expression, communicate an intention orfeeling. Gestures include body movements (e.g., palm-down, shoulder-shrug), and postures (e.g., angulardistance) and often occur in conjunction with speech.[1]

´ Touch gesture is becoming a part of our life and a lot ofdevices are using in daily bases like mobile, touch-tablets, touch monitor and many more.[9]

Examples of Gestures

Reference:https://www.google.ca/search?q=types+of+full+body+gestures&newwindow=1&biw =1264&b ih=607&source=lnms&tbm =isch&sa=X&ved=0ahUKEwjjxPmjjLTLAhXCMj4KHcA1 AY8Q_AUIBigB# imgrc=Q2KJMbShVXUqhM%3A

Body-gesture captured Model

Reference: Shoemaker G, Tsukitani T, Kitamura Y, et al. Body-centric interaction techniques for very large wall displays[C]//Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries. ACM, 2010: 463-472

How model work

Reference: Shoemaker G, Tsukitani T, Kitamura Y, et al. Body-centric interaction techniques for very large wall displays[C]//Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries. ACM, 2010: 463-472

Dynamic Light-Source Positioning

Reference: Shoemaker G, Tsukitani T, Kitamura Y, et al. Body-centric interaction techniques for very large wall displays[C]//Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries. ACM, 2010: 463-472

User Data Storage

Reference: Shoemaker G, Tsukitani T, Kitamura Y, et al. Body-centric interaction techniques for very large wall displays[C]//Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries. ACM, 2010: 463-472

Share User Data

Reference: Shoemaker G, Tsukitani T, Kitamura Y, et al. Body-centric interaction techniques for very large wall displays[C]//Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries. ACM, 2010: 463-472

Video

´ https://www.youtube.com/watch?v=hOAmreHmSVg

Types of Gestures on Surface:

´Touch mechanics´Touch activities

Touch mechanics[2]It is means that what will do in screen.[2]

a. Touch b. Double touch C. Drag, Swipe, or Fling

g. Pinch closedf. Pinch opene. Long-press drag

h. Two-finger touch

d. Long press

Further Details: https://www.google.com/design/spec/patterns/gestures.html#gestures-touch-mechanics”

Touch activities[2]

´ Activate

´ Cancel or Escape

´ Enable/Disable lights out

´ Drag or Swipe or Fling

´ Data selection

´ Data multi-selection drag

´ Pick up and move

Further Details: https://www.google.com/design/spec/patterns/gestures.html#gestures-touch-activities

´ Zoom in

´ Zoom out

´ Expand

´ Collapse

´ Rotate

Common Examples

Figure#2: Surface[3] Figura#3: Touch screen Monitor[4]

Figure#4: Mobiles[5]

WUW - Wear Ur World - A Wearable Gestural Interface[6]

How can we make our Laptop and Mobile phones in Big Display and used gesture?

´ This is wearable and gestures base device which acts on moving finger and combination of fingers.[6]

´ Any surface can turn into touch screen display.[6]

Figure#5: WUW[6]

Components

´ Projector

´ Camera

´ Colored Markers

´ Mobile / Laptop

´ Mirror

Figure#5b: WUW[6]

How It Works?

Capture data from finger

action using Webcam.[6]

Mobile receive data and send to projector.[6]

Mirror will reflect Image

and show it on desire

surface.[6]

Projector display

data.[6]

Used in deferent scenario

You can view map in large display and also make rotation of screen or zoom in /out and used thumb and index finger to navigate the map.[6]

Figure#6: Map Display [7]

Drawing Picture

Let the user draw any picture on any surface by using index finger.[6]

Hand Watch

Let the user see time on his/her hand[6]

Figure#7: Drawing Picture[6] Figure#8: Hand Watch[6]

Take Picture

User can take picture by making farm from four fingers.[6]

Video

User can make any surface to watch a video.[6]

Figure#9: Take Picture[6] Figure#10: Video[6]

Gestures on body

´ Types of skin gestures

´ What type of skin gestures are used most.

´ Common area for using skin gestures

´ Example

Types of skin gestures

a) Touch

b) Grab (Moving)

c) Pull

d) Press

e) Scratch

f) Shear (slide)

g) Squeeze

h) Twist (swap)Figure#11: Skin Gestures[8]

What type of skin gestures are used most´ In this research paper

researcher get data from 22 participant[8]

´ Most of the participant are using Multi-touching.[8]

´ Low usage of shearing and squeezing.[8]

Figure#12: Skin Gestures Usage[8]

Common area for using Skin gestures

´ Fingers

´ Forearm

´ Back of the hand

´ Palm

´ Upper arm

´ Elbow

Figure#13: Skin Gesture Areas[8]

Example: Skinput

• This device is used in hand and input will dedicate on the vibration of that area. [9]

• We can make call show watch and also maintain list.[9]

• This device also listen body vibration[9]

Figure#14: Skinput Device[9]

SKINPUT

´ Skinput used bioacoustics sensor to recognize finger taps.[9]

Figure#15: Skin as Touch[9]

Step of SKINPUT

Projector displays data

on arm.[9]

Tap from finger to

perform an action.[9]

Electronic signals

produced and perform action

on it.[9]

Vibration produced.[9]

Detector detected

armband.[9]

Limitation of Using Skin-Input

´ In cold weather people are pack with clothes so they unable to used skin

gestures devices.[9]

´ Consistent usage of skin device will be harmful to skin and that might

create any diseases.[9]

CONCLUSION AND FUTURE WORK

´ Surface gestures are more usable and convenient as comparing with

skin gestures.

´ Skin gesture devices is useful in few scenario while doing jogging,

climbing, cycling and etc.

´ Lot of working are in process to create small devices like Cup, table,

plate etc.

Reference: https://www.youtube.com/watch?v=bpXVuRulSHw

Microsoft Hololens

The Frist shipment of Microsoft Hololens on March 30. Yes it is builded and you can order from following link.[10]

Reference: https://www.microsoft.com/microsoft-hololens/en-us/development-edition

Activity

Questions?

Reference1. Shoemaker, G., Tsukitani, T., Kitamura, Y., & Booth, K. S. (2010, October). Body-centric interaction techniques for

very large wall displays. In Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries (pp. 463-472). ACM.

2. Gestures - Patterns - Google design guidelines. (n.d.). Retrieved March 05, 2016, from https://www.google.com/design/spec/patterns/gestures.html#gestures-touch-mechanics

3. Marshall, D. (n.d.). Win Microsoft Surface RT with Veeam : @VMblog. Retrieved March 05, 2016, from http://vmblog.com/archive/2013/07/25/win-microsoft-surface-rt-with-veeam.aspx#.VtkMlpwrLWI

4. Industrial Open Frame Touch Screen Monitors. (n.d.). Retrieved March 05, 2016, from http://www.caltronind.com/

5. Firemoon studios. (n.d.). Retrieved March 05, 2016, from http://www.firemoonstudios.com/android-application/

6. Mistry, P., Maes, P., & Chang, L. (2009, April). WUW-wear Ur world: a wearable gestural interface. In CHI'09 extended abstracts on Human factors in computing systems (pp. 4111-4116). ACM.

7. SixthSense - a wearable gestural interface (MIT Media Lab). (n.d.). Retrieved March 05, 2016, from http://www.pranavmistry.com/projects/sixthsense/

8. Weigel, M., Mehta, V., & Steimle, J. (2014, April). More than touch: understanding how people use skin as an input surface for mobile computing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 179-188). ACM.

9. Harrison, C., Tan, D., & Morris, D. (2010, April). Skinput: appropriating the body as an input surface. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 453-462). ACM.

Reference (cont..)

10. Microsoft HoloLens. (n.d.). Retrieved March 05, 2016, from https://www.microsoft.com/microsoft-hololens/en-us/development-edition

11. Goel, M., Lee, B., Islam Aumi, M. T., Patel, S., Borriello, G., Hibino, S., & Begole, B. (2014, April). SurfaceLink: using inertial and acoustic sensing to enable multi-device interaction on a surface. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems (pp. 1387-1396). ACM.

12. Jia, X., Lee, K. P., & Suk, H. J. (2011, September). Considerations of applying surface-based phone gestures to natural context. In Proceedings of the 13th international conference on Ubiquitous computing (pp. 545-546). ACM.

13. Tsandilas, T., Appert, C., Bezerianos, A., & Bonnet, D. (2014, April). Coordination of tilt and touch in one-and two-handed use. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 2001-2004). ACM.

14. Freeman, D., LaPierre, N., Chevalier, F., & Reilly, D. (2013, June). Tweetris: a study of whole-body interaction during a public art event. In Proceedings of the 9th ACM Conference on Creativity & Cognition (pp. 224-233). ACM.

15. Gunes, H., Piccardi, M., & Jan, T. (2004, June). Face and body gesture recognition for a vision-based multimodal analyzer. In Proceedings of the Pan-Sydney area workshop on Visual information processing (pp. 19-28). Australian Computer Society, Inc..