“It is the categorical verdict of all the nations and great seers of the world that national progress depends on education and training (of the people).”

AnalysisCollaborativeHaptics

Broadband

FelixHamza-Lup

Elena Clapan

JamesLaPlant

Carlos Sanchez

BenjaminLambeth

BradleySturz

OutlineWhat is Haptics ?The B.A.C.H. Project− Visual-Haptic Task− Haptic User Interface

Implementation− Gravitational Forces− Frictional Forces− Depth Cues: Cast Shadows

Assessment and Results Future WorkAcknowledgements

What is “Haptics” ?“Equipped with his five senses, man explores the universe around him and calls the

adventure Science.” Edwin P. Hubble, The Nature of Science, 1954.

Derived from the Greek word “haptesthai”(“touch” or “contact”)

Working with the sense of touch.

5 senses: sight, smell, taste, touch, and hearing

Haptic interfacesHaptics

Haptic Devices

Courtesy of Immersion Corporation

Courtesy of SensAble Technologies

PhantomDesktop

Phantom Omni

Falcon Novint

Cyber Glove

Phantom Omni• Robotic arm that tracks

the position and orientation of the user’s hand.

• Updates position and orientation information every ms (1KHz)

• Virtual representation of physical reference point within virtual application.

virtual reference

point

physical reference

point

Haptics - Application Domains

Medical− Remote Surgery − Telementoring/Training − Patient Rehabilitation

Entertainment− Haptic Games (booming)

Robotics− Hazardous Environments− Remote Manipulation (Telerobotics)

Education (HaptEK-16)− Simulation of Abstract Concepts

Academic Research− Multimodal Environments

Google in 2006: 446,000 hits for “haptic”. (0.19 seconds)Google in 2007: 1,030,000 hits for “haptic”. (0.28 seconds)Google in 2008: 1,840,000 hits for “haptic”. (0.27 seconds)

Haptics and InternetInternet is now being reengineered so that it can provide different levels of service for different types of traffic (e.g. VoIP, Video on Demand)

The ability to carry information relating to the haptic senses => potential for new applications.

Areas that will benefit:

− collaborative design [1]

− distance learning and training [2]

teaching by example (hand guide)- especially in the medical field

− remote guidance in virtual reality showrooms and museums [3]

− telerobotics [4]

Broadband Analysis of Collaborative Haptics (B.A.C.H.)

Project goal: − Research related to visual-haptic environments

Current focus:− Investigation of distributed visual-haptic environments

(i.e. force feedback distribution over the Internet)

How is haptic based collaborative task performance affected by:− Delay (variation - Jitter)

− Packet Loss (noise, error ratios …)

− Bandwidth (practical throughput between collaborative sites)

− Buffering algorithms (at different levels)

Haptic User Interface

• Completion time• No. of broken cubes• Quality of staking

OutlineWhat is Haptics ?The B.A.C.H. Project− Visual-Haptic Task− Haptic User Interface

Implementation− Gravitational Forces− Frictional Forces− Depth Cues: Cast Shadows

Assessment and Results Future WorkAcknowledgements

Implementation (APIs)

Application Programming Interface (API)

− Extensible 3D (X3D) – open standard for 3D scene design and implementation [5]

− Haptics 3D (H3D) – adds to the core X3D features, enabling force feedback programming [6]

− Python – object oriented scripting language – “glue”between H3D and X3D [7]

• H3D objects contain:– fields

• Fields– Example:

the DynamicTransform object has the following fields:

– Mass– Force– Torque

• An X3D scene graph is a directed acyclic graph (tree). – Nodes can contain specific fields

with one or more children nodes which participate in the hierarchy

Scene Graph

Part of the scene graph. Unfilled circles are nodes; filled circles are fields.

• Routes– routing one field to another causes

the second field to change when the first is changed

• Intermediate fields – can be used to “link” routes

between objects affecting their behaviour

• Example: When the object’s position changes, the new position is routed to a ShadowHandler field, which modifies the position’s y coordinate and routes it to the position of the object’s shadow.

Linking Fields

Simulating Frictional Forces

• Without friction we cannot lift objects.

• Friction is implemented through the “Slip-Slide” model [8]:– H3D calculates friction by

multiplying the static or dynamic coefficients of friction by the surface normal of the applied force.

G=mg

Fstatic

N

Fstatic ≤ µstatic * Normal

N = (-) Stick_Force

Fstatic

N

We generate a constant force on all the objects in the scene.

Since all cubes have the same mass, the same gravitational force will act on them.

Force value:

− 0.4N

Simulating Gravitational Forces

G=mg

We are currently simulating an elastic collision.

Objects in the scene are all rectangular boxes with 3 DOF

(only translations)

Simulating Collisions

Algorithm:

1: facedistance = |size(C1)/2 + size(C2)/2| - |center(C1) - center(C2)|2: for each direction in facedistance // direction = x, y, z3: if facedistance.direction < epsilon4: force.direction = -1 * force.direction5: momentum.direction = -1 * momentum.direction

Depth Cues: Cast Shadows

Cast shadows provide depth cues [9]

OutlineWhat’s Haptics ?The B.A.C.H. Project− Visual-Haptic Task− Haptic User Interface

Implementation− Gravitational Forces− Frictional Forces− Depth Cues: Cast Shadows

Assessment and ResultsFuture WorkAcknowledgements

Experimental SetupTask: stacking 4 cubes in a

virtual environment using haptic devices.

Three sets of experiments I. The user manipulates right/left haptic

devices

II. The user manipulates right/left haptic devices + 3D visualization

III. Two users (each with one device) collaborate to achieve the task goal

Goal: analyze the effect of network delay on task performance.

Experiment Setup

Simulated network latencyGenerated delays: 0, 25, 50, 100, 200 ms A set of warm-up trials and 10 test trials− The warm-up trials are for familiarization of the user

with the environment. Measures− Time to complete the task− Number of broken cubes − Quality of stacking

Preliminary Experiment (Spring 2008)

The BACH application is installed on one desktop machine

One user manipulates two haptic devices without shutter glasses

Participants: 4 male and 4 female students

2 warm-up trials at 0ms delay

Preliminary Experiment - Results

• Users are still in the process of learning during the 10 test trials

• Interpretation of the delay is not meaningful

First Five Last Five

Mea

n R

eact

ion

Tim

e(in

sec

onds

)

0

50

100

150

200

250

Experiment I (Fall 2008)

• Experiment setup similar

• 6 warm-up trials instead of 2 warm-up trials

• Participants: 2 male and 2 female students Network Delay

0ms 25ms 50ms 100ms 200ms

Mea

n Ti

me

to C

ompl

ete

Task

(in se

cond

s)

0

20

40

60

80

100

120

140

Experiment II

One user manipulates two haptic devices using a 3D Visualization system:− Crystal Eyes™ shutter glasses− NVidia FX Quadro− Infrared synchronizer

The same participants as in the previous experiment => No warm-up trials are necessary

OutlineWhat’s Haptics ?The B.A.C.H. Project− Visual-Haptic Task− Haptic User Interface

Implementation− Gravitational Forces− Frictional Forces− Depth Cues: Cast Shadows

Assessment and ResultsFuture WorkAcknowledgements

Experiment III – Network• Extending experiment to

the network

• Multiple Network Configurations• Single LAN• Two LANs• Three LANs• Internet

Experiment III - Application

Client Server Model

User Datagram Protocol

Force data

Position data

Send force of virtual pointer

Send update of cube position

Client 1 Client 2

Server

Experiment III - Monitoring

Implement Network Monitoring− Wireshark

Easy to implementLocated on ServerTimestamp

− NetFlowCisco 2500 RoutersMultiple network monitoring

Experiment III - Collaboration

Sense of Togetherness

AcknowledgementsThe preliminary phase of this research was supported by an AASU research and scholarship grant awarded to Dr. Bradley R. Sturz and Dr. Felix G. Hamza-Lup.

We would like to thank all the undergraduate and graduate students from computer science and psychology that helped with the implementation and participated in the experimental studies.

Sense Graphics (H3D)

Reach-In (http://www.reachin.se/)

References[1] Chris Gunn. Collaborative virtual sculpting with haptic feedback. Virtual Reality (2006)

vol 10, pp. 73-78

[2] Chebbi, B., Lazaroff, D., Bogsany, F., Liu, P.X., Liya Ni, Rossi, M.Design and implementation of a collaborative virtual haptic surgical training system. Mechatronics and Automation (2005), 2005 IEEE International Conference, vol.1(29), pp. 315-320

[3] McLaughlin, M. L., Sukhatme, G., Hespanha, J., Shahabi, C., Ortega, A., and Medioni, G. The haptic museum. Proc. EVA 2000 Conference on Electronic Imaging and the Visual Arts, (2000).

[4] Haptics in telerobotics: Current and future research and applications The International Journal of Computer Graphics (2007), vol 23(4), pp. 273-284

[5] http://www.web3d.org/x3d

[6] http://www.h3dapi.org/

[7] http://www.python.org/

[8] Hayward, V. and Armstrong, B. 2000. A New Computational Model Of Friction Applied To Haptic Rendering. In Experimental Robotics VI, Peter Corke and James Trevelyan (Eds), Lecture Notes in Control and Information Sciences, vol. 250, Springer-Verlag, pp. 403-412.

[9] Kersten, D., Mamassian, P., and Knill, D.C. Moving cast shadows induce apparent motion in depth. Perception, 26, 2 (1997), 171-192.

Thank you!