Teleoperation

Interfaces

Introduction Interface between

the operator and teleoperator!

Teleoperation interface is like any other HMI

H(mobile)RI = TI Also a HCI

Introduction mechanical

manipulation – ”included” interface

Modern methods Closed loop

teleoperation – MMI Supervisory control

– HCI Mixed interfaces

Interface types, Fong Direct

Closed loop Multimodal/multisensor

Multisensor/actuator control Supervisory control

Intelligent telerobot Novel

New ones

Interface - Direct Closed loop control Realtime operator

decision making is needed

Operator controls with hand controllers (like onboard)

High bandwith low delay communication

Multimodal/multisensor Complex robot in

dynamic situation Individual actuator

control, graphical feedback, coordinated motion

Integrated display with combined sensor information

Supervisory control Remarkable part of the control in the

teleoperator end Teleoperator is capable for more or less

autonomous task execution “task based teleoperation”

Control methods

H M I c o m p u te r

te le o p e ra to r ’sc o m p u te r

tra n sm is s io n

O P E R AT O R

TA S K

d isp la y c o n tro ls

a c tu a to rsse n so rs

H M I c o m p u te r

te le o p e ra to r ’sc o m p u te r

tra n sm is s io n

O P E R AT O R

TA S K

d isp la y c o n tro ls

a c tu a to rsse n so rs

H M I c o m p u te r

te le o p e ra to r ’sc o m p u te r

tra n sm is s io n

O P E R AT O R

TA S K

d isp la y c o n tro ls

a c tu a to rsse n so rs

Direct Supervisory

Novel interfaces ”novel” is relative gestures gazes brainwaves muscle

movements WEB interfaces

multimodal supervisory

Telepresence When sufficient amount of sensor

information (vision, sound, force) is brought from the teleoperator site to the operator he or she feels physically present in the teleoperator site

Called also tele-existence Important information is transferred

and dangerous/noise is filtered

Virtual presence (reality) Like tele-presence

except the sensor infromation is generated artificially by computer(s)

Simulators Games Models

Augmented reality Real information

(usually image data) is mixed with additional virtual information

Numerical information, real-time models, etc.

Telepresence Already camera monitor combination

creates some level of presence more sophisticated system is called for

in order to call it telepresence To provide a perfect telepresence, all

human senses should be transmitted from the teleoperator site to the operator site vision, hearing and sense are relatively easy smell and taste are more complicated

Vision Humans get 90% of their perception “To see is to believe” eyes are very complex opto-

mechanical systems FoV is (H)180 deg x (V)120 deg Focused area only few degrees Movements from whole area

Extremely difficult system to be imitated

Vision Head tracking Head following

cameras (2-3 DoF)

HMD => relatively

good feeling of presence

Hearing Human area 16 – 20000Hz Important in telepresence In case of mobile machine control

the noise can be filtered and the important sounds transferred with reasonable volume

Touch The most important human sense Human touch sensors –

mechanoreceptors – are activated by touch, i.e. by pressure on the tissues

Two basic classes tactile information (“touch”) kinesthetic information (“force”)

Tactile

referring to the sense of contact with the object, mediated by the responses of low-threshold mechanoreceptors innervating the skin (say, the finger pad) within and around the contact region

Kinesthetic referring to the sense of position

and motion of limbs along with the associated forces conveyed by the sensory receptors in the skin around the joints, joint capsules, tendons, and muscles, together with neural signals derived from motor commands

Force feedback (kinesthetic) force generated by the

teleoperator, usually a manipulator, is fed back to the operator in order to generate a real response in gripping and manipulation tasks

Also in virtual environments

Inbuilt in mechanical manipulators

Haptic feedback (tactile) haptic feedback,

the tactile skin sensors have the main role.

tactile sensing of the robot manipulator is fed back to the fingers of the operator

Other possibilities also

Vestibular sensors inside the inner ear angular acceleration and thus

rotation linear acceleration in the horizontal

and vertical plane, i.e. to gravity => position and movements of the

head to be detected Important in dynamic driving tasks

Vestibular feedback not usually used in

teleoperation not needed and

expensive to implement

usually in simulators to create presence

If vision and vestibular sensors mismatch => simulator sickness (=seasickness)

Simulator Sickness similar to motion sickness difference is that SS can occur

without any actual motion of the operator

Symptoms: apathy, general discomfort, headache, stomach awareness, nausea, etc.

encountered especially when HMD type displays are used

Simulator Sickness The most typical

reason of SS is the cue conflict

In cue conflict different nerves get different information from the environment

Typically conflict between visual and vestibular inputs

especially when HMD is used and the time lags in vision and control

Movingteloperator+

=>

What is Presence

Overview

Traditional explanation Based on tele(virtual)-presence Sheridan’s definition of telepresence

PRESENCE

Ability to modifythe envrionment

Control of relationof sensors to the

environment

Extent of sensoryinformation

Traditional explanation Extent of sensory information has

a much greater impact than the other two combined. These three factors however cannot describe presence alone. Task variables, such as task difficulty and degree of automation, also are important to presence.

Is presence only sensor information? It’s very easy to be unpresent in a

boring situation – like lectures What about dreaming while sleeping or

burying one’s head a book Presence can be very strong without

any sensor information ”a self generated Virtual Reality”

A book – a film from same book Maximum effect with hallucinogens

3 modes of presence

1. Really present, perceiving the existing environment

2. Tele or virtual presence presence is transferred/generated by ”cheating” human senses

3. ”Mental” presence, presence is generated in the mind without (direct) sensor information