Augmented assembly using a multimodal interface

Muscle ShowcaseSanni Siltanen, VTT

Alex Potamianos, TUC

Augmented assembly system

• PC/Laptop

• Light-weight HMD with camera

• Visual Markers

Display

Multimodal Dialogue Interface

• Multimodal interface includes:– Visual Control and Visual feedback– Spoken Input and Audio feedback– Gesture Recognition

• Speech+HMD vs. Keyboard+HMD– More natural and convenient communication

• Main components– Gesture Recognition– Speech Recognition and Speech Understanding– Multimodal Dialogue Manager– Multimodal Controller

Augmented instructions

• Speech Control• Visual Control