Tour Guide Robot Project

Face Detection and Face Orientation on The Mobile Robot Robotino

Gökhan Remzi YavuzAyşenur Bilgin

Outline

IntroductionHardwareProject RequirementsProject ProgressQuestions

Introduction

Human Computer Interaction - HCIDetecting human facesDetecting interested people

Face detection Face orientation and face rotation

Keeping people interested by interacting with them

Getting closer when a certain distance is exceeded

Hardware

Robotino: a mobile robot platform from Festo

To satisfy project's requirements, we are planning some extensions on Robotino: Cylindirical body, total height: 1.60m including camera A laptop computer, attached to body at 1.20m Second color camera on top of the laptop computer dedicated to HCI

9 infrared sensors3 omnidirectional drive unitsBump sensorColor camera

Control Unit: a PC with a 300 Mhz Processor and 128 MB SDRam

Hardware con’t

• Sketch of hardware modifications on the robot base Robotino

• 1 additional laptop computer, 2 additional color cameras, skeleton with steel sticks and not yet sketched cylindirical cover for better appearance

Project Requirements

A tour guide robot should:

be able to navigate autonomously and purposefully inside the building

operate in a dynamic environment

attract visitors' attention by interacting with them

be able to monitor its charge level and move to its charging dock on its own

be able to dynamically arrange tours by combining selected destinations

deliver information in forms of visual, audio and text messages

be able to synchronize with the CMPE web site, in order to get up to date information.

Project Progress

What we did? Architecture Pseudocode Examples

What we did not?What we encountered?What we achieved?

What we did?

Combining Robotino API and OpenCVModifying the open source code to satisfy our needsRunning the modified algorithm on the mobile robotTesting the behavior

Architecture

Robotino

RobotinoAPI

Application OpenCV RobotinoAPI

Watchdog Signal Action Requests Camera Requests Camera Data

Application

Action Requests Camera Data

OpenCV

Image Data Face Information

Wireless

Pseudocode

Connect to Robotino over Wireless

While bumpsensorvalue = false do

Retrieve camera image from RobotinoAPI

Process image

. OpenCV face detection using haarcascades, frontal and profile faces

. Compute approximate distances of faces and their angle to the robot

Decide action; rotate, move forward, attract attention, stop

Act

end While

Examples

Examples cont’d

What we did not?

Pose estimationTrackingFace recognitionCompletion of hardware modifications

What we encountered?

Major challenge : Erroneous detection of faces

Processing power Minor problems :

Relatively low (10 fps) performance when frontal and profile face detection algorithms are combined

Position of the camera

What we achieved?

Experiments held in laboratoryMoving towards the human face until the

distance is reduced to 1 meter approximately

Turning with respect to the weighted average of areas of detected faces

Questions???

Thank you!