Mobile Phones asPointing Devices

Rafael “Tico” Ballagas (RWTH Aachen)

Michael Rohs (ETH Zurich)

Jennifer Sheridan (University of Lancaster)

2/16

Tico Ballagas Mobile Phones asPointing Devices

Overview

Introduction / Motivation Large Public Displays

Phones vs. Other Interaction Techniques

New Phonecam Interaction Techniques Sweep

Point & Shoot

Future Directions Potential Improvements

Deployment Opportunities

Demo

3/16

Tico Ballagas Mobile Phones asPointing Devices

Interaction with Large Displays

Personal

Semi-Public Public

4/16

Tico Ballagas Mobile Phones asPointing Devices

Potential ApplicationsLarge Public Displays

Games Interactive art Digital bulletin boards Advertising

5/16

Tico Ballagas Mobile Phones asPointing Devices

Direct surface interaction

Clear affordance One-to-one mapping High SerendipityBut… Physical Security Scalability Perspective Sanitation / Maintanence Multi-user

6/16

Tico Ballagas Mobile Phones asPointing Devices

Mobile Phones

Inherent Multi-user support (1 device / person) Physical Security not an issue User familiarity Connectivity standards Many built-in

sensors / actuators

7/16

Tico Ballagas Mobile Phones asPointing Devices

Mobile Phones for PointingTasks

Nokia 6600 Phonecam

Two Camera-based Techniques Sweep Point & Shoot

8/16

Tico Ballagas Mobile Phones asPointing Devices

Sweep

Optical Flow Technique Allows the phone to be used

like an optical mouse

The joystick is used as aclutch Allows user to reposition

arm

User can focus attention onthe large display

9/16

Tico Ballagas Mobile Phones asPointing Devices

10/16

Tico Ballagas Mobile Phones asPointing Devices

Point and Shoot

Aim using cross-hair cursor on phone screen Take a picture Item that you selected through the camera lens becomes

selected on the large display.

11/16

Tico Ballagas Mobile Phones asPointing Devices

12/16

Tico Ballagas Mobile Phones asPointing Devices

Visual Codes

Arbitrary Orientation

Tilt

Rotation

In Point & Shoot they areused to derive coordinatesystem on the displaysurface.

Currently 83 bits

13/16

Tico Ballagas Mobile Phones asPointing Devices

Example Scenario

14/16

Tico Ballagas Mobile Phones asPointing Devices

Performance Evaluation

15/16

Tico Ballagas Mobile Phones asPointing Devices

Improving Interactions

Strategies for combiningsensors Camera +

Accelerometer

Improved Optical Flowalgorithms Higher Resolution

Faster Sample Rate

16/16

Tico Ballagas Mobile Phones asPointing Devices

Deployment Opportunities

eCampus REXplorer