1

Towards Pervasive Connectivity in Mobile

ComputingFrank Siegemund

European Microsoft Innovation Center

November 2006

2

Outline Motivation Related Work System Architecture Implementation Application Conclusions

3

Motivation Most middleware assumes basic connectivity

– Web Services– Jini– Corba

Problematic in mobile scenarios– Mobile P2P systems– Mobile CSCW

Heterogeneity– Bluetooth, WLAN, IrDA, GPRS– IPv4 vs. IPv6– NATs and firewalls

4

Mobile P2P Bootstrapping problem Establish connectivity

between mobile users Mobile devices as first-

class citizens in P2P applications

Exploit knowledge about connectivity in the group

More reliable/robust group establishment

5

Scenario Content sharing

between mobile users Conference scenario Skiing trip Existing group collects

connectivity parameters Sharing by means of

visual tags/glyphs Handheld-embedded

cameras for capturing tags

6

Outline Motivation Related Work System Architecture Implementation Application Conclusions

7

Related Work Capturing visual tags with handheld

devices and low-cost cameras– Rekimoto et al.

Visual tags in Pervasive Computing– Rohs and Zweifel– Many interesting scenarios

Visual tags to bypass Bluetooth device discovery– Scott et al.

8

Related Approaches Out-of-band discovery for Bluetooth

– IrDA– RFID

Collaborative approaches for device discovery– Siegemund and Rohs

Manual configuration Send SMS with information about group

– Complicated from user perspective

9

Outline Motivation Related Work System Architecture Implementation Application Conclusions

10

System Architecture

11

Collecting Connectivity Data Overlay between group members Exploit heterogeneity

– Various communication technologies in group– Different security constraints

Construct group connectivity profile Create tags to transmit connectivity profile Candidate group member captures tags Simplified user interaction

12

Group Overlay Overlay can contain both mobile and stationary

nodes Overlay provides multicast group Nodes in group have synchronized state Connectivity properties of nodes shared between

group members NATs can be dealt with via different group peers Direct connections not always possible Group used to circumvent connectivity

constraints

13

Visual Tags Tags captured from

video stream Captured by handheld-

embedded digital cameras

Direct user feedback about detected tags possible

Small video resolution requires displaying multiple tags

Concrete tag layout unimportant

14

Outline Motivation Related Work System Architecture Implementation Application Conclusions

15

Implemenation Heterogeneous environment

– Windows Mobile 5.0 smartphones– Window XP/Vista desktops

Implementation on desktops using Windows P2P Infrastructure

Mobile devices join/create P2P overlay– Socket interface – Lightweight database for data synchronization

Native implementation

16

Visual Tags Tag detection in

DirectShow Analyze single video

frames Basic algorithm

– Black and white conversion

– Region detection– Identify guide bars– Decode tags– Determine tag index– Finish if all tags have

been decoded– Join group

Similar to Rekimoto

et al.

17

Improvements Determine threshhold for black/white conversion

in feedback loop– Different lightning conditions

Mark tag in video frame to provide feedback to users – Tag detected or not

Synchronization– Tag contains time a single tag is displayed– Capturing device can calculate schedule for reading

tags

18

Evaluation 240 x 320 video frame resolution Implementation on 195 Mhz smartphone

– State of the art (no high-end model) 400 ms per frame

– Can be improved significantly !!!– Copy filter

90 ms for thresholding 250 ms for region detection Algorithm deals with small rotations of the tag More sophisticated tag detection systems

available

19

Outline Motivation Related Work System Architecture Implementation Application Conclusions

20

Example Application Mobile P2P chat

application Messaging for user

groups on mobile devices

Overlay to distribute connectivity parameters

Tags to exchange connectivity profiles

Deal with NATs and connectivity constraints

21

Outline Motivation Related Work System Architecture Implementation Application Conclusions

22

Conclusions Basic connectivity problematic in mobile scenarios Current middleware solutions insufficiently address

this problem Facilitate groups to establish connectivity in mobile

scenarios Groups create finger print of network environment NATs and security constraints can be dealt with Visual tags user friendly way to communicate

connection properties Useful for mobile P2P and CSCW scenarios Real-life applications

23

Contact

Frank SiegemundEuropean Microsoft Innovation CenterRitterstrasse 23, 52072 Aachenfranksie@microsoft.com