Composing Devices Prasun Dewan Department of Computer Science University of North Carolina...

Composing Devices

Prasun Dewan

Department of Computer Science University of North Carolina

[email protected]

2

Issues raised by UPnP

Scaling problem of multicast-based discovery Auto Shut off problem

Simple-minded search Lack of access and concurrency control No programmer-defined types High programming and maintenance costs Manual, procedural service composition Device-independent UI

3

Declarative Specifications Defined by declarations Higher-level than procedural

HTML vs. Toolkits User addresses what rather than how.

Not as general. Domain specific HTML vs. Toolkits

Form-based user interfaces No graphics, incremental feedback.

Composition domains for declarative specifications?

4

Database based composition

Queries over multiple devices For each rainfall sensor, average rainfall For each sensor in Tompkin county,

current rainfall For next 5 hrs, every 30 minutes, rainfall

in Tompkin county

5

Query KindsDevice Queries Historical

For each rainfall sensor, average rainfall Snapshot

For each sensor in Tompkin county, current rainfall

Long-running For next 5 hrs, every 30 minutes, rainfall in

Tompkin county New kind of query

Defining device databases and queries?

6

Cougar Device Model Embedded vs. Attached Stationary vs. Mobile Strongly vs. Intermittently connected Local area vs. wide area Device database work focuses on stationary

devices Sensors vs. arbitrary devices?

7

Cougar Device Operations Operation Model

acquire, store, and process data may trigger action in physical world return result

Synchronous operation returns result immediately

Asynchronous operation result(s) later

abnormal rainfall as event

Intermittently connected device only asynchronous operations possible device not guaranteed to be connected when

operation invoked

8

Defining Device DatabaseDevice DBMS vs. Traditional

Relational DBMS Device vs. Data collection

Computed vs. stored values

Distributed information sources Data needed not available

locally May not even be available

remotely for intermittent Long-running queries.

Not modelled by traditional DBMS

Solution Base device relations

One record for each device

Virtual Relations Records partitioned

over distributed nodes Includes results of

device operations Extended query

language over virtual relations

9

Base Relations

Collection of devices of a particular type.

one record per device attributes

device id X coordinate Y coordinate

ID X Y

10

Virtual RelationsPer device function

Attribute for each function

argument result global timestamp of

result device id

New record added for each new result

Append-only relation Each device

contributes to part of relation

a1 Valam TSID

f(a1,…, am ): T

11

ExampleRFSensor One function

getRainfallLevel() : int Base relation

RFSensors Virtual relation

VRFSensorsGetRainfallLevel

Value TS Device ID

RFSensors

VRFSensorsGetRainfallLevel

Device ID X Y

12

For next four hours, retrieve every 30 seconds, rainfall level of each sensor if it is greater than 50mm.

Query Q:

SELECT VR.value

FROM RFSensors R, VRFSensorsGetRainfallLevel VR

WHERE R.ID = VR.ID AND VR.value > 50 AND

AND $every(30)

Run Q for 4 hours

200 devices

R cardinality = 200

VR cardinality = 480

Long running queries

join selection

AND R.X = ...

IN ...

Why Join?

13

Execution Strategies

R Materialized VR

Q

No local knowledge

R VR VR VR

VR.value > 50R.ID = VR.ID

Val every 3O min

Local rate info, remote Q

R VR VR VR

if R.ID = VR.ID

VR.value > 50

Val every 3O min

Local Join, Remote Selection

R VR VR VR

R.ID = VR.ID

if VR.value > 50

Val every 3O min

Local Selection, Remote Join

14

Performance Metrics

Traditional Throughput Response time

Long query running time implementation independent

Sensor specific Resource usage

network, power Reaction time

Production to consumption time

15

Power Usage Components

CPU Memory access Sending message Sending Nbytes

Cost in joules = Wcpu*CPU + Wram*RAM + Wmsg*Msg + Wbytes*NBytes

16

Centralized Warehouse

No local knowledge Single location

monitors all sensors Queries are sent to

site

Works for historical queries

Wastes resources for long running queries Irrelevant sites Higher rate than

necessary What to monitor for long

running queries Camera direction?

Centralization Workload Bottleneck

17

Distributed Device Database All sensors together

form a distributed device database system

Individual nodes sense on demand do part of query

processing

Better resource utilization

Know what to monitor

Historical? based on some

long-running query

18

Remote Query Evaluation

All sensors send data

No relations sent to sensors

R VR VR VR

VR.value > 50R.ID = VR.ID

Val every 3O min

Local rate info, remote Q

19

Local Join

R VR VR VR

if R.ID = VR.ID

VR.value > 50

Val every 3O min

Local Join, Remote Selection

Only joined sensors send value

Whole relation send to device communication

overhead

20

Local Selection Only selected

sensors send data Whole relation not

send to device best approach

R VR VR VR

R.ID = VR.ID

if VR.value > 50

Val every 3O min

Local Selection, Remote Join

21

Composition Techniques Query approach

Data of sensors aggregated together Alternatives

Operations of devices aggregated Results of one device affect operations of

another

22

Automatic Composition Domains ICrafter and Local Work

Device pipes Camera/Scanner to printer

Proxy based connection compression

Adapter based connection transcoding

Composite commands DoAll

• Turn on all VCRs

Do Related• Turn on projector• Dim lights

23

ICrafter ApproachSystem selects

composer based on interfaces of components and registered interface patterns

DataP

roducer

DataC

onsumer PowerSwitchPowerSwitch

DataProducer, DataConsumer ProducerConsumerPipe

Interface Pattern “Generic” Composer

PowerSwitch PowerSwitchAll

24

Selecting Composer

Pattern described using “regular expressions” single producer consumer multiple producer consumers

Client requests composer for a set of devices Returns composer for each subset of devices

with matching pattern Composer provides user-interface for invoking

composite commands Switch on all lights Drag producer to consumer or vice versa

25

ICrafter: A Service Framework for

Ubiquitous Computing

Environments Shankar Ponnekanti, Brian Lee, Armando Fox, Pat Hanrahan, Terry Winograd (Stanford Univ.)

26

Aggregation

May seem individual UI’s can be combined. However ….

27

UI(S1+S2) != UI(S1)+UI(S2)

Good for individual operations Clumsy for compound operation: snap-and-

display

28

UI(S1+S2) != UI(S1) + UI(S2)

Transfer = snap and display in one click

29

Where We Stand

Suppose n services, m appliances and w workspaces O( F(n) × m × w ) UI’s F(n) is some combinatorial function of n

Existing ad-hoc interaction systems: Jini, UPnP, Hodes et al (Mobicom97, USITS99), Roman et al (WMCSA00)

Mostly focus on appliance heterogeneity

30

Generators: Handling Aggregation

Generators for multiple services Eg. {Camera, Display}, {Camera,

Display, Display} Generators for service patterns

Eg. {Camera, Display+}, {Projector* } Generators for service interface

patterns Eg. {DataProducer, DataDisplayer+}

31

Simplified Example

Request for {Camera, Display} Matches Camera generator, Display generator, and

{DataProducer, DataDisplayer} generator

32

Simplified Example

Aggregated using panels

33

Outline

What? Why? How?

34

Bootstrapping

35

Future Work

Authentication and synchronization Aggregation: more intelligent generator

set selection: Eg. Eliminate a {DataProducer,

DataDisplayer} generator if a {Camera, Display} generator exists

36

Interactive Workspaces Project Info

Some software already available Major release (including ICrafter) due

this month For software and other info:

http://iwork.stanford.edu/

http://iwork.stanford.edu/projects/iwork/software

37

Problems of ICrafter

Must manually connect devices select from a list drag and drop Automatic continuous like Unix pipes?

Equivalent of standard input and output? Interface/Class-based composition

rather than programming pattern Getter/setter

38

Problems of Interface-based Approach

Type of Data in Interfaces? Generic Object

Truly generic composer possible One composer for all producer/consumer pairs

• Rather than camera/printer, temperature sensor/aggregator pairs

Errors possible• Temperature piped to printer

Programmer-defined Type checking possible Need programmer-defined composers.

39

Problems of Interface-based Approach

Number of DoAll Composers At least one for each interface

Number of Interfaces Implemented? One of for every operation

power, play can share composers for different devices

• can turn on all cameras and VCRs

too many interfaces and associated composers One of each service

limited sharing of composers need one for each combination of device types

40

Summary of Problems of Interface-based Approach

Explosion of # of composers In producer/consumer connections

for each consumer/producer data type In do-all composition

if interface = operation• for every operation

if interface = service• for every service combination with overlapping operations

No automation really supported in composing Automation only in looking up and

“composing”composers No support for proxy and adapters.

41

Alternate: Pattern-based approach

Pattern-based vs. interface- based protocols Pattern-based

must follow conventions

42

DoAll Composition

capture()

on()on()

capture()

Do all: on() Do all: capture()

on()

43

Method-based doAll algorithm Find operation headers of all component devices For each operation header

create a doAll operation that invokes device-specific implementation of header on each device providing it.

Accidentally combining unrelated commands with same signature foo(), compute(), doIt(), mutate()

Not combining related commands with different signatures power() powerVCR() and on()

non combinable commands() serialize()

44

Overriding Default doAll()

doAll on():

vcr.powerVCR(), tv.powerTV() and camera.turnOn()

Access methods

on = power.Device.()

serialize = serialize()

Disable doALL

serialize()

Access methods

on = turnOn()

serialize = serialize()

Disable doALL

serialize()

45

doAll Access-Method Algorithm

Find access-method headers of all component devices For each header

create a doAll operation that invokes an implementation of header on each device providing it that does not disable its compositions

46

Producer-Consumer Composition Much harder problem

how to regularize producers and consumers? Enumerate and Generalize Examples

Continuous vs. discrete through composer Unix pipes of continuous data vs. ICrafter connection of discrete

data Continuous direct connection

push-pull based continuous proxy based connection

push-pull based continuous adapter based connection

push-pull based Direct connection vs. connection through composer

47

interface PictureProducer {

Image capture();

...

}

interface PictureConsumer {

void print (Image p);

…

}

Discrete: Single producer, single consumer

Connect (camera, printer)

new operation: printAndCapture

48


Image capture();

...

}



…

}

Discrete: Multiple producer, single consumer

Connect (cameras, printer)

new operation: captureAllAndPrint

49


Image capture();

...

}



…

}

Discrete: Single producer, multiple consumers

Connect (camera, printers)

new operation: captureAndPrintAll

50


Image capture();

void setPictureDestination(PictureConsumer c);

void sendToPictureDestination();

...

}



…

}

Continuous Push-based Connection

Connect (camera, printer):

camera.setPictureDestination(printer);

51


void capture();

void addPictureDestination(PictureConsumer c);

void sendToPictureDestinations();

...

}


void print (…);

…

}

Pushing to multiple destinations

Connect (camera, printer):

camera.addPictureDestination(printer);

52


void setPictureSource(PictureProducer c);

void getFromPictureSource();

...

}


void capture (…);

…

}

Direct Pull-based Continuous Connection

connect(camera, printer):

printer.setPictureSource(camera);

53


void addPictureSource(PictureProducer c);

void getFromSources();

...

}


void capture (…);

…

}

Pulling from multiple sources


printer.addPictureSource(camera);

54


void print();

...

}


void addPictureDestination(PictureConsumer c)

void capture (…);

}

Proxy based connection

interface PictureCompresser extends PictureConsumer {


void compress();

...

}

registerProxy(ACompressor)


compresser new ACompresser();

compresser.addPictureDestination(printer);


55



void getFromSources();

...

}


void capture (…);

…

}

Proxy-based pull connection

interface PictureCompresser extends PictureProducer {


void compress();

...

}




compresser.addPictureSource(camera);

printer.addPictureSource(camera);

56


void print();

...

}


void addPictureDestination(PictureConsumer c)

void capture (…);

}

Proxy vs. Adapter: Proxy

interface PictureCompresser extends PictureConsumer {


void compress();

...

}




compresser.addPictureDestination(printer);


57

interface BWPictureConsumer {

void print();

...

}

interface ColorPictureProducer {

void addPictureDestination(ColorPictureConsumer c)

void capture (…);

}

Proxy vs. Adapter: Adapter

interface PictureTranscoder extends ColorPictureConsumer {

void addPictureDestination(BWPictureConsumer c);

void transcode();

...

}


transcoder new APictureTranscoder();

camera.addPictureDestination(transcoder);

transcoder.addPictureDestination(printer);

58

Implementation of Continuous Connection

Motivated by composition of components of Collaboration Infrastructure called Collaboration Bus

Direct, Proxy based, and Adapter based continuous connections

Integration of registration/discovery and composition

Based on notion of facets (like pins on a chip)

Munson and Sunku

59

Facets public void add<FacetName>(<FacetType>) {...} [public remove<FacetName>(<FacetType>) {...}]

[public Enumeration enumerate<FacetName>() {...}]public void set<FacetName> (<FacetType>) {...}

[public <FacetType> get<FacetName> () {...}]public <FacetType> <facetName>;

FacetName (FacetType)

objectAssign (source, facet, destination)

source.addFacet(destination)

source.setFacet(destination)

source.facet = destination

60

Bi Directional Connect

connect (camera, pictureDestination, printer, pictureSource)

assign (camera, pictureDestination, printer) assign( printer, pictureSource, camera);

PictureDestination (PictureConsumer)

camera

printer

PictureSource(PictureProducer)

61

Uni Directional Connect

connect (camera, null, printer, PictureSource) assign (printer, pictureSource, camera)

camera

printer


62

Uni Directional Connect

connect (camera, pictureDestination, printer, null)

camera.setPictureDestination(printer)


camera

printer

63

Multiple Communication Primitives

May wish to pass reference to Object Socket Multicast channels

Pattern getReferenceFor<FacetName>() If this method exists

return value used instead of object defining the facet

64

Multiple Communication Primitives

connect (camera, PictureDestination, printer, PictureSource) assign (camera, PictureDestination,

printer.getReferenceForPictureSource()) assign( printer, PictureSource,

camera.getReferenceForPictureDestination());


camera

printer


65

Integrating connection and discovery

Connect (printer, PictureDestination) Integrate discovery and connection Discover an object whose type can be assigned to facet

Discovered objects must have been registered Registries distributed and connected to each other

Must now find peer facet public Facet get<FacetName>DestinationFacet().

Make bi-directional connect if such a facet exists, else unidirectional connect

66

Integrating registration and connection

Register(camera) For each auto connect facet, f

Connect (camera, f) Facet is auto connect if

member of AutoConnectFacetNames property Or if class of object

Implements AutoConnectAll interface

67

Proxy-based ConnectPictureDestination (PictureConsumer)

camera

printer


PictureDestination(PictureCosnumer)

Proxy


68

Proxy-based connectregisterProxy(p, “Proxy/facet1/type1/facet2/type2”)

connect (s, fs, d, fd)p = findProxy(“Proxy/fd/typeOf(s)]/fs/typeOf(d)”)

If (p != null)

connect (s, fs, p, fd), connect (p, fs, d, fd)

else if not (direct_connect(s, fs, d, fd))

adapterConnect(s, fs, d, fd)

;

69

Adapting the TypePictureDestination (ColorPictureConsumer)

camera

printer

PictureSource(BWPictureProducer)

PictureDestination

(BWPictureConsumer)Adapter

PictureSource

(ColorPictureProducer)

70

Adapting the FacetColorPictureDestination

(ColorPictureConsumer)

camera

printer

BWPictureSource

(BWPictureProducer)

BWPictureDestination

(BWPictureConsumer)Adapter

ColorPictureSource


connect (camera, ColorPictureDestination, printer, BWPictureSource) Cannot derive adapter facet names from

connect.

71

Unidirectinal: Adapting the FacetColorPictureDestination


camera

printer

BWPictureDestination

(BWPictureProducer)Adapter

connect (camera, ColorPictureDestination, printer, BWPictureSource) Can derive adapter from connect, but not

opposite facet

72

Adapting the TyperegisterProxy(p, “Proxy/facet1/type1/facet2/type2”)

adapterConnect (s, fs, d, fd)p = findProxy(“Proxy/fd/*/fs/*”)

connect (s, fs, p, fd), connect (p, fs, d, fd)

73

Adapting the FacetregisterProxy(p,

“Proxy/facet1/type1/facet2/type2/pfacet1/pfacet2”)

adapterConnect (s, fs, d, fd)

p = findProxy(“Proxy/fd/*/fs/*/pf1/pf2”)

connect (s, fs, p, pf1), connect (p, pf2, d, fd)

74

Creating proxiesregisterProxy(p,

“Proxy/facet1/type1/facet2/type2/pfacet1/pfacet2”)

registerProxy(pClass, “Proxy/facet1/type1/facet2/type2/pfacet1/pfacet2”)pClass instantiated automatically

registerProxy(pFactory, “Proxy/facet1/type1/facet2/type2/pfacet1/pfacet2”)

special method in pFactory to create proxy

75

Hybrid of Discrete and Continuous

Example set time of newly turned on vcr to that of clock

composite command: replicate clock time system calls getter method for clock property and

then setter method for vcr property when vcr service registered

Can generalize to arbitrary properties synchronize TV channels in a bar

user enters synchronize channel mode composer listens for channel properties and

synchronizes them Roussev’s coupler research valid here

76

Domain-specific Device Composition

Example Compose multiple devices, possibly of

different users, so that they work together to show a multimedia web presentation.

Websplitter Based on XML

77

WebSplitter: A Partial-View XML Framework For

Multi-Device Collaborative Web

BrowsingProfessor Rick HanDepartment of Computer ScienceUniversity of Colorado at [email protected]

78

Outline

A High-Level View of Pervasive Computing

WebSplitter Motivations Per-Device Partial Views Per-User Partial Views

WebSplitter’s XML Framework Demo Prototype

79

Motivation: Enabling A Pervasive Computing

World

80

WebSplitter Motivation #1: Generate Partial Views Per

Device Scenario: Web presentation in lecture hall controlled

by a wireless PDA

Lecturer

• Each device receives partial view• device’s capability• intended function

• Leverage other dev’s

• Composite view

Lecturer

81

WebSplitter Motivation #2: Partial-View Collaborative

Presentations Scenario: Online Web presentation to

audience – what you see is ~ what I see

Lecturer

Participant 2

Participant 1

82

Partial Views Per User WebSplitting generates subsets of a Web page

for each user, i.e. partial views

• Partial views limit access to components– To hide information– To prevent misdirection of a presentation

• Partial view scenarios:– Teacher giving presentation to students

(distributed)• Customized learning based on individual skill

levels, i.e. (teacher, pupil1, pupil2, …), not merely (teacher, students)

– Adversarial collaboration, e.g. lawyers

83

WebSplitter: Joint Multi-User Multi-Device Presentations

Scenario: Web presentation to online audience and a lecture hall via a remote control PDA

Lecturer

Lecturer

Participant 2

Participant 1

84

Multipleusers

MultipledevicesLecturer

Web

Server

SessionPresenter

WebSplitter

WebSplitter’s Architecture

WebSplitter Middleware

PolicyFile

---------------------------------------------

85

XML Framework:How does WebSplitter

use XML? Web pages are authored in XML

Tags correspond to components on Web page

Easy to manipulate tags & groups of tags – nested hierarchy

use XSL to filter/insert/delete tags “Policy file” implements partial views

defines tag mapping policies: which users and which devices have access to which XML tags

Written in XML External annotation file containing

authorization

86

Example: Splitting an XML Web page

<?xml version="1.0" encoding="UTF-8"?>

<sound xml:link="simple" href="music.wav"/>

<head>

<title>The online Shopping company</title>

<logo xml:link="simple" href="logo.gif"/>

<description>The most reliable and the best prices online</description>

</head>

WebSplitter

<HTML>

<BODY>

<center>

<table width="100%" cellpadding="0" cellspacing="0">

<tr>The online Shopping company</tr>

<tr><IMG SRC="logo.gif"></tr>

<tr>The most reliable and the best prices online</tr>

</table>

</center>

</BODY>

</HTML>

Title, Description, and GIF Logo

HTML to PocketPC

<?xml version="1.0" encoding="UTF-8"?>

<sound xml:link="simple" href="music.wav"/>

<head>

<logo xml:link="simple" href="logo.gif"/>

</head>

WAV Music and GIF Logo

XML to laptop

87

XML Policy File:Filtering Tags by User

Groups Define user privilege

groups <cmdb:group name=“ “

password=“ “ input_permitted=“ “>

Each privilege group receives a subset of tags <cmdb:taglist>

<cmdb:group name = ”lecturer” password = ”YYYYY” input_permitted = “yes”><cmdb:taglist>

presentation, head, title, nav_bar, slide, picture

</cmdb:taglist></cmdb:group>

<cmdb:group name = ”students” password = ”ZZZZZ” input_permitted = “no”><cmdb:taglist>

presentation, slide, picture

</cmdb:taglist></cmdb:group>

88

XML Policy File:Filtering Tags by Devices

Within a group, map subsets of tags to devices <cmdb:device

name=“ “> Standardizing

names

<cmdb:group name = ”lecturer” …<cmdb:device name = “IE||

Netscape”><cmdb:taglist> presentation,

head, title, nav_bar, slides, picture

</cmdb:taglist></cmdb:device><cmdb:device name =

“PocketIE”><cmdb:taglist> presentation,

head, title, nav_bar, notes </cmdb:taglist>

</cmdb:device></cmdb:group>

<cmdb:group name = ”students” ……………</cmdb:group>

89

XML Policy File: Filtering Tags By User &

By Device Define user

privilege groups Each privilege

group receives a subset of tags

Within a group, map subsets of tags to devices

<cmdb:group name = ”lecturer” …<cmdb:device name = “IE||

Netscape”><cmdb:taglist> presentation,

head, title, nav_bar, slides, picture

</cmdb:taglist></cmdb:device><cmdb:device name =

“PocketIE”><cmdb:taglist> presentation,

head, title, nav_bar, notes </cmdb:taglist>

</cmdb:device></cmdb:group>

<cmdb:group name = ”students” ……………</cmdb:group>

90

Implementation: WebSplitter Demo

Bluetooth

Bluetooth LAN Access

Point

EudoraWebLaptop2

WebSplitterproxy

Ethernet LAN

Laptop1

Web Server

XML Source & Policy File

2 privilege groups in policy file

91

WebSplitter Prototype Demonstrates:

Splitting to different browsers on different devices XML source Web page XML policy file

2 privilege groups Client-side component

on Palm, WinCE, Netscape

Static service discovery

92

Issues raised by WebSplitter

Integrate with Producer-Consumer Notion Splitting an output ~ connection to

multiple consumers Splitting an input ~ connection to multiple

producers Automatic vs. Declarative

93

Composition Techniques

Fully automatic doAll command pipes false positives/negatives possible

Semi-automatic disabling/enabling doAll composition Specifying proxies

Declarative WebSplitter XML file

94

Semi-Automatic vs. Declarative Fine line between semi-automatic and declarative Simple parameters (forms) rather than

specification language. Semi-automatic overrides fully automatic

Composing Devices Prasun Dewan Department of Computer Science University of North Carolina...

Documents

Transcript of Composing Devices Prasun Dewan Department of Computer Science University of North Carolina...