Tree Graph Views for a Distributed Pervasive...

Département des Sciences Informatiques

Context TGV : Tree Graph View TGV for a Pervasive Environment PADAWAN Prototype Conclusion

Tree Graph Views for a Distributed

Pervasive Environment

Tuyet Tram DANG NGOC ([email protected])Nicolas TRAVERS ([email protected])

ETIS Laboratory, University of Cergy-Pontoise



Outline

1 Context

2 TGV : Tree Graph View

3 TGV for a Pervasive Environment

4 PADAWAN Prototype

5 Conclusion



1 ContextMotivationData Integration Issues



4 PADAWAN Prototype

5 Conclusion



Motivation

Context

Integrate the pervasive vision of computing devices in order toallow any user anywhere in the world to query anything fromanywhere in the world.

Primergy

Doctor

Officer

Expert

Router

802.11

AP

IP

Sensor Network

Sink

Ad−Hoc Network

Internet

Fireman

Fireman

��

��

PADAWAN

PADAWAN

LDAP

SGBDWeb Server

News Feeder

Proxy

Emergency Unit

Proxy

ClientGIS



Motivation

Context

How to organize the graph, so a user with any type of profilecan connect on any node of the graph and access to otherdata that are dissiminated on nodes of the graph.

Nodes

Heterogeneousautonomous datasources

Proxies (both client andsource)

Heterogeneous clientapplications

Heterogeneous Network Links

S

P

C

S

S

S

S

S S

S

C

C

C

CS

S

C

C

C

S

S

S

S

CP

C

CP

P

P

S

Data Source

Client Terminal

Wireless Link

Wire Link

ProxyP

Legend



Data Integration Issues


SourceHeterogeneous Data(relational,semi-structured)→common querylanguage and model :XML/XQuery

Database

Web

Site

FROM person p, car c

WHERE age>60

AND p.id=c.driver

AND c.color LIKE "RED"

SELECT p.name, p.address

ORDER BY p.address

name address

Well

Doeuf Cergy

Paris

GET http://example.com/persons.xml

GET http://example.com/car.xml

<person>

<name>Well</name>

<fname>Rose</fname>

<age>80</age>

<address>Paris</address>

</person>

<person>

<name>Cover</name>

...





SourceHeterogeneous Data(relational,semi-structured)→common querylanguage and model :XML/XQuery +Wrapper Database

Web

Site

XMLXQuery

Wrapper

XMLXQuery

Wrapper





SourceAutonomous Sourcesqueries more or lesssupported

Database

Web

Site

FROM person p, car c

WHERE age>60

AND p.id=c.driver

AND c.color LIKE "RED"

SELECT p.name, p.address

ORDER BY p.address

name address

Well

Doeuf Cergy

Paris

GET http://example.com/persons.xml

GET http://example.com/car.xml

<person>

<name>Well</name>

<fname>Rose</fname>

<age>80</age>

<address>Paris</address>

</person>

<person>

<name>Cover</name>

...

Here are the whole documents,

make the processing yourself !

Here are the exact results





SourceAutonomous Sourcesqueries more or lesssupported , internalinformation not available(cost model, statistics,schema)→Source Description Database

Database

Wrapper Wrapper

*card(T2)*CPU(restr)*TIO(restr)=0.0001ms

TIO=0.000282ms

CPU(Join)=card(T1)

CPU

*card(T2)*CPU(restr)*TIO(restr)=0.0001ms

TIO=0.000282ms

CPU(Join)=card(T1)

CPU

Cost model ? Cost model ?

Cost model ?

Cost model ??

Tps(statistic)=3ms (Avg)Tps(Calibration)=4*CPU(Join)





Client Heterogeneityaccess permission, userpreferences, terminalcapabilities→Client profile + View





Data integrationevaluate query ondistributed node→mediation + Sourcedescription Wrapper

Source

Wrapper

Source

Wrapper

Source

Wrapper

Source

Mediator

Client





Source

Heterogeneous Data (relational, semi-structured)→common query language and model : XML/XQuery +WrapperAutonomous Sources queries more or less supported,internal information not available (cost model, statistics,schema)→Source Description

Client Heterogeneity access permission, user preferences,terminal capabilities→Client profile + View

Data integration evaluate query on distributed node→mediation + Source description



1 Context



4 PADAWAN Prototype

5 Conclusion



TGV : Tree Graph View [ Travers and Dang-Ngoc 2004-2007 ]

declare function local:f($doc as xs:string) as element()

{for $x in (doc(?rev.xml?)/review|doc(?$doc?)/catalog)

[. contains(?Robin Hobb?)]/book/[.//price > 15]

where

some $y in $x/comments

satisfies contains ($y, ?Excellent?)

order by $x/@isbn

return

<book>

{$x/@isbn} <price>{$x//price/text()}</price>

{ if (count($x/title) > 2)

then

{ for $z in doc(?books.xml?)/book

where $z/@isbn = $x/@isbn

return <title>{($z/title)[3]}</title>

}else<title/>

} </book>

}




XQuery has a (too) rich syntax :XPathConstraints, FilterQuantifiersDocument constructionNesting, Aggregates, SortsConditional operators, Set operatorsSequences, Function

Hard to parse, process and identify subpart and dependancies.

XQuery XML

Query

Processor




Model based on Tree Pattern matching that :

Supports Full untyped XQuery [ WEBIST2007 ]

Provides an intuitive representation [ DASFAA 2007 ]

Designed for distributed environment [ IBIS Journal 2006 ]

Support for optimisation and for evaluation [ ICEIS 2007 ]

logicalTGV

AnnotatedTGV

XQuery XML

Application

profileOptimization Evaluation

Rule Processing

XQueryCanonized

TGVEvaluationannotation



TGV : Example

”list every buildings occupied by more than 100 inhabitants, and for each, get the districtand the list of maximum temperature measured by the sensors located in the samedistrict.”

for $a in /buildings/buildingwhere $a/description/inhabitant > 100return

<districtMonitoring><location> {$a/district} </location><temperatures>

{ for $b in //sensorwhere

$b/deploymentArea/district = $a/districtreturn

<temperature>{$b/max temp}

</temperature>}</temperatures>

</districtMonitoring>

<districtMonitoring><location> Yellow Lake </location><temperatures>

<temperature> 14 </temperature></temperatures>

</districtMonitoring><districtMonitoring>

<location> Green Valley </location><temperatures>

<temperature> 163 </temperature><temperature> 25 </temperature><temperature> 43 </temperature>

</temperatures>




TGV : Example

”list every buildings occupied by more than 100 inhabitants, and for each, get the districtand the list of maximum temperature measured by the sensors located in the samedistrict.”

for $a in /buildings/buildingwhere $a/description/inhabitant > 100return

<districtMonitoring><location> {$a/district} </location><temperatures>

{ for $b in //sensorwhere

$b/deploymentArea/district = $a/districtreturn

<temperature>{$b/max temp}

</temperature>}</temperatures>


$a $b

> 100

temperatures

location

districtMonitoring

description

building

buildings

inhabitant

sensor

district

deploymentAreadistrict

temperature

$t

max_temp=



TGV : Generic Annotation for any type of

Information

A layer per informationtype

Information on any set ofTGV elements (includingannotation)

Any granularity, possiblerecovering

Generic Annotation (ex.several type of cost modelwith complex formulas)

4) Logical TGV

description

building

inhabitant

temperatures

locationdeploymentArea

deploymentArea max_temp

location

$a $b sensor

sensor$t

> 100

buildings

=

district

annotation

3) Physical TGV

annotation

with cost2) Physical TGV

annotation

1) Physical TGV

with evaluation

with location

description

building

district deploymentArea

$a $b sensorbuildings

=

deploymentArealocation

temperatures

> 100

inhabitant

sensor$t

max_temp

location

description

building

inhabitant

district

temperatures

location

deploymentArea max_temp

location

$a $b sensor

sensor$t

> 100

buildings

=

deploymentArea

temperatures

location

$t

buildings

=

deploymentArea

sensor

max_tempdeploymentArea

$b

sensorlocation

> 100

inhabitant

$a building

districtdescription

Yellow Lake | 1678

Rhode Forest | 1986

Green Valley | 82761

Yellow Lake | 14

Green Valley| 163

Green Valley| 25

Green Valley| 43

C2=cost_card*0.02cp=max(C1, C2)

cost=cp+Op

CS1=Cost(S1) Cost(S2)C1=CS1*sel*CS2*IO

Source1

Mediator

Source2, Source 3

Mediator



TGV : Rules

Rule

Annotated TGV Pattern → Annotated TGV Patterncondition” Conclusion”

Rules for :

Local Transformation

Optimization

Local Evaluation

Distributed Evaluation



TGV : Rules

=

card ($x)*card($y)*CPU_unit

CPU_unit=0,02

=

Rules for :


Optimization

Local Evaluation




TGV : Rules

person

name age refsrc person

name age refsrcFR_SRC2

card

50,000

Rules for :


Optimization

Local Evaluation




TGV : Rules

=$x $y

bind−join

card ($x) > card ($y)

=

bind−join

$y $x

card ($x)card ($y)

Rules for :


Optimization

Local Evaluation




TGV : Rules

= =$a $b $a $b

join (eval ($a), eval ($b))

Rules for :


Optimization

Local Evaluation




TGV : Rules

Rules for :


Optimization

Local Evaluation




1 Context


3 TGV for a Pervasive EnvironmentTGV EvaluationTGV*

4 PADAWAN Prototype

5 Conclusion



TGV Evaluation

TGV Mobile Agent

Initiated by the client

Evaluated on the fly

Parallel evaluation :

cloning the TGV on somenodes.evaluation rules located onthe node.

Leaves of the recursion tree onsource node. Source node fill theevaluation annotation layer.

P

S

S

S

SS

S

C

S

SP

TGV

TGV

TGV

TGV

TGV



TGV Evaluation

TGV Mobile Agent

Initiated by the client

Evaluated on the fly

Parallel evaluation :

cloning the TGV on somenodes.evaluation rules located onthe node.

Leaves of the recursion tree onsource node. Source node fill theevaluation annotation layer.

P

S

S

S

SS

S

C

S

SP

TGVeval

TGVeval

TGVeval

TGVeval

TGVevalTGV

eval



TGV Evaluation

TGV : Evaluation

Evaluation rules :

Evaluation onSourceWrappers

Join

Nest

Finalprojection

$a $b

> 100

temperatures

location

districtMonitoring

description

building

buildings

inhabitant

sensor

district


temperature

$t

max_temp=

Yellow Lake | 14Green Valley | 163Green Valley | 25Green Valley | 43

Yellow Lake | 678Rhode Forrest| 1986Green Valley | 82761



TGV Evaluation

TGV : Evaluation

Evaluation rules :


Join

Nest

Finalprojection

$a $b

> 100

temperatures

location

districtMonitoring

description

building

buildings

inhabitant

sensor

district


temperature

$t

max_temp=

Green Valley | 82761 | 163Green Valley | 82761 | 25Green Valley | 82761 | 43

Yellow Lake | 678 | 14



TGV Evaluation

TGV : Evaluation

Evaluation rules :


Join

Nest

Finalprojection

$a $b

> 100

temperatures

location

districtMonitoring

description

building

buildings

inhabitant

sensor

district


temperature

$t

max_temp=

Green Valley | 82761 | 163Green Valley | 82761 | 25Green Valley | 82761 | 43


14

163

25

43



TGV Evaluation

TGV : Evaluation

Evaluation rules :


Join

Nest

Finalprojection

$a $b

> 100

temperatures

location

districtMonitoring

description

building

buildings

inhabitant

sensor

district


temperature

$t

max_temp=

Green Valley | 82761 | 163 | | 25 | | 43




TGV*

TGV* : Sources Description

How to annotate the TGV?→TGV* to describe sources inan exhaustive manner.

Dataguide : source part(STP) of the TGV withannotation

Annotations

Set of rules (for operators)

TGV*

person

adressstreettownname

car

idimmmodel

appbrandcol

Rules

Join (cost−memoryà

Restriction (cost exec_time)

Join (cost−memory)

personn

id

immmodelbrandcol

name

person

town

id appnameadress

street modelimm

car

brandcol

car

app

streettown

adress

Cost (exec_time)

Dataguide

Cost (memory)

Location

personn car

adress

streettown

id

immmodel

app

brandcol

name



1 Context



4 PADAWAN PrototypeNodes and Agents

5 Conclusion



Nodes and Agents

PADAWAN : a Multi-Agent Plateform

Client Oracle Wrapper

Sensor WrapperRules Forwarders

Wrappers Rules

Forwarders

Web Site WrapperClientForwarders

Node 0

Node 2Node 1

Node 3

Node 4

Node 5Node 6

Forwarders

Node 7

ForwardersRulesCache

1 2

3

4

5

6

TGV *Profile

TGV

TGV *

TGV *

TGV

ProfileProfile

TGV *

It is the agentslocated on thenode that definethe functionalitiesof the node.

Client Node : Client and Presentation Agents

Source Node : Wrapper Agent

Proxy Node : Cache, Router, and Rules Agents

Repository Node : Agent Library

Prototype PADAWAN (a Plateform for All Devices Accessingthe World And Neighbourhood) on http ://padawan.ensea.fr/



1 Context



4 PADAWAN Prototype

5 ConclusionBibliographySimulation



Conclusion

TGV Mobile Agent

Full untyped-XQuery evaluation

Evaluation through the Multi-Agents PlatformPADAWAN graph

Dynamic Optimization

Current Work : TGV* Mobile Agent

describe data

generic enough to describe all information known on asource.

abstract TGV*



Bibliography

Publications on TGV and PADAWANDang-Ngoc, T.-T. and Travers, N. (2007).

Tree graph views for a distributed pervasive environment.In the 1st International Conference on Network-Based Information Systems (NBIS), Regensburg, Germany.

Liu, T., Dang-Ngoc, T.-T., and Laurent, D. (2007).

Cost framework for a distributed semi-structured environment.In in the proceedings of the International workshop Database Management and Application over Networks -DBMAN (APWeb/WAIM Workshop), Huangshan, China.

Travers, N. and Dang-Ngoc, T.-T. (2007).

An extensible rule transformation model for xquery optimization.In The 9th International Conference on Enterprise Information Systems (ICEIS), Madeira, Portugal.

Travers, N., Dang-Ngoc, T.-T., and Liu, T. (2006).

Tgv : an efficient model for xquery evaluation within an interoperable system.International Journal of Interoperability in Business Information Systems (IBIS), 3.ISSN : 1862-6378.

Travers, N., Dang-Ngoc, T.-T., and Liu, T. (2007a).

An efficient evaluation of xquery with tgv.In the 3rd International Conference of WEB Information Systems and Technologies (Web-IST), Spain.

Travers, N., Dang-Ngoc, T.-T., and Liu, T. (2007b).

Full untyped xquery canonisation.In in the proceedings of the International workshop on Emerging Trends of Web Technologies andApplications -WebETrends (APWeb/WAIM Workshop), Huangshan, China.

Travers, N., Dang-Ngoc, T.-T., and Liu, T. (2007c).

Tgv : a tree graph view for modelling untyped xquery.In the 12th International Conference on Database Systems for Advanced Applications (DASFAA), Thailand.



Simulation

Discrete Event Simulation

P1

S3

C1

C

S

S

P

PC

C

P2

S1

S2

r(C1,P1,P2)

r(C1,P1,P2)

r(C1,P1)

r(C1,P1)

4a−

SINK

r(C1)

R(C1,P1,P2)

R(C1,P1,P2)

R(C1,P1)

R(C1,P1)

R(C1)1−

2a−

2b−

6−

3c−

5a−

3a−

4b−

3b−

Sources alreadylocated located

route of the requestrandomly generated,using randomly 0-Nsources

time of service, andqueries generationparameterized



Simulation

Simulation

Mediation Architecture

Direct Access to sources

Bottleneck on mediator

nbclients = 3 nbproxy = 1 nbsources = 6

C3

C2

C1 S1

S2

S3

S4

S5

S6

P1

(a) Mediation Infrastructure



Simulation

Simulation

P2P Network

Each Peer is a mediator

Direct Access to sources

Load on each peer

Each peer knows how toresolve a whole request

nbclients = 3 nbproxy = 5 nbsources = 6

S1

S2

S3

S4

S5

S6

P2

P3

P4

P5

P1

C3

C2

C1

(b) Infrastructure PàP



Simulation

Simulation

Padawan Network

A peer has access to someother peers and eventuallyto some sources

Distribution of the queryevaluation

Evaluation depending onthe peers capabilities

Distributed waiting time

nbsources = 6nbproxy = 5nbclients = 3

C3

C2

C1S1

S2

S3

S4

S5

S6

P1

P2

P3

P4

P5

(c) Infrastructure PADAWAN



Simulation

Simulation Result

Times / Number of queries to process in the system 20 queriesgenerator (=clients), 100 sources, For P2P and PADAWAN :20 peers.

Mediator : Bottleneck

P2P Architecture betterthant PADAWANArchitecture

Linear

graph



Simulation

Simulation Result

Times / Number of queries to process in the system 20 queriesgenerator (=clients), 100 sources, For P2P and PADAWAN :20 peers.

Mediator : Bottleneck

P2P Architecture betterthant PADAWANArchitecture

Linear

But :Mean waiting time better inPADAWAN

graph-tw



Simulation

Simulation limitation

Opening connection delay

PADAWAN : few connection per proxy→all connections can be left opened.P2P : each peer can potentially reach any source→can’t maintain all opening/closing connection→consider connection delay

Nodes capabilities

In the major P2P networks, equivalent nodes with samecapabilities (or 2 or 3 categories with Superpeer andnormal peer)PADAWAN queries can be evaluated on nodes withdifferent capabilities

Tree Graph Views for a Distributed Pervasive...

Documents

Transcript of Tree Graph Views for a Distributed Pervasive...