Adaptive Content-based Routing In General Overlay Topologies
-
Upload
dara-holcomb -
Category
Documents
-
view
28 -
download
0
description
Transcript of Adaptive Content-based Routing In General Overlay Topologies
MIDDLEWARE SYSTEMSRESEARCH GROUP
Adaptive Content-based Routing In General Overlay Topologies
Guoli Li, Vinod Muthusamy
Hans-Arno Jacobsen
Middleware Systems Research Group
University of Toronto
http://padres.msrg.toronto.edu
Middleware 2008 @ Leuven, Belgium
MIDDLEWARE SYSTEMSRESEARCH GROUP
Distributed Publish/Subscribe
An acyclic overlay is sensitive to: Congestion Broker failures
Benefits of a general overlay: Routing around congestion and failures Handling imbalanced workloads
Publisher
Subscriber
Subscription
Publication
Advertisement
Subscriber
Applications Business process execution e.g., BPEL Business activity monitoring Service discovery and integration …
Middleware 2008 @ Leuven, Belgium
MIDDLEWARE SYSTEMSRESEARCH GROUP
Challenges With General Overlays
Subscriptions are routed in loops
Brokers receive duplicate subscriptions
Subscription copies exacerbate the problem
Same problem for publications
2
3 4
5
1
6
Adv 1
Adv 2
S X
S
S
Middleware 2008 @ Leuven, Belgium
MIDDLEWARE SYSTEMSRESEARCH GROUP
Agenda
Content-based routing protocol for general overlays Atomic and composite subscriptions Optimal publication routing
Evaluation Dynamic publication routing Adaptive composite subscription routing
Middleware 2008 @ Leuven, Belgium
MIDDLEWARE SYSTEMSRESEARCH GROUP
TID-based Approach
Each advertisement is assigned to a unique tree identifier (TID)
Each subscription has a TID predicate with a variable
2
3 4
5
1
6
Adv 1
Adv 2
XS
Middleware 2008 @ Leuven, Belgium
MIDDLEWARE SYSTEMSRESEARCH GROUP
Subscription Routing
2
3 4
5
1
6
Adv 1
Adv 2
XS: [class=stock][symbol=*] [TID=$Z]
At Broker 1:
Adv1:
[class=stock][symbol=IBM] [TID=Adv1]
Adv2:
[class=stock][symbol=HP] [TID=Adv2]
S matching Adv1:
[class=stock][symbol=*][TID=Adv1]
S matching Adv2:
[class=stock][symbol=*] [TID=Adv2]
SA2SA1
S
Middleware 2008 @ Leuven, Belgium
MIDDLEWARE SYSTEMSRESEARCH GROUP
Publication Routing
Each publication is assigned the TID of its matching advertisement e.g., p [class, stock][symbol,HP][TID,
adv_msg_id] Publications are routed:
Fixed TID routing: a publication is routed to subscribers along its advertisement tree.
Dynamic publication routing: a publication may be routed to subscribers across advertisement trees.
Middleware 2008 @ Leuven, Belgium
MIDDLEWARE SYSTEMSRESEARCH GROUP
Fixed TID Routing
Property No broker receives
duplicate publication messages
2
3 4
5
1
6
Adv 1
Adv 2
Sub
X
P
P
Middleware 2008 @ Leuven, Belgium
MIDDLEWARE SYSTEMSRESEARCH GROUP
Dynamic Publication Routing
Publication’s TID is changeable
Routing heuristic Util = Routput / Rsending
Property Changing a publication’s
TID while in transit will not change the set of notified subscribers.
2
3 4
5
1
6
Adv 1
Adv 2
Sub
X
P
Middleware 2008 @ Leuven, Belgium
MIDDLEWARE SYSTEMSRESEARCH GROUP
Advantages
Retains the publish/subscribe client interface Speeds up subscription and publication
matching Avoids duplicate subscriptions and
publications Routes publications dynamically across
multiple alternatives Enables routing around failures, congestion
and load imbalances
Middleware 2008 @ Leuven, Belgium
MIDDLEWARE SYSTEMSRESEARCH GROUP
Composite Subscription
AND
S1 S2
A composite subscription consists of atomic subscriptions linked by logical operators (e.g., AND, OR).
Composite subscription routing Topology-based routing Adaptive routing
e.g., CS= {[class=stock][symbol=YHOO][price>12]} AND
{[class=stock][symbol=MSFT][price<20]}
Middleware 2008 @ Leuven, Belgium
MIDDLEWARE SYSTEMSRESEARCH GROUP
Topology-based CS Routing
2
3 4
5
1
6Adv 1
Adv 2
7
8
9
CS={{S1 AND S2} ANDS3}CS
S2A2
S3A3
S1A1
CS’
Adv 3
CS’ ={S1 AND S2}
Broker 4 and 8 are the joint
point brokers
Middleware 2008 @ Leuven, Belgium
MIDDLEWARE SYSTEMSRESEARCH GROUP
Adaptive CS Routing CS’s joint points are determined according to
potential publication traffic, bandwidth, latency, etc.
2
1 3Adv 1
Adv 2
CS={S1 AND S2}
2
1 3Adv 1
Adv 2
CS={S1 AND S2}
Middleware 2008 @ Leuven, Belgium
MIDDLEWARE SYSTEMSRESEARCH GROUP
Cost Model Routing cost of CS
RC(CS)) = +
+
Subscription cardinality |P(S)| : The number of matching publicat
ions per unit of time.
|P(S)| =
|P(CS)| = |P(Sl)| + |P(Sr)| if op = or
subscription dest
Matching Engine
Routing Table
+
symbol=IBM B1
symbol=HP B2
input queue
output queue B1
output queue B2
Broker
Middleware 2008 @ Leuven, Belgium
MIDDLEWARE SYSTEMSRESEARCH GROUP
Adaptive CS Routing
2
3 4
5
1
6Adv 1
Adv 2
7
8
9
CS={{S1 AND S2} ANDS3}CS
S2A2
S3A3
S1A1
CS’
Adv 3
CS’ ={S1 AND S2}
CS’
Middleware 2008 @ Leuven, Belgium
MIDDLEWARE SYSTEMSRESEARCH GROUP
Evaluation
Setup Overlays of 32 brokers with different connection d
egrees Cluster (each node:1.86GHz, 4G) and PlanetLab Workloads:
Yahoo!Finance stock quote traces http://research.msrg.utoronto.ca/Padres/DataSets
Metrics End to end notification delay Network traffic
Middleware 2008 @ Leuven, Belgium
MIDDLEWARE SYSTEMSRESEARCH GROUP
Dense vs. Sparser Topologies
20%
4%
Note: The benefit is not proportional to the connection degree.
Middleware 2008 @ Leuven, Belgium
MIDDLEWARE SYSTEMSRESEARCH GROUP
Higher Publication Rate
stabilized
Middleware 2008 @ Leuven, Belgium
MIDDLEWARE SYSTEMSRESEARCH GROUP
With Broker Failures
1st failure
2nd failure
Middleware 2008 @ Leuven, Belgium
MIDDLEWARE SYSTEMSRESEARCH GROUP
Conclusions Enables routing around failures, congestion and
load imbalances Allows publications routing across alternative
paths Improves the notification delay by 20%
Enables flexible CS routing Reduces 80% publication traffic Improves the notification delay by 55%
Simplifies solutions for failure recovery and load balancing
Middleware 2008 @ Leuven, Belgium
MIDDLEWARE SYSTEMSRESEARCH GROUP
Questions?
A D R E SP
http://padres.msrg.utoronto.ca
Middleware 2008 @ Leuven, Belgium
MIDDLEWARE SYSTEMSRESEARCH GROUP
Effect of Subscriber Distance
Distance Fixed(ms) Dynamic(ms) Improvement
6 Hops 47.202 47.568 -0.78%
10 Hops 64.477 52.895 17.96%
12 Hops 74.416 60.598 18.57%
Max Diff 57.65% 27.39%
Middleware 2008 @ Leuven, Belgium
MIDDLEWARE SYSTEMSRESEARCH GROUP
Faster Matching with TIDs
Subscriptions are augmented with TIDs only once at the first broker.
Other brokers can route the subscription based on the TID alone.
Similar argument applies to publication routing.
Middleware 2008 @ Leuven, Belgium
MIDDLEWARE SYSTEMSRESEARCH GROUP
Advertisement Routing
Each advertisement forms a spanning advertisement tree
Duplicated advertisements are discarded by brokers
Each advertisement is assigned a unique tree identifier (TID) e.g., a [class,eq,stock]……[TID,eq,adv_msg_id]
Subscription Routing Table (SRT) A set of [advertisement, last hop]
Middleware 2008 @ Leuven, Belgium
MIDDLEWARE SYSTEMSRESEARCH GROUP
Subscription Routing
Each subscription has a TID predicate with a variable. e.g., s [class,eq,stock]……[TID,eq,$X]
The variable is bound to the TID of a matching advertisement
Publication Routing Table (PRT) A set of [subscription, {TID, last hop of
subscription }]