Dynamic Aggregation of Reservations forInternet Services

Roland BlessInstitute of Telematics, University of Karlsruhe

76128 Karlsruhe, GermanyPhone: +49 721 608 6413, Fax: +49 721 388097

bless@tm.uka.de

Abstract

Global availability of on-demand services with some level of guaran-teed quality-of-service (QoS) is still missing in the Internet today. TheDifferentiated Services Architecture achieves scalability in the data pathby subsuming all flows with a specific type of service into an aggregate.But in order to provide guaranteed end-to-end reservations, the man-agement of resource reservations by control entities (e.g. performingadmission control) has to be scalable as well.

The DARIS Architecture provides this essential linkage between scal-ability in packet forwarding path and scalability of QoS managementin the Internet. It comprises a novel concept of dynamic and hier-archical aggregation, which acts on the network level of autonomoussystems (AS). This relieves management entities in intermediate ASsof processing load by reducing their managed reservation states as wellas the number of signaling messages that have to be processed to aminimum. Moreover, novel and special support for aggregation by adedicated signaling protocol is provided.

1 Introduction

The Differentiated Services (often abbreviated as DiffServ or DS) ar-chitecture [2] promises a scalable solution to the concurrent need for

quality of service (QoS) support by several applications, and, will allowproviders a more competitive differentiation of service offerings. Thefundamental concept behind DiffServ is an aggregation of packet flowsin the data path: in contrast to the former Integrated Services (IntServ)approach [3], where every router distinguishes between individual end-to-end flows (i.e., a so called micro-flow that is characterized by proto-col, source and destination IP addresses and port numbers), DiffServrouters in the core network differentiate only between different forward-ing classes which are determined by a value in each packet’s header [1].Unavoidable complexity such as microflow classification and policing ispushed to the network edges, at which the number of forwarded flows isnot as high as in transit networks located in backbone and inner areasof the Internet, where many flows are aggregated into larger flows.

It is envisioned that an early deployment of DiffServ will use mainlystatically provisioned resources at first, whereas dynamically reservedresources for services should be added later on [2]. However, a majormissing piece for globally available on-demand services with an assuredquality-of-service from end to end is a scalable management structurewith a respective signaling protocol for requesting such services (a con-trol plane) from a domain of a provider. For the rest of the paper, it isassumed that QoS is provided by a pure DiffServ architecture from endto end.

Some future services, such as those which offer a guaranteed band-width for an end-to-end flow based on the Expedited Forwarding Per-Hop-Behavior [4], require admission control from end to end. Especially,this also includes the case where service requests have to be exchangedbetween domains. But per-flow admission control possesses obviouslypotential scalability problems in the control plane with respect to thenumbers of exchanged messages and reservation states. While manyexisting management approaches mainly concentrate solely on scalabil-ity within a single DiffServ domain, only few approaches address globalscalability in a multi-domain case. But, as described later, existingapproaches are not flexible enough.

In this paper, an approach for a scalable and integrated managementarchitecture is presented. The emphasis of this paper lies on the DARIS(Dynamic Aggregation of Reservations for Internet Services) concept to

achieve the required global scalability, and, on a related suitable andoptimized signaling protocol for this purpose.

Section 2 starts with a description of the overall management archi-tecture as well as the fundamental scalability problems. The DARISaggregation principle is described in section 2.1. Section 3 presents thenovel signaling protocol DMSP that has special support for aggrega-tion. Simulation results are presented and discussed in section 4. Thepaper closes with a summary and outlook on future work in section 5.

2 The DARIS Management Architecture

The DARIS management architecture follows others approaches thatare based on a separate resource management entity within a DiffServdomain. Letting dedicated entities instead of routers perform most re-source management functions, such as admission control, has severaladvantages: first, routers are relieved from carrying out these functionsin addition to routing and packet forwarding functions. Second, morecomplex functions such as policy-based admission control or such thatrequire keeping persistent state, e.g., accounting-related ones, shouldnot or cannot be built into routers due to lack of processing perfor-mance for these tasks or limitations of storing persistent states. Third,it allows a clear and better separation of control functions and datapacket forwarding functions which can evolve independently from eachother (in analogy to the separation of routing and forwarding functionsin routers). Usually, interior DS routers of a DiffServ domain only needan initial configuration (so-called “resource partitioning”) of their for-warding resources for different per-hop behaviors (PHBs), and, thus donot need to be involved in signaling or admission control functions forsingle or aggregated flows at all.

Well known examples for these approaches are the “Bandwidth Bro-ker” approach [5] or “Reservation Agents” [9]. In these approaches, alogically centralized agent controls resources of a whole domain. It re-ceives requests to provide a certain amount of resources for a particulardata flow through its domain. Based on its internal bookkeeping of re-served resources this agent admits or rejects the new request. Basically,this requires knowledge of the internal domain topology, resource ca-pacities and current selected routes. The latter can be provided by pas-

sively participating in link state-based interior routing protocols (suchas widely used OSPF or IS-IS) and BGP. After a positive intra-domainadmission control decision the agent requests resources from the agentof the next adjacent DiffServ domain on the path to the flow’s desti-nation. Thus, it also performs inter-domain signaling and negotiationin order to set up a path of reserved resources from end to end. Uponsuccessful resource negotiation, it creates or adapts traffic profiles inboundary routers accordingly in order to activate the actual service.

Router

DSDM DSDM

Network

Data Forwardi

ng Plane

Management Plan

e

Differentiated Services Domains

Differentiated Services Domain Manager

Autonomous System

controls

PolicyServer

DMSP DMSPDMSP

DMSP

Figure 1: Separation of control and data forwarding level

In the DARIS architecture, such a management entity or agent iscalled Differentiated Services Domain Manager (DSDM), because itperforms several additional functions besides “bandwidth brokerage”,which are not described in this paper, because it focuses on aggrega-tion and signaling issues. The basic two plane architecture is shownin figure 1. It is assumed that a larger Autonomous System (AS) maybe sub-divided into several DiffServ domains (usually correspondingto routing areas), each controlled by a single DSDM. A policy servermay be used to control resources of the whole AS, especially in orderto arbitrate resources between DSDMs in parts of shared control suchas common backbone areas. DSDMs communicate with each other andwith end-systems by a signaling protocol that is called Domain Manager

Signaling Protocol (DMSP). Some of its special features are describedlater in section 3.

But although DiffServ prevents scalability problems in the data for-warding plane, scalability problems with respect to the number of statesand signaling messages have to be expected in the control plane. Thisis illustrated by the following example. If one imagines that globaltelephone calls over IP use Differentiated Services (e.g., some EF-basedservice) to ensure a common QoS level, resources for every call haveto be admitted and reserved from end to end on demand. As shownin figure 2 those ASs which aggregate traffic (cf. ASs A and B) haveto process the sum of all signaling messages and to keep state for ev-ery reservation. Therefore, per micro-flow reservations are usually notconsidered to be scalable.

AutonomousSystems

A B C

DS Domain

DSDM

Reservation RequestData Flow

End System

Figure 2: Scalability problem in the control plane

The scalability problems can also be solved by using aggregation.For instance, the DSDM in AS A decides to combine the three existingreservations into a single aggregate. In this case, the aggregate spansASs A,B and C, that lie on the common path of all three reservations.The DSDM in AS B can now delete the three established states forthe single reservations and has to keep a state for the new aggregate.A new reservation that traverses the same partial path can then usethe already established aggregate if there is enough capacity in it to

integrate the additional flow. DSDM in AS A can directly forward thereservation request to the aggregation endpoint in AS C, therefore ASB can also save processing of the signaling messages.

New Reservation

ExistingAggregate

A B C

Figure 3: Scalability problem solved by using aggregates

Other approaches that considered scalability in the inter-domaincases mainly concentrated on aggregation along sink-based trees [8, 7],i.e., aggregation towards the same destination (network or end-system).The latter is a rather rare situation where many end-systems require re-served resources towards the same receiver or destination network. TheQBone Bandwidth Broker architecture [10] also allows aggregation, butonly from edge network to edge network, and hierarchical aggregationwas not considered.

The DARIS aggregation principle, that is presented in the next sec-tion, is more flexible, because it allows creation of aggregates betweenarbitrary domains and also allows to nest aggregates in a fully hierar-chical manner.

2.1 The DARIS aggregation principle

First of all, it has to be discussed what can be aggregated. The flexi-bility of the DiffServ architecture allows every provider to use differentcodepoints and PHBs to realize a certain per-domain behavior (PDB).For end-to-end service provisioning multiple domains have to agree on a

common service definition and its guaranteed parameters (for instance,a service that offers a guaranteed bandwidth within a specific interval).Therefore, the only possible unit of aggregation for the inter-domaincase can be service-specific resources of a commonly defined service. Ithas to be mentioned that the described aggregation is different from pos-sibly existing DiffServ aggregates between adjacent domains. Usuallythese aggregates have to be maintained independently of the reservationaggregates.

As noted before, aggregation of reservations is possible between arbi-trary DiffServ domains. It is assumed that every DSDM has knowledgeof the inter-domain routing table (e.g., as passive BGP peer) from whichan internal AS graph representation is created. In this graph, a noderepresents an AS and an edge a link between ASs. A DSDM also recordsevery established reservation in this graph along the reservation’s path,i.e., every node also contains a set of reservations that traverse thisAS or end in this AS. Surely, this graph represents only the view ofthis specific DSDM, therefore, it only sees directed paths from its ownnode to other ASs, and, it only records reservations for which it receivesrequests.

2.1.1 Establishing Aggregates

When a new reservation request is received, the DSDM checks whetherthere exist already other reservations along the path towards the des-tination. Thus, the number of existing reservations is checked in eachnode on the reverse path, i.e., backwards from the destination to theown node. If there exist at least k reservations, a new aggregate canbe created. This aggregate will subsume the capacity of k reservationsplus the resources of the newly requested reservation. Furthermore, thecapacity of the aggregated should be increased by an additional amountin order to include future reservations without having to increase theaggregate’s capacity first. The additional but possibly unused capacitycan nevertheless be utilized by low priority services such as the tra-ditional best-effort delivery. By starting from the destination networklonger aggregates should be created at first if possible, because longeraggregates save more states and signaling messages. However, only ag-gregates of at least two hops make sense, because creating a reservationaggregate between two directly adjacent does not have any benefit, but

only additional overhead. Therefore, an aggregate must comprise atleast one more node except starting point and end point.

In the following, the notation Ap(u, w) denotes an aggregate alongpath p (a sequence of edges) with node u as starting point and node w asend point of the aggregate. The value of Ap(u, w) is the capacity of theaggregate. Σ(Ap(u, w)) denotes the sum of resources of all flows that areaggregated in Ap(u, w), while ∆(Ap(u, w)) := Ap(u, w) − Σ(Ap(u, w))denotes the amount of “unused” resources in the aggregate Ap(u, w),i.e., these resources are not reserved by currently aggregated flows.

The aggregate is installed along path p by allocating the additionalcapacity that is required for the new reservation and the additional ca-pacity ∆(Ap(u, w)). If this allocation is successful, the states of aggre-gated flows are deleted between, but not including, starting point u andend point w. Usually, only directly adjacent DSDMs communicate witheach other. By successful establishment of the aggregate a new trustrelationship has also been built starting point and end point throughusing a chain of trust between adjacent DSDMs. Therefore, after ag-gregate establishment the request for establishing the single reservationcan be directly forwarded from the starting point u to w.

u v w

xq'

p

Reservation Request

yr

u v w

x

yr

u v w

x

yr

Reservation

Aggregation in v

Path

p'

Figure 4: Creation of an aggregate by the DSDM in AS v

Figure 4 shows an example where the DSDM in AS v decides to createan aggregate along path p′ upon receiving a new reservation request fora flow from u to w. Three existing flows and one new flow are aggregatedinto an aggregate Ap′(v, w).

existingaggregate Ap(u,w)

new aggregate Bq(r,x) to establishDSDM

u v w x u v wx

(a) Case 1: Creation of a longer aggregate

new aggregate to establishexisting aggregate

u v wx

(b) Case 2: Addinganother aggregate

new aggregate to establishexisting aggregate

u v wx

(c) Case 3: Aggrega-tion Conflict

Figure 5: Different cases of creating a new aggregate

However, some cases (which are illustrated in figure 5) have to beconsidered when new aggregates are created. Aggregate Ap(u, w) isalready established when a new aggregate Bq(r, x) should be created.The latter would aggregate some reservations that were already aggre-gated in Ap(u, w). In case 1 (cf. fig. 5(a)) a longer aggregate should becreated (r lies before u and x behind w or falls together with w). In thiscase all reservations of Bq(r, x) that are already aggregated in Ap(u, w)should be shifted to the new and longer aggregate, because it will savemore states and messages. The new aggregate Bq will be integrated intothe existing aggregate Ap. In case 2 (cf. fig. 5(b)) the new aggregateends at the starting point u of the already existing aggregate Ap(u, w).In u some reservations are aggregated simultaneously in both aggre-

s t u v w

Ap(s,u) Bq(s,w)

(a) Two nested aggre-gates, starting both ins

s t u v w

Ap'(s,w) Bq' (t,v)

(b) Two nested aggregates, thatpossess different starting and endpoints

Figure 6: Different kinds of nested aggregates

gates. But in case 3, where the new end point x falls into aggregate Ap

(excluding end points u and w), (cf. fig. 5(c)) the creation of the newaggregate will not be possible: if Bq(r, x) contains already aggregatedreservations, they are not known at the DSDM in x any longer, becauseall state information was deleted by the former aggregation process.This situation is called aggregation conflict. In this case, the DSDMin u must reject a request the form the new aggregate Bq. However,a new aggregation attempt can be made without the reservations thatare affected by the conflict.

As mentioned in case 1, in the DARIS approach, aggregates can behierarchically nested. Two types of nested aggregates are depicted infigure 6. Nested aggregates are visible within a DSDM at a startingpoint if two aggregates have the same starting point but different endpoints (cf. fig. 6(a)). In other cases, aggregate nesting is not visible for astarting point of an aggregate, if other DSDMs downstream embeddedthis aggregate into a larger one (cf. fig. 6(b)). As illustrated by case 1,hierarchical aggregates are important in order to let longer aggregatesand thus more efficient evolve over time and to preserve the indepen-dence of the aggregation decision within DSDMs. However, there is asmall chance that nested longer aggregates cannot be created for reser-vations that had an earlier conflict during establishment of the enclosingshorter aggregate. As simulations showed, these “indirect conflicts” arerare events, but nevertheless they have to be handled by DSDMs.

In summary, every DSDM makes its own autonomous decision whenand where to aggregate. Therefore, the autonomy of the serviceproviders is kept. It is possible that some reservations encounter con-flicts during the aggregation process. In this case, reservations that areaffected by a conflict cannot be aggregated along a certain path. Butit is often possible to establish another aggregate comprising a differentset of reservations.

2.1.2 Changing and Terminating Aggregates

The capacity of an aggregate may need to be changed over time. Es-pecially if a new reservation should be integrated into an already ex-isting aggregate, the aggregate’s capacity has to be increased first. Inorder to minimize aggregate changes and to save signaling messages,the requested additional amount should be larger than required, i.e.,∆(Ap(u, w)) > 0. This allows to integrate future reservations withouthaving to increase the aggregate’s capacity first.

Similarly, if an aggregated reservation is terminated, the aggregate’scapacity should not be decreased immediately. In this case a hysteresisfunction can be applied, to adapt the capacity only in larger intervalsand bigger steps. Thus, the capacity of the aggregate should be adaptedon a coarser time scale, e.g., in some few minutes.

u v wr xLink Failure

u' v'

y

Rp(r,y)

Ap'(w,y)

Figure 7: Inter-domain routing changes may cause new conflicts

Inter-domain route changes are not easy to handle, because theymight cause new conflicts. Figure 7 illustrates this situation. Reser-vation Rp(r, y) is re-routed via u′ and v′ due to a link failure between uand v. Unfortunately, in this case the new route ends within an existingaggregate Ap′(w, y) that aggregated Rp, too. Because the DSDM in xalready deleted all state information about Rp it cannot address Rp and

perform admission control again (necessary due to new ingress). Insteadof applying more complex re-reservation schemes, the DARIS approachreduces this situation to a simple case: the reservation is aborted andhas to be established again from end to end along the complete newpath.

Aggregates are terminated automatically some period after the lastreservation was terminated. This period is usually in the same timedomain as are the previously mentioned changing intervals.

3 The Signaling Protocol DMSP

Because aggregation causes also some overhead in signaling, one objec-tive of the DARIS architecture was to reduce it. In this section, someof the specialized and novel protocol mechanisms are presented.

First of all, it has to be noted, that signaling for resources of unicastflows in DiffServ networks must be sender-initiated. The reservationrequest must start at the sender and traverse the path downstreamto the receiver. During this first pass, admission control takes placeand resources are pre-reserved. The response is forwarded backwards(upstream) from the receiver back to the sender. During this secondpass, resources are actually reserved and unnecessary pre-allocated re-sources freed. Furthermore, traffic conditioning mechanisms have to beadjusted at this second pass. Adjusting them downstream would causepacket loss and QoS violations, e.g., if the rate of a traffic shaper atan egress boundary router is increased before the corresponding trafficprofile had been updated in the downstream ingress router. Neverthe-less, DMSP supports receiver-initiated reservations by sending such arequest directly to the sender which then initiates the actual reservationand sends back the result to the initiator.

The signaling sequence for the establishment of an aggregate is shownin figure 8. In step (1) the sender sends a SrvEstReq message to itsDSDM in order to establish a new reservation for a specific service.The DSDM in r now wants to create a new aggregate and signals acorresponding AggEstReq message (step (2)) to its neighbor in u. Theaggregate is established in the following steps (3)–(7) before the Srv-EstReq gets directly forwarded to the aggregation end point w in step(8). Finally, the aggregate end point forwards the request towards the

u vr wSender Receiver

(1) (2)

(5)

(8)

(9)

(10)

(11)

(12)

(3) (4)

(6)(7)

new aggregate to establish

SrvEstReqAggEstReq

Figure 8: Establishing an aggregate

receiver in step (9). The subsequent response (10) is also directly for-warded from the aggregate end point w to the aggregate starting pointr (11), which then forwards it to the sender that was initiator of therequest.

To avoid several trials when establishing an aggregate, the protocolsupports requests that carry parameter intervals. Thus, a lower boundof the requested resources for an aggregate Ap would be Σ(Ap(u, w)) andthe upper bound Ap(u, w). Consequently, any value that falls within thebounds will be accepted by the initiator, so that additional capacity isreserved if possible, and the SrvEstReq will not fail if there were enoughresources for the new single flow but not for the additional requestedaggregate resources ∆(Ap(u, w)).

Once an aggregate is established and a new reservation should beintegrated into the existing aggregate, the residual capacity of the lattermay not be sufficient to comprehend the resources of the new flow.In this case, the aggregate’s capacity (i.e., its reserved resources) hasto be increased at first. Furthermore, if the aggregate is embeddedinto another aggregate, this may also have to be increased. For thesesituations, a special support through the signaling protocol DMSP hasbeen developed.

In these cases, parallelism is used to reduce the reservation setup time.Figure 9 shows an example for this situation. After the SrvEstReq isreceived (1), the DSDM in r forwards it directly (strictly speaking aftera successful local admission control) to the aggregate end point w (2b).After performing a local admission control at w forwarding of the Srv-EstReq message is suspended until the AggChgReq message to increasethe aggregate arrives at w (4). This can be considered as an optimistic

u vr wSender Receiver

(1) (2a)

(7a)

(2b)

(5)

(6)

(7b)

(10)

(3) (4)

(8)(9)

aggregate to change

SrvEstReq AggChgReq

SrvEstReq

Figure 9: Integrating a new reservation into an existing aggregate byincreasing its capacity

strategy assuming that increasing the aggregate will be successful. Ifthe aggregate cannot be increased by the required amount, r simplysends a SrvAbortReq message to w, canceling the waiting SrvEstReqmessage. Thus, two signaling processes are started at w in parallel:increasing the aggregate capacity by sending a AggChgReq message tou (2a) and forwarding the SrvEstReq message (2b) to the aggregate endpoint. After the increment of the aggregate was considered successfulat w, the SrvEstReq message is forwarded towards the receiver (5). Atthis point it is possible to let the response AggChgRsp wait at w untilthe response for the single reservation arrives (6). Depending on theproviders policy, incrementing the aggregate may be likewise canceledwhen the response (6) for the single flow is negative. Alternatively,a successful increment of the aggregate may be preserved even whenestablishing the reservation for the single flow fails afterwards. The for-mer option will lead to a parallel sending of the AggChgRsp (7a) andthe SrvEstRsp (7b) message, while for the latter option AggChgRsp willbe sent back earlier. However, the SrvEstRsp will always wait at r onthe corresponding AggChgRsp to arrive, before it is finally forwardedto the sender (10).

In order support the withholding of a message until the requiredmessage arrives, DMSP supports so-called “forward waiting conditions”and“reply waiting conditions”. These can be seen as simple references toother messages whose arrival trigger the continuation of the suspendedmessage processing. It is sufficient to support only one“forward waitingcondition” and one “reply waiting condition” per message, because thereexists at most one superior aggregate.

r u v wSrvEstReq

SrvEstRsp

SrvEstRsp

SrvEstRspSrvEstReq

SrvEstReq

AggChgReq

AggChgReq

AggChgReq

AggChgRsp

AggChgRsp

AggChgRsp

Sender Receiver

Tp

D

(a) without parallelism

r u v wSrvEstReq

SrvEstRsp

SrvEstRsp

SrvEstRsp

SrvEstReq

SrvEstReqAggChgReq

AggChgReq

AggChgReq

AggChgRsp

AggChgRsp

AggChgRsp

Sender Receiver

2Tp+2HD

(b) with parallelism

Figure 10: Timed sequence diagrams of signaling sequence from figure 9without and with parallelism, D Delay, Tp Processing Time, H Numberof Hops between DSDMs.

Figure 10 illustrates the advantage of this solution for the examplegiven in figure 9. The total time for the parallelized version of the signal-ing sequence is clearly shorter than the non-parallel version (assumingthat partial delays and processing times are equal in both cases). Onecan coarsely quantify the gain: D denotes the average delay betweentwo directly adjacent DSDMs respectively between DSDM and end sys-tem, Tp denotes the mean processing time within a DSDM (neglectingthe processing times in end systems for the sake of simplicity). Simi-larly, it is assumed that D and Tp are equal for all systems. The numberof involved DSDMs is H − 1 (H is the number of hops). Then the totaltime for the complete sequence of figure 10(a) can be calculated as:

TNormal = 2(H + 2)(D + Tp) + 2HD (1)

and likewise for the sequence of figure 10(b):

TParallel = 2(H + 1)(D + Tp) + 2D (2)

The achieved gain (reduction of setup time) is the difference betweenequations 1 and 2, and thus TNormal−TParallel = 2Tp+2HD. Therefore,the savings will be more than one round trip time and they will increasewith a raising length of the path.

4 Evaluation

First of all, the applicability of the DARIS aggregation principle wasexamined. It is the question whether current AS paths in the Internetare long enough for aggregation. For this purpose, several BGP tabledumps from the routeviews project [6] were used and converted into agraph representation. Subsequently, for every AS that was not a puretransit AS the shortest path lengths to every other AS were calculatedand counted. The result is depicted as complementary distributionfunction P (X > x) = 1−P (X ≤ x) in figure 11: more than 92,16% of allpaths are longer than two hops, i.e., in 92,16% of all cases an aggregatecan span four or more ASs (saving states and message processing in atleast two ASs). Therefore, enough aggregation possibilities are given inthe real Internet topology.

The DARIS principle was simulated (including the special signalingsupport by DMSP) by using the OMNeT++ simulation environment[11]. In order to evaluate the DARIS aggregation a simple topologyas depicted in figure 12 was used. This topology consists of 13 DS-DMs, where DSDMs 5 and 6 have to handle messages from DSDMs 1and 2, respectively 3 and 4. Similarly, DSDM 7 has to handle reser-vations from DSDM 5 and 6. The right side is symmetric to the leftside, so analogous statements are valid for DSDMs 8–13. Every DSDM,except DSDM 7, has an end-system that creates randomly reservationrequests to every other end-system with equal probability. The time un-til a new request is issued at an end-system is exponentially distributed,while the duration of a “session” (which denotes an established reser-vation) is chosen from a pareto distribution with a mean duration of180 s. Every reservation requested a constant bandwidth of 64 kbit/s.Adaptation of an aggregate’s capacity is controlled by a combination ofseveral mechanisms that are not described in detail here. However, anexponential moving weighted average as well as a smoothed standarddeviation is used to track the development of Σ(Ap(u, w)). Further-

Empirical distribution function F(x)=P(X>x)

99,969892%

92,166284%

54,645003%

16,696200%

2,876996%0,282333% 0,015759% 0,000538% 0,000012% 0,000000%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1 2 3 4 5 6 7 8 9 10

AS path length (end to end)

Pro

babi

lity

18.8.199818.8.199918.8.200018.8.200118.8.2002

source: route-views.oregon-ix.net

Figure 11: Empirical distribution of end-to-end AS path lengths in theInternet

Represents oneor more end-systems

DSDM 1

DSDM 2

DSDM 3

DSDM 4

DSDM 5

DSDM 6

DSDM 7 DSDM 8

DSDM 9

DSDM 10

DSDM 11

DSDM 12

DSDM 13

Figure 12: A simple simulation topology

more, a smoothed gradient as well as quantization is used to calculatethe additional amount that should be used for incrementing the aggre-

gate’s capacity. Quantization and the average value are also used forthe hysteresis when the aggregate’s capacity has to be decremented. Anexample of the adaptation of an aggregate’s resources at DSDM 10 isgiven in figure 13.

0

2000

4000

6000

8000

10000

12000

0 2000 4000 6000 8000 10000 12000 14000

Ban

dwid

th [k

bit/s

]

Time [s]

Used Reserved Smoothed Average (EWMA)

Figure 13: Adaptation of an aggregate’s capacity

The simulations ran with different reservation initiation rates: eachend-system initiated 1000, 10000, 50000, 100000, 500000 and 1000000sessions with respective increasing mean rates of 0.05, 0.5, 2.5, 5, 25,and 50 sessions (or reservations) per second. Every configuration wasrun with 50 different random seeds. Furthermore, the minimum numberk of reservations that must exist before an aggregate can be created wasvaried over the values 2, 4, 8, and 100.

First of all, the number of transit states per DSDM is shown in figure14 for k = 2. Transit states stem from reservations that do not startor end within the own domain. Keeping states for reservations that areinitiated or terminated by end-systems of the own domain cannot beavoided. Consequently, DSDMs 1–4 and 10–13 do not have to keep anytransit states, because they are stub domains. The diagram shows the

101

102

103

104

105

106

107

1 2 3 4 5 6 7 8 9 10 11 12 13

Num

ber o

f tra

nsit

rese

rvat

ion

stat

es

DSDM No.

without DARIS with DARIS

Figure 14: Number of transit states per DSDM

averaged results for the configuration 1000000 sessions per end-systemwith a mean rate of 50 initiated sessions per second. By using theDARIS aggregation principle, several orders of magnitude of states canbe saved.

Figures 15 and 16 show the behavior of DARIS within DSDMs 5 and 7for the different configurations and k = 2. The plotted number of statesreflects the sum of states over the complete runtime of the simulation.The overhead due to own and foreign aggregates (aggregates initiatedby other DSDMs are contained in the number of transit reservations)very small and the number of states can be reduced to a scalable andmanageable amount with the DARIS concept.

Due to the symmetric topology results for DSDMs 6,8 and 9 aresimilar to those of DSDM 5.

In the next step, the number of signaling messages has to be exam-ined. Figures 17 and 18 depict the number of transit messages andaggregation-related signaling messages. Similarly, DARIS reduces thenumber of transit DMSP messages to a scalable amount. The over-

101

102

103

104

105

106

107

102 103 104 105 106

Num

ber o

f sta

tes

Number of sessions per end−system

DSDM 5

transit states without DARIS (no aggregation)transit state with DARIS (aggregation)own aggregates

Figure 15: Number of states within DSDM 5

101

102

103

104

105

106

107

102 103 104 105 106

Num

ber o

f sta

tes

Number of sessions per end−system

DSDM 7

transit states without DARIS (no aggregation)transit state with DARIS (aggregation)own aggregates

Figure 16: Number of states within DSDM 7

head caused by aggregation is small compared to the number of transitmessages without aggregation. It is obvious that with an increasingnumber of sessions the number of aggregation-related messages mustalso increase slowly: the aggregates have to be adapted more often overthe longer runtime.

1000

10000

100000

1000000

10000000

100000000

100 1000 10000 100000 1000000

Num

ber o

f tra

nsit

mes

sage

s fo

r DS

DM

5

Number of sessions per end−system

without DARISwith DARISaggregation−related messages

Figure 17: Number of messages within DSDM 5

So what about the mentioned aggregation conflicts? They occur onlyin early phases of aggregate creation. After most aggregates have beenestablished the situation stays stable. Figure 19 shows the developmentover time in DSDM 5 for k = 8 and 50000 sessions.

First trials to use a real Internet topology were made. Once morethe routeviews archive of BGP table dumps was used to extract anundirected AS graph. Although the BGP table represents always aspecific view of a particular AS, it can be assumed that this is the mostcomplete topology information at AS level about the Internet, and, a lotmore real than any randomly generated“Internet-like” topology. Due tomemory limitations only a topology of 1999 (of August 18th) could beused so far. It consists of 5561 ASs from which 4196 are stub ASs, i.e.,

1000

10000

100000

1000000

10000000

100000000

100 1000 10000 100000 1000000

Num

ber o

f tra

nsit

mes

sage

s fo

r DS

DM

7

Number of sessions per end−system

without DARISwith DARISaggregation−related messages

Figure 18: Number of messages within DSDM 7

100

101

102

103

104

105

10−2 10−1 100 101 102 103 104 105

Cum

ulat

ed n

umbe

r of s

tate

s

Time [s]

DSDM 5

Single reservationsAggregatesConflicts

Figure 19: Development of states and conflicts over time in DSDM 5

they are not able to (single-homed ASs) or are not configured to forwardtransit traffic. Because 64 ASs were classified as pure transit ASs, allother 5497 ASs had end-systems to initiate reservations. Every end-system initiated 5000 reservations with a mean rate of one reservationevery 3 seconds. k was set to 2 and the same parameters for the paretodistribution as within the small topology were used.

100

101

102

103

104

105

106

107

108

0 200 400 600 800 1000 1200 1400

Num

ber o

f Tra

nsit

Sin

gle

Res

erva

tions

DSDM

w/o Aggregationwith DARIS

Figure 20: Transit states per DSDM of an Internet topology from Au-gust 18th, 1999

Figure 20 shows the preliminary results of such a simulation. TheDSDMs were sorted with decreasing number of states. Due to the highnumber of stub ASs, only 1365 ASs carry transit traffic. In the averagesavings through applying the DARIS principle lie also in the range of oneto two orders of magnitude. However, because of the equally distributedreservations, there are merely established aggregates from end to end.Therefore, some highly connected ASs can reduce their transit statesonly to a very small degree, because there are no aggregates that includethem as intermediate AS. Nevertheless, further simulations have to bemade to understand the aggregation behavior better in real topologies.

A lot of memory is required for such large “real” topologies, becauseevery DSDM has to manage a routing table with paths to every otherAS. Currently, the whole simulation is ported to 64-bit platforms dueto the 4GB limit per process of traditional 32-bit platforms.

5 Summary and Outlook

Global availability of on-demand QoS-based services is one desirablefunctionality for a next generation Internet. Especially, scalability atthe inter-domain level with respect to the number of states and sig-naling messages of potential solutions has to be considered. DynamicAggregation of Reservations for Internet Services (DARIS) is a means tocouple aggregation techniques of the data path with those in the controlplane. DARIS allows a very flexible and fully hierarchical aggregationof resource reservations. A novel signaling mechanism is used to reducethe setup time for new reservations in combination with existing ag-gregates. The simulations confirm that DARIS reduces the number ofreservations states and messages considerably, even in a real Internettopology.

Currently, extensive efforts are made to examine the behavior of theDARIS concept for more current Internet topologies. The memory limitof 32-bit architectures for a single process is the main problem here, soa 64-bit version is currently built. Furthermore, the effects of usingDMSP’s waiting conditions on the average delay may be quantified infuture simulation scenarios.

One problem with aggregation is that not the aggregating and de-aggregating ASs profit from their initiated aggregates, but all interme-diate ASs within an aggregate. Therefore, some incentive for aggregatecreation has to be given. This may lead to novel cost models that haveto be developed for aggregation concepts.

References

[1] F. Baker, D. Black, S. Blake, and K. Nichols. Definition of the Dif-ferentiated Services Field (DS Field) in the IPv4 and IPv6 Headers.RFC 2474 (Standard), Dec. 1998.

[2] S. Blake, D. Black, M. Carlson, E. Davies, Z. Wang, and W. Weiss.An Architecture for Differentiated Services. RFC 2475 (Informa-tional), Dec. 1998.

[3] R. Braden, D. Clark, and S. Shenker. Integrated Services in theInternet Architecture: an Overview. RFC 1633 (Informational),June 1994.

[4] B. Davie, A. Charny, F. Baker, J. Bennet, K. Benson, J.-Y. L.Boudec, A. Chiu, W. Courtney, S. Davari, V. Firoiu, C. Kalmanek,K. Ramakrishnam, and D. Stiliadis. An Expedited ForwardingPHB (Per-Hop Behavior). RFC 3246, Mar. 2002.

[5] V. Jacobson, K. Nichols, and L. Zhang. A Two-bit DifferentiatedServices Architecture for the Internet. RFC 2638 (Informational),July 1999.

[6] D. Meyer. Route views project page. http://www.routeviews.org/,May 2002.

[7] P. Pan, E. L. Hahne, and H. Schulzrinne. BGRP: Sink-Tree-BasedAggregation for Inter-Domain Reservations. Journal of Communi-cations and Networks, 2(2):157–167, June 2000. Special Issue onQoS in IP Networks.

[8] O. Schelen and S. Pink. Aggregating resource reservations overmultiple routing domains. In Proceedings of IFIP Sixth Interna-tional Workshop on Quality of Service (IWQOS’98). IFIP, IEEE,May 1998.

[9] O. Schelen and S. Pink. Resource sharing in advance reservationagents. Journal of High Speed Networks, 7(3–4), 1998. SpecialIssue on Multimedia Networking.

[10] B. Teitelbaum and P. Cimento. QBone Bandwidth Broker Ar-chitecture. http://qbone.internet2.edu/bb/bboutline2.html, June2000. Working Document of the QBone Bandwidth Broker WorkGroup.

[11] A. Varga. ”omnet++ discrete event simulation system 2.2”. http://www.hit.bme.hu/phd/vargaa/omnetpp.htm, May 2002.

Roland Bless started to study Informatics at the University of Karlsruhe

in 1990 and finished 1996 with the diploma degree Dipl.-Inform. Since 1996

he is a research assistant at the Institute of Telematics at the University of

Karlsruhe. In February 2002 he got his doctoral degree Dr.-Ing. for his disser-

tation about “Integrated Management of QoS-based Communication Services

in the Internet”. His research interests are Quality-of-Service, QoS manage-

ment, Differentiated Services, Multicast, Mobility and QoS Signaling as well

as application of practical security techniques. He is actively participating in

IETF Working Groups DiffServ and NSIS and brought parts of his work into

the IETF standardization process. Mr. Bless is also a member of IEEE and

the German Gesellschaft fur Informatik (GI).