BGP: Inter-Domain Routing Protocol

Noah Treuhaft

U.C. Berkeley

The need for routing in the Internet

• Need to get packets from source to destination• How do you do this?

– Network is a collection of point-to-point links connected by routers

– Routers’ decisions determine which links you transit– Routing proceeds hop by hop (contrast with source

routing)– How do you determine the next hop?

• Could configure it statically• But the Internet needs a routing system and protocol to

exchange complex and changing routing info

External and internal gateway protocols

• Autonomous System (AS) – a single administrative domain (ISP, customer)

• External Gateway Protocols exchange routing information between routers of different AS’s.– Goal: support routing policies, scale

• Internal Gateway Protocols exchange routing information among an AS’s own routers– Goal: optimize route taken

Distance vector protocols

• <destination, metric> messages relative to sender– Essentially a routing table

• Contrast with link state protocols– <source, destination, metric> messages

“flooded” to all nodes– Shortest path first (Dijkstra) algorithm builds

routing table

Classless inter-domain routing

• Internet routing was once based on network classes

• Trading classes for variable-length prefixes allows aggregation– Greater flexibility in address allocation– Less routing information required

BGP communication

• A Border Gateway Protocol (BGP) session consists of a TCP connection between two routers– If connection fails, associated state is dropped.

• Message types: OPEN, UPDATE, NOTIFICATION, KEEPALIVE

• UPDATE format: <withdrawn routes, attributes, valid routes>

BGP attributes

• ORIGIN – BGP speaker’s unique identifier

• AS-PATH – AS’s that relayed this message

• NEXT-HOP

• MULTI-EXIT-DISCRIMINATOR (MED) – metric for multiple paths between 2 AS’s

• LOCAL-PREF – metric for multiple paths to same prefix

• COMMUNITY – update categorization

Internal-BGP

• Same messages, attributes as External-BGP• Different rules for readvertising prefixes

– Does not readvertise routes from one I-BGP speaker to another

– Prevents looping (E-BGP uses the ASN and AS-PATH for this)

• Route reflection: adding hierarchy for scalability• AS confederation: subdivision of a logical AS into

multiple AS’s

Route Information PropagationIn the Internet Using BGP

Matthew Denny

U.C. Berkeley

Introduction

• Internet consists of different Autonomous Systems (ASs), which consist of admin. defined domains of hosts (e.g. ISPs, universities, companies, etc.)

• Hosts in each AS must be able to send packets to any other host on the Internet

• ASs have routers which exchange routing info. with other ASs using BGP– How do ASs ensure “full reachability” of the Internet,

given no central authority?– How well does the current route information

propagation scheme perform?

Outline

• Introduction

• Structure of ASs

• BGP policies of ASs

• Why does it work?

• Now, does it really work?

Structure of ASs• Somewhat hiearchical (but becoming less so)• 3 Types of relationships

– Customer-Provider: customer AS pays provider AS for access to rest of Internet: provider provides transit service

• End customers pay ISPs, and ISPs in lower “tiers” pay ISPs in higher tiers

– Peers: ASs that allow each other transit service • ISPs on same tier, usually involves no fees

– Customer-Backup Provider: Provider if primary provider fails. May be peers otherwise

• Use BGP to communicate route info. at Network Exchange Points (NAPs) and private peering points

AS BGP Policies

• Customers export all of their routes and routes of their customers to providers, but not routes from peers or other providers

• Peers export their routes and routes of their customers to other peers, but not routes from peers or other providers

• Providers export all of their routes to customers• Usually, backup providers “promoted” to provider

from peer upon failure of primary• If an AS recieves 2 routes for same prefix, usually

exports the best by some path selection algorithm

AS BGP Policies

AS1 AS2

AS3 AS4

192.168.0.0/26

192.168.0.64/26 192.168.0.128/26

192.168.0.192/26

192.

168.

0.0/

26

192.

168.

0.19

2/26

192.168.0. 0/25

192.168.0. 128/25

192.168.0. 64/26

192.168.0. 128/25

192.168.0. 128/26

192.168.0. 0/25

192.168.64. 0/26

192.168.0. 128/25

192.168.0. 0/26

192.168.0. 192/26

192.168.0. 128/26

192.168.0. 64/26

192.168.0. 64/26

192.168.0. 128/25192.168.0. 64/26

AS BGP Policy Details

• Export Policy– To indicate priority of route, most ASs use

communities [Labovitz 2000b]

• Import Policy– AS Path loop detection, not usually selective. Use

communities to infer local pref

• Path Selection– If AS has multiple routes for same prefix, best route

decided by local pref; AS Path and MED are tiebreakers

How does it work?

• Full Reachability– Provider/Customer relationships form a DAG

• Assumes everyone below tier 1 has a provider

• Convergence– Can diverge, and checking for convergence is an NP

Complete Problem [Griffin 99]

– Assuming strict preferences on route selection and the above structure, [Gao 2000] proves that BGP systems will converge

– Will this hold as peering becomes more common?

Now, Does it Really Work?

• Potential Problems– Route Instability

• Large number of unneeded messages leads to router CPU flooding

– Routers lose Keep-Alive messages go “down”

• “Route Flap” Problem

– Route Convergence• Routes that change (e.g. failover to a backup

provider) may take a long time to propagate correctly through system

• Can cause intermittent loss of connectivity

Route Instability Study

• Labovitz et. al. performed a study to measure instability in BGP Updates [Labovitz 1997, 1999]– Logged BGP update messages at 5 NAPs 1996-

1998, and analyzed instability events• Routes withdrawn that are re-announced, and

“pathological” withdraws

• Some events due to route or policy instability, or pathological behavior

Initial Instability Findings

• In 1996, 45,000 prefixes, 1,500 unique AS paths, 1,300 ASs, 3-6 million BGP update messages/day

• Messages dominated by pathological withdraws• Redundant updates have strong periodicity of 60

sec.• Redundant Updates correspond with network

usage• Instability not dominated by small number of ASs

or routes

Many Problems due to Router Software Implementation

• Pathological withdraws due to “stateless BGP”– Announce withdraws to router peers that did not

originally receive an announcement

• Periodicity due to min. advertisement timer that was fixed in one BGP implementation

• New methods developed to prevent route flaps– BGP messages have higher priority than data, esp

Keep-Alives.

• Labovitz et. al. contacted router vendors, who released patches to fix these bugs

Follow-up Results

• June 1996, 2M pathological withdrawls/day, 10K in June 1998– Due to at least partial “stateful BGP” in most routers

• In 1998, duplicate announcements 40% of traffic– Bug in router software where non-transitive attribute is errantly

“propagated”– Min. advertisement timer allows routes to change back to original

value before transmission; router still sends these routes

• In 1998, vast majority of route fluctuation due to MED changes– Come from 2 ISPs that dynamically assign MEDs from IGP.

Effectively make IGP changes globally visible

Route Convergence

• For a set of real host addresses, Labovitz et. al. inject routing events and observe convergence behavior [Labovitz 2000]– Route failures, new routes, and routes with new

path– Simulated backup routes by inflating AS path– Analyzed BGP traffic– Tested faults affect on routes by sending ICMP

messages to web sites from set of addresses

Convergence Findings

• Delay in failovers ave. 3 min., but up to 15 min. (more than 30 sec. expected)

• Messages/event and convergence time/event varies from ISP to ISP

• Significant increase in packet loss and latency around faults

• Routers use per peer min. advertisement timers, which delays convergence

• Wait for min. advertisement timer to send updates that have loops in them; should use sender side loop detection instead

Conclusions

• Describe structure of ASs in Internet and how they exchange routing information via BGP

• Discuss work that shows that this structure should work given specific assumptions

• Discuss work that shows, in practice, Internet had instability and convergence problems, but many of these were due to implementation problems

References

• [Gao 2000] L. Gao and J, Rexford “Stable Internet Routing Without Global Coordination”, SIGMETRICS 2000

• [Labovitz 1997] C. Labovitz, G.R. Malan, F. Jahanian, “Internet Routing Instability”, SIGCOMM 97.

• [Labovitz 1999] C. Labovitz, G.R. Malan, F. Jahanian, “Origins of Internet Routing Instability”, INFOCOMM 1999

• [Labovitz 2000] C. Labovitz, A. Ahuja, A. Bose, F. Jahanian. “Delayed Internet Routing Convergence”, SIGCOMM 1999

• [Labovitz 2000b] C. Labovitz, R. Wattenhofer, S. Venkatachary, A. Ahuja. “The Impact of Internet Policy and Topology on Delayed Routing Convergence”. Microsoft Tech. Report MSR-TR-2000-74, 2000.