Telehealth user adoption: Blackholes & bullseyes - Douglas Shinsato, Anthill Ventures - TFSS Keynote
BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS...
Transcript of BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS...
![Page 1: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/1.jpg)
BGP Communities:
A measurement study
@RIPE77, Amsterdam
Florian Streibelt1, Franziska Lichtblau1, Robert Beverly2, Cristel Pelsser3,
Georgios Smaragdakis4, Randy Bush5, Anja Feldmann1
Oct. 2018
1 Max Planck Institute for Informatics (MPII), 2 Naval Postgraduate School (NPS),3 University of Strasbourg, 4 TU Berlin (TUB), 5 Internet Initiative Japan (IIJ)
![Page 2: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/2.jpg)
BGB-Communities:A weapon for the Internet!
RIPE 77 / Amsterdam2018.10.16
1
![Page 3: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/3.jpg)
Introduction
![Page 4: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/4.jpg)
BGP Community usage is increasing
●
●
●
●
●
●●
●●
2010 2012 2014 2016 2018
Year
●●
●●
● ● ● ● ●
2k
3k
5k
10k
20k
40k
70k
0.2B
0.3B
0.5B
1B
2B
4B
7B●
●
# Unique ASes in Communities# Unique Communities
# Absolute Communities# BGP table entries
Increasing usage warrants a closer look.
2
![Page 5: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/5.jpg)
BGP Community usage is increasing
●
●●
●
●●
●●
●
2010 2012 2014 2016 2018
Year
2k3k
5k
10k
20k
40k
70k
0.2B0.3B
0.5B
1B
2B
4B
7B●
# Unique ASes in Communities# Unique Communities
# Absolute Communities# BGP table entries
Increasing usage warrants a closer look.
2
![Page 6: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/6.jpg)
BGP Communities
• Optional Attribute in BGP message (32 bit field)
• Defined in RFC 1997
• By convention written ASN:VALUE
• ASN can be both sender or intended ’recipient’
• It’s up to the peers to agree upon ’values’ used
3
![Page 7: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/7.jpg)
BGP Large Communities
• Defined by RFC 8092 (usage recommendations ins RFC 8195)
• 12 byte attribute
• Enable networks with 4-byte ASNs to use communities
• The first 4 byte contain the ASN of the ”global administrator”
4
![Page 8: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/8.jpg)
BGP Large Communities
Sorry. . . as we only found a very small number of
occurrences1 we could not conduct any meaningful
measurements, yet.1283 individual large communities by 51 global administrators over the whole month of April 2018 at all available
route collectors at RIPE/RIS, Routeviews, Isolario and PCH
4
![Page 9: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/9.jpg)
BGP Communities: Usage
Informational Communities
(Passive Semantics)
• Location tagging
• RTT tagging
Action Communities
(Active Semantics)
• Remote triggered blackholing
• Path prepending
• Local pref/MED
• Selective announcements
Without documentation, you can not tell
if a community is active or passive!
5
![Page 10: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/10.jpg)
What This Talk Is About
Given the increasing popularity of BGP communities and the
ability to trigger actions as well as relay information,
the first question that comes to the mind of an
Internet measurement researcher is. . .
6
![Page 11: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/11.jpg)
What This Talk Is About
What could possibly go wrong?
6
![Page 12: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/12.jpg)
Propagation behavior
7
![Page 13: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/13.jpg)
Propagation behavior
• 14% of transit providers propagate received communities
(2.2k of 15.5k)
• Ratio seems small, but AS graph is highly connected
• RFC 1997: Communities as a transitive optional attribute
• RFC 7454: Scrub own, forward foreign communities
Still many people do not expect communities
to propagate that widely.
7
![Page 14: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/14.jpg)
Potential (for) misuse
• Propagated communities might trigger actions multiple
AS-hops away
• No way of knowing if intended or not, e.g., for traffic
management
• But are there also unintended consequences?
Our assessment is that there is a high risk for attacks!
8
![Page 15: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/15.jpg)
Observations
![Page 16: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/16.jpg)
Dataset
BGP updates and table dumps of April 2018 from publicly available
BGP Collector Projects: RIPE RIS, Routeviews, Isolario, PCH.
BGP messages 38.98 bn
IPv4 prefixes 967,499
IPv6 prefixes 84,953
Collectors 194
AS peers 2,133
Communities 63,797
More than 75% of all BGP announcements have at least one
BGP community set, 5,659 ASes are using communities.
9
![Page 17: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/17.jpg)
BGP Community Propagation Observations
●
●
●
●
●
●
●
●● ● ● ●
0 2 4 6 8 10
0.0
0.2
0.4
0.6
0.8
1.0
AS hop count
Frac
tion
of c
omm
uniti
es (E
CD
F)
• 10% of communities have a AS hop count of more than six
• More than 50% of communities traverse more than four ASes
• Longest community propagation observed: 11 AS hops
10
![Page 18: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/18.jpg)
BGP Community Propagation Observations
0 2 4 6 8 10
0.0
0.2
0.4
0.6
0.8
1.0
AS hop count
Frac
tion
of c
omm
uniti
es (E
CD
F)
• 10% of communities have a AS hop count of more than six
• More than 50% of communities traverse more than four ASes
• Longest community propagation observed: 11 AS hops
10
![Page 19: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/19.jpg)
BGP Community Propagation Observations
0 2 4 6 8 10
0.0
0.2
0.4
0.6
0.8
1.0
AS hop count
Frac
tion
of c
omm
uniti
es (E
CD
F)
• 10% of communities have a AS hop count of more than six
• More than 50% of communities traverse more than four ASes
• Longest community propagation observed: 11 AS hops
10
![Page 20: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/20.jpg)
BGP Community Propagation Behavior
AS1
AS4
AS3
AS2
• AS1 announces prefix p
, tagged with 3:123
• Community is intended for signaling towards AS3
• AS4 also receives this announcement
Off-path:
ASN from community is not on the observed AS-path at AS4.
11
![Page 21: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/21.jpg)
BGP Community Propagation Behavior
pp
pAS1
AS4
AS3
AS2
• AS1 announces prefix p
, tagged with 3:123
• Community is intended for signaling towards AS3
• AS4 also receives this announcement
Off-path:
ASN from community is not on the observed AS-path at AS4.
11
![Page 22: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/22.jpg)
BGP Community Propagation Behavior
pp
pAS1
AS4
AS3
AS2
3:1233:123
3:123
• AS1 announces prefix p, tagged with 3:123
• Community is intended for signaling towards AS3
• AS4 also receives this announcement
Off-path:
ASN from community is not on the observed AS-path at AS4.
11
![Page 23: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/23.jpg)
BGP Community Propagation Behavior
pp
pAS1
AS4
AS3
AS2
3:1233:123
3:123
• AS1 announces prefix p, tagged with 3:123
• Community is intended for signaling towards AS3
• AS4 also receives this announcement
Off-path:
ASN from community is not on the observed AS-path at AS4.
11
![Page 24: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/24.jpg)
BGP Community Propagation Behavior
pp
pAS1
AS4
AS3
AS2
3:1233:123
3:123
• AS1 announces prefix p, tagged with 3:123
• Community is intended for signaling towards AS3
• AS4 also receives this announcement
Off-path:
ASN from community is not on the observed AS-path at AS4.
11
![Page 25: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/25.jpg)
BGP Community Propagation Behavior
pp
pAS1
AS4
AS3
AS2
3:1233:123
3:123
p: 4, 2, 1 3:123
p: 3, 2, 1 3:123
• AS1 announces prefix p, tagged with 3:123
• Community is intended for signaling towards AS3
• AS4 also receives this announcement
Off-path:
ASN from community is not on the observed AS-path at AS4.
11
![Page 26: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/26.jpg)
BGP Community Propagation Behavior
pp
pAS1
AS4
AS3
AS2
3:1233:123
3:123
p: 4, 2, 1 3:123
"on−path"
p: 3, 2, 1 3:123
• AS1 announces prefix p, tagged with 3:123
• Community is intended for signaling towards AS3
• AS4 also receives this announcement
Off-path:
ASN from community is not on the observed AS-path at AS4.
11
![Page 27: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/27.jpg)
BGP Community Propagation Behavior
pp
pAS1
AS4
AS3
AS2
3:1233:123
3:123
"on−path"
p: 3, 2, 1 3:123
p: 4, 2, 1
"off−path"
3:123
• AS1 announces prefix p, tagged with 3:123
• Community is intended for signaling towards AS3
• AS4 also receives this announcement
Off-path:
ASN from community is not on the observed AS-path at AS4.
11
![Page 28: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/28.jpg)
On-path versus off-path
165
000
666
100 0
3000 2
1000
9498 200
1000 100 1
200
2000 10 2
3000 0
500
% c
omm
uniti
es o
bser
ved
0.0
0.2
0.4
0.6
0.8
1.0
1.2
off-path on-path
• Blackholing communities (e.g., :666) ’leaking’ off path
• But AS implementing RTBH
SHOULD add NO ADVERTISE or NO EXPORT (RFC7999)
Suggests ASes not implementing RTBH do not filter.
12
![Page 29: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/29.jpg)
Experiments
![Page 30: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/30.jpg)
Experimental setup
• Experiments conducted in a lab environment
• Validated on the Internet
Scenarios
• Remote Triggered Blackholing (RTBH)
• Traffic redirection attack
...for others see our paper.
13
![Page 31: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/31.jpg)
RTBH: how it works
• AS announces BH-prefix
to upstream
→ Provider blackholes prefix
AS5
AS1
AS3 AS4
AS2
Safeguards:
• Provider should check customer prefix before accepting RTBH
• Customer may only blackhole own prefixes
• Different policies for Customers/Peers
• On receiving RTBH, add NO ADVERTISE or NO EXPORT
(RFC7999)
14
![Page 32: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/32.jpg)
RTBH: how it works
• AS announces BH-prefix
to upstream
→ Provider blackholes prefix
AS5
AS1
AS3 AS4
AS2
BGP announcements
p
Safeguards:
• Provider should check customer prefix before accepting RTBH
• Customer may only blackhole own prefixes
• Different policies for Customers/Peers
• On receiving RTBH, add NO ADVERTISE or NO EXPORT
(RFC7999)
14
![Page 33: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/33.jpg)
RTBH: how it works
• AS announces BH-prefix
to upstream
→ Provider blackholes prefix
AS5
AS1
AS3 AS4
AS2
BGP announcements
Traffic flow
p
Safeguards:
• Provider should check customer prefix before accepting RTBH
• Customer may only blackhole own prefixes
• Different policies for Customers/Peers
• On receiving RTBH, add NO ADVERTISE or NO EXPORT
(RFC7999)
14
![Page 34: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/34.jpg)
RTBH: how it works
• AS announces BH-prefix
to upstream
→ Provider blackholes prefix
AS1 sends p, tagged 2:666
AS5
AS1
AS3 AS4
AS2
BGP announcements
Traffic flow
p2:666
Safeguards:
• Provider should check customer prefix before accepting RTBH
• Customer may only blackhole own prefixes
• Different policies for Customers/Peers
• On receiving RTBH, add NO ADVERTISE or NO EXPORT
(RFC7999)
14
![Page 35: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/35.jpg)
RTBH: how it works
• AS announces BH-prefix
to upstream
→ Provider blackholes prefix
AS2 continues announcing p
AS1 sends p, tagged 2:666
AS5
AS1
AS3 AS4
AS2
BGP announcements
Traffic flow
p2:666
Safeguards:
• Provider should check customer prefix before accepting RTBH
• Customer may only blackhole own prefixes
• Different policies for Customers/Peers
• On receiving RTBH, add NO ADVERTISE or NO EXPORT
(RFC7999)
14
![Page 36: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/36.jpg)
RTBH: how it works
• AS announces BH-prefix
to upstream
→ Provider blackholes prefix
AS2 continues announcing p
AS1 sends p, tagged 2:666
AS5
AS1
AS3 AS4
AS2
BGP announcements
Traffic flow
p2:666
Safeguards:
• Provider should check customer prefix before accepting RTBH
• Customer may only blackhole own prefixes
• Different policies for Customers/Peers
• On receiving RTBH, add NO ADVERTISE or NO EXPORT
(RFC7999)
14
![Page 37: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/37.jpg)
RTBH: how it works
• AS announces BH-prefix
to upstream
→ Provider blackholes prefix X
AS2 continues announcing p
AS1 sends p, tagged 2:666
AS5
AS1
AS3 AS4
AS2
BGP announcements
Traffic flow
p2:666
Safeguards:
• Provider should check customer prefix before accepting RTBH
• Customer may only blackhole own prefixes
• Different policies for Customers/Peers
• On receiving RTBH, add NO ADVERTISE or NO EXPORT
(RFC7999)
14
![Page 38: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/38.jpg)
RTBH: how it works
• AS announces BH-prefix
to upstream
→ Provider blackholes prefix X
AS2 continues announcing p
Traffic to p is dropped at AS2
AS1 sends p, tagged 2:666
AS5
AS1
AS3 AS4
AS2
BGP announcements
Traffic flow
p2:666
Safeguards:
• Provider should check customer prefix before accepting RTBH
• Customer may only blackhole own prefixes
• Different policies for Customers/Peers
• On receiving RTBH, add NO ADVERTISE or NO EXPORT
(RFC7999)
14
![Page 39: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/39.jpg)
RTBH: how it works
• AS announces BH-prefix
to upstream
→ Provider blackholes prefix X
AS2 continues announcing p
Traffic to p is dropped at AS2
AS1 sends p, tagged 2:666
AS5
AS1
AS3 AS4
AS2
BGP announcements
Traffic flow
p2:666
Safeguards:
• Provider should check customer prefix before accepting RTBH
• Customer may only blackhole own prefixes
• Different policies for Customers/Peers
• On receiving RTBH, add NO ADVERTISE or NO EXPORT
(RFC7999)14
![Page 40: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/40.jpg)
RTBH: how it should not work
BGP announcementsAS2
AS4
AS1
AS3
p
p
p
p
p
• AS on ’backup’ path adds RTBH-community
• Provider blackholes prefix
• Not only traffic traversing AS2 is dropped
15
![Page 41: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/41.jpg)
RTBH: how it should not work
AS1 announces p
BGP announcements
Traffic flow
AS2
AS4
AS1
AS3
p
p
p
p
p
• AS on ’backup’ path adds RTBH-community
• Provider blackholes prefix
• Not only traffic traversing AS2 is dropped
15
![Page 42: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/42.jpg)
RTBH: how it should not work
Community
Target
Attackee
Attacker
AS1 announces p
BGP announcements
Traffic flow
AS2
AS4
AS1
AS3
p
p
p
p
p
• AS on ’backup’ path adds RTBH-community
• Provider blackholes prefix
• Not only traffic traversing AS2 is dropped
15
![Page 43: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/43.jpg)
RTBH: how it should not work
Community
Target
Attackee
Attacker
AS2 tags p with AS3:666
AS1 announces p
BGP announcements
Traffic flow
AS2
AS4
AS1
AS3
p
p
p
p
AS3:666
pp
• AS on ’backup’ path adds RTBH-community
• Provider blackholes prefix
• Not only traffic traversing AS2 is dropped
15
![Page 44: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/44.jpg)
RTBH: how it should not work
Community
Target
X
Attackee
Attacker
AS2 tags p with AS3:666
Traffic to p is dropped at AS3
AS1 announces p
BGP announcements
Traffic flow
AS2
AS4
AS1
AS3
p
p
p
p
AS3:666
pp
• AS on ’backup’ path adds RTBH-community
• Provider blackholes prefix
• Not only traffic traversing AS2 is dropped
15
![Page 45: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/45.jpg)
RTBH: how it should not work
Community
Target
X
Attackee
Attacker
AS2 tags p with AS3:666
Traffic to p is dropped at AS3
AS1 announces p
BGP announcements
Traffic flow
AS2
AS4
AS1
AS3
p
p
p
p
AS3:666
pp
• AS on ’backup’ path adds RTBH-community
• Provider blackholes prefix
• Not only traffic traversing AS2 is dropped
15
![Page 46: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/46.jpg)
RTBH: how it should not work (with hijack)
Community
Target
X
Attackee
Attacker
Traffic to p is dropped at AS3
AS1 announces p
BGP announcements
Traffic flow
AS2 hijacks p, with AS3:666
AS2
AS4
AS1
AS3
p
p
AS3:666
pp
• Hijacker announces RTBH
• Prefix filters circumvented due to misconfiguration
• Provider blackholes prefix
16
![Page 47: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/47.jpg)
RTBH: Attack confirmed
Attack confirmed to work on the Internet, works multi hop
and is hard to spot
Triggering RTBH is possible for attackers because, e.g.,:
• BH prefix is more specific, accepted via exception
• Providers check BH community before prefix filters2
• NO ADVERTISE or NO EXPORT often is ignored / not set
• Problem: No validation for origin of community
2we found configuration guides with that bug
17
![Page 48: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/48.jpg)
Traffic redirection attack
AS3
AS6
AS4
AS2AS1
AS5
• Attacker AS2 uses community to add path-prepending in AS3
• AS6 routes traffic towards prefix p via AS5, AS4
• Network tap?
• Slow/Congested link?
• ...
18
![Page 49: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/49.jpg)
Traffic redirection attack
p
BGP−Announcements
AS3
AS6
AS4
AS2AS1
AS5
• Attacker AS2 uses community to add path-prepending in AS3
• AS6 routes traffic towards prefix p via AS5, AS4
• Network tap?
• Slow/Congested link?
• ...
18
![Page 50: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/50.jpg)
Traffic redirection attack
p
Trafficflow
BGP−Announcements
AS−Paths at AS6:
AS3
AS6
AS4
AS2AS1
AS5
p:
p: 5, 4, 2, 1
3, 2, 1
• Attacker AS2 uses community to add path-prepending in AS3
• AS6 routes traffic towards prefix p via AS5, AS4
• Network tap?
• Slow/Congested link?
• ...
18
![Page 51: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/51.jpg)
Traffic redirection attack
p
Attackee Attacker
Community TargetTrafficflow
BGP−Announcements
AS−Paths at AS6:
AS3
AS6
AS4
AS2AS1
AS5
p:
p: 5, 4, 2, 1
3, 2, 1
• Attacker AS2 uses community to add path-prepending in AS3
• AS6 routes traffic towards prefix p via AS5, AS4
• Network tap?
• Slow/Congested link?
• ...
18
![Page 52: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/52.jpg)
Traffic redirection attack
AS3:3x
p
pAttackee Attacker
Community TargetTrafficflow
BGP−Announcements
AS−Paths at AS6:
AS3
AS6
AS4
AS2AS1
AS5
p: 3, 3, 3,
p:
2, 1
5, 4, 2, 1
• Attacker AS2 uses community to add path-prepending in AS3
• AS6 routes traffic towards prefix p via AS5, AS4
• Network tap?
• Slow/Congested link?
• ...
18
![Page 53: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/53.jpg)
Traffic redirection attack
AS3:3x
p
pAttackee Attacker
Community TargetTrafficflow
BGP−Announcements
AS−Paths at AS6:
AS3
AS6
AS4
AS2AS1
AS5
p: 3, 3, 3,
p:
2, 1
5, 4, 2, 1
• Attacker AS2 uses community to add path-prepending in AS3
• AS6 routes traffic towards prefix p via AS5, AS4
• Network tap?
• Slow/Congested link?
• ...
18
![Page 54: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/54.jpg)
Traffic redirection attack
AS3:3x
p
pAttackee Attacker
Community TargetTrafficflow
BGP−Announcements
AS−Paths at AS6:
AS3
AS6
AS4
AS2AS1
AS5
p: 3, 3, 3,
p:
2, 1
5, 4, 2, 1
• Attacker AS2 uses community to add path-prepending in AS3
• AS6 routes traffic towards prefix p via AS5, AS4
• Network tap?
• Slow/Congested link?
• ...
18
![Page 55: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/55.jpg)
Traffic redirection attack
AS3:3x
p
pAttackee Attacker
Community TargetTrafficflow
BGP−Announcements
AS−Paths at AS6:
AS3
AS6
AS4
AS2AS1
AS5
p: 3, 3, 3,
p:
2, 1
5, 4, 2, 1
</>
• Attacker AS2 uses community to add path-prepending in AS3
• AS6 routes traffic towards prefix p via AS5, AS4
• Network tap?
• Slow/Congested link?
• ...
18
![Page 56: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/56.jpg)
Traffic redirection attack
AS3:3x
p
pAttackee Attacker
Community TargetTrafficflow
BGP−Announcements
AS−Paths at AS6:
AS3
AS6
AS4
AS2AS1
AS5
p: 3, 3, 3,
p:
2, 1
5, 4, 2, 1
• Attacker AS2 uses community to add path-prepending in AS3
• AS6 routes traffic towards prefix p via AS5, AS4
• Network tap?
• Slow/Congested link?
• ...
18
![Page 57: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/57.jpg)
Discussion: What now?
![Page 58: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/58.jpg)
BGP Communities Shortcomings Summarized
• Notation of ”ASN:value” is just convention
• No defined semantics: values can mean anything
• Used both for signaling and triggering of actions
• No cryptographic protection
• Attribution is impossible
• Large Communities have, in principle, similar limitations
19
![Page 59: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/59.jpg)
20
![Page 60: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/60.jpg)
BGP Communities: The Problem
• BGP Communities as they are used are not necessarily broken
• Secure usage requires good operational knowledge and
diligence
• While people in this room probably know what they are doing:
Based on experience we do not rely on that globally. . .
Do we need less fragile protocols and mechanisms?
21
![Page 61: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/61.jpg)
BGP Communities: The Problem
• BGP Communities as they are used are not necessarily broken
• Secure usage requires good operational knowledge and
diligence
• While people in this room probably know what they are doing:
Based on experience we do not rely on that globally. . .
Do we need less fragile protocols and mechanisms?
21
![Page 62: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/62.jpg)
Recommendations
• Filter incoming Informational Communities for your ASN
• Publish community documentation, to enable others to filter
• Monitor and log received communities to track abuse
• Talk to your Downstreams, so they filter
Action Communities for your ASN on ingress if neccessary
• Provide a looking glass (that shows communties!)
22
![Page 63: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/63.jpg)
Discussion: Authenticity
• Communities can be modified, added, removed by every AS
• No attribution is possible
• No cryptographic protection
• Still operators rely on their ’correctness’
• Large communities partially improve the situation
How can we achieve authenticity, or at least attribution?
23
![Page 64: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/64.jpg)
Discussion: Transitivity
• Communities can help in debugging
• Easy, low overhead communication channel
• Widely in use, but often only 1-2 hops
• But: High risk of being abused!
Are fully transitive communities still worth the clear risk?
24
![Page 65: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/65.jpg)
Discussion: Monitoring
• There is no global state in BGP
• Route collectors only see the ’end-result’
• Inferring modifications between origin-AS and collector:
almost impossible
• The meaning of a particular community can not be known
• No universal way for attribution of changes
Monitoring communities to detect abuse is extremely
difficult.
25
![Page 66: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/66.jpg)
Discussion: Standards
• There are limited standardized communities
• Many AS do not implement these
• Is the lack of standardized communities a problem?
• Are standards doing harm, by helping attackers?
• Security by obscurity never works
Standardization is necessary.
26
![Page 67: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/67.jpg)
Discussion: Documentation
There is no easy way to find meaning of a community:
• Some ASes document in the whois
• Some ASes document on their website
• Some ASes provide documentation only to customers
• Some ASes do not provide any documentation
Documentation is limited and fragmented.
27
![Page 68: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/68.jpg)
Summary
• Communities are widely in use
• Foundation of many policies
But:
• Relies heavily on mutual trust in capabilities
• No authenticity/security in place
• Attribution is impossible
• Hard to detect attacks
• While our prefix hijacks were reported,
no one reported our community attacks
It’s unknown if there are other unnoticed attacks.
28
![Page 69: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/69.jpg)
Get the preprint at:
https://people.mpi-inf.mpg.de/~fstreibelt/preprint/
communities-imc2018.pdf
Published at ACM IMC 2018
https://conferences.sigcomm.org/imc/2018/
29
![Page 70: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/70.jpg)
30
![Page 71: BGP Communities: A measurement study @RIPE77, Amsterdam · 2018. 10. 16. · RTBH: how it works AS announces BH-pre x to upstream!Provider blackholes pre x AS1 sends p, tagged 2:666](https://reader036.fdocuments.net/reader036/viewer/2022071106/5fdff6f2b1b5e85afc7c4e89/html5/thumbnails/71.jpg)
Contact:
Florian Streibelt <[email protected]>
Images:
Unicorn illustrations: Telegram stickers by Darya Ogneva:
https://tlgrm.eu/stickers/BornToBeAUnicorn
The Spanish Inquisition: by Miki Montllo
http://miquelmontllo.blogspot.com/2013/10/
the-spanish-inquisition-wallpaper.html
31