512K IPv4 Routes - NANOG Archive · Affected prefixes may represent equipment suffering from TCAM...
Transcript of 512K IPv4 Routes - NANOG Archive · Affected prefixes may represent equipment suffering from TCAM...
![Page 1: 512K IPv4 Routes - NANOG Archive · Affected prefixes may represent equipment suffering from TCAM exhaustion • Consensus routing table size was roughly 500,000 that morning •](https://reader034.fdocuments.net/reader034/viewer/2022050511/5f9b9050d10f535dc0768903/html5/thumbnails/1.jpg)
512K IPv4 Routes: The New Normal
Jim Cowie
Chief Scientist, Dyn Research
NANOG 62, Baltimore
7 October 2014
![Page 2: 512K IPv4 Routes - NANOG Archive · Affected prefixes may represent equipment suffering from TCAM exhaustion • Consensus routing table size was roughly 500,000 that morning •](https://reader034.fdocuments.net/reader034/viewer/2022050511/5f9b9050d10f535dc0768903/html5/thumbnails/2.jpg)
Pg. 2
Up And To The Right: IPv4 Routing Table Sizes
2009 2010 2011 2012 2013 2014…
512K
![Page 3: 512K IPv4 Routes - NANOG Archive · Affected prefixes may represent equipment suffering from TCAM exhaustion • Consensus routing table size was roughly 500,000 that morning •](https://reader034.fdocuments.net/reader034/viewer/2022050511/5f9b9050d10f535dc0768903/html5/thumbnails/3.jpg)
Pg. 3
Up And To The Right: IPv4 Routing Table Sizes
2009 2010 2011 2012 2013 2014…
512K
269K +30K
302K +33K
334K +32K
382K +48K
431K +48K
471K +40K
![Page 4: 512K IPv4 Routes - NANOG Archive · Affected prefixes may represent equipment suffering from TCAM exhaustion • Consensus routing table size was roughly 500,000 that morning •](https://reader034.fdocuments.net/reader034/viewer/2022050511/5f9b9050d10f535dc0768903/html5/thumbnails/4.jpg)
Pg. 4
Up And To The Right: IPv4 Routing Table Sizes
2009 2010 2011 2012 2013 2014…
512K
269K +30K
302K +33K
334K +32K
382K +48K
431K +48K
471K +40K
519K? +48K?
2015?
![Page 5: 512K IPv4 Routes - NANOG Archive · Affected prefixes may represent equipment suffering from TCAM exhaustion • Consensus routing table size was roughly 500,000 that morning •](https://reader034.fdocuments.net/reader034/viewer/2022050511/5f9b9050d10f535dc0768903/html5/thumbnails/5.jpg)
Pg. 5
“512K day” …. November 2014 (predicted) Let’s Take a Closer Look
Up And To The Right: IPv4 Routing Table Sizes
2009 2010 2011 2012 2013 2014…
![Page 6: 512K IPv4 Routes - NANOG Archive · Affected prefixes may represent equipment suffering from TCAM exhaustion • Consensus routing table size was roughly 500,000 that morning •](https://reader034.fdocuments.net/reader034/viewer/2022050511/5f9b9050d10f535dc0768903/html5/thumbnails/6.jpg)
Pg. 6
“Consensus” is Broadening
2013 2014
Today’s consensus estimate: 507K (October 6th 2014) 61% of our peers believe it’s higher than consensus 23% believe it’s higher than 512K already
512K
Today: 507K
![Page 7: 512K IPv4 Routes - NANOG Archive · Affected prefixes may represent equipment suffering from TCAM exhaustion • Consensus routing table size was roughly 500,000 that morning •](https://reader034.fdocuments.net/reader034/viewer/2022050511/5f9b9050d10f535dc0768903/html5/thumbnails/7.jpg)
Pg. 7
2013 2014
512K
Today: 507K
Invisible at 1hr resolution. This was a “normal intraday fluctuation” …Except that we’re so close to 512K. Global table pushed across the line….
“512K Day” Panic: 12 August 2014 (7:49:30 UTC)
![Page 8: 512K IPv4 Routes - NANOG Archive · Affected prefixes may represent equipment suffering from TCAM exhaustion • Consensus routing table size was roughly 500,000 that morning •](https://reader034.fdocuments.net/reader034/viewer/2022050511/5f9b9050d10f535dc0768903/html5/thumbnails/8.jpg)
Pg. 8
One-second BGP announce/withdraw rates (logscale)
06h00-09h00 UTC 12 August 2014
• Advertisement rate peaks first • Withdrawal rate briefly rivals advertisement rate
![Page 9: 512K IPv4 Routes - NANOG Archive · Affected prefixes may represent equipment suffering from TCAM exhaustion • Consensus routing table size was roughly 500,000 that morning •](https://reader034.fdocuments.net/reader034/viewer/2022050511/5f9b9050d10f535dc0768903/html5/thumbnails/9.jpg)
Pg. 9
Affected prefixes may represent equipment suffering from TCAM exhaustion
• Consensus routing table size was roughly 500,000 that morning
• VZ (AS701) deaggregated almost 30,000 /24s at 7:48:38 UTC
• Consensus routing table size hits 528,000 for 10 minutes
• 528K > 512K; All hell breaks loose
• Aggregates withdrawn, table reverts to safely less than 512K
![Page 10: 512K IPv4 Routes - NANOG Archive · Affected prefixes may represent equipment suffering from TCAM exhaustion • Consensus routing table size was roughly 500,000 that morning •](https://reader034.fdocuments.net/reader034/viewer/2022050511/5f9b9050d10f535dc0768903/html5/thumbnails/10.jpg)
Pg. 10
One-second BGP withdrawal rates, logscale
Most of the withdrawals in the hour are AS701 deaggregates… But 24,280 other prefixes are affected as well.
Rate climbs 60x in 10m
![Page 11: 512K IPv4 Routes - NANOG Archive · Affected prefixes may represent equipment suffering from TCAM exhaustion • Consensus routing table size was roughly 500,000 that morning •](https://reader034.fdocuments.net/reader034/viewer/2022050511/5f9b9050d10f535dc0768903/html5/thumbnails/11.jpg)
Pg. 11
Affected Prefixes, By Country
It’s not your circulatory system
2479
4494
1014
571
492 389
1084
860
727
398
595 Relatively significant impacts in Iran, Thailand, Indonesia, Argentina, Russia
![Page 12: 512K IPv4 Routes - NANOG Archive · Affected prefixes may represent equipment suffering from TCAM exhaustion • Consensus routing table size was roughly 500,000 that morning •](https://reader034.fdocuments.net/reader034/viewer/2022050511/5f9b9050d10f535dc0768903/html5/thumbnails/12.jpg)
Pg. 12
Affected routers in critical places
Russian provider Vimpelcom’s AS21332 suffers some significant impairment during the event, as seen through the lens of traceroute completion failure.
![Page 13: 512K IPv4 Routes - NANOG Archive · Affected prefixes may represent equipment suffering from TCAM exhaustion • Consensus routing table size was roughly 500,000 that morning •](https://reader034.fdocuments.net/reader034/viewer/2022050511/5f9b9050d10f535dc0768903/html5/thumbnails/13.jpg)
Pg. 13
Affected routers in critical places
Bayan (PH) loses its Pacnet transit.. twice
![Page 14: 512K IPv4 Routes - NANOG Archive · Affected prefixes may represent equipment suffering from TCAM exhaustion • Consensus routing table size was roughly 500,000 that morning •](https://reader034.fdocuments.net/reader034/viewer/2022050511/5f9b9050d10f535dc0768903/html5/thumbnails/14.jpg)
Pg. 14
Conclusions
• Affected countries/networks are likely to suffer again as consensus table reaches 512K organically, “for real”
• We project that this will begin occuring broadly later this month, with consensus 512K emerging by mid-November
• Hopefully this brief event in August was enough to encourage people to upgrade and/or reallocate TCAM appropriately…..
![Page 15: 512K IPv4 Routes - NANOG Archive · Affected prefixes may represent equipment suffering from TCAM exhaustion • Consensus routing table size was roughly 500,000 that morning •](https://reader034.fdocuments.net/reader034/viewer/2022050511/5f9b9050d10f535dc0768903/html5/thumbnails/15.jpg)
Pg. 15
Thank you! Jim Cowie @jimcowie
Dyn Research http://research.dyn.com