Status Report on US networks at the Turn of the Century
description
Transcript of Status Report on US networks at the Turn of the Century
1
Status Report on US networks at the Turn of the Century
Les Cottrell – SLAC & Stanford U.www.slac.stanford.edu/grp/scs/net/talk/us-net-status-2000.htm
Presented at CHEP00, Padua Italy, February 9, 2000
Partially funded by DOE/MICS Field Work Proposal on Internet End-to-end Performance Monitoring (IEPM), also supported by IUPAP
2
Overview• U.S. Networks:
– Internet 2
– Federal networks, in particular ESnet
– Commercial
• Performance seen from U.S.: – Compare Internet2 vs. ESnet vs Commercial
– Performance to Europe and rest of world
– Trends
• Summary
3
US National Networks• Internet 2 - universities
– Abilene & vBNS backbones
• Federal backbone networks– ESnet (DoE), DREN (DoD), NREN (NSF), NSI
(NASA) ...
• Commercial Internet Service Providers (ISPs)
• Interconnection points (MAEs, NAPs, NGIXs, & colocation points …)
4
Internet 2• A project by consortium of universities (UCAID) to:
– foster development of advanced internet applications;
– foster development internet technology itself;
– provide a high performance network for general research.
• Not a Government project; no direct Federal subsidy
• Not a network itself:
– The NSF-funded vBNS evolved into a ‘pre’ Internet-2 backbone;
– Abilene is the UCAID-sponsored Internet-2 backbone.
5
Internet 2 Membership• Internet 2
– 170 universities– ~ 10-12 non-university members including CERN– other networks can connect as affiliates;– U.S. National Labs are not members of UCAID:
• National Lab Internet-2 participation assumed thru ESnet
• vBNS– 101 institutions connected– 22 peer networks
• Abilene– 96 participants, 75 connected– 15 peer networks
6
Internet 2 AUP• Internet 2 Acceptable Use Policy (AUP)
prohibits transit traffic between affiliates:– ESnet site with Internet-2 member site is ‘OK’ to use I2;– ESnet site to other affiliated network site via Internet-2
backbone is NOT permitted.
• Internet 2 AUP also prohibits “commodity” internet traffic!– Traffic to another Internet-2 site routed via Internet-2
backbone;– Traffic to a non-Internet-2 site must be routed via
commercial ISP;– Requires universities to have a separate Internet
connection for commodity traffic.
7
• 5500 miles deployed, backbone operates at speeds up 2.4Gbps. Interconnections at 155 & 622Mbps
Abilene
Seattle San Fran.
CalRen UCB
Stanford
UCSFUCDavis
UCSC
• Universities connect to GigaPoPs, GigaPoPs connect to backbone
• Peer with: APAN/Transpac, CA*net-2, DANTE, DFN, DREN, ESnet, ILAN, INFN, JANET, NACSIS, NORDunet, NISN, NREN, RENATER, SingAREN, SURFnet, vBNS …
Red=N.America, Blue=Europe, Green=Asia
ESnet
8
Abilene Network, Jan-2000
9
vBNS• ATM-based OC12 (622Mbps) backbone, with 2.4 Gbps IP-
over-SONET segments on parts of the backbone:– 45Mbps is the minimum connection line speed;– 98 connections as of 6/15, with 7 more pending;
• Run by MCI; subsidized out of NSF NGI funds;– NSF vBNS contract expires April 1st, 2000;
– June 99 MCI announces vBNS+ a 5 year agreement with EDUCAUSE
• 1750 higher education institutions members• vBNS+ no restrictive NSF imposed AUP, • Connection speeds from 1.5Mbps - 2.5Gbps• peering: with vBNS requires NSF authorization, none at
moment with Abilene, will offer to FEDnet and International nets
10
vBNS (101 institutions)
11
ESnet• DOE Energy Sciences Research Network
• Connects DOE ER labs and universities with major DoE funded projects– ~ 50 sites– Mainly 155 Mbps backbone with some 622Mbps links
• Peers with other major networks– 13 Internet Interconnect points– Peering exchange at MAE-West, MAE-East, Sprint NAP,
Ameritech NAP, PacBell NAP
• International connections– CERN, DFN, INFN, JAERI, KEK, Moscow, NIFS
12
ESnet• 100% growth/year since 1990
• New contract, Sprint did not bid– New contract with Qwest announced January 4, 2000 – 1 year transition, 2 concurrent contracts for coming year
means funding tight– Same supplier as US-CERN link (KPN-Qwest)– Initial deployment ATM based
• ESnet3 backbone to be Tbit/sec by 2003-2005– 5 major hubs see next transparency ...
13
ESnet-3 Initial Configuration
SNV
ALBORN
NYC
CHI
LANL
SNLA
BNLTELEHOUSE
OC48-ATM
OC12-ATM
OC3-ATM
OC48-SONET
OC3-SONET
T3-ATM
T3
MIT
CHI-NAP PPPL
ORNL
ATL
SRS
ANL
FNAL
AMES
PANTEX
JLAB
GTN
ASIG
60-HUD
DCOffices
DC
SNLL
LLNL
LBNL
NERSC
FIX-W
PB-NAP
MAE-W
OC12-SONET
SLAC
MAE-E
GA
JGI
PNNL
GA
(SDSC)
YUCCA-MT
(BECHTEL)
SEA
(SAIC)
FULL-MESHEDATM CORE
INEEL
JAnetSURFnet
NORDUnetAbilene
DFNINFN
DANTE
OC3?
CanadaFranceCERNKEK/China
Japan/Russia
Courtesy of Jim Leighton/ESnet
14
Testbeds for U.S. NRENs• “Foster Development of Internet Technology”
– IPv6 = Next Generation Internet• volunteers see [email protected]
– QoS - VoIP, multimedia and data transfer– Computing & Data grids– Collaboratories - video, virtual reality, electronic
notebooks, multicast ..– Middleware - PKI, directories ...
15
Commercial Internet• Important to HENP
– ~ 18% of US HEP universities still rely solely on commercial ISPs for internet access:
• Internet-2 participation, even subsidized, isn’t cheap…
– Critical information needed from commercial sites
• Quality of US commercial Internet Service is improved:– Commercial ISPs have been keeping their backbone
capacity in line with (or ahead of…) demand;– Network Access Point (NAP) congestion is down.
• ISPs match research networks technologically:– ISPs are ahead in rollout of high bandwidth links;– ISPs are pursuing Quality of Service solutions
16
Commercial traffic: Ames Internet Exchange (AIX)
https://anala.caida.org/AIX/• TCP 90%, UDP 10%, Web ~ 55%, FTP ~ 5%, mail
~ 3%• Game traffic represents noticeable portion
– Quake & Starcraft account for 5% in summer, 2-3% term-time
– Fairly constant (0.5%)
• Real audio declining (factor 2 in 6 months, now 1%)• IPSEC traffic, small (< 0.2%) but growing (factor 3
in 6 months)• Spikes in ICMP (security scans?)
17
Performance• Measurements from
– 28 monitors in 15 countries
– Over 500 remote hosts– 72 countries (covers all 56 PDG booklet countries)
– Over 1200 monitor-remote site pairs
• Over 50% of HENP collaborator sites are explicitly monitored as remote sites by PingER project– Atlas (37%), BaBar (68%), Belle (23%), CDF (73%),
CMS (31%), D0 (60%), LEP (44%), Zeus (35%), PPDG (100%), RHIC(64%)
18
How are the U.S. Nets
doing?
In general performance is good (i.e. <= 1%).
Edu (vBNS/Abilene) is catching up with ESnet
XIWT (70% .com) 3-5 times worse than ESnet | I2
19
Europe seen from U.S.
650ms
200 ms
7% loss10% loss
1% loss
Monitor siteBeacon site (~10% sites)HENP countryNot HENPNot HENP & not monitored
20
Asia seen from U.S.
3.6% loss
10% loss
0.1% loss
640 ms
450 ms
250ms
21
Latin America, Africa & Australasia4% Loss
2% Loss
350 ms
700ms
170 ms
220 ms
22
Bulk transfer - Performance TrendsBandwidth TCP < 1460/(RTT * sqrt(loss))
Note: E. Europe NOT catching up
23
Summary• US HEP research network research environment is
improving…
• The Internet-2 project is already having positive results for collaborative research:– More research universities have high bandwidth, low
latency access to major U.S. research facilities;– Mission-specific research networks have something to
direct improving university access efforts at…;– Testbed projects emerging for new network technologies.
• International performance from US to sites outside W. Europe, Japan, Korea is generally poor to bad
24
More Information• ESnet home page
– http://www.es.net/
• Internet 2 home page– http://www.internet2.edu/
• vBNS+ home page– http://www.vbns.net/vBNS+/index.html
• IEPM/PingER home site– http://www-iepm.slac.stanford.edu/
• ICFA-SCIC Homepage– http://www.hep.net/ICFA/index.html
25
Peering• Not always optimal
– paths may go through congested exchange points -increased loss
– paths may be very indirect (e.g. KEK to SLAC was via NY) - adds 80 msec to RTT
Performance28 monitors in 15 countriesOver 500 remote hostsOver 1200 pairs72 countriesOver 50% HENP sites are monitored directly