Internet2 IPv6 Update Matt Zekauskas, [email protected] APAN IPv6 Task Force 2007-August-28.
Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis...
-
Upload
amie-mitchell -
Category
Documents
-
view
220 -
download
3
Transcript of Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis...
![Page 1: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/1.jpg)
Measurement on the Internet2 Network: an evolving story
Matt ZekauskasJoint Techs, Minneapolis
11-Feb-2007
![Page 2: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/2.jpg)
2
Outline
• The Internet2 Observatory• What we are measuring today
• The perfSONAR vision• What is happening in the near term• LHC OPN “e2emon”
![Page 3: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/3.jpg)
3
The Observatory
• Collect data for operations• Understanding the network, and how well it
is operating• How we started
• Collect data for research
• Part of Internet2’s long-standing commitment to network research
![Page 4: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/4.jpg)
4
The Observatory
• Two components• Data collected by NOC and Internet2 itself• Ability for researchers to collocate
equipment when necessary
![Page 5: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/5.jpg)
5
The New Internet2 Network
• Expanded Layer 1, 2 and 3 Facilities• Includes SONET and Wave equipment• Includes Ethernet Services• Greater IP Services
• Requires expanded Observatory
![Page 6: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/6.jpg)
6
In Brief
• Extends to all optical Add/Drop Sites• Add capability:
• Run the control software• Other out-of-band mgmt. tasks
• Refresh of Observatory• Refresh PCs• 10G capabilities on IPO• 10G capability on Ciena Network
(planned, next year)• Experimental NetFPGA Cards
(planned, next year)
• Standing up each node as it is installed
![Page 7: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/7.jpg)
7
The New Internet2 Observatory• Seek Input from the Community, both Engineers and
Network Researchers• Current thinking is to support three types of services
• Measurement (as before)• Collocation (as before)• Experimental Servers to support specific projects - for
example, Phoebus (this is new)
• Support different types of nodes:• Optical Nodes• Router Nodes
![Page 8: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/8.jpg)
8
Existing Observatory Capabilities• One way latency, jitter, loss
• IPv4 and IPv6 (“owamp”)• Regular TCP/UDP throughput tests – ~1 Gbps
• IPv4 and IPv6; On-demand available (“bwctl”)• SNMP
• Octets, packets, errors; collected 1/min• Flow data
• Addresses anonymized by 0-ing the low order 11 bits• Routing updates
• Both IGP and BGP - Measurement device participates in both• Router configuration
• Visible Backbone – Collect 1/hr from all routers• Dynamic updates
• Syslog; also alarm generation (~nagios); polling via router proxy
![Page 9: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/9.jpg)
9
Observatory FunctionsDevice Function Detailsnms-rthr1 Measurement BWCTL on-demand 1 Gpbs router
throughput, Thrulay
nms-rthr2 Measurement BWCTL on-demand 10 Gbps router throughput, Thrulay
nms-rexp Experimental NDT/NPAD
nms-rpsv Measurement Netflow collector
nms-rlat Measurement OWAMP with locally attached GPS timing
nms-rpho Experimental Phoebus 2 x 10GE to Multiservice Switch
nms-octr Management Controls Multiservice Switch
nms-oexp Experimental NetFPGA
nms-othr Measurement On-demand Multiservice Switch 10 Gbps throughput
![Page 10: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/10.jpg)
10
Router Nodes
![Page 11: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/11.jpg)
11
Router Nodes
![Page 12: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/12.jpg)
12
Optical Nodes
![Page 13: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/13.jpg)
13
Optical Nodes
![Page 14: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/14.jpg)
14
Observatory Hardware
• Dell 1950 and Dell 2950 servers• Dual Core 3.0 GHz Xeon processors• 2 GB memory• Dual RAID 146 GB disk• Integrated 1 GE copper interfaces• 10 GE interfaces
• Hewlett-Packard 10GE switches• 9 servers at router sites, 3 planned at optical
only sites (initially 1 - control)
![Page 15: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/15.jpg)
15
Observatory Databases – Datа Types
•Data is collected locally and stored in distributed databases•Databases•Usage Data•Netflow Data •Routing Data • Latency Data • Throughput Data •Router Data • Syslog Data
![Page 16: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/16.jpg)
16
Lots of Work to be Done
• Internet2 Observatory realization inside racks set for initial deployment, including planning for research projects (NetFPGA, Phoebus)
• Software and links easily changed• Could add or change hardware depending on
costs• Researcher tools, new datasets• Consensus on passive data
![Page 17: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/17.jpg)
17
New Challenges
• Operations and Characterization of new services• Finding problems with stitched together VLANs• Collecting and exporting data from Dynamic Circuit
Service...• Ciena performance counters• Control plane setup information• Circuit usage (not utilization, although that is also nice)
• Similar for underlying Infinera equipment
• And consider inter-domain issues
![Page 18: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/18.jpg)
18
Observatory Requirements Strawman
• Small group: Dan Magorian, Joe Metzger and Internet2
• See document off of http://measurement.internet2.edu/
• Want to start working group under new Network Technical Advisory Committee• Interested? Talk to Matt or watch NTAC Wiki on
wiki.internet2.edu; measurement page will also have some information…
![Page 19: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/19.jpg)
19
Strawman: Potential New Focus Areas
• Technology Issues• Is it working? How well? How debug
problems?
• Economy Issues – interdomain circuits• How are they used? Are they used
effectively? Monitor violation of any rules (e.g. for short-term circuits)• Compare with “vanilla” IP services?
![Page 20: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/20.jpg)
20
Strawman: Potential High-Level Goals
• Extend research datasets to new equipment• Circuit “weathermap”; optical proxy• Auditing Circuits• Who requested (at suitable granularity)• What for? (ex: bulk data, streaming media,
experiment control)• Why? (add’l bw, required characteristics,
application isolation, security)
![Page 21: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/21.jpg)
21
Inter-Domain Issues Important
• New services (various circuits)
• New control plane
• That must work across domains
• Will require some agreement among various providers
• Want to allow for diversity…
![Page 22: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/22.jpg)
22
Sharing Observatory Data
We want to make Internet2 Network Observatory Data:
• Available:• Access to existing active and passive measurement
data• Ability to run new active measurement tests
• Interoperable:• Common schema and semantics, shared across
other networks• Single format• XML-based discovery of what’s available
![Page 23: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/23.jpg)
23
What is perfSONAR?
• Performance Middleware• perfSONAR is an international consortium in which
Internet2 is a founder and leading participant• perfSONAR is a set of protocol standards for
interoperability between measurement and monitoring systems• perfSONAR is a set of open source web services
that can be mixed-and-matched and extended to create a performance monitoring framework
![Page 24: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/24.jpg)
24
perfSONAR Design Goals
• Standards-based• Modular• Decentralized• Locally controlled• Open Source• Extensible• Applicable to multiple generations of network
monitoring systems• Grows “beyond our control”• Customized for individual science disciplines
![Page 25: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/25.jpg)
25
perfSONAR Integrates
• Network measurement tools
• Network measurement archives
• Discovery
• Authentication and authorization
• Data manipulation
• Resource protection
• Topology
![Page 26: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/26.jpg)
26
perfSONAR Credits•perfSONAR is a joint effort:
• ESnet• GÉANT2 JRA1• Internet2• RNP
•ESnet includes:• ESnet/LBL staff• Fermilab
•Internet2 includes:• University of Delaware• Georgia Tech• SLAC• Internet2 staff
•GÉANT2 JRA1 includes:• Arnes• Belnet• Carnet• Cesnet• CYNet• DANTE• DFN• FCCN• GRNet• GARR• ISTF• PSNC• Nordunet (Uninett)• Renater• RedIRIS• Surfnet• SWITCH
![Page 27: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/27.jpg)
27
perfSONAR Adoption
•R&E Networks• Internet2• ESnet• GÉANT2• European NRENs• RNP
•Application Communities• LHC• GLORIAD Distributed
Virtual NOC• Roll-out to other
application communities in 2007
•Distributed Development• Individual projects (10
before first release) write components that integrate into the overall framework
• Individual communities (5 before first release) write their own analysis and visualization software
![Page 28: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/28.jpg)
28
Proposed Data to be made available via perfSONAR
• First Priorities• Link status (CIENA data)• SNMP data• OWAMP• BWCTL
• Second Priorities• Additional CIENA data
• Ethernet stats• SONET (Severely errored seconds, etc.)• Light levels
• Similar Infinera data• Later: Flow data
• Feedback? Alternate priorities?
![Page 29: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/29.jpg)
29
What will (eventually) consume data?
• We intend to create a series of web pages that will display the data
• Third-party Analysis/Visualization Tools• European and Brazilian UIs• SLAC-built analysis software• LHC OPN E2EMON• More …
• Real applications• Network-aware applications
• Consume performance data• React to network conditions• Request dynamic provisioning
• Future Example: Phoebus
![Page 30: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/30.jpg)
JRA4 E2EMon slides
From Mauro Campanella, GARR, 2006-Nov
Demo: http://cnmdev.lrz-muenchen.de/e2e/html/G2_E2E_index.html
![Page 31: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/31.jpg)
Connect. Communicate. Collaborate
31The Italian Research and Education Network
Hopi Meeting3 Nov 2006
Problem space
Point A
PointB
Domain A
Domain B
Domain C
Goal: (near) real-time monitoring (link status) of constituent DomainLinks
(and links between domains) and whole end-to-end Link A-B.
The following applies to the GÉANT2+ service and the cross border fibres.
E2ELink A-B
![Page 32: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/32.jpg)
Connect. Communicate. Collaborate
32The Italian Research and Education Network
Hopi Meeting3 Nov 2006
Divide & conquer(JRA4 E2Emon info model) Connect. Communicate. Collaborate
JRA4 view of world:
note WDM systems,
& static lambdas
![Page 33: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/33.jpg)
Connect. Communicate. Collaborate
33The Italian Research and Education Network
Hopi Meeting3 Nov 2006
Approach
Point A
PointB
Domain A
Domain B
Domain C
E2ELink A-B
perfSONARMP or MA
perfSONARMP or MA
E2Emoncorrelator
perfSONARMeasurement Point (MP)
or Measurement Archive (MA)
DomainLink and(partial) ID_Link info
“Weathermap”view for users
E2ECUoperators
![Page 34: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/34.jpg)
Connect. Communicate. Collaborate
34The Italian Research and Education Network
Hopi Meeting3 Nov 2006
GARR
SWITCH
CNAFXBO
MIPDKARLSRUHE
X
DFN
XWDM
WDM
WDM
Manno
• e2e lightpath
from CNAF (Bologna, Italy) to Karlsruhe (Germany)• The logical topology built for the e2e monitoring system
abstracts the internal topology of each domain and produces a simpler topology.
LHC-OPN e2e Monitoring
![Page 35: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/35.jpg)
Connect. Communicate. Collaborate
35The Italian Research and Education Network
Hopi Meeting3 Nov 2006
Domain1 Domain2 Domain3, Domain4 Domain5
EP Demarcation Point DP DP End Point
ID Link Domain Link Other Domain LinksID Link ID Link
GARR
SWITCH
CNAFXBO
MIPDKARLSRUHE
X
DFN
XWDM
WDM
WDM
Manno
LHC-OPN e2e Monitoring
![Page 36: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/36.jpg)
Connect. Communicate. Collaborate
36The Italian Research and Education Network
Hopi Meeting3 Nov 2006
Network 2Network 2Network 1Network 1Network nNetwork n
Domain 1 MPDomain 1 MP Domain 2 MPDomain 2 MP Domain n MPDomain n MP
Domain 1 MADomain 1 MADomain 2 MADomain 2 MA Domain n MADomain n MA
E2E Monitoring SystemE2E Monitoring System User
User
web services
script
pollsacquisition
domain aggregation and xml generation
interdomain aggregation
LHC-OPN e2e Monitoring
![Page 37: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/37.jpg)
Connect. Communicate. Collaborate
37The Italian Research and Education Network
Hopi Meeting3 Nov 2006
GEANT2
GINS (the GARR network monitoring system) checks the status of the logical circuits in the GARR domain and provides the result to the GARR MP. The central e2e measurement system queries each domain and provides the global e2e status. This shows the domain independency, the possibility to easily aggregate the information and its scalability.
GARR
end point
IP Link
X
X
IL MONITORING
GARR monitoring domain
CNAF GINS e2e Monitor
XML Data
GARR MP E2E MSMPLS LSP IP/L2 Link
CNAF - CERNGARR monitoring flow
![Page 38: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/38.jpg)
Connect. Communicate. Collaborate
38The Italian Research and Education Network
Hopi Meeting3 Nov 2006
IP MPLS lambda
GARR
SWITCH
CNAFXBO
MIPDKARLSRUHE
DFN
WDM
WDM
WDM
MannoX X
lambda lambda GINS e2e Service
check the status of segments
GINSUser
GINSUser
E2E Monitoring System
E2E Monitoring System
(status aggregation)
GARR monitoring domain
![Page 39: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/39.jpg)
Connect. Communicate. Collaborate
39The Italian Research and Education Network
Hopi Meeting3 Nov 2006
GARR User Interface
![Page 40: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/40.jpg)
Connect. Communicate. Collaborate
40The Italian Research and Education Network
Hopi Meeting3 Nov 2006
VISUALIZZAZIONECNAF - CERNE2E MS user interface
![Page 41: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/41.jpg)
Connect. Communicate. Collaborate
41The Italian Research and Education Network
Hopi Meeting3 Nov 2006
VISUALIZZAZIONE
(Slides from Marco Marletta , Giovanni Cesaroni GARR)
CNAF - CERNGARR GINS user interface
![Page 42: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/42.jpg)
Connect. Communicate. Collaborate
42The Italian Research and Education Network
Hopi Meeting3 Nov 2006
Measurement System Future work - wish list
• Define & implement “degraded” link status• Add scheduled maintenance indication• Add more detail to data model
– Break down DomainLink into constituent parts?(e.g. OCh trails)
– use more info from equipment
![Page 43: Measurement on the Internet2 Network: an evolving story Matt Zekauskas Joint Techs, Minneapolis 11-Feb-2007.](https://reader036.fdocuments.net/reader036/viewer/2022081516/56649f205503460f94c38647/html5/thumbnails/43.jpg)
43