“Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613...
Transcript of “Internet2Presentaon”€¦ · Project(Route(Total PennREN Fiber route is estimated at 1,613...
“Internet2 Presenta,on” Brian Stengel, Director of Opera-ons, Kinber Peter J. Heverin, Project Manager, Kinber
Mike Carey, Network Engineer, Kinber Jon Paul Herron, Director of Engineering, GlobalNOC
Luke Fowler, Senior Manager, Systems Engineering, GlobalNOC Marianne Chitwood, Director of Opera-ons, GlobalNOC
September 20th, 2012
What is PennREN
• Capital Budget - $128,958,031 – Federal Stimulus Funds - $99,660,678 – Matching Funds - $29,297,353
• Outside Plant Infrastructure Constructed for PennREN – 48 Stands of NZD Fiber optic Cable – 1700± Route Miles – Outsourced Fiber Maintenance
• 13 Optical Regeneration Service Nodes • 56 Service Distribution Access Nodes
a high-speed, state-wide, research & education serving healthcare, ���K-20 and the public good
Project Route
Total PennREN Fiber route is estimated at 1,613 miles. The route consists of the following;
• 1,086 miles of new aerial construction. • 486 miles of aerial overlash. • 14 miles of new underground construction. • 27 miles of leased underground conduit.
Network Backbone Engineering is 95% complete overall.
3
Project Route
4
PennREN Loca,ons Service Nodes
Lehigh University
401 N. Broad
Penn State Hershey Medical Center
Indiana University of Pennsylvania
Allegheny Center Mall
Slippery Rock University of Pennsylvania
Penn State Erie (Behrend)
Clarion University of Pennsylvania
Penn State Dubois
Penn State – State College
Windstream
Bucknell University
University of Scranton
Amp Nodes East Stroudsburg University of Pa
Kutztown University of Pa
West Chester University of Pa
Millersville University of Pa
Shippensburg University of Pa
PiQ Johnstown
Community College of Beaver Co
Allegheny College
PiQ Titusville
Penn State Hazelton
Access Nodes 46 Loca,ons on PennREN fiber
Project Teams
• Hardware procurement • Design • Integra,on • Project Management • Professional Services
• Plan/Design • Op,cal engineering • Field services • Professional services
• Staging • Shipping • Configura,on • Tes,ng • Turn-‐up • Field Services
• Plan/Design • Consul,ng • Engineering • Support • Training
• Fiber OSP • Building entrances • Cabinets, Installa,on • Splicing • Tes,ng • Last mile solu,ons
• Earthlink sourcing for OOB MPLS Network
• Last mile solu,ons
• Service Desk, Tier 1, Tier 2 • Systems and Network Tools • Network Engineering • Net, Perf, Change Management • Opera,ons support
Node Installa,on Status
Segment in Produc,on
Equipment Installed
Under Construc,on
Co-‐Loca,on at Member Sites
Construc,on Diagrams
Op,cal Core/DWDM System
Packet Core – MPLS
Plan – Build – Integrate
Peering Points (Ini,al)
Internet
R&E
R&E
Internet
Network Management
External Networks
External Networks
Out-‐of-‐band access
In-‐band management
Performance Measurement Servers at every Service Node
PS1 PS2 PS3
DB
Opera,ons -‐ Support GRNOC provides to KINBER: • Service Desk – 24x7x365 call center support, ,cket management, technical
support coordina,on, and workflow support • Network Engineering – Expert network engineers work with the Service Desk
to ensure fast problem resolu,on, provisioning, and strategic engineering and planning
• Sofware and Systems – Provides support through a fully integrated system of network management, measurement, and visualiza,on tools
The GlobalNOC at Indiana University provides carrier-‐grade opera,ons, tools, and network exper,se while placing a singular focus on the unique requirements of the research and educa,on (R&E) Community GRNOC supports 20+ R&E networks across the country
Opera,ons -‐ Maintenance
• Warranty • Hardware replacement
(pre-‐ship) • Sofware maintenance
• Warranty • Hardware
replacement (pre-‐ship)
• Sofware maintenance
• Fiber Maintenance • Emergency Restoral • Rou,ne
Maintenance • OSP Records
• Host IT/Site teams provide remote eyes and hands support upon request
• KINBER Network Engineers
Founda,on for Services
West DWDM Ring
East DWDM Ring
MPLS PE Switches
ASBR Router
Mul,-‐Degree ROADMs
External Networks
External Networks
3rd Party Providers
On-‐Net: Delivery
Internet
R&E R&E
Internet
Access Node
Access Node
Access Node
Access Node
Access Nodes
Off-‐Net: Delivery
Internet
R&E R&E
Internet
Member Off-‐Net
First/Last Mile Commercial
Co-‐Lo
CX Carrier
Private
NNI
PennREN
EPC – Ethernet Port Connec,on Service
Customer Router
Service Node: PE Switch
Customer Subscribes to 1/10GE Ethernet Port(s)
R&E networks
VPLS (pt – pt, mul,point)
KMEX
Commodity Internet
Services • Services available to a customer with an EPC
Member A
Member A Member C
Member B
Services • Community-‐wide, distributed service for members to
exchange traffic across a common network. Similar to an Internet Exchange
• Member-‐to-‐Member peering, ad-‐hoc R&E ac,vi,es • Best-‐effort traffic exchange within the community
• Transit service to major R&E networks such as Internet 2, ESNet, NLR…
• Provided by KINBER affiliates
• Virtual private networks with commiYed bandwidth can be established using VPLS instances in the PennREN network
• VPWS – Virtual Private Wire Service – Point to Point • VPLS – Virtual Private LAN Service -‐ Mul,point
• Access to commodity Internet service is available over the PennREN network
Op,cal – Wave/Lambda Services
• Op,cal Waves 10G can be provisioned across the network Member A Member A
Member A Member B Member B
• PerfSONAR Measurement Archives allow exchange of data with other network operators
• Regularly scheduled tes,ng across the backbone
• User-‐ini,ated tes,ng for applica,ons such as problem diagnosis
• Mul,ple rou,ng tables allow us to support both 1G and 10G testpoints on a single host
Performance Measurement
Performance Measurement
Deploying performance measurement servers at each service node
3 servers per site:
– Active throughput measurement (1G and 10G) – Active latency measurement – local data collection / ad hoc performance measurement Specific performance measurement tools include:
BWCTL OWAMP MaDDash
24
Network Measurement
• GlobalNOC tool-‐set updated to support all PennREN devices, including: – SNAPP – High-‐resolu,on SNMP-‐based network u,liza,on data
– LLAMA – DWDM layer performance measurement data
– Central storage of other passive data like syslog, configura,on, flow, etc. for regular and ad-‐hoc processing & analysis.
PennREN Service Desk
• GlobalNOC Specialized Support Technician • Footprints PennREN project created – integrated to
TickMon, Opera,ons Calendars, Trouble Tickets • Telephone number for PennREN customers integrated
into shared GlobalNOC phone queue • Email established [email protected] to receive customer
inquiries and or communica,ons from vendors….this is monitored 24x7x365
PennREN Service Desk
• Network and Member impact guidelines defined
• Web form for customers/vendors to submit trouble ,ckets into Footprints
• Change management process/form created • Internal documenta,on developed/published for staff training and reference
PennREN Service Desk
• Pro-‐ac,ve network monitoring • Support for scheduled maintenances and changes
• Vendor coordina,on • Customer install process • Repor,ng • Security • Tools and Communica,ons
Weighing network needs
Reach
Time
Cost
Capability
A new network is more than a construc,on project
• It’s a system: – Goals – Infrastructure – People – Services – Opera,ons – Business
• Make decisions and plans early (also, there will be more decisions than you think, so watch for boQlenecks)
• The work on all of these areas starts right away
Service Defini,on
• Unrealis,c to imagine services will be completely defined from the beginning of the design
• But, enough understanding is needed to guide the design and build plan
• Early “anchor” users help a lot!
Communica,ons
• Communica,on needs: – to be high-‐bandwidth – Early is good – Changing staff can be disrup,ve, do it carefully. – Informal is good – Mul,-‐channel is good – reliable technologies are good – Face to face is good
Documenta,on
• Documenta,on: – Have a place for documents – Keep the place for documents clean – Keep the purpose of each document clear/dis,nct
Opera,ons
• Opera,ons Preparedness: – Start early, there are a LOT of things to think about, especially:
• Turn-‐up/acceptance process • Expecta,ons for facili,es • How and where to keep network data
Build-‐outs make for lots of informa,on
• Need to be ready to put it somewhere where it’ll be usable later
• Data entry is cheap • Entering/documen,ng is best when the informa,on and its context are fresh
The lowly management network
• Some,mes doesn’t get enough aQen,on • It can get crazy complicated or crazy expensive
• NOTE: in an SDN world, this becomes even MORE important!