Network & Services OverviewJune 2012

Jeff Ambernjambern@grnoc.iu.edu

Agenda• Indiana GigaPOP Overview• Services• Commodity Internet Usage Trends• Monon100 (100Gbps to Chicago)

Indiana GigaPOP

• Established in Indianapolis in 1998

• Partnership between Indiana University and Purdue University

• Advanced high-speed, high-availability, feature-rich network

• Drives down the costs and increases the connection speeds for Indiana’s top research colleges and universities

• Serves as an aggregation point for Indiana’s universities to access (regional, national and international) R&E and Commodity Internet.

GigaPOP Participants• Indiana University • Purdue University • I-Light • Notre Dame • RLHEC

• SLIAC • ENA • CSPAN Archives• NCAA

GigaPOP Providers & PeersProviders (Commodity)• Cogent• TimeWarner• Iquest (Local only)• Smithville (Local only)• Wintek (Local only)• Internet2 TRCPS• WiscNet RPS• Akamai (Cache)• Google (Cache)

Peers (R&E)• Internet2 • National LambdaRail• MREN• CIC• ESnet

• Juniper MX960 (Core Nodes)• Fully redundant chassis hardware (cooling, power supplies, Routing

Engines, SCB’s)

• HP 5400 (Layer2 Backbone Switches)• Brocade MLXe (Layer2 Backbone Switch)• HP 3500 (Management Switches for OOB)• Mix of Cisco Routers (Terminal Servers, VPN, Mgt.)

Network Elements

Current Topology

GigaPOP Capabilities• Layer3

• R&E Access (National, Regional and Local)• Commodity Internet Access• L3VPN/MPLS• Multicast

• Layer2 • L2VPN/MPLS• VLANs

• Layer1/2• 100G Ethernet (Fiber)• 10G Ethernet (Fiber)• 1G Ethernet (Fiber, Copper)

Access to Research NetworksR&E (Base Service)

• National/International/Regional Networks (Internet2 (100G), NLR,CIC,

MREN, Esnet)• Local R&E (Gigapop Participants, State Universities)

R&E Service includes IPv4 plus: • IPv6 (we can provide IPv6 address space to members)• Multicast• Jumbo Frames (9000 bytes)

Commodity Internet (Optional to Participants)

• Full Transit• Complete Internet Routing Table• ~410K Routes

• Settlement-Free Internet• Partial Internet Routing Table• ~240K Routes• Commercial Entities agree to peering arrangements with I2 (TRCPS)

or other GigaPOPs to save on upstream BW charges• Internet Caching Servers

• Similar to a Settlement-Free peering arrangement but involving only one entity (i.e. Google, Akamai, Netflix (future peering))

• Server Farm and cached data is local to customer• Does not consume Internet Transit bandwidth for cached data (except

during incremental updates)

Caching Services

Why is caching beneficial to Participants?• Offloading commercial traffic to local servers

reduces our costs that we would have to pay to our per Mbps transit providers

• Reduces the amount of Commodity traffic that would congest our upstream links and force us into costly upgrades (I2 TRCPS, Cogent and TWTC)

• Reducing our costs allows us to pass savings on to our Participants

Caching Services

Google Caching Services added 12/2011• 1.3Gbps 95%• 730Mbps average

Akamai server farm added late 2009• 1Gbps 95%• 500Mbps daily average we would have to pay to our upstream

providers or consume BW on our settlement-free links

Netflix Caching Services / Peering - Late 2012• Currently working with Netflix on traffic analysis• Depending on usage will either Peer at IXP or install cache

node in Indpls.

Commodity Usage - Gbps• Cogent (10G) & TimeWarner

(10G)• Full Internet Transit • Fully Redundant

Upstreams • Full IPv4 & IPv6 Transit

• Internet2 TRCPS Commodity Peering Service (10G)

• Reduces cost through aggregation and settlement-free peering

• WisNet Regional Peering Service (10G) – Via our CIC membership

• Akamai Caching Service (10G)• Google Caching Service (10G)

Bandwidth TrendsCommodity Internet Usage Trends (Aggregate Upstream)• 40-70% increase in usage per year 2009-2012

• Sept 2009 – 2.5G Avg (Cost/Gbps $10000) – 25K/Month• Sept 2010 – 6.5G Avg (Cost/Gbps $6000)• Sept 2011 - 9.5G Avg (Cost/Gbps $5000)• July 2012 – 13.0G Avg (New Pricing $1750/Gbps) – 22.27K/Month

• Able to keep member cost down due to decreases in provider pricing and Settlement-Free Peers

Commodity Bandwidth 2009-2012 24hr Avgs

Monon100

Monon100• Monon100 is named after the Monon Rail line that connected Indiana's higher

education institutions to the rest of the world through Chicago

• The Monon Rail served six colleges and universities along its line (from Chicago to Louisville):o Purdue University in West Lafayette, Indianao Wabash College in Crawfordsville, Indianao DePauw University in Greencastle, Indianao Indiana University in Bloomington, Indianao Butler University in Indianapolis, Indianao St. Joseph's College in Rensselaer, Indiana

• Links the Indiana GigaPoP to Internet2, NLR, CIC, MREN and other networks.

• 10 times faster than our previous network path to Chicago

• Resource available to all Indiana GigaPoP members

Gigapop - Internet2 Update• Connected at 100G to Internet2 in Chicago

• Upgrading optical nodes between Indianapolis and Chicago to support 100G

• Temporary Internet2 100G Wave until Optical upgrades completed

Optical Transition• Phase 1

o Rebuild IPGrid Optical System to be Monon100 Optical System

• Phase 2 o Transition services from temp I2 DWDM to Monon100 Optical System

• Indianao Future growth will include 100GE to IUB via I-Light

• CICo Future growth will include 100GE to CIC in Chicago

Monon100 – Core Upgrades• Installed new Brocade MLXe 16 slot switch in Chicago

• Core Juniper routers have been upgraded to 12.1R1.9o Power supplies upgraded to 4100Wo Enhanced MX SCB (3 per router)o Added High Capacity Fan Trays 100G Cards online (3 at ICTC and 1 at LL)o MPC Type 3 - Modular Port Concentratoro CFP-100G-LR4

Monon100/Indiana GigaPOP

Ilight to ChicgoI-light access to Chicago

Questions?