EARNEST: The Future of Routing & Networking Technology

HEAnet National Networking Conference 200715 November 2007, Kilkenny, Ireland

Slide 2

› EC-funded GN2 project (35 NRENs including HEAnet ):

› Build and operate GÉANT2, the pan-European research and education backbone.

› Joint Research Activities (JRA 1-5) to investigate and develop network enhancements.

› Service Activities (SA 1-6) to procure network and connect NRENs.

› Networking Activities (NA 1-7) to provide user support, dissemination, addressing digital divide, coordination activities, conferences and workshops, and undertake foresight study.

› GN2-NA4 = Education And Research Networking Evolution STudy (EARNEST)

EARNEST Background

Slide 3

› Aims to identify trends, developments, and to make recommendations for future research and education networks.

› Seven sub-studies:

› Organisational and Governance issues

› Economic issues (move to dark fibre, and provision of new services)

› Researchers’ needs (what type of network and services are required?)

› Other users’ needs (e.g. schools, healthcare, arts & humanities)

› Geographic issues (examining and quantifying digital divide)

› Campus issues (infrastructure, services, expertise and collaboration)

› Technical issues (transmission, control plane & routing, network virtualisation, operations and performance, middleware)

› All sub-study reports plus final conclusions available shortly.

› http://www.terena.org/activities/earnest/

EARNEST Background

Slide 4

Technical Study Areas

› Transmission Technologies

› Fibre provisioning, transmission protocols, equipment evolution.

› Control Plane & Routing Technologies

› Switching & routing developments (optical & IP), IPvX, multicasting.

› Operations and Performance

› QoS vs overprovisioning, end-to-end performance, network management (optical & IP), network monitoring, PERT.

› Middleware

› AAIs, identity management, federations, mobility.

› Network Virtualisation

› Customer-enabled networks utilising virtual routers, lightpaths, VPNs, VLLs controlled by UCLP, DRAC etc..

Slide 5

Methodology & Caveats

› Technical panel with expertise in specific areas advised on important or emerging technologies.

› Interviews with key personnel from 11 vendors, 3 research institutes, and a number of NRENs.

› Technological briefings and research papers also used.

› Primary goal was to investigate technologies applicable to NRENs, although attempts to address other types of network as well.

› R&E networks often have different requirements to telco and ISP sectors, and usually have fewer legacy issues.

Slide 6

Transmission Technology Findings

Slide 7

Ethernet or SDH?

› No obvious path for SDH beyond OC-768 (40 Gbps), and likely to become legacy technology in coming years.

› All manufacturers developing 40 and/or 100 Gigabit Ethernet because of cost advantages, and because packet-based services are increasingly prevalent.

› Vendors don’t wish to repeat experience of having to support different variants of 10 GE (i.e. LAN-PHY, WAN-PHY).

› Was initially expected that 100 GE would be next standard, but this is proving to be technically difficult.

Slide 8

Ethernet or SDH?

› 100 GE implementations not expected before 2010, and likely later.

› Initially likely to be 4 x 25 Gbps and restricted to short-haul applications.

› Full serial implementations not expected until at least 2012.

› 40 GE may be interim solution as implementations possible by 2009

› Expected to be 40% the cost of OC-768.

› Supposedly intended for data centre applications, but some vendors talking about WAN capabilities (80 km before amplification/2000 km before regeneration).

Slide 9

Ethernet Enhancements

› Ethernet scalability initially addressed with IEEE 802.1Q and 802.1ad.

› PBB (IEEE 802.1ah) aims to greatly increase number of customer networks, and defines protocols for connecting provider-bridged networks.

› Carrier-grade OAM&P and virtual circuit functionality is also currently being added:

› PBBTE (802.1Qay) will support point-to-point circuits over Ethernet.

› CFM (802.1ag) will support hop-by-hop detection, isolation of connectivity problems

› Shortest-Path Bridging (IEEE 802.1aq) being developed as alternative to Spanning Tree for loop-free forwarding.

http://www.terena.org/activities/ngn-ws/ws1/docs/061107-jacobs-TERENA-NGN-WS-01.pdf

Slide 10

DWDM Systems

› Trade-off between number of wavelengths, faster line rates and longer reaches due to CD, PMD, XPM and FWM.

› New modulation techniques (e.g. DP-QPSK) are becoming practical and promise longer reaches at 40 Gbps+ speeds, whilst minimising need for EDCM.

› Most manufacturers focusing on 50 GHz spacing for DWDM channels (i.e. ~80 channels per fibre). This has been found to provide optimal performance with respect to faster line rates and longer reaches.

› Tunable lasers, VOAs, EDCMs, multi-degree ROADM technology, and PIC-based OEOs promise easier-to-facilitate (and potentially cheaper) DWDM systems. Also make meshed optical networks possible.

› Passive Optical Networks (PONs) being trialled.

Slide 11

DWDM Systems

› Questions to ponder:

› There was a lot of hype about DWDM five years ago, but actually how important is this to NRENs?

› Dark fibre is increasingly available to NRENs, but few fully exploit DWDM possibilities.

› Why is the take-up of DWDM by NRENs so slow?

› Is being ‘faster’ or ‘fatter’ more important to NRENs?

Slide 12

Control Plane & Routing Findings

Slide 13

IP Routing

› Routing scalability becoming problematic (again).

› Huge rise in number of hosts, fragmentation of service provider hierarchy, increase in multihoming, and amount of traffic.

› Global routing table now >230,000 entries, which generates around 400,000 BGP updates per day.

› Concern that growth is starting to outstrip router chipset and memory developments, but more specifically the cost of provisioning these.

› IPv6 doesn’t help as end-users unwilling to use provider-assigned addresses, or renumber when changing service providers.

Slide 14

IP Routing

› Not immediate cause for concern, but IAB/IETF looking for efficiencies.

› Multihoming and traffic engineering should be possible.

› Addresses should be provider-independent

› Work with IPv6, and ideally IPv4.

› Proposals based on splitting IP addresses into identifiers and locators.

› End hosts would have unique identifier regardless of location (EID)

› Locator used for intermediate routing, but is dynamically allocated in accordance with network location (RLOC)

› Locator would be provider dependent and allow for better aggregation.

Slide 15

IP Routing

› How to do it?› NAT sort of achieves the same thing, but uses private IP

addresses which introduce other problems.

› e-FIT would use EIDs within user networks, and encapsulate packets using RLOCs in transit networks. Routing separated between networks.

› LISP also uses encapsulation, but this happens at edge routers. Is transparent to BGP.

› Six/One based on 8+8/GSE and shim6. Lowest 64-bits are unique identifer, but top 64-bits written by edge router (although hosts can suggest route).

› EID-to-RLOC mapping: APT (all nodes hold limited database with default routes), NERD (all nodes hold complete database), and LISP-CONS (distributed query)

› What part of the network should have global overview?

› How to determine default routes?

› How dynamic can/should mapping be?

› Security?

Slide 16

IPv6

› Core IPv6 specifications and related protocols largely completed some years ago.

› Most NRENs already support IPv6 in dual-stack systems, but also tend to have more IPv4 address space.

› Some router and user equipment still has limited support.

› Still limited support in most campuses.

› New predictions suggest IPv4 address space could be exhausted in 3-5 years.

› Regional Internet Registries discussing rationing measures.

Slide 17

IP Multicasting

› Never really taken off in past 20 years, but IPTV in context of ‘triple play’ increasing interest amongst service providers.

› Increased availability to end-users may make it easier to deploy across Internet.

› Inter-domain multicast routing still complex, although SSM may improve situation.

› Automatic Multicast Tunnelling (ATM) allows hosts to find convenient multicast relay if native multicast not available.

› Many peer-to-peer applications already multicast at application layer.

Slide 18

Dynamic-control of hybrid networks

› Lightpaths are still largely manually configured.

› Optical and IP domains still managed separately.

› GMPLS offers possibility of integrating IP routing and WDM control planes (amongst other things).

› In development for long time, but only just starting to be deployed using vendor-specific solutions.

› Still signalling and interoperability issues to resolve, especially between domains.

› Peer or overlay model?

› Probably necessary for fully exploiting hybrid networks, but introduces more complexity.

Slide 19

Network Virtualisation Findings

Slide 20

Network Virtualisation

› Virtualisation concepts starting to be used across all networking layers.

› Basic virtualisation already implemented in certain modern routers to enable upgrades and troubleshooting of specific interfaces, and programmable features.

› NRENs (e.g. CANARIE, CESNET) pioneered customer-empowered network concept, where resources on NREN-provisioned infrastructure can be managed by customers to build logical networks.

› Deployment of UCLP, DRAC and similar technologies are first step towards full network virtualisation.

› Need for technology agnostic infrastructure, although most users still want IP connectivity as part of service.

Slide 21

Network Virtualisation

› MANTICORE and FEDERICA projects aim to develop network virtualisation to allow disruptive technologies to be tested over production infrastructure.

› US-based GENI initiative extends concept to wireless and sensor networks as well.

› EARNEST study revealed there was little knowledge in wider R&E community about virtualisation initiatives, but lot of potential interest.

› TERENA NGN Workshop (06/11/07) had session on network virtualisation/customer-empowered networks.

› Generated much discussion.

› Support for information exchange and coordination activity (e.g. task force).

› Need a better term to describe all this though!

Slide 22

Operations & Performance Findings

Slide 23

Layer 0-2 Management

› NRENs have traditionally only managed Layer 3 and above, so have limited experience at the optical level (WDM systems and/or SDH).

› Limited tools for managing Network Layers 0-2, and expensive.

› Although some R&E developments such as TL1 Toolkit and NDL.

› Management of Layers 0-2 is currently labour intensive and relies heavily on documentation.

› NRENs have not really made extensive use of WDM systems to-date, and the management of much so-called dark fibre is often outsourced.

› Is this something to investigate further?

Slide 24

Overprovisioning vs QoS

› Core networks likely to continue to be overprovisioned as bandwidth is (relatively) cheap.

› Some edge networks do need to undertake traffic engineering though, so QoS transparency should be supported.

› Increasing availability of dark fibre allows R&E networks to operate hybrid networks, enabling dedicated links to be provisioned for demanding customers using C/DWDM.

› Should encourage innovation through network neutrality, subject to traffic engineering requirements.

Slide 25

End-to-End Connectivity

› Most end-to-end performance issues are due to problems at customer sites.

› Middleboxes such firewalls, NATs, rate shapers, caches and other ‘black box’ solutions are responsible for many of these problems.

› This is due to instrinic architecture, misconfigurations, or simply intentional behaviour.

› They encourage workarounds that circumvent what the box is trying to achieve in the first place.

› Consider improving network transparency, either through protocol support, or moving functionality closer to end-hosts.

› Filtering and firewalling should also be weighed against reduction in innovation capabilities within research environment.

› Buggy or sub-optimally tuned software also responsible for some problems (e.g. TCP stacks for large file transfers).

› Consider evolution of PERT concept.

Slide 26

Middleware Findings

Slide 27

› Identity federations are solution for supporting user access to remote services.

› Most NRENs have identity federation or are establishing one. Others should plan to do so within next couple of years.

› NRENs are natural candidates for supporting technical organisation within their countries, as well as representing national federations.

› User-centric identity (e.g. OpenId) management also growing, and abstract identity framework also being worked on. NRENs should monitor developments.

› Already integrations of identity federation and OpenId

Identity Federations

Slide 28

Interoperability

› Inter-operability of identity federation happening:› SAML 2.0 is today choice for exchanging identity data for web-based

applications.

› All the identity federations technologies are SAML2.0-compatible or they migrating to be SAML2.0-compatible.

› Schemas such as eduPerson or SCHAC becoming more important to facilitate inter-operability.

› In order to be able to handle different AAIs it is recommended that NRENs support multiple trust infrastructures: › X.509 certificates used quite a lot.

› SAML signed tokens, coming up.

› It is recommended that NRENs try to minimise number necessary (e.g. by reusing existing PKIs).

› Still open issue: No well established standard for communicating identity data to applications. › NRENs should be proactive about this (possible task force?)

Slide 29

Further Information

› EARNEST Reports

› http://www.terena.org/activities/earnest/publications.html

(Draft Technical Report available, minus some updates)

› TERENA NGN Workshop

› http://www.terena.org/activities/ngn-ws/ws1/

› Thanks to:

Alcatel-Lucent, Calient, Cisco, DTU-COM, DANTE, Extreme Networks, Force10, i2CAT, IBM, Juniper, Liberty Alliance, MERLIN Project, Nortel, Sun Microsystems & SxIP

plus the Advisory Panellists