Resource Virtualisation of Network Routers

Ross McIlroy, Joe Sventek

(ross@dcs.gla.ac.uk, joe@dcs.gla.ac.uk)

Department of Computing Science

University of Glasgow

QoS in the InternetProposed ApproachImplementation of QuaSARExperimental ResultsExperience GainedConclusions

The Problem

The Internet provides an inherently best effort service

However, many recent Internet applications require Quality of Service guarantees Isochronous network flows

Voice over IP Streaming Video

High Throughput applications Grid Utilities Streaming Video Peer to Peer

Previous Research into Network QoS

Integrated / Differentiated Services Traffic Engineering (e.g. MPLS) Resource Reservation Protocols (e.g. RSVP)

However, less research into how router resources (e.g. processing time) should be provisioned to guarantee QoS to network flows Over provisioning typically used

Expensive Inflexible

Weighted Fair Queuing Complex calculation for each packet Simple queuing of packets within flows

QoS in the InternetProposed Approach

Implementation of QuaSARExperimental ResultsExperience GainedConclusions

Overall Approach

Each QoS network flow is assigned its own virtual router - routelet

Each routelet runs on its own virtual machine Routelets are assigned a proportion of the

router’s resources required to support the flow’s QoS requirements

This provides resource usage partitioning between network flows

QuaSAR Components

Main Router Controls creation of routelets and their assignment

to QoS flows Routes best effort traffic

Routelets Process and route packets from a QoS MPLS flow

Packet Demultiplexing Identifies packets from QoS flows Directs QoS packets to the correct Routelet for

processing

QuaSAR Architecture Overview

Pac

ket

Dem

ult

iple

xer

Main Best-Effort Router

Idle Routelet Pool

QoS Routelets

Inp

ut

NIC

Ch

ann

els

Ou

tpu

t N

IC C

han

nel

s

Possible Applications

Per QoS flow partitioning on access networks

Virtual trunking of traffic on access / medium area networks

Application specific traffic processing for different traffic flows

Packet scheduling within traffic flows

QoS in the InternetProposed Approach

Implementation of QuaSARExperimental ResultsExperience GainedConclusions

Virtualisation in QuaSAR

Virtualisation divides the resources of a computing system into multiple execution environments

A virtualisation layer provides the illusion of a real machine to multiple virtual machines

QuaSAR uses the Xen Virtual Machine Monitor to isolate QoS Routelets within virtual machines Xen chosen due to increase in performance and scalability

caused by paravirtualisation approach Paravirtualisation presents an idealised virtual machine

interface, rather than attempting to fully emulate the real hardware

QuaSAR Hierarchically

Main Best-Effort Router

Idle Routelet Pool

QoS Routelets

Inp

ut

NIC

Ch

ann

els

Ou

tpu

t N

IC C

han

nel

s

Pac

ket

Dem

ult

iple

xer

Packet Demultiplexer

RealHardware

VirtualisationLayer

VirtualMachinesVirtual Machine

Virtual NIC

Routelets Routelets perform simple packet

processing using Click Modular Router Each Routelet only processes one MPLS LSP,

therefore only simple processing needed Routelets could be specialised for certain types of

traffic

Main Router Routes any best effort traffic using standard

Linux Routing Software Creates a Pool of Routelet VMs (initially all idle) Assigns QoS flows to routelets when new flow

creation is signalled using RSVP messages Chooses a routelet for this new flow Modifies Routelet’s packet forwarding engine for this flow Assigns a proportion of the machine’s resources (CPU,

NIC bandwidth, etc) to routelet based upon flow’s flowspec

Updates Demultiplexer to pass packets from this flow to the chosen routelet

Demultiplexing Framework

Click Modular Router used to create a switch which passes packets to the routelet processing their flow Peeks at MPLS label and compares this with a

table of Label / Virtual Network Device pairs Passes packet to the Virtual Network Device

corresponding to the packet’s label Any packets which don’t match are passed to the

best effort router Routelets are linked to the demultiplexer

through Virtual Network Devices Bypasses the Linux Network Stack

Restricting Resource Usage

CPU Usage Xen’s Virtual Machine Scheduler decides when routelets can

make progress in processing packets Soft Real-time Scheduler could provide guarantees about

times to process packets associated with a certain routelet

Network Bandwidth Usage Credit based system to limit each routelet’s maximum

outgoing network usage

Memory Usage Not significantly important with QuaSAR (Linux uses most of

each routelet’s assigned memory) Statically assigned at routelet startup

QoS in the InternetProposed ApproachImplementation of QuaSAR

Experimental ResultsExperience GainedConclusions

Results - Latency

Results - Partitioning

QoS in the InternetProposed ApproachImplementation of QuaSARExperimental Results

Experience GainedConclusions

Problems with QuaSAR

Linux OS running for each routelet Context switch overhead between routelets Memory protection between domains Much of the work per packet occurs during

classification This is assigned to the best effort router, therefore much of

the flow’s work does not occur in its routelet

Routelets have no direct access to physical network devices

No soft real-time scheduler for Xen available during this project

QoS in the InternetProposed ApproachImplementation of QuaSARExperimental ResultsExperience Gained

Conclusions

Conclusions

Virtualisation can be used to partition a router into several distinct routelets QoS flow partitioning opportunities Virtual segregation of network for management

purposes

Application specific routelets can be built for different classes of network traffic

It is possible to build a router which uses virtualisation without a huge loss in performance

QuaSAR Hierarchically (Actual)

Packet Demultiplexer

RealHardware

VirtualisationLayer

VirtualMachines

Domain 0

Domain 0

Demultiplexer - Click Architecture

best effort router

best effort router

best effort router

best effort router

Results - Jitter

Results - Throughput