Breaking the Terabit Barrier - · PDF fileA Sandvine Technology Showcase ... mitigate attack...

15
A Sandvine Technology Showcase Contents Executive Summary ................................... 1 Introduction ............................................ 2 Network Policy Control and Network Functions Virtualization ....................................... 2 Network Functions Virtualization Benchmarks and Milestones ...................................... 3 Building a One Terabit per Second Policy Control VNF ...................................................... 4 Components ......................................... 4 Configuration .................................... 5 Testing ............................................... 6 Achieving Sufficient Load ...................... 7 Ensuring a Realistic Test ....................... 7 Results ............................................... 8 Performance Scale ............................. 10 Performance Efficiency ....................... 11 Conclusions ............................................ 13 Optimizations and Extensions ................... 13 Related Resources................................. 14 Invitation to Provide Feedback ................. 14 Executive Summary This document explains how Sandvine, Dell®, and Intel®, using standards-based virtualization technologies, have achieved data plane performance scale and efficiency within a policy control virtual network function (VNF) that exceeds those of proprietary network appliances. Data Plane Inspection Throughput: 1.1 Tbps (using realistic traffic, including diverse protocols, encapsulation and tunneling) Rack-Space: 10 rack units (RU) (includes all compute, storage, and network components) Peak Power Consumption: 4995 W Performance Density: 110 Gbps/RU Power Efficiency: 4.5 W/Gbps These figures are all superior to those provided by network appliance vendors, and should effectively put to rest any concerns or questions surrounding the practical and commercial viability of network functions virtualization for policy control use cases. This demonstration proves that network functions virtualization is a cost effective alternative to proprietary network appliances, and that virtualized policy control deployments can meet the scale, efficiency, and business objectives of communications service providers. Breaking the Terabit Barrier: A One Terabit per Second Policy Control Virtual Network Function

Transcript of Breaking the Terabit Barrier - · PDF fileA Sandvine Technology Showcase ... mitigate attack...

Page 1: Breaking the Terabit Barrier - · PDF fileA Sandvine Technology Showcase ... mitigate attack traffic, manage congestion, ... software applications performing network functions share

A Sandvine Technology Showcase

Contents

Executive Summary ................................... 1

Introduction ............................................ 2

Network Policy Control and Network Functions

Virtualization ....................................... 2

Network Functions Virtualization Benchmarks

and Milestones ...................................... 3

Building a One Terabit per Second Policy Control

VNF ...................................................... 4

Components ......................................... 4

Configuration .................................... 5

Testing ............................................... 6

Achieving Sufficient Load ...................... 7

Ensuring a Realistic Test ....................... 7

Results ............................................... 8

Performance Scale ............................. 10

Performance Efficiency ....................... 11

Conclusions ............................................ 13

Optimizations and Extensions ................... 13

Related Resources................................. 14

Invitation to Provide Feedback ................. 14

Executive Summary This document explains how Sandvine, Dell®, and Intel®, using

standards-based virtualization technologies, have achieved data

plane performance scale and efficiency within a policy control

virtual network function (VNF) that exceeds those of proprietary

network appliances.

Data Plane Inspection Throughput: 1.1 Tbps (using

realistic traffic, including diverse protocols,

encapsulation and tunneling)

Rack-Space: 10 rack units (RU) (includes all compute,

storage, and network components)

Peak Power Consumption: 4995 W

Performance Density: 110 Gbps/RU

Power Efficiency: 4.5 W/Gbps

These figures are all superior to those provided by network

appliance vendors, and should effectively put to rest any

concerns or questions surrounding the practical and commercial

viability of network functions virtualization for policy control use

cases.

This demonstration proves that network functions virtualization

is a cost effective alternative to proprietary network appliances,

and that virtualized policy control deployments can meet the

scale, efficiency, and business objectives of communications

service providers.

Breaking the Terabit Barrier: A One Terabit per Second Policy Control Virtual Network Function

Page 2: Breaking the Terabit Barrier - · PDF fileA Sandvine Technology Showcase ... mitigate attack traffic, manage congestion, ... software applications performing network functions share

A 1 Tbps Policy Control VNF

Page 2

Introduction Traditionally, the processing demands of network policy control (e.g., stateful packet processing,

complex decision-making, etc.) required proprietary hardware solutions, but technology advances

mean that virtualization now provides a commercially viable alternative.

Transitioning from a purpose-built, proprietary hardware component – a model in which a vendor likely

controls every aspect – to a virtualized commercial off-the-shelf (COTS) model is a formidable

challenge. In the latter model, performance is dependent on various network elements and compute

node density, and drivers and data plane acceleration capabilities vary by hardware manufacturer. We

have previously discussed these challenges in Implementing Policy Control as a Virtual Network

Function: Challenges and Considerations.

This document explains how Sandvine, Dell, and Intel have achieved a major performance milestone: a

single policy control virtual network function (VNF) that delivers 1.1 Tbps of data policy control

throughput, with a compact rack-space footprint and exceptional power efficiency.

Network Policy Control and Network Functions Virtualization Network policy control (also called policy management) refers to technology that enables the definition

and application of business and operational policies in networks.

Network policy control works by:

identifying conditions (e.g., subscriber entitlement, current network conditions, data traffic

identity, etc.);

evaluating decisions (e.g., determining if the network is congested, deciding whether certain

traffic constitutes a distributed denial of service attack, etc.); and

enforcing actions (e.g., record the usage into a database, decrement from a prepaid wallet,

mitigate attack traffic, manage congestion, etc.).

Policy control powers many innovative subscriber services, network management actions, and business

intelligence (e.g., big data, customer experience management, analytics, etc.) initiatives.

Network Functions Virtualization (NFV) is a carrier-led effort to move away from proprietary hardware,

motivated by several desires:

1. To increase agility and enable faster service launches

2. To reduce operational costs by leveraging elastically scalable, API-driven architectures: in an

NFV environment, software applications performing network functions share execution,

storage, and network resources, and these resources can be turned on and off as needed

3. To reduce capital costs by harmonizing on COTS hardware: by using standard x86 COTS

hardware for everything – that is, by running all vendor solutions on a common hardware

architecture – an operator needs fewer spare parts, can standardize the provisioning systems,

lower their operational expenditure, and can simplify their supply chain

Oftentimes, it is assumed that virtualizing a network appliance may lead to greater operational

budgets in the data center due to larger footprint demands and the use of less efficient hardware

components than those found within specialized network appliances.

But is this really the case? This paper proves otherwise.

Page 3: Breaking the Terabit Barrier - · PDF fileA Sandvine Technology Showcase ... mitigate attack traffic, manage congestion, ... software applications performing network functions share

A 1 Tbps Policy Control VNF

Page 3

Network Functions Virtualization Benchmarks and Milestones With any new technology wave, there is a gradual progression from a theoretical foundation, to proving

fundamental concepts (e.g., functional interaction, performance explorations, etc.), and finally to

demonstrations of commercial practicality. From there, chasms are crossed and the technology is

adopted until it eventually becomes ubiquitous wherever practical.1

Virtualization for network policy control functions is no exception:

In May 20132, Sandvine teamed up with Heavy Reading to explore some of the theoretical issues

around software-defined networking and NFV in a joint whitepaper Policy Control & SDN: A

Perfect Match? In the press release announcing the paper, Sandvine stated that, “Some

technical details will need to be worked out. Traffic needs to be steered in and out of the

network functions cloud with full subscriber, state and resource capacity awareness. To

provide a consistent level of service to subscribers on a per-session basis requires session-

aware load balancing, or partitioning, across resources.”

In October 20133, Sandvine became the first network policy control vendor to demonstrate a

completely virtualized solution, at the SDN & OpenFlow World Congress

In February 20144, Sandvine announced a business services offering based on a virtualized

policy control platform5

In July 20146, the Sandvine Virtual Series had been deployed by a number of CSPs

Up to this point, the commercial deployments were relatively smaller scale, with customers deploying

the Sandvine Virtual Series in locations with relatively low data plane throughput requirements and for

which high-scale proprietary equipment was not economical.

With functionality proven, the next wave of benchmarks and milestones dealt with performance.

In March 2015, a Sandvine blog post, “The PTS Virtual Series: Virtualized Performance in the

Real World,”7 detailed how the PTS Virtual Series had achieved 155 Gbps in a single system Dell

PowerEdge R730x with Intel Xeon® E5-2699 v3 processors

This was an important milestone, as it showed that virtualization could deliver the same performance

density, at scale, as proprietary equipment.

However, real-world carrier deployments often demand many hundreds of gigabits per second of

performance. To achieve this scale, many hardware elements must be clustered and load-balanced

together, cooperating to achieve the singular objective of implementing network policy control.

It is this performance milestone – a policy control VNF of several hundred Gbps – that remained

uncrossed…until now.

The remainder of this paper will explain how this milestone was achieved, including the equipment

used, the configuration parameters, and the nature and results of the performance tests.

1 Referring of course both to the technology adoption lifecycle and Geoffrey Moore’s adaptation in Crossing the Chasm. 2 The announcement can be found here. 3 The announcement can be found here. 4 The announcement can be found here. 5 The Cloud Services Policy Controller, which subsequently won the Network Appliances category at the Intel Network Builders

Network Transformation Contest. 6 The announcement can be found here. 7 The blog post can be found here.

Page 4: Breaking the Terabit Barrier - · PDF fileA Sandvine Technology Showcase ... mitigate attack traffic, manage congestion, ... software applications performing network functions share

A 1 Tbps Policy Control VNF

Page 4

Building a One Terabit per Second Policy Control VNF The subsections below explain the components, configuration, and test methods used to establish this

performance benchmark.

Components Table 1 details the components that were used to implement the virtualized deployment.

Table 1 - Solution Components

Component Description/Notes

Sandvine Policy Traffic Switch Virtual Series v7.20

The virtualized version of Sandvine’s Policy Traffic Switch (PTS).

In a policy control deployment, the PTS performs three critical functions: traffic classification, policy decision-making, and policy enforcement.

The PTS Virtual Series uses the Intel DPDK v2.1.0 API8.

1x Dell PowerEdge9 M1000e10 blade enclosure

The PowerEdge M1000e blade chassis enclosure is the robust foundation for the PowerEdge M series blade solution, enabling significant data center density, with power and cooling efficiency, on a platform that is easy to deploy and manage. Includes:

Redundant Chassis Management Controller (CMC)

6x3000 W capable AC Power Supply Units (PSU)

6x Dell Networking Force10 MXL11 switches

Ethernet switches provide multiple converged 10GbE interface data transfers in M1000e blade deployments and a range of 40GbE data transfers North/South of the enclosure.

The six MXL units provide a total complement of 192x10GbE switch ports for blade server interconnect and 36x40GbE for top-of-rack connectivity.

For this test, 112 of the available 10GbE switch ports were used by the PTS Virtual Series data plane.

14x Dell PowerEdge M63012 blades

Half-height, two-socket 13th-generation blade servers, with:

2x Intel Xeon Processor E5-2699 v3 18-core CPUs

1x Intel X71013 4x10GbE Network Daughter Card (NDC)

2x Intel X54014 2x10GbE Mezzanine Card

64GB RAM

120GB SSD

Although two Xeon processors are populated, one was disabled for this test; importantly, this means that there is substantial room for growth in higher-complexity work per packet.

8 More information is available here. 9 More information is available here. 10 More information is available here. 11 More information is available here. 12 More information is available here. 13 More information is available here. 14 More information is available here.

Page 5: Breaking the Terabit Barrier - · PDF fileA Sandvine Technology Showcase ... mitigate attack traffic, manage congestion, ... software applications performing network functions share

A 1 Tbps Policy Control VNF

Page 5

All blades’ host operating system run the Intel Open Network Platform (ONP) v1.4 Server Software15 reference compute node configuration, which is based on Fedora 21 Server, with the following software versions:

Linux: Kernel v3.17.8-300.fc21.x86_64

QEMU-KVM: v2.1.3-8.fc21

Libvirt API: v1.2.9.3

Note that two of the M1000e half-height server slots remain unused (see Figure 1), allowing expansion or for use of control-plane or reporting components.

Figure 1 shows the populated Dell PowerEdge M1000e.

Figure 1 - Front (left) and back (right) view of the populated Dell PowerEdge M1000e; note the two empty

half-height slots visible in the front view.

Configuration Each PTS Virtual Series guest instance was deployed on the Dell PowerEdge M630 Servers as a single

virtual machine, using one socket. The other socket was not used, leaving room for future expansion.

Eight 10GbE interfaces were used across 4 bridge groups (4x2 10GbE) for data plane inspections to each

of the 14 Dell PowerEdge M630 blade servers.

The eight Intel 10GbE Converged Network Interfaces per server were assigned to the guest VNF via PCI

Passthrough using the Linux VFIO kernel module.

On each blade, only one of the two available Xeon processor sockets had its 18 physical cores pinned as

virtual CPUs (vCPUs) to the guest instance virtual machine. NUMA cell integrity was preserved between

CPU and NIC mapping. The remaining Xeon processor socket remained idle and running the Host Fedora

OS specified in Intel Open Network Platform (ONPv1.4) Server Software.

15 More information is available here.

Page 6: Breaking the Terabit Barrier - · PDF fileA Sandvine Technology Showcase ... mitigate attack traffic, manage congestion, ... software applications performing network functions share

A 1 Tbps Policy Control VNF

Page 6

16GB of the available 64GB of each host RAM was assigned to the guest PTS Virtual Series instance.

Each PTS Virtual Series instance includes different network components within its running functions to

perform different tasks, including:

PTS Module (PTSM): traffic forwarding and encapsulation support

PTS Daemon (PTSD): application detection and enforcement

Load Balancer: one such instance is typically configured per bridge group and load balances the

traffic to a PTSM and PTSD pair via a virtual NIC, maintaining session awareness and preserving

processing core affinity

For the purpose of the virtual configuration, we have pinned the guest vCPU (which are host physical

cores):

1 vCPU for each Load Balancer process; 4 vCPU in total

1 vCPU pair for each PTSM/PTSD process pair; 7 pairs in total, for 14 vCPU

The vCPU allocation as represented from the PTS command line:

PTS> show system cpu allocation

NumberOfVCPUs : 18

NumberOfInspectionInstances: 7

NumberOfLoadBalancers : 4

Figure 2 - Inside a PTS Virtual Series instance

Testing Table 2 details the components that were used to test the solution.

Table 2 - Test Components

Component Description/Notes

1x Dell PowerEdge R730

Rack server to function as a traffic generator, equipped with:

1x Intel X710 4x10 GbE NIC

6x Intel 82599ES 2x10 GbE NIC

DPDK pktgen traffic generator16

Traffic generator powered by Intel’s DPDK.

An application packet payload functionality was added to the traffic generator software by Sandvine.

16 More information about pktgen is available here.

Page 7: Breaking the Terabit Barrier - · PDF fileA Sandvine Technology Showcase ... mitigate attack traffic, manage congestion, ... software applications performing network functions share

A 1 Tbps Policy Control VNF

Page 7

Achieving Sufficient Load To sustain traffic across all 14 PTS Virtual Series instances, a novel traffic generation approach was

used.

Traffic patterns from the R730 traffic generator were encapsulated into a GRE tunnel. Since the PTS

Virtual Series has inspection support for a multitude of tunnel types, including GRE, the traffic will be

inspected within the tunnel (i.e., preserving application awareness).

Tunnel broadcasting17 is employed to have the packet stream traverse the bridge groups of all PTS

Virtual Series instances. While not a true IP broadcast, this method employs Non Broadcast Multicast

Address (NBMA) as an outer Destination IP address for the GRE tunnel endpoint, and is used in some

VPN deployment methods.

To complement this approach, the Dell Force10 MXL switches were configured to operate in Layer 2

mode, with their IGMP snooping capability disabled. This configuration causes the L2 domain to flood

the multicast stream to all the switch VLAN member interfaces, essentially replicating the multicast

traffic of each individual stream (which in this scenario is a GRE-encapsulated payload) across the 14

interfaces connected to the PTS Virtual Series’ bridge ports.

By replicating this configuration across all legs of the bridge ports (i.e., eight in total), the test

platform generates 80 Gbps of traffic from the R730 traffic generator, across eight 10GbE interfaces,

each of which being a multicast source to one of eight VLANs on the MXL switches, which are each in

turn mapped to a bridge port of a PTS.

Ultimately, this approach transforms each source of 10 Gbps traffic stream generation into 80 Gbps of

traffic input into the Force10 MXL, which subsequently forwards the multicast across all VNF bridge

ports of the 14 PTS instances. The total traffic forwarded across the six Force10 MXLs and through the

single PTS VNF thus totals 1,120 Gbps (80 Gbps times 14 instances).

Ensuring a Realistic Test To have practical relevance in the real world, the test must be as realistic as possible. An important

element of realism is the actual traffic that is passed through the PTS Virtual Series VNF.

In order to generate a real packet payload across all bridge groups of the PTS Virtual Series cluster and

to require use of the PTS Virtual Series’ rich traffic classification capabilities, the DPDK pktgen

software was employed and its traffic “Range” function was used to generate a valid range of IP

traffic:

IP ranges: 254 different Source and Destination address increments in each of eight streams

TCP port ranges: 254 each of Source and Destination increments in each stream

Packet payload sizes: a range of packet payload size sweep, from 400 to 1518 bytes, to allow

for real life application payload insertion18

Different start point of each port, IP address and packet size in each stream

Applications: A per-thread iteration of multiple applications from four application categories

(Real-Time Entertainment, Social Media, Peer-to-Peer, and Web Browsing)19.

17 Explained somewhat here: http://linux-ip.net/gl/ip-tunnels/node9.html 18 If the reader is looking for exclusively 64-byte packet performance, then it is sufficient to examine DPDK performance alone,

as there would be no payload. Such performance information is available here. 19 These four categories typically account for more than 80% of all Internet traffic. Check out Sandvine’s Global Internet

Phenomena Reports for detailed statistics.

Page 8: Breaking the Terabit Barrier - · PDF fileA Sandvine Technology Showcase ... mitigate attack traffic, manage congestion, ... software applications performing network functions share

A 1 Tbps Policy Control VNF

Page 8

Putting it all together, the traffic flowing through the PTS Virtual Series bridge groups consisted of:

An application payload, within

An IP TCP packet with SRC and DST port and IPv4 address sweeping through 254 possible values,

within

An IP type 47 GRE packet, with

An IPv4 multicast destination address and valid IPv4 source address, with

An 802.1q VLAN ID with static value per port, within

An L2 datagram with a Source MAC corresponding to the NIC in use, and Destination Multicast

Ethernet address

Figure 3 shows a Wireshark capture of a 1000-byte test packet.

Figure 3 – Wireshark capture of a test packet

For the sake of clarity, the description above omits other standard fields that were present in the

traffic, such as Preamble, Ether Types, and CRC/FCS,

To further test the PTS Virtual Series under realistic conditions, a number of instances were configured

with GRE tunnel awareness (i.e., were configured to inspect within the tunnel itself), while others

were left with a default configuration where the GRE tunnel was the expected traffic detection result

(i.e., the traffic classification processes would not look within the tunnels).

Results We confirmed that the traffic generation from pktgen was at line-rate, from both the generator host

and the Dell Force10 MXL switch port interface statistics. Each reported valid/expected data:

Multicast packets being received

Interface Bitrate (excluding pre-amble and packet gap): input 9818.00 Mbps per 10GbE

Packet rate: 1271245 packets/s

Switch port capacity: 99.90% of line-rate

Note that the same interface statistics were observed on each of the 112 10GbE switch ports of the

MXL switches.

On the PTS VNF, the individual interface groups were observed to bridge the incoming traffic:

PTS> show interface rate

DATA INTERFACES

===============

Port In(bps) Out(bps) PacketsIn(pps) PacketsOut(pps)

----- ------- -------- -------------- ---------------

1-3 9.79G 9.79G 1.26M 1.26M

1-4 9.79G 9.79G 1.26M 1.26M

Page 9: Breaking the Terabit Barrier - · PDF fileA Sandvine Technology Showcase ... mitigate attack traffic, manage congestion, ... software applications performing network functions share

A 1 Tbps Policy Control VNF

Page 9

1-5 9.79G 9.79G 1.26M 1.26M

1-6 9.79G 9.79G 1.26M 1.26M

1-7 9.79G 9.79G 1.26M 1.26M

1-8 9.78G 9.78G 1.26M 1.26M

1-9 9.79G 9.79G 1.27M 1.26M

1-10 9.78G 9.78G 1.26M 1.27M

TOTAL 78.30G 78.30G 10.10M 10.10M

The CLI command output of the interface rates above represent the 8x 10GbE Ethernet interfaces as

data ports available on each instance of PTS Virtual Series.

The PTS Load Balancer module handles traffic from each of the four bridges per PTS, and balances the

traffic via the virtual NIC across the seven processing vCPU pairs (i.e., a pair consisting of one PTSM

and one PTSD process).

The traffic balancing was confirmed:

PTS> show interface rate lb

Port In(bps) Out(bps) PacketsIn(pps) PacketsOut(pps)

-------- ------- -------- -------------- ---------------

lb0.ptsm 0.00 19.49G 0.00 2.52M

lb1.ptsm 0.00 19.50G 0.00 2.53M

lb2.ptsm 0.00 19.50G 0.00 2.52M

lb3.ptsm 0.00 19.49G 0.00 2.53M

TOTAL 0.00 77.98G 0.00 10.10M

PTS> show interface rate module

Port In(bps) Out(bps) PacketsIn(pps) PacketsOut(pps)

-------- ------- -------- -------------- ---------------

ptsm0.lb 10.82G 0.00 1.40M 0.00

ptsm1.lb 10.82G 0.00 1.40M 0.00

ptsm2.lb 10.82G 0.00 1.41M 0.00

ptsm3.lb 10.82G 0.00 1.39M 0.00

ptsm4.lb 10.83G 0.00 1.41M 0.00

ptsm5.lb 10.81G 0.00 1.39M 0.00

ptsm6.lb 10.84G 0.00 1.42M 0.00

TOTAL 75.76G 0.00 9.83M 0.00

Note that the throughput and packet rates vary across all tables, due to the varying packet sizes, gap,

preamble, and payloads.

With default payload detection enabled, each PTS Virtual Series instance showed the following traffic

classification within each tunnel:

PTS> show traffic

ApplicationType SessionRate Downstream(bps) Upstream(bps) Total(bps)

--------------------- ----------- --------------- ------------- ----------

Tunneling 0.00 37.59G 37.58G 75.17G

Miscellaneous 0.00 0.00 0.00 0.00

RealTimeEntertainment 0.00 0.00 0.00 0.00

Page 10: Breaking the Terabit Barrier - · PDF fileA Sandvine Technology Showcase ... mitigate attack traffic, manage congestion, ... software applications performing network functions share

A 1 Tbps Policy Control VNF

Page 10

PeerToPeer 0.00 0.00 0.00 0.00

WebBrowsing 0.00 0.00 0.00 0.00

Email 0.00 0.00 0.00 0.00

RealTimeCommunication 0.00 0.00 0.00 0.00

BulkTransfer 0.00 0.00 0.00 0.00

Gaming 0.00 0.00 0.00 0.00

NetworkStorage 0.00 0.00 0.00 0.00

SocialNetworking 0.00 0.00 0.00 0.00

TOTAL 0.00 37.59G 37.58G 75.17G

When GRE tunnel inspection was enabled, each PTS Virtual Series instance showed the following traffic

classification:

PTS> show traffic

ApplicationType SessionRate Downstream(bps) Upstream(bps) Total(bps)

--------------------- ----------- --------------- ------------- ----------

WebBrowsing 0.00 9.29G 9.70G 18.99G

RealTimeEntertainment 0.00 9.86G 8.90G 18.76G

SocialNetworking 0.00 8.66G 9.97G 18.63G

PeerToPeer 0.00 9.66G 8.89G 18.55G

Miscellaneous 0.00 0.00 0.00 0.00

Email 0.00 0.00 0.00 0.00

RealTimeCommunication 0.00 0.00 0.00 0.00

BulkTransfer 0.00 0.00 0.00 0.00

Tunneling 0.00 0.00 0.00 0.00

Gaming 0.00 0.00 0.00 0.00

NetworkStorage 0.00 0.00 0.00 0.00

TOTAL 0.00 37.47G 37.46G 74.93G

The delta between traffic rates without and with GRE inspection enabled is because the payload rate

reported in the latter configuration excludes all encapsulation, preamble, and CRC/FCS prior to the

inspected and counted payload.

Performance Scale Aggregating the results shown above (i.e., the 78.30Gbps traffic rate) across all fourteen PTS Virtual

Series instances totals to 1.096 Tbps (excluding preamble) observed at the PTS bridge groups.

The PTS Virtual Series VNF performed policy control (i.e., real-time traffic inspection and

classification) on IP payload for 1.052 Tbps and 1.049Tbps, respectively, for GRE level detection and

GRE-aware application inspection.

For all test iterations, the total aggregated throughput measured at the Dell Networking Force10 MXL

Ethernet switch including preamble and packet gaps was 1.1 Tbps.

Figure 4 shows the PTS Virtual Series deployment as viewed in Sandvine’s Control Center policy and

operations management graphical user interface. Of particular note:

The navigation pane (on the left) shows all 14 PTS Virtual Series instances

The Flow Capture pane (bottom-middle) shows a real-time snapshot of the flows traversing the

system

The Statistics pane shows the real-time packet and byte rates

Page 11: Breaking the Terabit Barrier - · PDF fileA Sandvine Technology Showcase ... mitigate attack traffic, manage congestion, ... software applications performing network functions share

A 1 Tbps Policy Control VNF

Page 11

Figure 4 - Control Center screen showing the aggregate deployment

Performance Efficiency Scale without efficiency is insufficient, so how did this demonstration fare in terms of efficiency, and

how does the achieved efficiency compare to existing commercial network policy control appliances?

Throughout the functional tests, the Dell PowerEdge M1000e Chassis Management Controller was polled

for its Power Budget and Power Management environmental metrics. These values were polled via the

Dell CMC CLI racadm getpbinfo and racadm getpminfo commands:

The total server power budget during the test peaked at 4995 W for the Server infrastructure;

average power consumption was ~3500 W

The remaining chassis power budget (MXL switches, Fans, KVM, and chassis control functions)

peaked at 1420 W

Combining the achieved scale with the observed power consumption (and using the peak power to keep

things conservative), we see that compute node power efficiency is an impressive 4.5 W/Gbps of policy

control function.

These values compare very favorably (see Figure 5, Figure 6, and Figure 7) to competing hardware

appliance vendors in the network policy control space: other policy control vendors who have pursued

an appliance-based solution deliver best-case scenario performance between 8.3 W/Gbps and 16.2

W/Gbps, depending upon the vendor and the particular appliance, with variable maximum scalability.

For the sake of comparison, these figures also include the PTS 32000, a two rack unit 100 GbE unit

capable of intersecting 400 Gbps.

Page 12: Breaking the Terabit Barrier - · PDF fileA Sandvine Technology Showcase ... mitigate attack traffic, manage congestion, ... software applications performing network functions share

A 1 Tbps Policy Control VNF

Page 12

Figure 5 - Performance cost, in terms of power consumption per unit of performance

Figure 6 - Performance efficiency, in terms of throughput per unit of power consumption

Figure 7 – Performance density, in terms of throughput per rack unit

3.8 4.5

8.3

16.2

02468

1012141618

PTS32000

PTSVirtual Series

CompetitorAppliance A

CompetitorAppliance B

Watt

s /

Gbps

Performance Cost

260

220

120

62

0

50

100

150

200

250

300

PTS32000

PTSVirtualSeries

CompetitorAppliance A

CompetitorAppliance B

Mbps

/ W

att

Performance Efficiency

187.5

110

43 36

0

50

100

150

200

PTS32000

PTSVirtualSeries

CompetitorAppliance A

CompetitorAppliance B

Gbps

/ R

U

Performance Density

Page 13: Breaking the Terabit Barrier - · PDF fileA Sandvine Technology Showcase ... mitigate attack traffic, manage congestion, ... software applications performing network functions share

A 1 Tbps Policy Control VNF

Page 13

Conclusions This demonstration proves that network functions virtualization is a cost effective alternative to

proprietary network appliances, and that virtualized policy control deployments can meet the scale,

efficiency, and business objectives of communications service providers.

Table 3 - Summary of Benchmarks

Metric Performance

Data Plane Inspection Throughput 1.1 Tbps

Rack-Space 10 rack units (RU)

Power Consumption 4995 W

Performance Cost 4.5 W/Gbps

Performance Efficiency 220 Mbps/W

Performance Density 110 Gbps/RU

These figures are all superior to those provided by network appliance vendors, and should effectively

put to rest any concerns or questions surrounding the practical and commercial viability of network

functions virtualization for policy control use cases.

Optimizations and Extensions It is worth noting that the four PTS Load Balancer instances ran at an average of 70% CPU usage per

instance; this means that at 20 Gbps full line rate on the bridge group, a single physical Xeon core can

handle the packet forwarding rate with cycles to spare, as is often demonstrate by Intel DPDK packet

forwarding benchmarks.

Additionally, each PTSM/PTSD pair ran at an average of 57% CPU usage per instance. Importantly, this

result means that there is still significant capacity within the pairs to carry out additional policy

control functions well beyond identifying and measuring traffic in real-time (e.g., network security,

online charging, etc.).

This CPU utilization also demonstrates that we could have delivered even more impressive power

efficiency, had we elected to use five PTSM/PTSD pairs per PTS Virtual Series instance (which could

have been accommodated by the remaining CPU capacity) instead of the seven that we used in the

test. This configuration would have freed up four physical cores and their associated power, per PTS

Virtual Series instance, from the overall power budget.

However, we instead chose to implement as realistic a test as possible, as a genuine proof of the viable

economics of virtualized policy control solutions, rather than to slant the configuration to deliver an

even more favorable (but less representative) result.

Additionally, as noted in Table 1, although two Xeon processors are populated, one was disabled for

this test; like the 57% CPU utilization, this means that there is substantial room for growth in higher-

complexity work per packet.

Page 14: Breaking the Terabit Barrier - · PDF fileA Sandvine Technology Showcase ... mitigate attack traffic, manage congestion, ... software applications performing network functions share

A 1 Tbps Policy Control VNF

Page 14

Related Resources In addition to the extensive list of footnotes included within this document, readers interested in

virtualization and specific use case reference implementations should visit https://nfv.net/, the online

home of the NFV Leaders Forum.

Invitation to Provide Feedback Thank you for taking the time to read this technology showcase. We hope that you found it useful, and

that it helped you understand how we, together with Dell and Intel, achieved this important

performance milestone.

While this paper is fairly technical, it is not meant to be a complete reference publication of the

metrics, configuration, and procedures used for the demonstration. Readers interested in more detail

are encouraged to contact Sandvine.

If you have any feedback or have questions that have gone unanswered, then please send a note to

[email protected]

Page 15: Breaking the Terabit Barrier - · PDF fileA Sandvine Technology Showcase ... mitigate attack traffic, manage congestion, ... software applications performing network functions share

Copyright ©2015 Sandvine

Incorporated ULC. Sandvine and

the Sandvine logo are registered

trademarks of Sandvine Incorporated

ULC. All rights reserved.

European Offices

Sandvine Limited

Basingstoke, UK

Phone: +44 0 1256 698021

Email: [email protected]

Headquarters

Sandvine Incorporated ULC

Waterloo, Ontario Canada

Phone: +1 519 880 2600

Email: [email protected]