Now and Then, How and When? June 16 th , 2009 Stephen Donnelly Technologist | Endace Technology

SHARKFEST '09 | Stanford University | June 15–18, 2009

Now and Then, How and When?June 16th, 2009

Stephen DonnellyTechnologist | Endace Technology

SHARKFEST '09Stanford UniversityJune 15-18, 2009


Endace

• Potted history– 1996 The University of Waikato– 2001 Endace created– 2005 Publically Listed

• Specialists in packet capture– High data/packet rates– Accurate time stamping– Wide variety of network interfaces


Network Monitoring Interfaces

• DAG cards cover many network technologies

• 8000 bps to 39813120000 bps

• TDM - T1/E1/J1• PDH - T3/E3• SONET/SDH - OC-3, 12,

48, 192, 768• InfiniBand – SDR, DDR


Platforms and Appliances

• Open Platforms– Full access

• Managed Appliances– Packet Capture– Trace Replay– Applied Watch IDS– Flow Export– Lawful Intercept– CACE Pilot


Lossless Packet Capture

• Capture all packets on link– Categorize– Filter– Present to user

• Debugging• Security• Forensics• Lawful Intercept


Network Interface Cards

• Designed to provide inexpensive network connectivity for diverse applications– Web, Email, File transfer

• Generally applications are the bottleneck– E.g. a web server generating content

• Protocols are fault tolerant so NIC need not be• LAN traffic is bursty


NIC Device Model

NIC

Tx DescriptorRing

Rx DescriptorRing

Packet Buffers

Driver

NetworkStack

PacketFilter

Libpcap

Application


Performance Testing

• Simple Libpcap app counting packets– Packets Captured vs. Applied– CPU Load

• Single processor core• AMD Opteron 248 (2.2GHz)• 2GB DDR 400 DRAM• Linux 2.6.12


DAG cards

• Optimized for packet capture and replay– Efficient transfer to and from user applications

• Capture 100% of received packets– Full or partial packet capture– Account for any packet loss that does occur

• Record accurate timestamps– Synchronized clocks for timestamp comparisons

• ERF Format with rich per-packet metadata


DAG 8.1SX


Features only on subset of cards

DAG Internals

FPGA

1 to n Network Physical Layer

Interface/s

LEDs

Sync Connector

Clock Oscillator

Network Interface /

Framer

Power Supply Circuits

CPLD

ROM

JTAG / Test Connector/s

Processor RAM

Coprocessor

Bus Connector

FIFO


DAG Stream Buffer

• Large Static Ring Buffers– 4MB to 2GB each

• Window-based Handshaking– Minimize per-packet

overhead

• Memory-mapped to User space– Zero copy

ReadingFilled

Empty

Writing


DAG Device Model

DAG

Tx Stream Rx Stream

Driver

NetworkStack

PacketFilter

Libpcap

Application

Rx Stream

Libdag


Extensible Record Format


Accurate time stamps

• Debugging/Benchmarking/Optimization– QoS/SLA– Service response time– Storage networks– Network equipment– HPC

• Financial services– Time=Money, Latency=Risk


Resolution

Network Packet Rate (64 Byte)

Packet Time (64 Byte)

Byte Time

10BASE-T 14,880 67,200ns 800ns

100BASE-TX 148,809 6,720ns 80ns

1000BASE-SX 1,488,095 672ns 8ns

10GBASE-SR 14,880,952 67.2ns 0.8ns

OC-768c (POS) 69,721,043 14.3ns 0.2ns

100GBASE-SR10 148,809,520 6.7ns 0.08ns


Reference Clocks

• GPS– Worldwide– Clear view of sky

• CDMA– Works indoors– Limited coverage– Unknown distance to tower

• Radio (Shortwave)– Limited by RF Propagation


Reference Clock Sources

Reference Accuracy (Est.)

GPS 100nsCDMA 10,000nsRadio 1,000,000ns


Clock Transports

Transport Accuracy (Est.)

Hardware 100ns

IEEE 1588 (LAN) 1,000nsNTP (LAN) 1,000,000nsNTP (WAN) 10,000,000ns

Now and Then, How and When? June 16 th , 2009 Stephen Donnelly Technologist | Endace Technology

Documents

Transcript of Now and Then, How and When? June 16 th , 2009 Stephen Donnelly Technologist | Endace Technology