Slide 1SHARKFEST ‘10 | Stanford University | June 14–17, 2010
To the Terabyte and Beyond! Leveraging Pilot and Wireshark to Analyze Truly Massive Packet Traces
June 17, 2010
Packet Aquisition
Capture Card
Filtering capability in the card normally not really useful
Unless in some unusual conditions, the application wants to see everything
PCI bus is the only resource that card filtering optimizes
Any tap nowadays can do basic filtering
Small packets is the worst condition
CACE Turbocap
No unnecessary features (who needs filtering?)
Affordable price
CPU
Bottlenecks
Multi-threading hard to leverage when capturing and processing network packets
Network monitoring is intrinsically sequential
Locking is evil
At 10Gbps, cache coherency is a big deal
Small packets is the worst condition
SHARKFEST ‘10 | Stanford University | June 14–17, 2010
Disk
Bottlenecks
Solid State? Not a good idea yet
Single disk performance is not really the bottleneck
Cost is an important factor when you build a system with tens of disks
Reliability not as proven as the old magnetic disks
SHARKFEST ‘10 | Stanford University | June 14–17, 2010
Disk write speed based on position
SHARKFEST ‘10 | Stanford University | June 14–17, 2010
I can capture a lot of packets. Now what?
Read of packets must be non-disruptive!
Even if I stop the capture process, since I was writing at full speed, reading the data is going to take around the same time of writing it
Read needs to be localized
I need high level visibility to reach the point I need
Indexing
Standalone card vs. kit
to build a functional packet
capture system.
Indexing
On a trace file, after the fact
Summary of the network traffic
Volume, talkers and protocol information
Coordinated with the packet store
“Netflow on steroids”
Designed to be extremely efficient in terms of disk usage
Coordinated with the packet store
SHARKFEST ‘10 | Stanford University | June 14–17, 2010
Indexing
into the memory buffer.
storage system. Create time
RAID level 0Packet Storage SystemLive network traffic
Content Indexing Thread
Captures packets off the live interface and creates an index that
speeds up “index-friendly” Views.
View
View
Pilot Indexed Views Threads
Return summary information about terabytes of traffic in a matter of
seconds.View