Apache Traffic Server Internals

Phil Sorber - Comcast

How I learned to stop worrying and love ATS

About Me

(Why you should listen to me) •  Principal Engineer @ Comcast on IpCDN team •  Committer to ATS •  Generally cool guy [citation needed]

About This Talk

(Why you shouldn’t leave just yet) •  CPU •  Memory •  Storage •  Network •  Abstractions

A brief History

•  Inktomi (1996) •  Yahoo! (2002) •  Apache Incubator (2009) •  Apache TLP (2010)

CPU

•  Multi-Threaded •  Asynchronous •  Event Driven •  Continuations •  CPU Affinity (NUMA)

CPU

(continued)

CPU

(continued) •  Thread Types

§  Net §  AIO §  Task §  Accept §  SSL §  DNS §  Remap §  Plugins can also do thread things

Memory

•  RAM Cache (not RAM disks) •  NUMA! NUMA! •  Freelists

§  Allocators §  ClassAllocators (Global with atomics) §  ProxyAllocators (Thread) §  Arenas

•  Huge Pages?

Memory

(continued)

Head

Version

Next Next Next

Memory

(continued)

Socket 1 Socket 2

Storage

•  Raw Devices •  Unique Caches •  Distinct RAM Caches •  Consistent Hashing •  Circular Cache

Storage

(continued)

Vol 1 Vol 5 Vol 4 Vol 3 Vol 2

RC

Consistent Hash

RC RC RC RC

Storage

(continued)

Vol 1 Vol 5 Vol 4 Vol 3 Vol 2

RC

Consistent Hash

RC RC RC RC

Network

•  Connection Pools •  Tiered Caches •  Parent Selection

Network

(continued) Origin

Mid 2

Mid 1

Edge 2

Edge 3

Edge 1

Client 1

Client 2

Client 3

Client 4

Edge 4

Client 5

Client 6

Network

(continued) Origin

Mid 2

Mid 1

Edge 2

Edge 3

Edge 1

Client 1

Client 2

Client 3

Client 4

Edge 4

Client 5

Client 6

Abstractions

•  Processors §  HostDB §  DNS §  Net §  SSL §  Cache §  Remap

•  State Machines •  Clustering

Abstractions

(continued)

Questions?

Seriously, ask some.

•  Phil Sorber •  sorber@apache.org •  @PhilSorber •  http://trafficserver.apache.org •  #traffic-server on freenode (PSUdaemon)