Mobile TCPVladimir Kirillov

@darkproger

August 2012

! int fd;! int rc;! struct sockaddr_in sin;

fd = socket(AF_INET, SOCK_STREAM, 0);! assert(fd >= 0);

! bzero(&sin, sizeof(sin));! sin.sin_family = AF_INET;! sin.sin_port = htons(5000);! sin.sin_addr.s_addr = inet_addr("127.0.0.1");

! rc = connect(fd, (const struct sockaddr *)&sin, sizeof(sin));! assert(rc == 0);

! write(fd, "hello", sizeof("hello"));

DTrace!

DTrace Secret

#pragma D option flowindent

syscall::SYSCALL:entry{! self->mon = 1;}

fbt:::/self->mon/{}

syscall::SYSCALL:return/self->mon/{! self->mon = 0;}

% cat systrace.d

% nc -l 5000 &% dtrace -C -DSYSCALL=connect -s systrace.d -c ./a.out

BPF(4) BSD Kernel Interfaces Manual

NAME bpf -- Berkeley Packet Filter

SYNOPSIS pseudo-device bpf

DESCRIPTION The Berkeley Packet Filter provides a raw interface to data link layers in a protocol independent fashion. All packets on the network, even those destined for other hosts, are accessible through this mechanism.

SEE ALSO

tcpdump(1)

2012-08-16 12:44:01.903179 IP (tos 0x0, ttl 64, id 24809, offset 0, flags [DF], proto TCP (6), length 64) 127.0.0.1.49454 > 127.0.0.1.5000: Flags [S], seq 4171303565, win 65535, options [mss 16344,nop,wscale 4,nop,nop,TS val 337494866 ecr 0,sackOK,eol], length 02012-08-16 12:44:01.903252 IP (tos 0x0, ttl 64, id 18653, offset 0, flags [DF], proto TCP (6), length 64) 127.0.0.1.5000 > 127.0.0.1.49454: Flags [S.], seq 620969591, ack 4171303566, win 65535, options [mss 16344,nop,wscale 4,nop,nop,TS val 337494866 ecr 337494866,sackOK,eol], length 02012-08-16 12:44:01.903266 IP (tos 0x0, ttl 64, id 53159, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.49454 > 127.0.0.1.5000: Flags [.], seq 1, ack 1, win 9186, options [nop,nop,TS val 337494866 ecr 337494866], length 02012-08-16 12:44:01.903278 IP (tos 0x0, ttl 64, id 17812, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.5000 > 127.0.0.1.49454: Flags [.], seq 1, ack 1, win 9186, options [nop,nop,TS val 337494866 ecr 337494866], length 02012-08-16 12:44:01.903302 IP (tos 0x0, ttl 64, id 29616, offset 0, flags [DF], proto TCP (6), length 58)

127.0.0.1.49454 > 127.0.0.1.5000: Flags [P.], seq 1:7, ack 1, win 9186, options [nop,nop,TS val 337494866 ecr 337494866], length 62012-08-16 12:44:01.903317 IP (tos 0x0, ttl 64, id 48096, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.5000 > 127.0.0.1.49454: Flags [.], seq 1, ack 7, win 9186, options [nop,nop,TS val 337494866 ecr 337494866], length 02012-08-16 12:44:01.903337 IP (tos 0x0, ttl 64, id 50325, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.49454 > 127.0.0.1.5000: Flags [F.], seq 7, ack 1, win 9186, options [nop,nop,TS val 337494866 ecr 337494866], length 02012-08-16 12:44:01.903353 IP (tos 0x0, ttl 64, id 9153, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.5000 > 127.0.0.1.49454: Flags [.], seq 1, ack 8, win 9186, options [nop,nop,TS val 337494866 ecr 337494866], length 02012-08-16 12:44:01.903362 IP (tos 0x0, ttl 64, id 10452, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.49454 > 127.0.0.1.5000: Flags [.], seq 8, ack 1, win 9186, options [nop,nop,TS val 337494866 ecr 337494866], length 02012-08-16 12:44:01.903898 IP (tos 0x0, ttl 64, id 40658, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.5000 > 127.0.0.1.49454: Flags [F.], seq 1, ack 8, win 9186, options [nop,nop,TS val 337494866 ecr 337494866], length 02012-08-16 12:44:01.903921 IP (tos 0x0, ttl 64, id 10996, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.49454 > 127.0.0.1.5000: Flags [.], seq 8, ack 2, win 9186, options [nop,nop,TS val 337494866 ecr 337494866], length 0^C

11 packets captured

# tcpdump -i lo0 -nn -vv -tttt -K 'tcp port 5000'

TCP

•Transmission Control Protocol•sliding window•Automatic Repeat reQuest (ARQ)•asynchronous

HTTP

queue = NSOperationQueue.new

queue << lambda do HTTP.get "http://share.darkproger.net/bundleblob.tar.gz" end

# rake device

xxx.xxx.31.176 - - [16/Aug/2012:12:35:14 +0000] "GET /bundleblob.tar.gz HTTP/1.1" 200 4644089 "-" "iostcp/1.0 CFNetwork/602 Darwin/13.0.0"

Sniffing iPhone

% udid=$(system_profiler SPUSBDataType \ | awk '/iPhone/{go=1} /Serial/ {if (go) print $3; go=0}')276cb9530201bcehelloworldcd55560ed015d00

% rvictl -s $udid

Starting device 276cb9530201bcehelloworldcd55560ed015d00 [SUCCEEDED]

% ifconfig rvi0rvi0: flags=3005<UP,DEBUG,LINK0,LINK1> mtu 0

% tcpdump -w iostcp.pcap -i rvi0

Protocol Analysis

• CAPTCP• http://research.protocollabs.com/captcp/• https://github.com/proger/captcp

• 1988f4b support OSX rvi RAW datalink type pcap dumps

• tcpstat• http://www.frenchfries.net/paul/tcpstat/

• tcptrace• http://www.tcptrace.org/

gnuplot p0rn

TCP Features

• Congestion Control• Initially designed for wired network• dropped packets mean congestion

Congestion Control

(TCP Reno)

• Congestion avoidance• Additive Increase Multiplicative Decrease

• Retransmit Timeout (RTO)• Slow Start• ACK-clocking• Fast retransmit• Fast recovery• Improvements• Selective ACK, NewReno• TCP Vegas, ECN, RED

Tuning Methodolody

• define application workload types • use defaults• gather stats on client/server sides

• snoop(1)/tcpdump(1) data off the wire• think, change tunables

• rinse, repeat

iOS

•setsockopt(2)•adjust window size•socket buffer sizes•TCP_NODELAY•etc•getsockopt(2)•monitoring•low-latency responding to socket events•watch your CFRunLoop

Server Tuning

•setsockopt(2)

•Solaris/Illumos•ipadm show-prop tcp•ndd -get /dev/tcp \?•Linux•/proc/sys/net/ipv4/tcp*•getsockopt(SOL_TCP, TCP_INFO, ...)•BSD•sysctl(8)

•Do not forget to establish persistent monitoring on server boxes :-)

More Batteries

• Percona Toolkit pt-tcp-model• http://www.percona.com/doc/percona-toolkit/2.1/pt-

tcp-model.html• Extracting Performance and Scalability Metrics From

TCP• http://regions.cmg.org/regions/scmg/fall_11/

scalability-performance-tcp.pdf

More Stuff

• Practical achievements• Next time

• come to itjam.ciklum.net• Kiev• September 1st• NSK Olympiysky

kthxbai@darkproger