Staying Afloat with Buoy: A High-Performance HTTP Client
61
Louis-Philippe Gauthier Director, Platform Engineering Staying afloat with Buoy: a high-performance HTTP client
Transcript of Staying Afloat with Buoy: A High-Performance HTTP Client
Louis-Philippe Gauthier Director, Platform Engineering
Staying afloat with Buoy: a high-performance HTTP client
2
Buoy's raison d'être
Story time
3
1. A • 2011: ibrowse • 2012: dlhttpc (fred’s dispcount lhttpc fork) • 2013: dlhttpc 2.0 (my fork) • 2017: buoy!
Problems with dlhttpc 2.0
4
• no pipelining! • reconnects on request • complex code base with broken test suite • hacks on hacks on hacks
5
Buoy’s architecture
Buoy’s architecture
6
1. A
2. C
process shackle_pool
buoy_pool_utils
7
Shackle framework
1. A 2. B 3. C
Problem
8
application service
???
Service…?
9
• communicates over socket • ascii or binary protocols • synchronous or asynchronous protocols
Goals?!
10
• reusability • speed (low overhead) • concurrency • safety
11
Shackle’s overview
Features (last year)
12
1. A • multi-protocol support (TCP / UDP) • fast pool implementation (random, round_robin) • request pipelining • backpressure via backlog (OOM protection) • smart reconnect mechanism (exp. backoff)
Architecture (last year)
13
serviceprocess
msg
sockets
ETS
clients(proc_lib)
pool options + backlog
Shackle features (today)
14
1. A • Managed timeouts • SSL support • handle_data callback error • optimizations:
• granderl • cache pool options with module • remove setelement/3 calls
2. C
Shackle’s architecture (today)
15
serviceprocess
msg
sockets
ETS
clients(metal)
backlog
Shackle features (today)
16
2. C
Shackle features (future)
17
1. A • SCTP network protocol support • more optimizations! • better test suite • better documentation
2. C
Shackle usage
18
19
buoy_client
=init/0
setup/0
handle_request/2
handle_data/2
terminate/2
20
buoy_client
21
1. A
2. C
buoy_client
22
1. A
2. C
buoy_client
23
1. A
2. C
buoy_client
24
1. A
2. C
buoy_protocol request
25
1. A
2. C
buoy_protocol response
26
• small HTTP parser (< 150 lines) • all-at-once vs streaming parser • fast! (10x hackney’s parser)
• minimize function calls • minimize sub-binary creation • minimize setelement/3 calls
• uses binary:split/{2,3} 2. C
buoy_protocol response
27
1. C
buoy_protocol response
28
1. A
• split 25.7 μs • split-hipe 20.7 μs • split2 16.3 μs
buoy_protocol response
29
buoy_protocol response
30
1. A • implements a subset of RFC2616 • unsupported features:
• arbitrary number of new lines in headers • random capitalization of headers • protection against malicious servers • chunked encoding
2. C
buoy_compiler
31
• generates module buoy_pool_utils • function cases generated from ETS state • uses erl_syntax to generate AST • hot-load with code:load_binary/3
2. C
buoy_compiler
32
1. A
2. C
buoy_compiler
33
1. A
2. C
buoy_compiler
34
1. A
2. C
35
Buoy’s performance
profiling
36
• make profile target • fprofx (fork of fprof with suspend and GC) • qcachegrind / kcachegrind
2. C
profiling
37
profiling
38
extrospect-beam
41
1. A • sampling profiler using perf events (linux only) • by Julian Squires (AdGear) • erlang-write-perf-map PID
2. C
perf top -p PID
42
erlang-sample --only-erlang PID
43
44
HTTP client benchmarks
httpc_bench
45
1. A • timing_hdr • HDR histogram by Gil Tene (Azul Systems) • hdr_histogram_erl (NIF) • It’s on Github, send PRs!
2. C
httpc_bench
46
1. A • dlhttpc 2.0 • hackney 1.6.2 • httpc (erlang 18.3.3) • ibrowse 4.4.0 • katipo 0.4.0 • buoy 0.1.0
2. C
httpc_bench
47
1. A • bench servers:
• 48 Xeon cores @ 2.20GHz • 256 GB ram • SSDs
• latency between servers < 0.1 ms • Erlang 18.3.3
2. C
httpc_bench
48
1. A • iterations / # requests • concurrency / # processes • pool_size/ # connections
2. C
dlhttpc
49
https://gist.github.com/lpgauth/f9a0ce1f32437b819c4a9ea252896f9b
hackney
50
https://gist.github.com/lpgauth/f9a0ce1f32437b819c4a9ea252896f9b
httpc
51
https://gist.github.com/lpgauth/f9a0ce1f32437b819c4a9ea252896f9b
ibrowse
52
https://gist.github.com/lpgauth/f9a0ce1f32437b819c4a9ea252896f9b
katipo
53
https://gist.github.com/lpgauth/f9a0ce1f32437b819c4a9ea252896f9b
buoy
54
https://gist.github.com/lpgauth/f9a0ce1f32437b819c4a9ea252896f9b
buoy
55
httpc_bench results
56
1. A • buoy (max: 605K/s) • katipo (max: 96K/s) - 6.3x • dlhttpc (max: 70K/s) - 8.6x • ibrowse (max: 59K/s) 10.2x • hackney (max: 33K/s) - 18.3x • httpc (max: 14K/s) - 43x
2. C
57
Tips & Tricks
Performance tips
58
• pattern match • use iolist() • keep your server loops lean (specialized) • use ETS for shared state (R/W) • compile modules for read only states
Performance tips
59
• bin_opt_info compile opt • measure everything
Links
60
• http://github.com/lpgauth/buoy • http://github.com/tokenrove/extrospect-beam • http://github.com/tokenrove/granderl • http://github.com/lpgauth/httpc_bench • http://github.com/lpgauth/metal • http://github.com/lpgauth/shackle • https://github.com/lpgauth/timing
@lpgauth
Thank you!
