Supercharging Content Delivery with

VarnishSamantha Quiñones

Day Camp for Developers: Performant PHP

@ieatkillerbees

http://samanthaquinones.com

What is Varnish

• Web Application Accelerator

• Caching Reverse Proxy

• Written in C, initially built by Poul Henning Kamp

• Open-source at http://www.varnish-cache.org

• Supported at http://www.varnish-cache.com

Computer Storage

• Primary Storage can be accessed directly by the CPU

• Secondary Storage is accessed via an I/O channel or controller

Virtual Memory Management

• As early as the 1950s, computer scientists were experimenting with virtual memory.

• By the 1970s, virtual memory was common in commercial computers

• Virtual memory is an abstraction that allows secondary storage to extend primary storage

• The operating system cooperates with specialized hardware to manage the paging of data in and out of virtual memory.

Format Time (s) Equivalent Distance Equivalent Time

1 CPU Cycle 0.3 ns 1 m (1 step) 1 second

L1 Cache 0.9 ns 3 m (3 steps) 3 seconds

Main Memory 120 ns360 m

(width of US Capitol Grounds)

6 minutes

SSD 50 µs 170 km (NYC to Wilm., DE) 2 days

HDD 5 ms 13,000 km (Hong Kong) 5 months

Virtual Memory is a Cache

• In essence, virtual memory is a cache

• The operating system swaps data between high-speed primary storage and slower secondary storage based on factors like age and access frequency

• Commonly accessed data is kept “hot” and ready while rarely-needed data can be quickly retrieved when called for

What does Varnish do?

• Varnish allocates a heap of memory up front

• Objects stored in that heap are managed by the OS

• OS Virtual Memory Managers are very sophisticated

• Why reinvent the wheel?

How Varnish Works

• Varnish creates a “workspace” in its memory space

• Workspace contains pointers to cached objects, headers, etc

• Varnish prioritizes worker threads by most recently used

• These factors combine to reduce overall disk & memory ops

varnishd

• varnishd has two processes

• manager runs as root and starts the child (which does all the work)

• manager monitors child and restarts it if it fails

• manager interacts with the varnish cli interface (varnishadm)

• child runs with more limited permissions and handles traffic

varnishadm (varnish CLI)

• Allows administrators to interact with a running varnish

• Secured by PSK

• Designed to be “scriptable”

varnishlog

• Provide access to logs stored in memory

varnishstats

• Provides access to in-memory statistics (cache hit/miss rate, resource usage, etc)

Varnish Config Language

• Configuration DSL that is translated to C and compiled

• We do not “configure” Varnish so much as write policies for handling types of traffic

Malloc Storage

• Memory is allocated as startup in KB, Mb, Gb, or Tb

• Defaults to unlimited

• Overflows to swap

• Extremely fast performance

File Storage

• Space allocated in KB, Mb, Gb, Tb, or as a percentage of available space on the device

• Defaults to 50% of available space

Transient Storage

• Special storage space for short-lived objects

• Defaults to an unlimited malloc

• Threshold TTL is configurable (default: 10s)

Sizing

• Understand the size of your “hot” dataset

• Size of homepage (including images) + size of linked pages/objects

• Cost to produce objects

Installing Varnish

• https://www.varnish-cache.org/docs#install

Installing Varnish

# apt-get install apt-transport-https

# curl https://repo.varnish-cache.org/ubuntu/GPG-key.txt | apt-key add -

# echo "deb https://repo.varnish-cache.org/ubuntu/ precise varnish-4.0" >> /etc/apt/sources.list.d/varnish-cache.list

# apt-get update

# apt-get install varnish

Important Commands

• service varnish restart — Stops and restarts Varnish. Clears all cache

• service varnish reload — Reloads the currently active VCL

• varnishadm vcl.load <name> <filename> — Loads a VCL

• varnishadm vcl.use <name> — Makes VCL named <name> active

• varnishadm param.set <param> <value> — Sets parameters

Default Config

Edit /etc/default/varnish

DAEMON_OPTS="-a :6081 \ # Listen Address

-T localhost:6082 \ # Manage Address

-f /etc/varnish/default.vcl \ # Config File

-S /etc/varnish/secret \ # PSK

-s malloc,256m" # Storage Config

Backends

• Define origin servers, pools, and clusters

Varnish & Cookies

• By default, varnish will not cache if the request has a Cookie header or if the response has a Set-Cookie header

• NB: It is better to not cache content, or to cache multiple copies, than to deliver content to the wrong person.

Dealing with Cookies

• If possible, strip any cookies you do not need. If there are none left, cache

• Create url schemes based on whether cookies are needed or not

• Never cache Set-Cookie

Stripping Cookies

sub vcl_recv { if (req.url ~ "wp-admin|wp-login") { return (pass); }

unset req.http.cookie; }

Varnish Config Language

• C-Derived Domain-Specific State Engine

• Processes requests in isolation

• return(action) exits one state and moves to the next

• Default VCL is present beneath your code and is appended during compilation

VCL Syntax - Types

• Strings - "foo"

• Integers/Real Numbers - 2, 3.14

• Booleans - true or false

• Time - A time_t wrapper

• Durations - 5m, 2h

VCL Syntax - ACLs

• Access Control Lists - Struct-like types used to match client addresses

acl local { "localhost"; // myself "192.0.2.0"/24; // and the local network ! "192.0.2.23"; // except for the dial-in router }

VCL Syntax - Operators

Operator Value

= Assignment

== Comparison

~ Match against RegEx or ACL

! Negation

&& And

|| Or

VCL Syntax - Subroutines

• Structural element used to group code (for reusability or readability)

sub strip_cookies { unset req.http.cookie; }

VCL Syntax - Call

• call <subroutine> - Transfer execution to the named subroutine

call strip_cookies;

VCL Syntax - Return

• return (<state>) - Trigger a state transition to <state>

return (pass);

VCL Syntax - Synth

• synth(<statuscode>, <message>) - Special state that terminates by sending an HTTP response to the client

return (synth(403, 'forbidden'));

VCL Syntax - Loops

• Nope! (think about it!)

Finite State Machines

State: Power On

State: Power Off

Press Button

Press Button

VCL Functions• regsub(<str>, <regex>, <sub>) — Replace the first match of <regex> in <str> with <sub>

• regsuball(<str>, <regex>, <sub>) — Replace all matches of <regex> in <str> with <sub>

• ban(<regex>) — Invalidate all cached objects that match <regex>

• call(<subroutine>) — Call a subroutine

• hash_data(<input>) — Adds data to the hash input. By default, Host and URL of the request are used

• new() — Creates a new object

• rollback() — Restore request headers

• synthetic(<string>) — Prepares a synthetic response

• return(<action>) — Terminate a subroutine

vcl_recv• Called at the start of a request after the request has been parsed.

• Access to request object

• Normalize input

• Make backend routing decisions

• Re-write client data

• Manage caching policy

• Access controls & security

vcl_recv - State Transitions• pass (→vcl_pass) — Bypass the cache, send request to the backend and

return the response

• pipe (→vcl_pipe) — Switch to a proxy-like mode

• hash (→vcl_hash) — Attempt a cache lookup, possibly entering new data in the cache

• synth — Generate a synthetic error response and abandons the request

• purge (→vcl_hash→vcl_purge) — Purge the object and any variants

sub vcl_recv { if (req.restarts == 0) { if (req.http.x-forwarded-for) { set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip; } else { set req.http.X-Forwarded-For = client.ip; } } if (req.request != "GET" && req.request != "HEAD" && req.request != "PUT" && req.request != "POST" && req.request != "TRACE" && req.request != "OPTIONS" && req.request != "DELETE") { /* Non-RFC2616 or CONNECT which is weird. */ return (pipe); } if (req.request != "GET" && req.request != "HEAD") { /* We only deal with GET and HEAD by default */ return (pass); } if (req.http.Authorization || req.http.Cookie) { /* Not cacheable by default */ return (pass); } return (lookup); }

Request Object• req.backend_hint — Set backend to this if we attempt to fetch

• req.hash_always_miss — (bool) Force a cache miss for this request. If set to true Varnish will disregard any existing objects and always (re)fetch from the backend

• req.http.[header] — The corresponding HTTP header

• req.method — The request type (e.g. "GET", "HEAD")

• req.restarts — Count of how many times this request has been restarted

• req.url — The requested URL

• req.xid — Unique ID of this request

vcl_backend_response

• Called after the response headers have been received from a backend

• deliver (→vcl_deliver) — Deliver the response, possibly caching it

• abandon — Abandons the request and returns an error

• retry — Retries the backend request. When the number of retries exceeds max_retries, Varnish will return an error.

Backend Response Object• beresp.backend.ip — IP of the backend this response was fetched from

• beresp.backend.name — Name of the backend this response was fetched from

• beresp.grace — Set to a period to enable grace

• beresp.http.[HEADER] — The corresponding HTTP header

• beresp.proto — The HTTP protocol version used the backend replied with

• beresp.reason — The HTTP status message returned by the server

• beresp.status — The HTTP status code returned by the server

• beresp.storage_hint — Hint to Varnish that you want to save this object to a particular storage backend

• beresp.ttl — The object's remaining time to live, in seconds. beresp.ttl is writable

• beresp.uncacheable — (bool) Marks the response as uncacheable

vcl_hit

• Called when a cache lookup is successful

• deliver(→vcl_deliver) — Deliver the object. Control passes to vcl_deliver

• synth(status code, reason) — Return the specified status code to the client and abandon the request.

• restart — Restart the transaction

vcl_miss• Called after a cache lookup if the requested document was not found in the cache

• synth(status code, reason) — Return the specified status code to the client and abandon the request

• pass (→vcl_pass) — Switch to pass mode

• fetch (→vcl_backend_fetch) — Retrieve the requested object from the backend

• restart — Restart the transaction sub vcl_miss {

return (fetch);

}

vcl_pass

• Called upon entering pass mode. In this mode, the request is passed on to the backend, and the backend's response is passed on to the client, but is not entered into the cache

• synth(status code, reason — Return the specified status code to the client and abandon the request

• pass — Proceed with pass mode

• restart — Restart the transaction

vcl_hash• Defines the unique characteristics of a request

sub vcl_hash {

hash_data(req.url);

if (req.http.host) {

hash_data(req.http.host);

} else {

hash_data(server.ip);

}

return (lookup);

}

vcl_deliver

• Called before a cached object is delivered to the client

• deliver — Deliver the object to the client

• restart — Restart the transaction

vcl_backend_fetch

• Called before sending the backend request

• fetch — Fetch the object from the backend.

• abandon — Abandon the backend request and generates an error.

vcl_backend_error

• This subroutine is called if we fail the backend fetch

• deliver — Deliver the error

• retry — Retry the backend transaction

Calculating TTL

• The s-maxage variable in the Cache-Control response header

• The max-age variable in the Cache-Control response header

• The Expires response header

• The default_ttl parameter (120s).

• Cached Statuses: 200, 203, 300, 301, 302, 307, 404, 410

Managing TTLs

sub vcl_backend_response { if (bereq.url ~ ".png$|.gif$|.jpg$") { set beresp.ttl = 8h; }

if (!beresp.http.Cache-Control) { set beresp.ttl = 1h; } }

Modifying Responses

• Edit response objects before they are sent

• Reattach or re-write headers

• Add additional information to the response

Adding a Header

sub vcl_deliver { if (obj.hits > 0) { set resp.http.X-Cache = "HIT"; } else { set resp.http.X-Cache = "MISS"; } }

Backends

backend default { .host = "45.55.140.8"; .port = "80"; }

Multiple Backendsbackend default { .host = "45.55.140.8"; .port = "80"; }

backend varnishorg { .host = "www.varnish-cache.org"; .port = "80"; }

Hinting & Routingbackend foo { .host = “vhost.example.com”; .port = "8080"; } backend bar { .host = “vhost.example.com"; .port = "8081"; } sub vcl_recv { if (req.http.host ~ "foo.com") { set req.backend_hint = foo; } elsif (req.http.host ~ "bar.com") { set req.backend_hint = bar; } }

Directors

• Logical groupings of backends

• Random or round-robin routing of requests

• Set periodic health checks and manage health status of backends

backend server1 { .host = "45.55.140.8"; } backend server2 { .host = "45.55.163.196"; } backend server3 { .host = "45.55.163.193"; }

import directors;

sub vcl_init { new vdir = directors.round_robin(); vdir.add_backend(server1); vdir.add_backend(server2); vdir.add_backend(server3); }

sub vcl_recv { set req.backend_hint = vdir.backend(); }

Health Checks

• Varnish can monitor backends for health and direct traffic to healthy servers.

• Define health probes which are attached to backends

• Monitor backend health from varnished

Grace Mode

• Allows varnish to serve stale content under certain circumstances.

• Acts as a supplemental TTL

sub vcl_backend_response { set beresp.grace = 2m; }

• This is useful for…

Anti-Stampeding• Varnish uses request coalescing

• When many requests come for the same object, varnish places them on hold and makes a single request to the backend.

• When the object becomes available, it is delivered to many clients.

• This can mean thousands of threads waking at the very same moment!

• Grace Mode will serve stale content while the object is refreshed asynchronously!

if (obj.ttl >= 0s) { // A pure unadultered hit, deliver it return (deliver); } if (obj.ttl + obj.grace > 0s) { // Object is in grace, deliver it // Automatically triggers a background fetch return (deliver); } // fetch & deliver once we get the result return (fetch); }

Rescuing Requests

• Varnish allows retying requests that don’t meet your expectations.

sub vcl_backend_response { if (beresp.status == 200 && beresp.http.content-length == “0”) { return(retry); } }

Cache Busting

• Purging - Explicitly removing an object from cache

• Banning - Instructing Varnish not to serve certain cached objects

Purging Objectsacl purge { "127.0.0.1"; }

sub vcl_recv { if (req.method == "PURGE") { if (!client.ip ~ purge) { return(synth(403,"Forbidden")); } return(purge); } }

Banning

• Bans act as filters on objects which tell Varnish not to return cached objects that meet certain criteria

• Bans are checked when a cache hit is made

• Bans can be set from CLI or with custom VCL

• varnishadm ban req.req.url ~ “\.png$” bans all *.png files

• Banned content remains in cache, memory is not freed

Varnish Log

• Varnish does not write logs directly, but streams them in memory

• varnishlog is the client used to access logs

• varnishlog can be used to capture logs to disk and to filter logs

* << Request >> 196714 - Begin req 196713 rxreq - Timestamp Start: 1428517714.523155 0.000000 0.000000 - Timestamp Req: 1428517714.523155 0.000000 0.000000 - ReqStart 64.236.208.25 40521 - ReqMethod GET - ReqURL / - ReqProtocol HTTP/1.1 - ReqHeader Host: blog.tembies.com - ReqHeader Connection: keep-alive - ReqHeader Cache-Control: max-age=0 - ReqHeader Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 - ReqHeader User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.68 Safari/537.36 - ReqHeader Accept-Encoding: gzip, deflate, sdch - ReqHeader Accept-Language: en-US,en;q=0.8 - ReqHeader Cookie: wp-settings-1=editor%3Dtinymce%26libraryContent%3Dbrowse; wp-settings-time-1=1428174206 - ReqHeader X-Forwarded-For: 64.236.208.25 - VCL_call RECV - ReqUnset Cookie: wp-settings-1=editor%3Dtinymce%26libraryContent%3Dbrowse; wp-settings-time-1=1428174206 - VCL_return hash - ReqUnset Accept-Encoding: gzip, deflate, sdch - ReqHeader Accept-Encoding: gzip - VCL_call HASH - VCL_return lookup - Hit 2147876896 - VCL_call HIT - VCL_return deliver - RespProtocol HTTP/1.1 - RespStatus 200 - RespReason OK - RespHeader Date: Wed, 08 Apr 2015 18:23:48 GMT - RespHeader Server: Apache - RespHeader X-Powered-By: PHP/5.5.9-1ubuntu4.4 - RespHeader X-Pingback: http://blog.tembies.com/xmlrpc.php - RespHeader Vary: Accept-Encoding - RespHeader Content-Encoding: gzip - RespHeader Content-Length: 6971 - RespHeader Content-Type: text/html; charset=UTF-8 - RespHeader X-Varnish: 196714 393248 - RespHeader Age: 286 - RespHeader Via: 1.1 varnish-v4 - VCL_call DELIVER - RespHeader X-Cache: HIT - VCL_return deliver - Timestamp Process: 1428517714.523595 0.000439 0.000439 - Debug "RES_MODE 2" - RespHeader Connection: keep-alive - RespHeader Accept-Ranges: bytes - Timestamp Resp: 1428517714.523782 0.000627 0.000187 - Debug "XXX REF 2" - ReqAcct 478 0 478 382 6971 7353 - End

* << BeReq >> 65678 - Begin bereq 65677 fetch - Timestamp Start: 1428517428.357995 0.000000 0.000000 - BereqMethod GET - BereqURL / - BereqProtocol HTTP/1.1 - BereqHeader Host: blog.tembies.com - BereqHeader Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 - BereqHeader User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.68 Safari/537.36 - BereqHeader Accept-Language: en-US,en;q=0.8 - BereqHeader X-Forwarded-For: 64.236.208.25 - BereqHeader Accept-Encoding: gzip - BereqHeader X-Varnish: 65678 - VCL_call BACKEND_FETCH - VCL_return fetch - BackendOpen 22 server3(45.55.163.193,,80) 45.55.163.206 48492 - Backend 22 vdir server3(45.55.163.193,,80) - Timestamp Bereq: 1428517428.359096 0.001101 0.001101 - Timestamp Beresp: 1428517428.362346 0.004352 0.003251 - BerespProtocol HTTP/1.1 - BerespStatus 200 - BerespReason OK - BerespHeader Date: Wed, 08 Apr 2015 18:23:48 GMT - BerespHeader Server: Apache - BerespHeader X-Powered-By: PHP/5.5.9-1ubuntu4.4 - BerespHeader Content-Length: 0 - BerespHeader Content-Type: text/html - TTL RFC 120 -1 -1 1428517428 1428517428 1428517428 0 0 - VCL_call BACKEND_RESPONSE - TTL VCL 300 10 0 1428517428 - VCL_return retry - BackendClose 22 server3(45.55.163.193,,80) - Timestamp Retry: 1428517428.362532 0.004537 0.000185 - Link bereq 393248 retry - End

* << BeReq >> 393248 - Begin bereq 65678 retry - Timestamp Start: 1428517428.362532 0.004537 0.000000 - BereqHeader X-Varnish: 393248 - VCL_call BACKEND_FETCH - VCL_return fetch - BackendClose 14 server1(45.55.140.8,,80) toolate - BackendOpen 14 server1(45.55.140.8,,80) 45.55.163.206 58585 - Backend 14 vdir server1(45.55.140.8,,80) - Timestamp Bereq: 1428517428.363330 0.005335 0.000798 - Timestamp Beresp: 1428517428.475972 0.117978 0.112643 - BerespProtocol HTTP/1.1 - BerespStatus 200 - BerespReason OK - BerespHeader Date: Wed, 08 Apr 2015 18:23:48 GMT - BerespHeader Server: Apache - BerespHeader X-Powered-By: PHP/5.5.9-1ubuntu4.4 - BerespHeader X-Pingback: http://blog.tembies.com/xmlrpc.php - BerespHeader Vary: Accept-Encoding - BerespHeader Content-Encoding: gzip - BerespHeader Content-Length: 6971 - BerespHeader Content-Type: text/html; charset=UTF-8 - TTL RFC 120 -1 -1 1428517428 1428517428 1428517428 0 0 - VCL_call BACKEND_RESPONSE - TTL VCL 300 10 0 1428517428 - TTL VCL 300 120 0 1428517428 - VCL_return deliver - Storage malloc s0 - ObjProtocol HTTP/1.1 - ObjStatus 200 - ObjReason OK - ObjHeader Date: Wed, 08 Apr 2015 18:23:48 GMT - ObjHeader Server: Apache - ObjHeader X-Powered-By: PHP/5.5.9-1ubuntu4.4 - ObjHeader X-Pingback: http://blog.tembies.com/xmlrpc.php - ObjHeader Vary: Accept-Encoding - ObjHeader Content-Encoding: gzip - ObjHeader Content-Length: 6971 - ObjHeader Content-Type: text/html; charset=UTF-8 - Fetch_Body 3 length stream - Gzip u F - 6971 21147 80 80 55698 - BackendReuse 14 server1(45.55.140.8,,80) - Timestamp BerespBody: 1428517428.476646 0.118651 0.000674 - Length 6971 - BereqAcct 385 366 751 417 6971 7388 - End

Varnish Stats

• Run-time statistics about an instance

ESI

• Edge-side Includes

• Varnish supports:

• esi:include

• esi:remove

• <!--esi ...-->

VMODs

• Shared libraries with C functions that can be called from VCL code

• directors

• vmod_std

• VMOD Directory: https://www.varnish-cache.org/vmods (check the version!)

• Commercial VMODs (DeviceAtlas)

vmod_std

• std.querysort(req.url) — Sorts the query string

• std.healthy(backend) — Returns TRUE is a backend is healthy

• strstr(stringA, stringB) — Returns the substring if the second string is a substring of the first string

• man vmod_std

Embedded C

• C{ #include <dragons.h> }C

• C code included in VCL is compiled with the VCL and dynamically linked to Varnish in memory.

• Embedded C essentially becomes part of the Varnish process

• If your code produces a segfault, Varnish will crash

• Holy crap, don’t do this, why are you still reading this?

Further Reading

• https://www.varnish-cache.org/docs/4.0/index.html

• http://www.mobify.com/blog/beginners-guide-to-http-cache-headers/

• https://www.varnish-cache.org/trac/wiki/VCLExamples

Further Help

• @ieatkillerbees on Twitter

• tembies on freenode IRC

• #varnish on irc.linpro.no