Varnish bof
58
Varnish BOF!
-
Upload
marcus-deglos -
Category
Technology
-
view
4.708 -
download
1
description
#drupaldevdays birds of a feather talk on Varnish and ESI - Edge Side Includes - at Brussels.
Transcript of Varnish bof
VarnishBOF!
What's this talk about?
• A basic intro to Varnish and reverse proxies• Some cool ESI stuff• How to setup, implement + configure
Varnish/ESI in Drupal• Teh Awesomeness of speed
Why Varnish?
Drupal performs…just…
How much faster?
20 reqs/sec?
200 reqs/sec?
Drupal
Varnish
When personalisation goes wrong
Dries logs in
Drupal creates a personalised page
Logged in as dries
The page gets cached
Varnish
When webchick hits the site
The page is fetched from the cache
Varnish
The personalisation is incorrect
Logged in as dries
WTF?
WTF?
No-cache headers can prevent this
Varnish
But now it's slow :-(
EdgeSideIncludes
ESI: Tales from the past
• The idea originated in SSI• SSI, but moved to the edge• ESI is a W3C standard• ESI is a bunch of xml tags/attributes.• Supported by:– Varnish (partial support – no ESI variables)– Akamai– Some Nginx support– ???
Dries is visiting the site
The page is requested
Varnishhttp://example.com/node/1
The page includes an ESI tag
<esi src= "http://example.com/esi/user/0" />
The proxy sends a second request
Varnish
http://example.com/esi/user/0
http://example.com/node/1
The ESI URL sends a HTML snip
<div id="userinfo"> <a title="View & edit your user profile">Logged in as dries</a> <a href="/logout">Logout</a></div>
The ESI code is merged with the page
Proxy forwards the page on
Varnish
http://example.com/esi/user/0
http://example.com/node/1
This doesn’t sound quicker
The Importance of Tests
• It's important to measure the results(I'll come to measuring tools later!)
• The ESI snippets can be cached too
Cached ESIs
1st Page = 2 Requests
Varnishhttp://example.com/node/1
http://example.com/esi/user/0
2nd Page = 1 request
Varnishhttp://example.com/node/2
1 request for each new page
• 10 pages = 11 requests• Each page is cached• The ESI tag is cached
The next visitor gets the page for free
VarnishPage fetched from cache
http://example.com/node/1
1st page = 1 request (for the ESI)
Varnish
http://example.com/esi/user/0
Page fetched from cachehttp://example.com/node/1
Following pages all served from cache
Varnish
1 request for each new page
• 10 pages = 11 requests• The ESI tag is cached• Each page is cached• Subsequent requests have 1 request for the
ESI tag• 20 pages = 12 requests (10 pages + 2 ESI tags)
Fast enough yet?
Warm the cache
A modern infrastructure
Varnish HA Proxy
TehIntertubes
Service webheadadmin/cron/emails/??
Spidering the site
Varnish HA Proxy
TehIntertubes
Service webheadadmin/cron/emails/??
Link-checker /spider app.
Dries is back for more
Each user generates 1 request
Varnish
http://example.com/esi/user/0
Page fetched from cachehttp://example.com/node/1
5 users, 40 pages
• 5 backend requests!
Theory Over
Basics Varnish
+Drupal
Installing Varnish
• sudo apt-get install varnishsudo yum install varnish
• Edit /etc/default/varnish• Edit /etc/varnish/default.vcl• Usual config:– Varnish listens on port 80– Apache listens on port 8080 (or any available port)– SSL is more complex – sometimes Pound is used
for SSL decryption
VCLs are awesome!
• Varnish configuration is controlled by VCL: Varnish Configuration Language.
• Documented athttp://www.varnish-cache.org/trac/wiki/VCL
• Drupal VCL examples can be found in:– Mercury (Amazon AMI)– ESI module
• You can write C code in VCLs. Oh my.
Keeping the cache current
Varnish
Varnish admin port
Varnish Management Port
• The Varnish admin port uses a plain-text protocol to command Varnish
• Test by:telnet localhost 6093
• List commands by typing help• Purge cached pages: url.purge <regexp>
E.g. url.purge .*• The Varnish module provides an API.
Use with caution!
The default behaviour of the Varnish module is to
clear the entire cachewhen you edit a node.
Enter…the Expires module
• Came from the Boost module• Clears specific cache pages – not the entire
cache• Supports path-alias URLs as well as node/1
paths.• Provides hook_expire_cache_alter to add URLs
(e.g. nodes displayed in a page-view)• Supports other cache modules, not just Varnish.
I can haz ESI?
• The ESI module provides block integration
Configuring ESI
• Download the module• Configure the blocks to serve via ESI, and their
TTL• Modify the Varnish VCL (the ESI module comes
with example VCLs to use• Coming soon:– D7– Panels integration
Benchmarking
• Some tools:– J-meter– ab (apache-benchmark)– Seige
• Approaches:– Hammer your URL(s)
ab –n 100 –c5 http://example.com/– Replay your log files
Pitfalls await
• VCLs are complex• Per-user cache rules are
even scarier!• Varnish may be slower• Risk exposing private
data
Alternatives
• Boost module– No need to install a new service on the server– Static file cache
• Advanced cache module• Authcache module• Other proxies:– Squid– Nginx– Akamai?
Summary
How hard?
Teh Awesomeness
Varnish
Varnish module
Expire module
ESI module
Picture creditsThanks to these CC-licenced flickr users• Mac Book Pro - http://www.flickr.com/photos/wicho/• F1 car - http://www.flickr.com/photos/w3i_yu/• Banger car - http://www.flickr.com/photos/photoplod• Dries - http://www.flickr.com/photos/boris• Dries+laptop - http://www.flickr.com/photos/azugaldia• Webchick - http://www.flickr.com/photos/mortendk • Webchick - http://www.flickr.com/photos/puregin• Fire - http://www.flickr.com/photos/benwatts• Hourglass - http://www.flickr.com/photos/opendemocracy• Stopwatch -
http://www.flickr.com/photos/purplemattfish
