LCA 2012

Creative Techniques for Loading Web Pages Faster

vs.low hanging fruit micro-optimization

Trevor Parscal -Roan Kattouw -

@trevorparscal@catrope

LCA 2012

People

Trevorhuman

Roanrobot

LCA 2012

People

Front-endBack-end

LCA 2012

Software

MediaWikiplatform

Wikipediaproject

LCA 2012

LCA 2012

Resources

0KB 100KB 200KB 300KB 400KB 500KB 600KB

HTML JavaScript CSS Images

1 2229 4

LCA 2012

Resources

javascriptScripts

cssStyles

jsonTranslations

LCA 2012

Origins

most everything elseExtensions

Math.random();Users

skins & common stuffCore

LCA 2012

Optimizations

file concatenationCombination

whitespace removalMinification

multi-graphic imagesSprites

LCA 2012

Caching

one version for everythingVersioning

NEW

wait for cache expiryPurging

versions in the htmlQuery String

?v=2

LCA 2012

Being a developer sucked

messy and boringMaintaining

clunky and slowUsing

good luck!Creating

LCA 2012

Traffic

2008 2009 2010 20110M

100M

200M

300M

400M

500M

Unique Visitors per Month

LCA 2012

Cache Invalidation

site-wide resourcesAdmins

site-wide messagesTranslators

user-specific resourcesUsers

LCA 2012

Approach

no micro-optimizationEasy Gains

developers use this stuffEase of Use

it’s where the magic isClient Focus

LCA 2012

Modules

Scripts, styles& messages

LCA 2012

Modules

b ca

a, b & c

LCA 2012

Scripts

debug, skin or languageConditions

whitespace removalMinification

{ }delayed executionWrapping

LCA 2012

Styles

free rtl supportFlipping

relative urls still workRemapping

data uri sweetnessEmbedding

LCA 2012

Styles

whitespace removalMinification

skin specific stylesConditions

one requestBundling

LCA 2012

Messages

Bundlingone request

Conditionslanguage

LCA 2012

Startup Module

site-wide settingsConfiguration

if ( IE5 ) { giveUp(); }Sanity check

module manifestDependencies

b c

a

d e

LCA 2012

Client-side Loader

calculate dependenciesResolution

b c

a

give it to me all at onceBatching

ca +

run in correct orderExecution

a

LCA 2012

Caching

?v=time

per-moduleVersioning

30 daysResources

5 minutesStartup Module

LCA 2012

So, it turns out...

LCA 2012

/html

/body

!DOCTYPE html

html

head

body

/head

Page contentp /p

script src=”all.js?version=123” /script

Order matters

LCA 2012

script src=”startup”

/script

var registry = { /* manifest of all modules */ };document.write(‘<script src=”all.js?v=’ + version + ’”></script>’);

/html

/body

!DOCTYPE html

html

head

body

/head

Page contentp /p

LCA 2012

script src=”startup”

/script

var registry = { /* manifest of all modules */ };document.write(‘<script src=”all.js?v=’ + version + ’”></script>’);

/html

/body

!DOCTYPE html

html

head

body

/head

Page contentp /p

LCA 2012

script src=”startup”

/script

var registry = { /* manifest of all modules */ };document.write(‘<script src=”base.js?v=’ + version + ’”></script>’);

/html

/body

!DOCTYPE html

html

head

body

/head

Page contentp /p

mw.loader.load([‘extras’]); // uses document.write

script

/script

LCA 2012

script src=”startup”

/script

var registry = { /* manifest of all modules */ };document.write(‘<script src=”base.js?v=’ + version + ’”></script>’);

/html

/body

!DOCTYPE html

html

head

body

/head

Page contentp /p

mw.loader.load([‘bottom’]); // adds to head using DOM methods

script

/script

mw.loader.load([‘top’]); // adds to head using document.write

script

/script

LCA 2012

Embedding is sweet

Test: embedding images in the Vector skin, 27.3% reduction in total size after compression, 97.2% reduction in requestsCSSMin: http://tinyurl.com/CSSMin-php

0KB

4KB

8KB

12KB

16KB

Normal (35 requests) Embedded (1 request)

LCA 2012

JSMin.php is slow

Test: minifying jQuery development distribution, 160KB of uncompressed code; 0.5% larger size than JSMin output after compressionJavaScriptMinifier: http://tinyurl.com/JavaScriptMinifier

0s

0.25s

0.50s

0.75s

1.00s

JSMin.php JavaScriptMinifier.php

LCA 2012

$humans--;seriously, go robots!

is Awesome

Portedfrom python

CSSJanus: http://tinyurl.com/CSSJanus

LCA 2012

Balance is important

a & b b, c & d

Batchingduplicate data may be sent

Groupscontrolled fragmentation

c & da b

LCA 2012

It Works!

0KB 100KB 200KB 300KB 400KB 500KB 600KB

HTML JavaScript CSS Images

1 47 2

LCA 2012

It’s Efficient!

Requests90k req/s peak load

Servers~400 servers

For resources: 4 app, 9 cache of which 40k are for resources

and 73 are cache missescache hit rate: 98.2%

LCA 2012

ResourceLoader

no micro-optimizationEasy Gains

developers use this stuffEase of Use

it’s where the magic isClient Focus

LCA 2012

ResourceLoader

happy serversLightweight

happy peopleSnappy Pages

happy developersEasy to Make

LCA 2012

Thanks!

http://wikitech.wikimedia.org/view/Presentations

http://www.mediawiki.org/wiki/ResourceLoader

Trevor Parscal -Roan Kattouw -

@trevorparscal@catrope

LCA 2012

Work @ Wikimedia

http://jobs.wikimedia.org

LCA 2012

Questions?

http://wikitech.wikimedia.org/view/Presentations

http://www.mediawiki.org/wiki/ResourceLoader

Trevor Parscal -Roan Kattouw -

@trevorparscal@catrope