Boomerang at the Boston Web Performance meetup

Why measure?boomerang

data data data

Measuring the web with boomerang

Philip Tellis / [email protected]

Boston Performance Meetup / 2010-09-16

Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang

http://www.meetup.com/Web-Performance-Boston/calendar/14510230/



data data data

$ finger philip

Philip [email protected]

@bluesmoonyahoogeekhttp://bluesmoon.info/


http://twitter.com/bluesmoon

http://bluesmoon.info/



data data data

The slow webMeasurementsMeasuring with javascript




data data data


Less than 20% of page load time is something we can measureand fix during development




data data data


It’s what we can’t control that bites us




data data data


Too many variations

browsers




data data data


Too many variations

plugins




data data data


Too many variations

OSes




data data data


Too many variations

viruses




data data data


Too many variations

antiviruses




data data data


Too many variations

microwaves




data data data


Too many variations

baby monitors




data data data


Too many variations

naughty neighbours




data data data


Too many variations

file shares




data data data


Too many variations

governments




data data data


Too many variations

rodents




data data data


Too many variations

Try simulating all that in the lab!




data data data


We need to measure real end-user performance




data data data


We need to measure real end-user performance from the realend-user’s box




data data data


Ask the user?




data data data


While this might work, it isn’t necessarily representative




data data data


Why not just use javascript in each page that you want tomeasure?




data data data


But javascript can’t measure everything... we get as close aswe can




data data data


It also adds page weight




data data data


However, it is available everywhere and gives us flexibility




data data data

What?How does it work?Accuracy

boomerang is...

A piece of javascript that you add to your web page where itmeasures and beacons back to you, the end user’s perceivedperformance of your page




data data data


How?

<script src="boomerang.js" type="text/javascript"></script><script type="text/javascript">BOOMR.init({

user_ip: "<user’s ip address>",beacon_url: "http://yoursite.com/beacon.php"

});</script>




data data data


What does it do?

About once a week, measures user’s bandwidth andlatency to your serverOn (almost) every request, measures the time it took toload the current pageBeacons these results back to your server




data data data


How does it do it?

Let’s take that one at a time




data data data


Measuring latency

Download a 32 byte gif 10 times in sequenceMeasure the time to download eachDiscard the first measurement because it’s overpricedCalculate the arithmetic mean, standard deviation andmargin of error of the rest




data data data


Wait, did you say overpriced?

The first image might require a DNS lookup and TCPhandshakeSlow start is not an issue since 32 bytes fits in 1 packet




data data data


Measuring bandwidth

After the latency test is done, we download progressivelylarger imagesStop at the first image that times outRedownload that image a few more timesCalculate the median, standard deviation and margin oferror of the largest images




data data data


Measuring latency before bandwidth helps here

Those 10 latency images do a lot to widen the TCPwindow sizeThe bandwidth images make much better use of bandwidthThe image we end with uses the most bandwidth




data data data


How do we measure page load time?

In the onbeforeunload event, measure the time andstore it in a cookieIn the onload event, check the cookie, and measure thedifference with the current timeWe also make sure that the page that set the cookie is thereferrer of the current page




data data data


What? Two pages?

Yes, this needs two pages and cookies. If those aren’tsupported, we try to use the WebTiming API.




data data data


How accurate is it?

Latency measurements are very accurate (±1%)Bandwidth is to an order of magnitude. For badconnections can be ±30%

Page load time sometimes has outliers, you needpost-filteringThe margin of error tells you how good your data is




data data data

FilteringGroupingData

What do we do with the data?

Sanity checking to:Remove fake dataRemove abusive dataMaybe just rate limiting




data data data


What do we do with the data?

Statistical analysis to:Remove outliersAggregate based on bandwidth blocksMeasure trends over time and correlate them with codechanges




data data data


Bandwidth blocks

0-100 kbps100-300 kbps300-2000 kbps2-6 Mbps6+ Mbps




data data data


Bandwidth blocks

Group page load times based on bandwidth block




data data data


Bandwidth blocks

Ref: Analysing Bandwidth & Latency – YUI Blog


http://www.yuiblog.com/blog/2010/04/08/analyzing-bandwidth-and-latency/



data data data


Bandwidth blocks

Data points from some countries may require narrower bands




data data data


Geographic data

Looking at latency from different geographic locations can tellyou where to put your next CDN




data data data


ISPs

Grouping data by ISP can tell you who’s behaving badly




data data data


Storing the data

We log all beacon requests to apache’s log access fileLow traffic sites could write directly to a DBOthers have suggested using CouchDB as the beaconserverDaily summaries can be sent across to ShowSlow




data data data


More data

Write plugins to get more performance dataWe already have a DNS pluginI’m thinking of an IPv6 v/s IPv4 pluginWhat about a full WebTiming plugin?Can we measure connection setup time?




data data data


You decide

Once you have the data, you can do anything with it




data data data


Thank youhttp://github.com/yahoo/boomerang

http://yahoo.github.com/boomerang/doc/


http://www.github.com/yahoo/boomerang/

http://yahoo.github.com/boomerang/doc/



data data data


Photo credits

flickr.com/photos/21233184@N02/4389412851


http://www.flickr.com/photos/21233184@N02/4389412851/



data data data


Contact me

Philip Tellisyahoogeek@bluesmoonhttp://bluesmoon.info/slideshare.net/[email protected]


http://twitter.com/bluesmoon

http://bluesmoon.info/

http://www.slideshare.net/bluesmoon



data data data


References

github.com/yahoo/boomerangMore bandwidth doesn’t matter (much) – Mike BelsheAnalysing Bandwidth & Latency – YUI BlogIt’s the latency, stupid – Stuart CheshireThe statistics of web performance


http://github.com/yahoo/boomerang/

http://www.belshe.com/2010/05/24/more-bandwidth-doesnt-matter-much/

http://www.yuiblog.com/blog/2010/04/08/analyzing-bandwidth-and-latency/

http://www.stuartcheshire.org/rants/Latency.html

http://www.slideshare.net/bluesmoon/index-3441823


Boomerang at the Boston Web Performance meetup

Technology

Transcript of Boomerang at the Boston Web Performance meetup