RUM for Breakfast - distilling insights from the noise
-
Upload
buddy-brewer -
Category
Technology
-
view
1.140 -
download
0
description
Transcript of RUM for Breakfast - distilling insights from the noise
![Page 2: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/2.jpg)
RUM for Breakfast
Buddy Brewer, Carlos Bueno, Philip Tellis
Velocity 2012 / 2012-06-26
Velocity 2012 / 2012-06-26 RUM for Breakfast 2
![Page 5: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/5.jpg)
https://github.com/lognormal/boomerang/
Velocity 2012 / 2012-06-26 RUM for Breakfast 5
![Page 7: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/7.jpg)
Buddy built the navtiming plugin
Velocity 2012 / 2012-06-26 RUM for Breakfast 7
![Page 8: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/8.jpg)
tl;dr
1 Measure a bunch of stuff in the browser2 Use high school stats that we vaguely remember3 Randomly invent insights
Velocity 2012 / 2012-06-26 RUM for Breakfast 8
![Page 10: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/10.jpg)
![Page 11: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/11.jpg)
![Page 12: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/12.jpg)
![Page 13: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/13.jpg)
![Page 14: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/14.jpg)
![Page 15: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/15.jpg)
![Page 16: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/16.jpg)
![Page 17: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/17.jpg)
![Page 18: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/18.jpg)
![Page 19: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/19.jpg)
![Page 20: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/20.jpg)
![Page 21: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/21.jpg)
![Page 22: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/22.jpg)
![Page 23: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/23.jpg)
![Page 24: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/24.jpg)
![Page 25: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/25.jpg)
![Page 26: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/26.jpg)
![Page 27: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/27.jpg)
![Page 28: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/28.jpg)
![Page 29: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/29.jpg)
![Page 30: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/30.jpg)
![Page 31: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/31.jpg)
![Page 32: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/32.jpg)
![Page 33: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/33.jpg)
![Page 34: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/34.jpg)
![Page 35: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/35.jpg)
![Page 38: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/38.jpg)
Log-Normal Distribution
The logarithm of the x-axis follows a Normal distribution
Velocity 2012 / 2012-06-26 RUM for Breakfast 11
![Page 39: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/39.jpg)
Log-Normal Distribution
Use the Geometric Mean for pure Log-Normal distributions
Velocity 2012 / 2012-06-26 RUM for Breakfast 12
![Page 40: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/40.jpg)
Log-Normal Distribution
Performance data does not always follow a "pure" Log-Normal
distribution
Velocity 2012 / 2012-06-26 RUM for Breakfast 13
![Page 41: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/41.jpg)
Look at the entire spread
. . .
Velocity 2012 / 2012-06-26 RUM for Breakfast 14
![Page 42: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/42.jpg)
Look at the entire spread
which often approaches an infinite width
Velocity 2012 / 2012-06-26 RUM for Breakfast 14
![Page 44: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/44.jpg)
• 0.8% of hits are fake/abusive• 0.2-0.5% of hits are from a stale cache• 0.1% of hits are absurd• Timestamps in the future (or past depending on how you
interpret it)• Bots ignore robots.txt across domains• "Interesting" caches/copies
Velocity 2012 / 2012-06-26 RUM for Breakfast 16
![Page 45: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/45.jpg)
Even with beacons, you need to sanitize your input
Velocity 2012 / 2012-06-26 RUM for Breakfast 17
![Page 47: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/47.jpg)
Band-pass filtering
• Strip everything outside a reasonable range• Bandwidth range: 4kbps - 4Gbps• Page load time: 0ms - 600s
• You may need to relook at the ranges all the time
Velocity 2012 / 2012-06-26 RUM for Breakfast 18
![Page 49: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/49.jpg)
IQR filtering
Derive the range from the data
Velocity 2012 / 2012-06-26 RUM for Breakfast 19
![Page 52: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/52.jpg)
MoE & Sample size
There is an inverse square root correlation between sample size
and margin of error
Velocity 2012 / 2012-06-26 RUM for Breakfast 22
![Page 53: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/53.jpg)
How big a sample is representative?
Select n such that���1.96 σ√n
��� ≤ 5%µ
Velocity 2012 / 2012-06-26 RUM for Breakfast 23
![Page 54: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/54.jpg)
This needs to be at your lowest drilldown level
Velocity 2012 / 2012-06-26 RUM for Breakfast 24
![Page 56: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/56.jpg)
How does performance impact human behavior?
![Page 57: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/57.jpg)
8 million pages
1.5 million visits
50 different dimensions
![Page 58: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/58.jpg)
0%
17.50%
35.00%
52.50%
70.00%
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
very fast sessions had high bounce rates
![Page 59: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/59.jpg)
0%
17.50%
35.00%
52.50%
70.00%
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29
bounce rate vs. load time
![Page 60: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/60.jpg)
0%
17.50%
35.00%
52.50%
70.00%
1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 6.5 7 7.5 8 8.5 9 9.5 10 10.5 11 11.5 12 12.5
bounce rate vs. DOM interactive
![Page 61: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/61.jpg)
0%
20.00%
40.00%
60.00%
80.00%
0.5 2 3.5 5 6.5 8 9.5 11 12.5 14 15.5 17 18.5 20 21.5 23 24.5 26 27.5 29
bounce rate vs. front end time
![Page 62: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/62.jpg)
http://www.flickr.com/photos/21560098@N06/3796822070
is my web site performance toxic to my users?
![Page 63: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/63.jpg)
http://www.flickr.com/photos/thecosmopolitan/6117530924
LD50 - when do half the users bounce?
![Page 64: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/64.jpg)
Back end time
DOM Loading
DOM Interactive
Front end time
DOM Complete
Load event
1.7 sec
1.8 sec
2.75 sec
3.5 sec
4.75 sec
5.5 sec
Bounce rate >=50%
![Page 65: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/65.jpg)
What is the LD50 for your site?
Other bounce rates? 40%? 30%?
Other variables? (critical content visible, etc)
Other behaviors? Conversions, revenue, pages per session, actions, when do people make tea?
Future directions
![Page 67: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/67.jpg)
Questions?
Buddy Brewer @bbrewerPhilip Tellis @bluesmoon
Carlos Bueno @archivd
![Page 69: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/69.jpg)
![Page 70: RUM for Breakfast - distilling insights from the noise](https://reader033.fdocuments.net/reader033/viewer/2022051610/54829838b07959570c8b4822/html5/thumbnails/70.jpg)