Metrics, metrics everywhere(but where the heck do you start?)Tammy Everts & Cliff CrockerVelocity Santa Clara 2015

@tameverts @cliffcrocker

#velocityconf

Who cares about performance today?How do I measure performance?

How fast am I?How fast should I be?How do I get there?

The myth of a single metric

Start render DNS TCP TTFB

DOM loading DOM ready Page load Fully loaded

User timing Resource timing Requests Bytes in

Speed Index Pagespeed score 1s = $$ DOM elements

DOM size Visually complete Redirect SSL negotiation

Who cares about performance?

“47% of consumers expect a web page to load in 2 seconds or less.”

Akamai, 2009

1s = $27M DNS144ms

Start render1.59s

Hero image render2.01s

How do I measure performance?

Andr

oid

devi

ce fr

agm

enta

tion

Ope

nSig

nal,

Augu

st 2

014

RUM versus plus synthetic

RUM 101

Technology for collecting performance metrics directly from the end user’s browser

Involves instrumenting your site via JavaScript

Measurements are fired across the network to a collection point through a small request object

(beacon)

What makes RUM great

Always on Every user, every browser, everywhere Able to capture human behavior/events Only getting better

Questions your RUM data can answer

What are my users’ environments?

How do visitors move through my site?

How are my third-party scripts

performing in real time?

What impact does performance have

on my business?

Synthetic Monitoring 101

Uses automated agents (bots) to measure your website from different physical locations

A set “path” or URL is defined

Tests are run either ad hoc or scheduled, and data is collected

What makes synthetic monitoring great Rich data collected (waterfall, video,

filmstrip, HTTP headers) Consistent “clean room” baseline Nothing to install Doesn’t require users / ability to

measure pre-production and competition

Questions your synthetic data can answer

How do I compare to the competition?

How does the design of my pages affect

performance?

How does the newest version of my site compare to previous versions?

How well am I sticking to my performance budget?

What if my third parties fail?

Original: 3.5s

SPOF: 22.7s

37© 2014 SOASTA CONFIDENTIAL - All rights reserved.

Common things we hear about RUM & synthetic

Why are these numbers so different?I don’t trust your data. Your numbers are wrong.How are you calculating page load time?I can’t share two sets of numbers with the business?

“But it loads so much faster for me!?!”

2015 Macbook ProWarm browser cache

FIOS

X86 – Windows 7 VMCompletely cold cache/DNS

Throttled bandwidth

boomerang.js

Episodes

W3C Performance Working Group

How fast am I?

Navigation Timing API

Browser support for Navigation Timing

!

46© 2014 SOASTA CONFIDENTIAL - All rights reserved.

Role-specific use cases

Network operationsFront-end developerMarketing and site operationsDesignerC-level

Use case: Measure network performance

I need visibility into… issues with authoritative DNS servers impact of denial of service attacks

on end users efficiency of connection re-use tier 1 connectivity issues (load balancer,

web server)

Start render DNS TCP TTFB

DOM loading DOM ready Page load Fully loaded

User timing Resource timing Requests Bytes in

Speed Index Pagespeed score 1s = $$ DOM elements

DOM size Visually complete Redirect SSL negotiation

Measuring DNS and TCP

function getPerf() { var timing = window.performance.timing; return {

dns: timing.domainLookupEnd - timing.domainLookupStart};

connect: timing.connectEnd - timing.connectStart};}

What’s with all those zeros! Connection reuse DNS caching Prefetching

Focus on higher percentiles

85th percentile

Median (50th)

Use case: Measure front-end browser events

How do I… understand the impact of back-end

versus front-end on page speed? investigate how DOM complexity affects

performance? measure a “fully loaded” page?

Start render DNS TCP TTFB

DOM load event DOM ready Page load Fully loaded

User timing Resource timing Requests Bytes in

Speed Index Pagespeed score 1s = $$ DOM elements

DOM size Visually complete Redirect SSL negotiation

Isolate front-end vs. back-end

Isolate front-end vs. back-end

function getPerf() { var timing = window.performance.timing; return {

ttfb: timing.responseStart - timing.connectEnd};basePage: timing.responseEnd - timing.responseStart};frontEnd: timing.loadEventStart -

timing.responseEnd};}

Front-end

Back-end

Investigate DOM eventsfunction getPerf() { var timing = window.performance.timing; return {

DLoading: timing.domLoading – timing.navigationStart};

DInt: timing.domInteractive – timing.navigationStart};

DContLoaded: timing.domContentLoadedEventEnd – timing.navigationStart;

DContLoadTime: timing.domContentLoadedEventEnd – timing.domContentLoadedEventStart};

DComplete: timing.domComplete - timing.navigationStart};

PLoad: timing.loadEventStart - timing.navigationStart};}

2618 DOM nodes

86 DOM nodes

Visualizing DOM complexity

Use case: Measure object-level performance

I need to understand… how third-party content affects my

performance how long a group of assets takes to

download how assets served by a CDN are

performing

Start render DNS TCP TTFB

DOM loading DOM ready Page load Fully loaded

User timing Resource timing Requests Bytes in

Speed Index Pagespeed score 1s = $$ DOM elements

DOM size Visually complete Redirect SSL negotiation

Resource Timing interface

Browser support for Resource Timing

Cross-Origin Resource Sharing (CORS)

Start/End time only unless Timing-Allow-Origin HTTP Response Header defined

Timing-Allow-Origin = "Timing-Allow-Origin" ":" origin-list-or-null | "*"

Resource Timingvar rUrl = ‘http://www.akamai.com/images/img/cliff-crocker-dualtone-150x150.png’;var me = performance.getEntriesByName(rUrl)[0];var timings = { loadTime: me.duration, dns: me.domainLookupEnd - me.domainLookupStart, tcp: me.connectEnd - me.connectStart, waiting: me.responseStart - me.requestStart, fetch: me.responseEnd - me.responseStart}

Measuring a single resource

Other uses for Resource Timing Slowest resources Time to first image (download) Response time by domain Time a group of assets Response time by initiator type (element type) Cache-hit ratio for resources

For examples see: https://github.com/lognormal/beyond-page-metrics

Using Resource Groups

PLT impact not due resource groups

PLT impact correlates with improvement from image domains

Use case: Measure the user experience

I just need to understand… when users perceive the page to

be ready how long until my page begins

to render when content above the fold is visible

Start render DNS TCP TTFB

DOM loading DOM ready Page load Fully loaded

User timing Resource timing Requests Bytes in

Speed Index Pagespeed score 1s = $$ DOM elements

DOM size Visually complete Redirect SSL negotiation

The fallacy of “First Paint” in the wild Support for First Paint is not standardized between browsers Metric can be misleading (i.e. painting a white screen)

First Paint is not equal to Start Render!Chrome – “First Paint” True Start Render

Start Render and filmstrips

User Timing Interface Allows developers to measure performance of

their applications through high-precision timestamps

Consists of “marks” and “measures” PerformanceMark: Timestamp PerformanceMeasure: Duration between

two given marks

Measure duration between two marksperformance.mark(“startTask”);

//Some stuff you want to measure happens here

performance.mark(“stopTask”);

//Measure the duration between the two marks

performance.measure(“taskDuration”,“startTask”,“stopTask”);

How long does it take to display

the main product image on my site?

Record when an image loads

<img src=“image1.gif” onload=“performance.mark(‘image1’)”>

For more interesting examples, see:

Measure hero image delayhttp://www.stevesouders.com/blog/2015/05/12/hero-image-custom-metrics/

Measure aggregate times to get an “above fold time”http://blog.patrickmeenan.com/2013/07/measuring-performance-of-user-experience.html

How do I measure performance when the onload event no longer matters?

Use case: Measure performance of SPAs

onload event

visible resources

Measuring SPAs• Accept the fact that onload no longer matters• Tie into routing events of SPA framework• Measure downloads during soft refreshes• (support in boomerang.js for Angular and other

SPA frameworks)

See: http://www.soasta.com/blog/angularjs-real-user-monitoring-single-page-applications/

How fast should I be?

Use case: Measure business impact

I need to understand… how performance affects business KPIs how our site compares to our

competitors

Start render DNS TCP TTFB

DOM loading DOM ready Page load Fully loaded

User timing Resource timing Requests Bytes in

Speed Index Pagespeed score 1s = $$ DOM elements

DOM size Visually complete Redirect SSL negotiation

2% increase in conversions for every 1 second of improvement

Cut load times in halfIncreased sales by 13%

So what?You must look at your own data.

Not all pages are created equalTop of funnel impact (“browse” pages)

For a typical ecommerce site, conversion rate drops by up to 50% when “browse” pages increase from 1 to 6 seconds

Not all pages are created equalBottom of funnel impact (“checkout” pages)

However, there is much less impact to conversion when “checkout” pages degrade

Conversion Impact Score

How do I get there?

Create a performance budget

See:

Setting a Performance Budgethttp://timkadlec.com/2013/01/setting-a-performance-budget/

Performance Budget Metricshttp://timkadlec.com/2014/11/performance-budget-metrics/

Set meaningful, page-specific SLAs

“Response time measured using resource timing from Chrome browsers in the United States should not exceed a median (50th percentile) of 100ms or a 95th percentile of 500ms for a population of more than 500 users in a 24-hour period.”

“Vendor will make an effort to ensure average response times for content is within reasonable limits.”

Chapter 8: Changing Culture

at Your Organization

performancebacon.com

performancebeacon.com

Thanks!

Meet us at booth #801