MetricsHvordan arbejder du effektivt med metrics?

2

Lidt om mig

• Iværksætter på den anden side af bordet• Finance, hardware/software, bl.a. i Kina• Har arbejdet ca. 4 år i VHHR• Arbejder udelukkende med fokus på early stage

tech startups• Ser omkring 100 startups/år• Arbejder med omkring 50

• Arbejder med mange startups omkring metrics

3

Hvor møder jeg metrics

4

5

6

Metrics er ikke til for dine investorer!• De er til for at du kan styre din virksomhed

mest effektivt

• Men der er SÅ mange forskellige ting man kan se på – hvor skal man starte?

• Det kommer an på din type af virksomhed og hvilket stadie du er på!

7

Lad os tage et par eksemplerType:• SaaS løsning• 2-sidet markedsplads• Socialt netværk• Rejsesøgemaskine

Stadie:• Jo tidligere du er: Jo mere kvalitativt og jo mere fokus på brug af dit

produkt

8

Det er blevet ”let”(ere) at måle

• Software startups (og virksomheder der sælger på nettet) har en række analytics redskaber der gør det lettere at måle metrics.• Google Analytics• Mixpanel/Flurry/etc. etc. etc.

• Men de kræver stadig at du har defineret hvilke metrics der er vigtige før du kan bruge dem til noget!

T he O nl y Metric That Matters b y a Partner atGreylock Partners

www.productschool.com

Elman

The Only MetricThat MattersJosh

We’re all here to build products thatmatter.

We want to make an impacton our user’s lives.

We start with a launch. Thatmeans zero users. Hopefully theycome (or we bring them).

We start hearing stories.Anecdotes.

The plural of anecdote is data.

We have so much data now..

..it can be overwhelming

There are lots of data points…

“ We just topped “250 million PVs inthe last 30 days”200,000 sign ups ”

“We have over1 million downloads”

“Our D7 retention is 60%.D30 is 22%. Is that

good?”“ We have a 50%DAU/MAU ratio ”

So where do you start?

Just focus on one thing:

Arepeople using your product?

OK, but how can Itell?

1 Purpose: Whydo they use your product?2 Core Action: What is the key behavior?3 Cycle: Do they use it at the frequency you expect?

Key metric: How many times usersperform the core action on theexpected cycle

LinkedInCirca2005

PurposeFindand be found.

Core Action & CycleFind: search for other2-3x a week

members

Befound: respond to inquirieswhen you receive them 1-2x ayear

Yelp

PurposeFindlocal businesses/services. Andreview them.

Core Action & CycleSearchers: search for businesses 2-3x a week.

Reviewers: write a new review1x/month

Facebook

PurposeStay connected with friends.

Core Action & CycleCheckin with friends and seewhat they are up to.

Sharing 2-3x/day

DiscordPurpose: Connect and communicate with friends around playing PC games.

Core Action & CycleOpen the app to see who wants toplay games and get on a live channel.

3-4x/week (depends on howoften you play)

Airbnb

PurposeBook a place to stay.

Core ActionSearch and book trip options.

Cycle3-5x / year

Your “core users” are your best users

Top of Mind: They come directly, not because ofreminder, notification, or special deal

a1

Track: % users with direct traffic / engagement

Recurring: Do they come back again and again2Track: % users who return weekly/monthly

Referring: Do they share it with others?3Track: k-factor, and % users who are referrers

Arepeople really, reallyusing your product?

Step 1: Analyze month over monthusage

# days visited inmonth 1

%w

hovi

site

dN

days

inm

onth

2

Step 2: Determine threshold for core

Cold Casual Core

Step 3: Understand20%

activation buckets90% 8

%Core

40%40

% Casual

Curious

New 1

%10% 4

%Cold 50

%40% 95

%

dataStep 4a: Find out why — from your

1/3 - mutualfollows

2/3 - one-wayfollows

Following morethan 30 accounts

Step 4b: Find out why — talk to themAsk these questions:1)in2)to3)

Whatprompted you to sign up and try out our productthe first place?What did not meet your expectations or what was hardfigure out?Whydid you come back to give the product anothertry?

4) Whatworked the second time that got you using itmore frequently?

Cherish your core users. They areyour superheroes.

IhaveI

a few core users.want more!

Problem 1Not enough people show up.

Clone your core users through virality

DemonstrationVirality

InfectiousVirality

Word-of-mouthVirality

Problem 2People sign up, but no one stays.

Onboard users all the way to core.

Learn Flow: Teach your users your product as theysign up. Take your time. They are there to learn. Don’trush it.

Teaches users what the product

does

Shows a human curated list of the

best content

Encourages new users to engage and

create content

Welcome Week: If users don’t do it the first day,nudge them to come back and keep going

Use email to educateusers on the product.

new

Social Pressure: If friends are on the system, usethem to welcome, teach, and connect with the newfolks

Problem 3People

astay,little

but only forwhile.

Goback to the core analysisand extend it over quarters.

Understanding churn iscritical to your long termsuccess.

One metric to rule them all..

How many people are reallyusing your product?

44

Framework 2: Pirate metrics

• Her ser vi også meget på hvordan vi får folk ind og bruge produktet!

* reminder: National Talk Like a Pirate Day is Sept 19th!

Startup Metrics for Pirates:AARRR!!!

(Startup Metrics for Product Marketing & Product Management)

Dave McClureMaster of 500 Hats

blog: http://500hats.typepad.com/website: http://www.500hats.com/

slides: http://slideshare.net/dmc500hats/

Customer Lifecycle: 5 Steps to Success

• cquisition: users come to the site from various channels

• ctivation: users enjoy 1st visit: "happy" user experience

• etention: users come back, visit site multiple times

• eferral: users like product enough to refer others

• evenue: users conduct some monetization behaviorAARRR!

A

A

R

R

R

Customer Lifecycle / Conversion Behavior

Networks

1. ACQUISITION

Subscriptions, etc

Website.com

Ads, Lead Gen,Blogs, Content

Biz Dev

System Events & Time-based Features

ProductFeatures

Emails &widgets

Emails & Alerts

Homepage / Landing Page

Campaigns, Contests

Domains

EmailDirect, Tel,

TVApps & Widgets

AffiliatesBlogs

Social BizDev

PRSEMCampaigns,

ContestsSEO

Example Conversion Metrics(note: *not* actuals; your mileage may vary…)

Category User Status Conv % Est. ValueAcquisition Visit Site

(or landing page, or external widget)100% $.01

Acquisition Doesn't Abandon(views 2+ pages, stays 10+ sec, 2+ clicks)

70% $.05

Activation Happy 1st Visit(views X pages, stays Y sec, Z clicks)

30% $.25

Activation Email/Blog/RSS/Widget Signup(anything that could lead to repeat visit)

5% $1

Activation Acct Signup(includes profile data)

2% $3

Retention Email Open / RSS view -> Clickthru 3% $2Retention Repeat Visitor

(3+ visits in first 30 days)2% $5

Referral Refer 1+ users who visit site 2% $3Referral Refer 1+ users who activate 1% $10Revenue User generates minimum revenue 2% $5Revenue User generates break-even revenue 1% $25

Customer Lifecycle / Conversion Behavior

Networks

1. ACQUISITION

Marketing Channels:•••

largest-volume (#)lowest-cost ($)best-performing (%)

Website.com

Domains

EmailDirect, Tel,

TVApps & Widgets

AffiliatesBlogs

Social BizDev

PRSEMCampaigns,

ContestsSEO

Customer Lifecycle / Conversion Behavior

Activation Criteria:•

•

•

•

10-30+ seconds

2-3+ page views

3-5+ clicks

1 key feature usage

1. ACQUISITION

Do LOTS of landing pagetests & A/B tests - just make lots of dumb guesses & iterate QUICK

Website.com

ProductFeatures

Homepage / Landing Page

Domains

EmailDirect, Tel,

TVApps & Widgets

AffiliatesBlogs

SocialNetworks

BizDev

PRSEMCampaigns,

ContestsSEO

Customer Lifecycle / Conversion Behavior

Networks

1. ACQUISITION

Automated emails are simple & easyretention feature• lifecycle emails @ +3, +7, +30d

• status emails weekly/monthly

• event-based emails as they occur

Website.com

Blogs, Content

System Events & Time-based Features

ProductFeatures

Emails & Alerts

Homepage / Landing Page

Domains

EmailDirect, Tel,

TVApps & Widgets

AffiliatesBlogs

Social BizDev

PRSEMCampaigns,

ContestsSEO

Customer Lifecycle / Conversion Behavior

Networks

1. ACQUISITION

Only encourage usersrefer *after* they have

to

“happy” user experience;avg score >= 8 out of 10

Website.com

Blogs, Content

System Events & Time-based Features

ProductFeatures

Emails &widgets

Emails & Alerts

Homepage / Landing Page

Campaigns, Contests

Domains

EmailDirect, Tel,

TVApps & Widgets

AffiliatesBlogs

Social BizDev

PRSEMCampaigns,

ContestsSEO

Customer Lifecycle / Conversion Behavior

Networks

1. ACQUISITION

widgets

Subscriptions, etc

This is the part *you*have to figure out…I don’t know jack about your business

Website.com

Ads, Lead Gen,Blogs, Content

Biz Dev

System Events & Time-based Features

Emails &ProductFeatures

Emails & Alerts

Homepage / Landing Page

Campaigns, Contests

Domains

EmailDirect, Tel,

TVApps & Widgets

AffiliatesBlogs

Social BizDev

PRSEMCampaigns,

ContestsSEO

Types of Metrics & MeasurementQualitative: Usability Testing / Session Monitoring•

– Watch what users do, figure out problems & solutions from small # of users

of users

are most effective

• Competitive: Monitoring & Tracking Competitors– Track competitor activity & compare against yours; if possible compare channels,

keyword traffic, demographic targeting, user satisfaction, etc.

(see slide notes for links to tools & vendors)

• Quantitative: Traffic Analysis / User Engagement– Report what users do, track usage & conversion %'s for all or empirical sample #

• Comparative: A/B, Multivariate Testing– Compare what users do in one scenario vs another, see which copy/graphics/UI

Quantitative & ComparativeMeasurement

Conversion Criteria:•–––

best-performing (%) channels / campaigns / copylargest-volume (#) channels / campaigns / copy lowest-cost ($) channels / campaigns / copy

Quantitative & ComparativeMeasurement

Conversion Criteria:•–––

best-performing (%) channels / campaigns / copylargest-volume (#) channels / campaigns / copy lowest-cost ($) channels / campaigns / copy

• Measurement Components:–––––

Audience Segment (young women, regional metro, older singles)Channel Source (social network, SEM, organic, PR, etc) Campaign Theme / Brand Promise (“find a job”, “learn to cook”) Landing Page & CTACopy & Graphics

Activation Happy 1st Visit 30% $.25(views X pages, stays Y sec, Z clicks)

Activation Email/Blog/RSS/Widget Signup 5% $1(anything that could lead to repeat visit)

Activation Acct Signup 2% $3(includes profile data)

Retention Email Open / RSS view -> Clickthru 3% $2

Retention Repeat Visitor 2% $5(3+ visits in first 30 days)

Referral Refer 1+ users who visit site 2% $3

Referral Refer 1+ users who activate 1% $10

Revenue User generates minimum revenue 2% $5

Revenue User generates break-even revenue 1% $25

Example Conversion Metrics(note: *not* actuals; your mileage may vary…)

Category User Status Conv % Est. ValueAcquisition Visit Site

(or landing page, or external widget)100% $.01

Acquisition Doesn't Abandon(views 2+ pages, stays 10+ sec, 2+ clicks)

70% $.05

Retention Email Open / RSS view -> Clickthru 3% $2

Retention Repeat Visitor 2% $5(3+ visits in first 30 days)

Referral Refer 1+ users who visit site 2% $3

Referral Refer 1+ users who activate 1% $10

Revenue User generates minimum revenue 2% $5

Revenue User generates break-even revenue 1% $25

Example Conversion Metrics(note: *not* actuals; your mileage may vary…)

Category User Status Conv % Est. ValueAcquisition Visit Site

(or landing page, or external widget)100% $.01

Acquisition Doesn't Abandon(views 2+ pages, stays 10+ sec, 2+ clicks)

70% $.05

Activation Happy 1st Visit(views X pages, stays Y sec, Z clicks)

30% $.25

Activation Email/Blog/RSS/Widget Signup(anything that could lead to repeat visit)

5% $1

Activation Acct Signup(includes profile data)

2% $3

Referral Refer 1+ users who visit site 2% $3

Referral Refer 1+ users who activate 1% $10

Revenue User generates minimum revenue 2% $5

Revenue User generates break-even revenue 1% $25

Example Conversion Metrics(note: *not* actuals; your mileage may vary…)

Category User Status Conv % Est. ValueAcquisition Visit Site

(or landing page, or external widget)100% $.01

Acquisition Doesn't Abandon(views 2+ pages, stays 10+ sec, 2+ clicks)

70% $.05

Activation Happy 1st Visit(views X pages, stays Y sec, Z clicks)

30% $.25

Activation Email/Blog/RSS/Widget Signup(anything that could lead to repeat visit)

5% $1

Activation Acct Signup(includes profile data)

2% $3

Retention Email Open / RSS view -> Clickthru 3% $2Retention Repeat Visitor

(3+ visits in first 30 days)2% $5

Revenue User generates minimum revenue 2% $5

Revenue User generates break-even revenue 1% $25

Example Conversion Metrics(note: *not* actuals; your mileage may vary…)

Category User Status Conv % Est. ValueAcquisition Visit Site

(or landing page, or external widget)100% $.01

Acquisition Doesn't Abandon(views 2+ pages, stays 10+ sec, 2+ clicks)

70% $.05

Activation Happy 1st Visit(views X pages, stays Y sec, Z clicks)

30% $.25

Activation Email/Blog/RSS/Widget Signup(anything that could lead to repeat visit)

5% $1

Activation Acct Signup(includes profile data)

2% $3

Retention Email Open / RSS view -> Clickthru 3% $2Retention Repeat Visitor

(3+ visits in first 30 days)2% $5

Referral Refer 1+ users who visit site 2% $3Referral Refer 1+ users who activate 1% $10

Example Conversion Metrics(note: *not* actuals; your mileage may vary…)

Category User Status Conv % Est. ValueAcquisition Visit Site

(or landing page, or external widget)100% $.01

Acquisition Doesn't Abandon(views 2+ pages, stays 10+ sec, 2+ clicks)

70% $.05

Activation Happy 1st Visit(views X pages, stays Y sec, Z clicks)

30% $.25

Activation Email/Blog/RSS/Widget Signup(anything that could lead to repeat visit)

5% $1

Activation Acct Signup(includes profile data)

2% $3

Retention Email Open / RSS view -> Clickthru 3% $2Retention Repeat Visitor

(3+ visits in first 30 days)2% $5

Referral Refer 1+ users who visit site 2% $3Referral Refer 1+ users who activate 1% $10Revenue User generates minimum revenue 2% $5Revenue User generates break-even revenue 1% $25

MarketingQ: what channels / who do you market to?

••

Design & Test Multiple Mktg ChannelsSelect & Focus on

VolumeConversion

Channels with:–––

HighHighLow Cost

• Measure *deeper*not just to website

down the conversion/ landing page

funnel,

• Segment & Select channels & customers byconversion @ deepest possible level (ideally $$$)

ProductQ: how do you choose what to build?

• Choose features for conversion improvement––

80% on existing feature optimization20% on new feature development

•••

Just guess, then A/B test… A LOTMeasure conversion improvementRinse & Repeat

Founder/CEOQ: What metrics do you choose to watch?

• Hypothesize Customer Lifecycle & Refine––

Choose 5-10 conversion stepsLess, not More is better

•••

BUT measure & iterateFocus on conversion improvementDelegate each Metric to someone to OWN

65

Konklusioner

• 2 overordnede typer metrics• Eksterne (Hvordan kommer de til dig)• Interne (Brugen af dit produkt)

• Hvilke metrics der er vigtigst kommer an på din type startup og dit stadie

66

Lean Analytics framework

• Empathy (MVP)• Stickiness (Engagement + retention)• Virality/WOM• Reveneu (CAC/CLTV)• Scale

67

Metrics for investorer

• Hvilke metrics er relevante for investorer?• SaaS – SEED• Social network – SEED• Marketplace - SEED

68

69

70

Lad os tage nogle eksempler fra jer!