[GrowthHacker Conference '16] Annabell Satterfield Senior PM, Growth at BitTorrent: Engaging...

February 18, 2016

Annabell Satterfield

Annabell Satterfield @als355


Core Value

Product

Growth

Product = delivery device for the core

value

This includes acquisition funnels,

engagement mechanics*Consumer products


Core Value

Product

Growth

Growth-driven product culture

*Andy Johns

Foundational strategy set with data

Beyond features

Early deposits to “compounding growth”

*

Pic of client team


Agenda

Processing data > insight with BucketingScreening + Tradeoff decision modelingQuantify utilityA/B test to maximize profit

In order to:

Prioritize next stepsInternationalize Prioritize (paid) featuresChoose prices

Getting from data > insights > action


Warning: Outcomes here are real. Most data here is not.


Master framework to get from Data > Insights > Action


Work backwards. 1. Define the question

Question



2. Define outcome

Outcome



2. Define outcome3. Is what I need out there? Go get it.

Data



2. Define outcome3. Is what I need out there? Go get it.

4. Process the data

Process


Let’s take this to the streets


Case 1- Next stepsData: Survey Process: Bucketing


Question

“What features will you build to drive paid app revenue?”


Question

“What features will you build to drive paid app revenue?”Among these, what will drive the most paid app revenue:

price testing, adding features, building awareness?Annabell Satterfield @als355

Outcome


Awareness

Price

Features #1

#3

#2

Ranked opportunities

Outcome

Audience: Core users, top geosfor paid productPay for stuff


Awareness

Price

Features #1

#3

#2


Data

Does this data exist? No.


Data

Does this data exist? No. Let’s go make it!


Data

Does this data exist? No. Let’s go get it!


Data


Process


Price

Features

Awareness #1

#2

#3


+92% paid app revenue lift


Let’s take it to the next level.


Insert Page # Here

Case 2- Expanding internationally: Data: External Databases, Product Data Process: Screening + Tradeoff


Question

Which geos and languages should we prioritize?


Outcome

#1 Country 1- Language #2 Country 2- Language#3 Country 3- Language

--#X Country X- Language

Highest-potential… based on our goals.Annabell Satterfield @als355

Data

Does this data exist? Yes! Let’s go get it!


Data

Address these elements:Market sizeLikelihood to purchase/monetize Annabell Satterfield @als355

Data

Choose your data based on those elements:Market size: Installed Android user base (all apps), Install base for the BitTorrent Desktop clientsLikelihood to monetize: Relative value of mobile adverts Annabell Satterfield @als355

Process

Choose a weighting for each:Market size: Installed Android user base [3], install base for the BitTorrent desktop clients [4]Likelihood to monetize: Value of adverts by geo [2] Annabell Satterfield @als355

Step 1- Screen

Min Criteria 1

Min Criteria 3Min Criteria 2Meets

all 3


Step 1- Screen

Mobile ad Value > $X eCPM

Android User Base >ZMM

BitTorrent Desktop Users >YMM

*Fake listAnnabell Satterfield @als355

Step 1- Screen

Android User Base >ZMM

BitTorrent Desktop Users >YMM

*Fake list

Mobile ad Value > $X eCPM


Step 2- Prioritize

(Decile x

(1

..

(10

Android User BaseWeight)

3)

..

3)

Top

90%

..

10%

=

3

..

30

=

=

x

x


Step 2- Prioritize

(Decile x

1

..

10


3

..

3

+

Ad valueTop

90%

..

10%

+

Desktop Users

=


Step 2- Prioritize

(Decile x

1

..

10


3

..

3

+

Ad valueTop

90%

..

10%

+

Desktop Users

=

Value

68

..

90


And… do your due diligence

Think about re-adding:BrazilRussiaIndiaChina...others?

Ordered by value…*#1 United States- English: 90#2 Canada- English: 89#3 Spain- Spain Spanish: 70 #4 S. Korea- Korean: 68…

*Fake listAnnabell Satterfield @als355

Spain +93% w/wBrazil +43%Italy +32%Russia +23%

Installs in high-potential geos targeted for growth with translation:


Case 3- [Paid] Feature PrioritizationData: Survey Process: Quantifying Utility (+ ROI analysis )


Question

Will more users be willing to buy Pro if we released more features? Which ones?


Outcome

Feature #1- Utility value of X Feature #2- Utility value of Y

--Feature #N- Utility value of Z


Outcome

Feature #1- Utility value of X Feature #2- Utility value of Y

--Feature #N- Utility value of Z

Impact of new feature on likelihood to buy

Audience: Core users, top geosfor paid productPay for upgrades


Data



Remember this survey?

#2 Opportunity


Data


Process

Calculate weighted value of each feature…

http://bit.ly/GHC2016


Calculated weight of each Pro feature:

Feature X: 5.78 Ranked #1

Feature Y: 5.59 Ranked #2

Feature Z: 5.20 Ranked #3

10+ features considered and prioritized with data… priceless


Outcome…

Battery-Saver: +47% Daily paid Revenue Auto-shutdown: +20% Daily paid Revenue

uTorrent Android- top 5 grossing (for paid upgrades] in its category in 71 geos including US, UK, CA, AU*

Happy paying users

* App Annie, April 2015 (before moved to desktop team)Annabell Satterfield @als355

Sum

Case 4- PricingData: Experiment Process: A/B testing


Question

What is the profit-maximizing price point for the paid app?


Outcome

In the United States, $39.95- Profit per user$29.95- Profit per user

--$4.95- Profit per user


Data



Process

Experimentation..

… essentially, pretotyping*

*Via Alberto Savoia, pretotyping.org


BitTorrent, Inc. |

Optimizely…

Only differenceis price


BitTorrent, Inc. |

Track toactual sale


BitTorrent, Inc. |

For each:

[(units x price)-(units x COGS)]

impressions


Outcome…

Hit desktop team’s 2013 paid revenue goal

No change to product


Data>insights>action framework = a great habit to build.


These are not substitutes for experiments or MVPs.They inform experiments.


Don’t let anything keep you from being data-driven where it counts.


Qualitative data can fill in the gaps where other data ends.



In order to:


These tools are multifunctional



You can also:Isolate problems with feedback Prioritize B2B features w/survey Develop keep/kill feature list Pretotype unbundled paid productsMore…



Product + Growth


These are not substitutes for experiments or MVPs.They inform experiments.


Don’t let anything keep you from being data driven where it counts.


Qualitative data can fill in the gaps where other data ends.



In order to:





You can also:Isolate problems with feedback Prioritize B2B features w/survey Develop keep/kill feature list Pretotype unbundled paid productsMore…



Product + Growth



Q & A

Slides:http://bit.ly/GHC2016



Bingo

Talking to users Open-ended surveysOPR (OP Research)

Product dataDatabases

Product dataSurveys*

ExperimentsOPRTalking to usersDatabases

ExperimentsPretotypes

Surveys*Databases

*term used loosely

Choose data based on what you understand about your problem…

Paid Feature Prioritization- how to calculate weighted value…


Raw data


Sum


Divide


Multiply


Sum


https://growthhackers.com/videos?utm_source=slideshare&utm_medium=SlidesEndCTA&utm_campaign=AnnebellSlide

[GrowthHacker Conference '16] Annabell Satterfield Senior PM, Growth at BitTorrent: Engaging...

Marketing

Transcript of [GrowthHacker Conference '16] Annabell Satterfield Senior PM, Growth at BitTorrent: Engaging...