It Worked for Ustream
-
Upload
balazs-kereskenyi -
Category
Data & Analytics
-
view
209 -
download
0
description
Transcript of It Worked for Ustream
![Page 1: It Worked for Ustream](https://reader034.fdocuments.net/reader034/viewer/2022052618/554bcb23b4c905706a8b46ec/html5/thumbnails/1.jpg)
!
It Worked for UstreamProduct Development @ Ustream
![Page 2: It Worked for Ustream](https://reader034.fdocuments.net/reader034/viewer/2022052618/554bcb23b4c905706a8b46ec/html5/thumbnails/2.jpg)
Balázs Kereskényi product manager
API, discovery
Gergely Schmidt product manager,premium products
![Page 3: It Worked for Ustream](https://reader034.fdocuments.net/reader034/viewer/2022052618/554bcb23b4c905706a8b46ec/html5/thumbnails/3.jpg)
Making decisions by assumptions:“Let’s do this, because it worked for Spotify!” - you will never know why it worked for them.
![Page 4: It Worked for Ustream](https://reader034.fdocuments.net/reader034/viewer/2022052618/554bcb23b4c905706a8b46ec/html5/thumbnails/4.jpg)
Making decisions by experiments:“Let’s try this out, measure effectiveness,and see if it works!
![Page 5: It Worked for Ustream](https://reader034.fdocuments.net/reader034/viewer/2022052618/554bcb23b4c905706a8b46ec/html5/thumbnails/5.jpg)
![Page 6: It Worked for Ustream](https://reader034.fdocuments.net/reader034/viewer/2022052618/554bcb23b4c905706a8b46ec/html5/thumbnails/6.jpg)
What to test?
Icon & text: +12% conversion.
Myth #13: Icons enhance usability
![Page 7: It Worked for Ustream](https://reader034.fdocuments.net/reader034/viewer/2022052618/554bcb23b4c905706a8b46ec/html5/thumbnails/7.jpg)
What to test?
• CTA microcopies: “Get started”, “Try out for free”, “Sign up”, “Learn more” etc.
• Layout: arrangement of form fields, mixing elements on the whole page.
• “Meta parameters” of a product: trial period length, price.
• ...
![Page 8: It Worked for Ustream](https://reader034.fdocuments.net/reader034/viewer/2022052618/554bcb23b4c905706a8b46ec/html5/thumbnails/8.jpg)
How to measure effectiveness?
Conversion (basically any kind of your metrics):
• Increase in purchases,
• Engagement,
• Number of clicks,
• Number of signups,
• Form completions,
• ...
![Page 9: It Worked for Ustream](https://reader034.fdocuments.net/reader034/viewer/2022052618/554bcb23b4c905706a8b46ec/html5/thumbnails/9.jpg)
Always question assumptions
but be prepared that only 1 out of 8 A/B testswill drive you to significant changes.*
*based on the tests done by appsumo.com
![Page 10: It Worked for Ustream](https://reader034.fdocuments.net/reader034/viewer/2022052618/554bcb23b4c905706a8b46ec/html5/thumbnails/10.jpg)
FAMOUS!A/B!
TESTS
![Page 11: It Worked for Ustream](https://reader034.fdocuments.net/reader034/viewer/2022052618/554bcb23b4c905706a8b46ec/html5/thumbnails/11.jpg)
Marissa Mayer:41 shades
of blue
![Page 12: It Worked for Ustream](https://reader034.fdocuments.net/reader034/viewer/2022052618/554bcb23b4c905706a8b46ec/html5/thumbnails/12.jpg)
How Obama raised $60M by running a simple experiment
Optimizely, Dan Siroker
![Page 13: It Worked for Ustream](https://reader034.fdocuments.net/reader034/viewer/2022052618/554bcb23b4c905706a8b46ec/html5/thumbnails/13.jpg)
Who to blame for the slow loading up? Facebook or iOS?
![Page 14: It Worked for Ustream](https://reader034.fdocuments.net/reader034/viewer/2022052618/554bcb23b4c905706a8b46ec/html5/thumbnails/14.jpg)
How to set up a test?• Targeting (when the test should start, visitor characteristics &
segmentation, technical circumstances etc.),
• Experiment type (A/B, multivariate, multi page),
• Traffic allocation,
• Goals,
• Integration with other kind of analytics.
![Page 15: It Worked for Ustream](https://reader034.fdocuments.net/reader034/viewer/2022052618/554bcb23b4c905706a8b46ec/html5/thumbnails/15.jpg)
Technical background of a test• Frontend
• Quick and dirty methods
• Always having a usable fallback is anything goes wrong
• Backend for bigger and more complicated tests
• Measurement
• GA and Optimizely events are our best friends
![Page 16: It Worked for Ustream](https://reader034.fdocuments.net/reader034/viewer/2022052618/554bcb23b4c905706a8b46ec/html5/thumbnails/16.jpg)
Understanding your resultsChance to beat baseline.
…or you can simply use isvalid.org anytime.
![Page 17: It Worked for Ustream](https://reader034.fdocuments.net/reader034/viewer/2022052618/554bcb23b4c905706a8b46ec/html5/thumbnails/17.jpg)
USTREAM!CASE!
STUDIES
![Page 18: It Worked for Ustream](https://reader034.fdocuments.net/reader034/viewer/2022052618/554bcb23b4c905706a8b46ec/html5/thumbnails/18.jpg)
![Page 19: It Worked for Ustream](https://reader034.fdocuments.net/reader034/viewer/2022052618/554bcb23b4c905706a8b46ec/html5/thumbnails/19.jpg)
Changing directions: SaaS vs. content.
Started with concluding tiny preliminary experiments: less content, pro CTAs. Ended up with split testing the whole page.
editorializedpro purchases++
![Page 20: It Worked for Ustream](https://reader034.fdocuments.net/reader034/viewer/2022052618/554bcb23b4c905706a8b46ec/html5/thumbnails/20.jpg)
Ustream homepage test results
• Tested on approx. 700k unique visitors.
• No important general metrics harmed (pages / visit time on site & time on homepage).
• 1% increase in purchases starting from the home page.
• Showing 1/3 of content resulted only in halving the clicks.
![Page 21: It Worked for Ustream](https://reader034.fdocuments.net/reader034/viewer/2022052618/554bcb23b4c905706a8b46ec/html5/thumbnails/21.jpg)
A DIFFERENT APPROACH FOR VALIDATION:!GHETTO TESTING
http://bit.ly/ghettozynga
![Page 22: It Worked for Ustream](https://reader034.fdocuments.net/reader034/viewer/2022052618/554bcb23b4c905706a8b46ec/html5/thumbnails/22.jpg)
Ghetto test: Pay with PayPal
• Built by frontend engineers, only triggered from Optimizely
• Decided not to build it because of low conversion rate (~2%)
![Page 23: It Worked for Ustream](https://reader034.fdocuments.net/reader034/viewer/2022052618/554bcb23b4c905706a8b46ec/html5/thumbnails/23.jpg)
Pro Purchase Flow: 1 or 2 steps?
• Built by frontend engineers, only triggered from Optimizely
• Currently live, analysis is on the way
![Page 24: It Worked for Ustream](https://reader034.fdocuments.net/reader034/viewer/2022052618/554bcb23b4c905706a8b46ec/html5/thumbnails/24.jpg)
This is not a test
![Page 25: It Worked for Ustream](https://reader034.fdocuments.net/reader034/viewer/2022052618/554bcb23b4c905706a8b46ec/html5/thumbnails/25.jpg)
!
https://ustream.tv/platform/unlimited!
Thanks!
@bkereskenyi@sgergely