A/B Testing - In data we trust
-
Upload
pedro-marques -
Category
Data & Analytics
-
view
400 -
download
1
Transcript of A/B Testing - In data we trust
![Page 1: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/1.jpg)
A/B TESTINGIn data we trust!
Pedro Marques CI&T Digital Day
![Page 2: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/2.jpg)
PEDRO MARQUESUX Designer @ booking.com @pmarquees
![Page 3: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/3.jpg)
700kProperties
135Offices
207Countries
42Languages
74Currencies
8.5kEmployees
8kServers
perl + html templatesCore
3º MAIORe-commerce do mundo
![Page 4: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/4.jpg)
![Page 5: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/5.jpg)
A/B TestingWTF?
![Page 6: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/6.jpg)
Emojiana
![Page 7: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/7.jpg)
Emojiana Hippo
![Page 8: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/8.jpg)
50%
50%
11%
2%
![Page 9: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/9.jpg)
Born at GoogleEarly 00’s
![Page 10: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/10.jpg)
PharmaceuticalTo test new drugs
![Page 11: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/11.jpg)
But why A/B Test?
![Page 12: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/12.jpg)
No Regression
![Page 13: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/13.jpg)
But bear in mind:
![Page 14: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/14.jpg)
A/B Test will prove you wrong
![Page 15: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/15.jpg)
Data never lies
![Page 16: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/16.jpg)
Define your KPI’s
![Page 17: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/17.jpg)
Correlation does NOT imply causation
![Page 18: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/18.jpg)
Conver$ion
![Page 19: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/19.jpg)
User performs a given action
![Page 20: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/20.jpg)
Define your KPI's
![Page 21: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/21.jpg)
What should I test?
![Page 22: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/22.jpg)
![Page 23: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/23.jpg)
Test all your hypothesis
![Page 24: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/24.jpg)
Assumption is the mother of all fuck ups
![Page 25: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/25.jpg)
Framing Effect
![Page 26: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/26.jpg)
A/B Testing Kills creativity
![Page 27: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/27.jpg)
Junior designers X Sr. Designes
https://medium.com/the-year-of-the-looking-glass/junior-designers-vs-senior-designers-fbe483d3b51e
![Page 28: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/28.jpg)
![Page 29: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/29.jpg)
![Page 30: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/30.jpg)
Would like fries with that?http://blog.booking.com/hamburger-menu.html
![Page 31: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/31.jpg)
![Page 32: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/32.jpg)
No significant impact.At all.
![Page 33: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/33.jpg)
NICE!But how do I do that?
![Page 34: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/34.jpg)
Coin flip
![Page 36: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/36.jpg)
Now what?Data analysis is the key
![Page 37: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/37.jpg)
Some advicesThese are mine, test and see if it works for you =)
![Page 38: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/38.jpg)
Shit X ShitNope, a big nope.
![Page 39: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/39.jpg)
Ask questionsEven before starting the experiment
![Page 40: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/40.jpg)
Dont trust the gurusThis is science, not religion
![Page 41: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/41.jpg)
Flip the coin properlyCalculate the amount of time and sample that you need
https://vwo.com/ab-split-test-duration/
![Page 42: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/42.jpg)
A/B Testing is a big subject
![Page 43: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/43.jpg)
![Page 45: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/45.jpg)
![Page 46: A/B Testing - In data we trust](https://reader034.fdocuments.net/reader034/viewer/2022042907/5870a3a01a28abcb078b540f/html5/thumbnails/46.jpg)
Thank you folks! <[email protected]@pmarquees