A/B Testing with React · o Introduction o Our project • About the project • The stack •...
Transcript of A/B Testing with React · o Introduction o Our project • About the project • The stack •...
![Page 1: A/B Testing with React · o Introduction o Our project • About the project • The stack • Hypothesis- and goal-driven development/Workflow o Story-time o What is A/B testing?](https://reader033.fdocuments.net/reader033/viewer/2022042302/5eccd5f41cdec34d366aa545/html5/thumbnails/1.jpg)
A/B Testing with React
Eivind Arvesen and Lars Dølvik
![Page 2: A/B Testing with React · o Introduction o Our project • About the project • The stack • Hypothesis- and goal-driven development/Workflow o Story-time o What is A/B testing?](https://reader033.fdocuments.net/reader033/viewer/2022042302/5eccd5f41cdec34d366aa545/html5/thumbnails/2.jpg)
o Introductiono Our project
• About the project• The stack• Hypothesis- and goal-driven development/Workflow
o Story-timeo What is A/B testing?
• react-ab-test (library)▪ Isomorphism
o Evaluation• Measurements (GA: scrolling and events…)• Interpreting the numbers
o Practical demonstration
Outline
![Page 3: A/B Testing with React · o Introduction o Our project • About the project • The stack • Hypothesis- and goal-driven development/Workflow o Story-time o What is A/B testing?](https://reader033.fdocuments.net/reader033/viewer/2022042302/5eccd5f41cdec34d366aa545/html5/thumbnails/3.jpg)
o A recipe focused application
o Runs on Google Cloud Platformo Isomorphico Built with React and Relay in fronto GraphQL as data query languageo Uses ElasticSearch to hold our contento Enonic CMS to publish content
o Hypothesis- and goal-driven development/Workflow
Our project
![Page 4: A/B Testing with React · o Introduction o Our project • About the project • The stack • Hypothesis- and goal-driven development/Workflow o Story-time o What is A/B testing?](https://reader033.fdocuments.net/reader033/viewer/2022042302/5eccd5f41cdec34d366aa545/html5/thumbnails/4.jpg)
o Tactical goalo Action What?o Hypothesis Why?o Measurement How?o Estimated effect
Workflow
![Page 5: A/B Testing with React · o Introduction o Our project • About the project • The stack • Hypothesis- and goal-driven development/Workflow o Story-time o What is A/B testing?](https://reader033.fdocuments.net/reader033/viewer/2022042302/5eccd5f41cdec34d366aa545/html5/thumbnails/5.jpg)
Story-time
![Page 6: A/B Testing with React · o Introduction o Our project • About the project • The stack • Hypothesis- and goal-driven development/Workflow o Story-time o What is A/B testing?](https://reader033.fdocuments.net/reader033/viewer/2022042302/5eccd5f41cdec34d366aa545/html5/thumbnails/6.jpg)
Story-time
![Page 7: A/B Testing with React · o Introduction o Our project • About the project • The stack • Hypothesis- and goal-driven development/Workflow o Story-time o What is A/B testing?](https://reader033.fdocuments.net/reader033/viewer/2022042302/5eccd5f41cdec34d366aa545/html5/thumbnails/7.jpg)
A/B-TESTING!
![Page 8: A/B Testing with React · o Introduction o Our project • About the project • The stack • Hypothesis- and goal-driven development/Workflow o Story-time o What is A/B testing?](https://reader033.fdocuments.net/reader033/viewer/2022042302/5eccd5f41cdec34d366aa545/html5/thumbnails/8.jpg)
A form of statistical hypothesis testing.
What is A/B testing?
Illustration by Maxime Lorant
![Page 9: A/B Testing with React · o Introduction o Our project • About the project • The stack • Hypothesis- and goal-driven development/Workflow o Story-time o What is A/B testing?](https://reader033.fdocuments.net/reader033/viewer/2022042302/5eccd5f41cdec34d366aa545/html5/thumbnails/9.jpg)
Usual steps:
● Have an idea or supposition about our solution● Verify or falsify the hypothesis via experiment(s)
○ Collect various measurements
What is A/B testing?
![Page 10: A/B Testing with React · o Introduction o Our project • About the project • The stack • Hypothesis- and goal-driven development/Workflow o Story-time o What is A/B testing?](https://reader033.fdocuments.net/reader033/viewer/2022042302/5eccd5f41cdec34d366aa545/html5/thumbnails/10.jpg)
Why?
It’s normal to be wrong.
Measuring how our development impacts our users makes us able to make decisions based on facts!
What is A/B-testing?
![Page 11: A/B Testing with React · o Introduction o Our project • About the project • The stack • Hypothesis- and goal-driven development/Workflow o Story-time o What is A/B testing?](https://reader033.fdocuments.net/reader033/viewer/2022042302/5eccd5f41cdec34d366aa545/html5/thumbnails/11.jpg)
React-AB-Test (on npm)
https://github.com/pushtell/react-ab-test
What is A/B-testing?
![Page 12: A/B Testing with React · o Introduction o Our project • About the project • The stack • Hypothesis- and goal-driven development/Workflow o Story-time o What is A/B testing?](https://reader033.fdocuments.net/reader033/viewer/2022042302/5eccd5f41cdec34d366aa545/html5/thumbnails/12.jpg)
Legg gjerne inn et portrett av personen du siterer på dette lysbildet!
”The only true wisdom is in knowing you know nothing.Socrates - allegedly...
● Be skeptical● Don’t measure too much● Have a good basis for comparison
![Page 13: A/B Testing with React · o Introduction o Our project • About the project • The stack • Hypothesis- and goal-driven development/Workflow o Story-time o What is A/B testing?](https://reader033.fdocuments.net/reader033/viewer/2022042302/5eccd5f41cdec34d366aa545/html5/thumbnails/13.jpg)
o Google Analytics APIo Scroll and time trackingo Event tracking
o Tracking the correct evento Follow a standardo Write documentationo Less is more(ish)o Evaluating the numbers
Evaluation
![Page 14: A/B Testing with React · o Introduction o Our project • About the project • The stack • Hypothesis- and goal-driven development/Workflow o Story-time o What is A/B testing?](https://reader033.fdocuments.net/reader033/viewer/2022042302/5eccd5f41cdec34d366aa545/html5/thumbnails/14.jpg)
o A good grounds for comparisono Avoid noise in your datao Supplement qualitatively...
"Lies, damned lies, and statistics"
Evaluation
![Page 15: A/B Testing with React · o Introduction o Our project • About the project • The stack • Hypothesis- and goal-driven development/Workflow o Story-time o What is A/B testing?](https://reader033.fdocuments.net/reader033/viewer/2022042302/5eccd5f41cdec34d366aa545/html5/thumbnails/15.jpg)
Evaluation
Let’s say one of the media elements is an autoplaying video. Clicks lead to another recipe.
What is the user’s real intent?
![Page 16: A/B Testing with React · o Introduction o Our project • About the project • The stack • Hypothesis- and goal-driven development/Workflow o Story-time o What is A/B testing?](https://reader033.fdocuments.net/reader033/viewer/2022042302/5eccd5f41cdec34d366aa545/html5/thumbnails/16.jpg)
Evaluation
![Page 17: A/B Testing with React · o Introduction o Our project • About the project • The stack • Hypothesis- and goal-driven development/Workflow o Story-time o What is A/B testing?](https://reader033.fdocuments.net/reader033/viewer/2022042302/5eccd5f41cdec34d366aa545/html5/thumbnails/17.jpg)
Evaluation
![Page 18: A/B Testing with React · o Introduction o Our project • About the project • The stack • Hypothesis- and goal-driven development/Workflow o Story-time o What is A/B testing?](https://reader033.fdocuments.net/reader033/viewer/2022042302/5eccd5f41cdec34d366aa545/html5/thumbnails/18.jpg)
Let’s see how we can perform A/B-testing in practice: look at code and run it in the browser.
https://github.com/eivind88/ab-testing-react-meetup/tree/master
Demo
![Page 19: A/B Testing with React · o Introduction o Our project • About the project • The stack • Hypothesis- and goal-driven development/Workflow o Story-time o What is A/B testing?](https://reader033.fdocuments.net/reader033/viewer/2022042302/5eccd5f41cdec34d366aa545/html5/thumbnails/19.jpg)
o It is not as complicated as it might seem• The thought process is more the half the job
o Keep it simple and structured• Follow a standard and write documentation
o Keep your experiments limitedo You don’t know it if you haven’t proven it
In summary