Optimize delivery and quality with the right combination of tools

MyHeritage

6,600,000,000Historical Records

2,500,000,000Profiles

85,000,000Users

Who are we?

• Personal goals - We should all want to produce and release a high quality app

• Poor quality impacts your bottom line• User satisfaction leads to high retention rates• Brand reputation • Pride

Why Quality Matters?

• Keeping the foot on the gas paddle when it comes to releasing new features and versions while maintaining high quality.

• Quality isn’t determined just by crashes, need a way to determine that your users get the experience you want them to.

Challenges In a Fast Pace Environment

• High device fragmentation that leads to greater complexity

Challenges In a Fast Pace Environment

• The need to support a high number of OS versions

Challenges In a Fast Pace Environment

• Keeping track of your app versions and their contents over long periods of time.

• Preserving your API’s integrity over time.

Challenges In a Fast Pace Environment

• Visibility• Controlled releases• Regression Prevention

Keys to Success

Visibility

• Design and implement your analytics strategy in a way that provides you the most visibility and information.

• Set a unique user ID so that you can monitor the same user over multiple analytic platforms

Visibility

Visibility

• Monitoring crash reports• Monitoring version adoption

Controlled App Releases

• Try to stick to a pre-defined version release schedule• Periodic releases - New version every 2-3 weeks• Feature flags • Staged Rollout

Regression Prevention

• Beta/Alpha testing• Unit tests – Build machine • End to end Automations• API automations

Regression prevention – Nightly Builds

Nightly Unit Tests End to End UI Test

Nightly Coverage

How it all fits togetherVersion Release

New Feature Development

Testing

Alpha/Beta Phase

How it all fits together

• Writing unit tests • Writing E2E/Integration/Automation tests • Implement feature flags • Fixing the top crashes that were collected from previous version

New Feature Development Phase

How it all fits together

• Running Unit Test from nightly build machine• Running E2E/Integration from nightly build machine• Manual QA• Monitor test usage to see if the user experience is what was intended• Analyze user feedback

Testing and Beta analysis

How it all fits together

• Set feature flags to their production values • Final QA sanity check • Staged rollout over about 2 days – Bug fixes if needed

Version Release Phase

How it all fits together

• Crash report monitoring – Collect top crashes• Version adoption monitoring • Key app event monitoring (e.g. in-app purchases) • Monitoring new feature analytics for quality and usage insights • Monitoring Google play reviews and support tickets

Post Version Release Analysis

• Visibility• Controlled releases• Regression Prevention

Summary

Thank you

Guy Tsypeguy.tsype@myheritage.com