Mobile App User Experience Myths, Debunked

© 2016 Apteligent • www.apteligent.com

What Data Science Can Teach You About App Performance & User Experience • March 2017

Mobile App User Experience Myths, Debunked


Agenda

Myth #1: Crash Data is All I Need

Myth #2: Release Planning Happens Once

Myth #3: User Behavior & Business Insights are separate from App Performance

Finally: Where is this data coming from?

Andrew LevyCo-Founder & Chief Strategy [email protected]@andrewmlevy


Myth #1:

Crash Data is All I Need

Cras

hes +

Erro

rsNet

work P

roble

msSlo

w Perfo

rman

ceDev

ice Pr

oblem

sUI Is

sues

User C

ontex

t

Intractable Number of Permutations of Issues Impacting User Experience

Even if you don’t buy into this, what can data science teach us about the cause of crashes?


20% of Crashes are Correlated with a Network Issue

Network issues account for a large percentage of crashes


88% of Network Calls Were Successful Before a Crash

Mobile developers are typically careful to detect and handle a failed network call.

What seems counterintuitive is that the majority of crashes happen after successful network calls


10% of Successful Network Calls Before a Crash Returned No Data

89% of the time speeds were lower than expected72% of the time latency was lower than average

Since latency was lower than average, we would expect the speed to increase. Since the majority of calls were successful, the lower speed must be driven by a lower than expected bytes received.

bpl (speed) = (bytes out + bytes in) / latency


Myth #2:

Release Planning

Happens Once

• Where are we relative to our competitors?• Gather relevant KPIs & performance benchmarks based on geography and App Store

category

• What are currently the most popular hardware & software configurations?• Which platforms should you support and how will your app perform in those environments?• Plan for new device and operating system launches

• Understand global bandwidth constraints

• Choose service providers and SDKs for analytics, performance, and monetization

Take a Data-Driven Approach to Drive Your Experience Strategy


Example: Launch and Android Shopping App in India

Key Performance, Device & OS Metrics. Sample pulled from Apteligent Release Planner

• 60% of Android Shopping apps have a launch time of < 2.0 seconds

• The overall average launch time is 3.09 seconds. Android Shopping apps have an average crash rate of 3.38%, this is 35% lower than the global average of 5.22%

• 52% of apps in this category have less than < 1.5% crash rate

• The most popular operating system is Android 6.0 with 25% usage share.

• The top 3 devices in India make up 12% of the market.

• Of the top 20 devices, the worst performing by latency are: Samsung Galaxy Note 3 (675 ms), Intex Cloud Q11 4G (649 ms), and the OnePlus 2 (599 ms).


New Device Releases 2H 2016


Compare Yourself Against Relevant Competition & Prioritize Dev Time

89% of the time speeds were lower than expected


Myth #3:

User Behavior &

Business Insights are

Separate from App

Performance

Understand Why User Behavior Changes and the Relationship Between Performance and Business Metrics

Users drop at every step of the customer acquisition funnel


Poor Performance Drives a Huge Increase in Churn

• Crashes increase churn by as much as 6x

• Users are up to 8x less likely to return the next day after a crash

• Revenue critical App Store categories like Shopping and Finance are very susceptible

Rolling Retention

Nth Day Retention

“Hard Churn”

Day-1 Retention: “Did they come back the next day?”


Relationship Between App Loads and Crashes : Churn


Example mCommerce Scenario

It is much more expensive to acquire a new user than to retain an existing user, anywhere from 5x to 25x

• Customer acquisition cost is $50 x 600 means $30,000 just to win back another 600• Assume $300 LTV, you just lost $180,000 in future revenue• If you had a goal to gain 300 customers, and you’ve lost 600, you’ve just tripled

your customer acquisition cost to reach that goal

Quickly isolated the issue, spent X hours of an engineers time (and $Y dollars, much less than the numbers outlined above)

A crash just caused your app to lose 600 customers


About Apteligent

Data Generated from Tens of Thousands of Apps

Userflow Failure Network Services Error Crashes Slow apps

Apteligent Data Platform


About Apteligent

Data Generated from Tens of Thousands of Apps

Transaction Failure Network Services Error Crashes Slow apps

Apteligent Data Platform Optimize Business Result

Get Ecosystem Insight

Manage App Lifecycle

• Retention/Churn

• Detect Issues• Determine root

cause• Assess impact

• Prioritize• Manage releases• Monitor vital signs

• Benchmarks and trends

• Carrier QoS

• Device adoption• Bespoke analysis

• Engagement• Monetization• Data ingestion

• Correlation• Prediction• Recommendation


Feel free to contact me:[email protected]

Mobile App User Experience Myths, Debunked

Data & Analytics

Transcript of Mobile App User Experience Myths, Debunked