7 Steps to Pragmatic Mobile Testing · © 2017 SPLUNK INC. © 2017 SPLUNK INC. CONFIDENTIAL...
Transcript of 7 Steps to Pragmatic Mobile Testing · © 2017 SPLUNK INC. © 2017 SPLUNK INC. CONFIDENTIAL...
© 2017 SPLUNK INC.
© 2017 SPLUNK INC. CONFIDENTIAL INFORMATION. DO NOT DISTRIBUTE.
© 2017 SPLUNK INC.
7 Steps to
Pragmatic Mobile Testing
@TomChavez [email protected] Sr. Manager, Developer Marketing
Feb 14, 2020
© 2017 SPLUNK INC.
© 2017 SPLUNK INC. CONFIDENTIAL INFORMATION. DO NOT DISTRIBUTE.
7 Steps to Pragmatic Mobile Testing A path to faster, better testing
© 2017 SPLUNK INC.
© 2017 SPLUNK INC. CONFIDENTIAL INFORMATION. DO NOT DISTRIBUTE.
▶ Expanding Test Plans
Introduction Changes affecting the testing process
© 2017 SPLUNK INC.
© 2017 SPLUNK INC. CONFIDENTIAL INFORMATION. DO NOT DISTRIBUTE.
▶ Expanding Test Plans
▶ Shorter Product Cycles
Introduction Changes affecting the testing process
© 2017 SPLUNK INC.
© 2017 SPLUNK INC. CONFIDENTIAL INFORMATION. DO NOT DISTRIBUTE.
▶ Expanding Test Plans
▶ Shorter Product Cycles
▶ More Device Platforms
Introduction Changes affecting the testing process
© 2017 SPLUNK INC.
© 2017 SPLUNK INC. CONFIDENTIAL INFORMATION. DO NOT DISTRIBUTE.
▶ Expanding Test Plans
▶ Shorter Product Cycles
▶ More Device Platforms
▶ A Changing Focus
Introduction Changes affecting the testing process
© 2017 SPLUNK INC.
© 2017 SPLUNK INC. CONFIDENTIAL INFORMATION. DO NOT DISTRIBUTE.
▶ Expanding Test Plans
▶ Shorter Product Cycles
▶ More Device Platforms
▶ A Changing Focus
▶ An Emphasis on Analysis
Introduction Changes affecting the testing process
© 2017 SPLUNK INC.
© 2017 SPLUNK INC. CONFIDENTIAL INFORMATION. DO NOT DISTRIBUTE.
Time
Duration
Location
Intent
TIP: Work with your product owner, review market
research, conduct user focus groups, and verify/validate
with real user monitoring
Step 1 – Know your user Understand the most common user patterns
© 2017 SPLUNK INC.
© 2017 SPLUNK INC. CONFIDENTIAL INFORMATION. DO NOT DISTRIBUTE.
Step 2 – Know your app Understand the application flow
Developers intended execution paths
User common use paths
© 2017 SPLUNK INC.
© 2017 SPLUNK INC. CONFIDENTIAL INFORMATION. DO NOT DISTRIBUTE.
Step 2 – Know your app Understand the application flow
Developers intended execution paths
User common use paths
User revenue generating patterns
Current problem areas
TIP: Use real user data to create test cases
© 2017 SPLUNK INC.
© 2017 SPLUNK INC. CONFIDENTIAL INFORMATION. DO NOT DISTRIBUTE.
RUM = Real User Monitoring Monitoring and measure all (or at least some) users
© 2017 SPLUNK INC.
© 2017 SPLUNK INC. CONFIDENTIAL INFORMATION. DO NOT DISTRIBUTE.
Core functions and experiences
Core differentiators
Edge functions
Complicated
Repetitive
Measure manual effort
Know your person days per cycle, per area
TIP: Use your matrix to know which tests to automate
Step 3 – Know your test matrix All tests are not created equal
© 2017 SPLUNK INC.
© 2017 SPLUNK INC. CONFIDENTIAL INFORMATION. DO NOT DISTRIBUTE.
Rank active devices
Analyze device usage
Weight matrix to device
Know problem areas per device
Core devices to own
Edge devices to borrow/rent
TIP: Choose which devices you must own
for testing and which might be optional but available
based on real user monitoring
Step 4 – Know your devices Understand your device matrix
© 2017 SPLUNK INC.
© 2017 SPLUNK INC. CONFIDENTIAL INFORMATION. DO NOT DISTRIBUTE.
Using RUM to Build Device Matrix
© 2017 SPLUNK INC.
© 2017 SPLUNK INC. CONFIDENTIAL INFORMATION. DO NOT DISTRIBUTE.
Build a Device Lab
© 2017 SPLUNK INC.
© 2017 SPLUNK INC. CONFIDENTIAL INFORMATION. DO NOT DISTRIBUTE.
Or Use a Remote Device Cloud
© 2017 SPLUNK INC.
© 2017 SPLUNK INC. CONFIDENTIAL INFORMATION. DO NOT DISTRIBUTE.
Start ASAP
Simple
Time Consuming
Mundane
Rhythm
TIP: Goal is Continuous Integration execution
Step 5 – Know how to Automate Return person days to your test cycle
© 2017 SPLUNK INC.
© 2017 SPLUNK INC. CONFIDENTIAL INFORMATION. DO NOT DISTRIBUTE.
Device reaction to app functionality
Battery, CPU, Memory…
Common user path
Speed per screen or page
Separate back and front end performance
Performance to revenue correlation/transaction or task completion
TIP: Prioritize tests to highest revenue producing/completion paths
Step 6 – Know your mobile performance
© 2017 SPLUNK INC.
© 2017 SPLUNK INC. CONFIDENTIAL INFORMATION. DO NOT DISTRIBUTE.
Measuring Performance at Load Load Testing from the Cloud
© 2017 SPLUNK INC.
© 2017 SPLUNK INC. CONFIDENTIAL INFORMATION. DO NOT DISTRIBUTE.
Time to Market
Functional
Competitive
TIP: Prioritize which strategy you will apply with the time saved from
test automation
Step 7 – Know your edge Understand ahead of time what you will do when you can
© 2017 SPLUNK INC.
© 2017 SPLUNK INC. CONFIDENTIAL INFORMATION. DO NOT DISTRIBUTE.
7 Steps to Pragmatic Mobile Testing A path to faster, better testing
© 2017 SPLUNK INC.
© 2017 SPLUNK INC. CONFIDENTIAL INFORMATION. DO NOT DISTRIBUTE.
© 2017 SPLUNK INC.
@TomChavez [email protected]