Seattle AT&T Hackathon

AT&T Application Resource Optimizer (ARO)

Doug Sillars

@dougsillars

Seattle Hackathon

4/24/2015

Testing Mobile Apps

Testing Mobile Apps is Challenging

There are 19,000 Android User Agents reported by Akamai

There are Screen Size differences

And there are Processor/Memory constraints

It isn’t easy!

Many Do Not Invest Enough in Testing44% of all issues are reported by users

20% issues were only found because of complaints in app reviews

34% of Users use ratings as primary influence in choosing an app

http://info.perfectomobile.com/rs/perfectomobile/images/why-apps-fail-infographic.pdf

Users Expect Things to Just Work

Defects vs. Performance

Is Performance REALLY that important? Yes!

https://t.co/7bMT4A1PIt

Top App Complaints from Consumers

1.User Interface Issues (58%)

2.Performance (52%)

3.Functionality (50%)

4.Device Compatibility (45%)

Performance Importance?How Important is Speed and Responsiveness of Mobile Apps?


How Fast Should an App Startup?

72%

<3 seconds

49%

<2 seconds

91%

<4 seconds


Testing For Performance

http://smarterappsguidelines.gsma.com/

http://knowmyapp.org/

http://www.appqualityalliance.org/aqua-performance-test-criteria

http://developer.att.com/application-resource-optimizer

Guidelines

Performance Testing With ARO

• ARO Records and Analyzes Network Interactions of App

• Grades Network interactions against 25 Best Practices

Network use

accounts for

40-70% of

battery drain!

ARO Requirements

• Open Source and Free

Developer.att.com/ARO

• iOS

- Requires iOS device tethered to Mac (with Xcode)

• Android

- Requires Android device tethered to Mac/PC (with

Android SDK)

Testing With ARO

In ARO Analyzer App:

Start Data collector

Name the test

Capture Video or not

Phone View of TraceARO establishes a VPN

connection on your Android

Device (click ok)

ARO shows Screen on your

computer (if video is being

recorded)

Click Stop to end ARO trace

Files copied to PC and

analyzed

ARO Analysis

All network traffic

mapped to processes

Stats about trace

Best Practices

© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks

contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific

company or entity is not an endorsement by AT&T.

Duplicate Content

This 2.8 KB file should be cached for 1 year. yet

downloaded twice in ~2 minutes.

Closing Connections

Blue Burst Color indicates

Packets closing connections

GPS/Bluetooth

Green Color Indicates

active GPS usage

Text Compression

Other Considerations in Testing

Background traffic

Many Apps wake up in the background to update data or communicate with

servers.

Just 10 background connections = 0.5% battery/day

And it adds up FAST!

Background Traffic

AT&T Corp

Email

Facebook

Exercise app

ARO For Mobile Apps

• Performance testing is crucial

• Good tools and test plans are essential

• AT&T Application Resource Optimizer (ARO)

• Testing manually

• Testing with automation

• Analysis and filing bugs




http://bit.ly/HighPerfAndroidApps




Q&A

http://smarterappsguidelines.gsma.com/

http://knowmyapp.org/

http://www.appqualityalliance.org/aqua-performance-test-

criteria

http://developer.att.com/application-resource-optimizer

http://bit.ly/HighPerfAndroidApps

ARO Automation – Headless collectionBeta product: release planned July 2015

Command Line and Java APIs to run ARO

./aro --startcollector vpn_android --output /users/demo/MCoC_testing --video yes

ARO Automation – Analysis

• Open trace in Analyzer tool:

• Analyze headlessly into JSON or HTML

./aro --analyze /users/demo/MCoC_testing --output

/users/demo/MCoC_testing/report.html --format html

Seattle AT&T Hackathon

Mobile

Transcript of Seattle AT&T Hackathon