Post on 26-Mar-2020
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Introduction to Mobile Performance Testing Shlomi Zalma, Wilson Mar DT3338 at Noon, June 13, 2013
HOL330 in Sands 305
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 3
Mobile apps are now:
Why performance test mobile apps?
• how customers locate stores and information
• how brands get exposure many times a day
• the “face” of the organization
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 4
Slow app = Slow product?
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 5
Baseline vs. impact of network latency, etc.
[P01] from HP Network Virtualization by Shunra
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 6
Possible points of risk in landscape
Storage DB
Servers App
Servers Web
Servers
Load Balancers
3rd Party
Cloud Services
CDNs & CoLo.
Carriers
Internet Browsers
Platforms & Versions
Firewalls VPNs
Device Features
2G,3G,4G Cellular
Mobile Components
Screen Sizes
Web Services
LANs
ISPs
DB Indexes & Logs
Resource Usage
Latency & Loss
Pooling & Threading
Server Config.
SysAdmin: DBA: Back-end: Dev: Network:
Data Caching
mDev: Encrypt, Compress.
Load Bal. Algorithm
Network: Network:
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 7
HP Live Network
Collaborate 2
HP LoadRunner components and workflow
Virtualize Network 5
End User Application Server(s) Internet/WAN Database
Analyze, Compare, Report 8
Client(s)
Generate Load 6
Author Scripts 1
Measure Impact 7
Test Scenario
Plan and Execute Tests 3
Virtualize Services 4
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 8
Scan for more information
LoadRunner Network Virtualization
Service
Virtualization Mobile
Testing
HP Codescan app
http://www.hp.com/go/...
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 9
Apple App
Store
Play. google
.com
iPhone, iPad Android “feature phones”
Windows PC
Mobile technology terms
Safari browser
URL
m.ft.com mobile
site
ft.com/ws web
services
ft.com HTML5
responsive design
Native app
internet explorer
ft.com HTML4
URL
minimal browser
URL Hybridapp
Chrome browser
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Mobile LoadRunner Scripting
https://connections.houston.hp.com/docs/DOC-15086
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 11
Create a New Script, Mobile Category
HP LoadRunner VuGen 11.5 SP1+
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 12
Multiple Approaches Available
Browser or hybrid app:
Ajax? > TruClient Ajax Mobile scripting protocol using User Agent spoofing in Firefox
Simple > Mobile HTTP/HTML scripting protocol
Native app: > Mobile HTTP/HTML scripting protocol
Android:
No device > Record Emulator using Android Virtual Device (AVD)
Have rooted device > Mobile Recorder generate lrpcap to Analyze
iOS & others:
> LoadRunner Proxy on same subnet/Wi-Fi hub (the newest technique)
> Record & Analyze WinPcap capture on the mobile app server
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 13
Recording Method TruClient Ajax
TruClient Ajax HP LoadRunner VuGen 11.51+
App web server
User Agent in Firefox >
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 14
TruClient recording
HP LoadRunner VuGen 11.51+
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 15
Devices with User Agent strings
Count: 27
HP LoadRunner VuGen11.52
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 16
Sample User Agent for Samsung Galaxy S3
User Agent string example
Mozilla/5.0 (Linux; U; Android 4.0; xx-xx; GT-I9300 Build/IMM76D) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30
HP LoadRunner VuGen11.52 More at http://www.useragentstring.com/
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 17
Android Versions
4.1 (16)
4.2 (17)
2.3 (9,10)
4.0 (15)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 18
Mobile Device Manager Tool in VuGen
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 19
VuGen Scripting Process
from HP Virtual User Generator User Guide 11.52 page 73
1. Configure recording settings
2. Consider run-time settings
3. Object identification using script levels
4. Specify Transaction Names
5. Specify Completion End Events Logic
6. Insert flow control
7. Run-time logic
8. Vary data based on parameters
9. Using C script functions in a library
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 20
Random Think-time in TruClient
Arguments:
http://www.perftesting.co.uk/ajax-truclient-tips-and-tricks-part-ii/2012/05/02/
Math.floor( 11 * Math.random()+5 );
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 21
TruClient Toolbox drag and drop
window.alert( “randWait=” + randWait );
Arguments:
http://www.perftesting.co.uk/ajax-truclient-tips-and-tricks-part-ii/2012/05/02/
Math.floor( 11 * Math.random()+5 ); var randWait =
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
HTTP/HTML Mobile Recording
https://connections.houston.hp.com/docs/DOC-15086
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 23
Mobile HTTP vs. HTML
HP LoadRunner VuGen
mode=“HTML” mode=“HTTP”
Parse HTML to automatically identify additional resources to download
For use with web services calls. Does not parse for resource requests.
• CSS • JavaScript
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 24
Record Mobile HTTP/HTML
HP LoadRunner VuGen11.52
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 25
HTTP/HTML
TruClient
HTTP/HTML Recording Methods Wizard
Proxy Recording
Record Emulator
Analyze Traffic
Record & Analyze Traffic
HP LoadRunner VuGen11.52
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Proxy Recording
https://connections.houston.hp.com/docs/DOC-15086
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 27
Proxy Recording
Android
LoadRunner Proxy
HTTP/HTML
Proxy Recording
HP LoadRunner VuGen11.5x
App web server
Port 80/443
Proxy Port 8080
APK
iOS IPA
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 28
Same subnet via Wi-Fi router
HTTP/HTML
Ethernet cables
Wi-Fi router on private LAN
Proxy Recording
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 29
Proxy Recording
HP LoadRunner VuGen11.52
HTTP/HTML
Proxy Recording
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 30
Proxy Recording Setup on iOS
1.
Source: http://www.amsys.co.uk/2012/blog/how-to-setup-proxy-servers-in-ios/
4. 3. Scroll down 2.
Eleanor
Password:
ncjm-q3ey-t1dd
Eleanor 192.168.1.2
255.255.255.0
192.168.1.1
192.168.1.3
8080
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 31
Proxy Recording Setup on Android Galaxy S3 (4.1.2)
1. 4. 3. 2. Long press
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Record Emulator
https://connections.houston.hp.com/docs/DOC-15086
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 33
Record Emulator Android Recording Method
Android
device emulator
HTTP/HTML
Record Emulator
APK
HP LoadRunner VuGen11.5x
App web server
Async.
APK
@device ADT file
Android… \sdk\tools
File manager
Backup
Play. Google
.com
Install
Copy
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 34
Record Emulator Requirements
1. USB cable to device
2. USB driver for device
3. Android app APK file for Android app under test
4. Android file download manager app to extract APK file from device
5. JDK (Java Development Kit)
6. ADT (Android Development Toolkit) plug-in for Eclipse
7. AVD (Android Virtual Device) within Eclipse for device under test
8. JAVA_HOME and PATH Windows environment variables
9. HP LoadRunner 11.50 and SP2 patch to upgrade to 11.52
HTTP/HTML
Record Emulator
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 35
Record & Analyze Traffic
Android
network sniffer
iOS
HTTP/HTML
Record & Analyze Traffic
Ad hoc network
.cap, .pcap
HP LoadRunner VuGen11.5x
App web server
APK
IPA
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Analyze Traffic
https://connections.houston.hp.com/docs/DOC-15086
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 37
Analyze Traffic
Mobile Recorder
Android
.lrpcap
HTTP/HTML
Analyze Traffic
Android 4.1+ (rooted)
App web server
HP LoadRunner VuGen11.52
APK
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 38
Android Mobile Recorder 1. Start Recording 3. Stop and send email 2.
Perform user steps.
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 39
Android Mobile Recorder Settings
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 40
Recording Methods Recap
Mobile Recorder
Android
.lrpcap
LoadRunner Proxy
device emulator
network sniffer
iOS
Proxy Port 8080
HTTP/HTML
Proxy Recording
Record Emulator
Analyze Traffic
Record & Analyze Traffic
Port 80/443
User Agent in Firefox >
Android 4.1+ (rooted)
Ad hoc network
APK
APK
.cap, .pcap
TruClient Ajax HP LoadRunner VuGen11.52
App web server @device ADT file IPA
Play. Google
.com
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Mobile scenarios
https://connections.houston.hp.com/docs/DOC-15086
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 42
How can we help?
Run test cases
Provision & deploy environment
Execute automated build
Application Security Assessment
Test results
Security Code Scan Fortify Source Code Analysis
UFT / BPT LoadRunner
WebInspect
Continuous Delivery Automation (CDA)
(Jenkins)
HP integrated testing suite
ALM Analysis
Executive Scorecard
Performance Center
Service Virtualization
Network Virtualization
Development Agile Manager
Business Service Management
Monitoring & Diagnostics
Analytics
App. Lifecycle Intelligence
ALM Lab Management
Tear down environment Continuous Delivery Automation (CDA)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 43
Shopping process
Interconnected component services
Pay Buy Ship Track
Product
Search
Customer profiles Payment Shipping
Composite Managed cloud
Public cloud
Traditional
Track package
Private cloud
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 44
Concurrency and rendezvous
[P10] Illustrative data
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 45
Key take-aways
• For mobile browser apps (jQuery Mobile, Sencha, PhoneGap, etc.), little scripting is needed using User Agent spoofing via Firefox TruClient Mobile
• JavaScript executes against a DOM for each virtual user
• Traffic captured between native apps and server are converted to VuGen scripts
• Android native app traffic on rooted devices can be recorded on device using LoadRunner Mobile Recorder
• iPhone and iPad (iOS) -- all non-rooted apps – can be recorded using LoadRunner Proxy
• Monitor and Diagnose using HP Business Service Management
• Consider HP Network Virtualization and Services Virtualization
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Thank you