Webinar: How to Load Test for Your Mobile Peak
-
Upload
jennifer-finney -
Category
Technology
-
view
155 -
download
2
Transcript of Webinar: How to Load Test for Your Mobile Peak
![Page 1: Webinar: How to Load Test for Your Mobile Peak](https://reader035.fdocuments.net/reader035/viewer/2022070514/5881e3d91a28ab36088b560b/html5/thumbnails/1.jpg)
©2 016 S OA S TA , A l l r i gh t s r es e rv ed .
Tom Chavez @tomchavezMike Ostenberg @mikeostenberg
How to Load Test for your Mobile Peak
![Page 2: Webinar: How to Load Test for Your Mobile Peak](https://reader035.fdocuments.net/reader035/viewer/2022070514/5881e3d91a28ab36088b560b/html5/thumbnails/2.jpg)
©2 016 S OA S TA , A l l r i gh t s r es e rv ed .
• How important is Mobile?• Mobile Web and Apps• The need for load testing• Measure everything• What to test?• Test early and often – shift left and right• How to test?
![Page 3: Webinar: How to Load Test for Your Mobile Peak](https://reader035.fdocuments.net/reader035/viewer/2022070514/5881e3d91a28ab36088b560b/html5/thumbnails/3.jpg)
Importanceof Mobile
![Page 4: Webinar: How to Load Test for Your Mobile Peak](https://reader035.fdocuments.net/reader035/viewer/2022070514/5881e3d91a28ab36088b560b/html5/thumbnails/4.jpg)
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 4
Mobile is growing!
• In 2015, mobile commerce
accounted for $115B worldwide
• In 2016, mobile commerce
is project to reach $142B
![Page 5: Webinar: How to Load Test for Your Mobile Peak](https://reader035.fdocuments.net/reader035/viewer/2022070514/5881e3d91a28ab36088b560b/html5/thumbnails/5.jpg)
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 5
How is your site now?
• mCommerce is globalo US is not the benchmark
• 39% of mobile users
are unhappy with their online experienceso Top reasons: slow pages and site freezes/crashes
• 57% have experienced problems when using mobile
• 3 out of 4 consider mobile slower than desktop
• 46% will not return to a site that performs poorly
• 27% of smartphone users exceed data cap every month
![Page 6: Webinar: How to Load Test for Your Mobile Peak](https://reader035.fdocuments.net/reader035/viewer/2022070514/5881e3d91a28ab36088b560b/html5/thumbnails/6.jpg)
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 6
When is your Peak Season?
• Peak is year-roundo Super Bowl Sunday – 110 million viewers
o Bi-weekly or monthly payroll processing
o Valentine’s Day, President’s Day, Mother’s Day, Father’s Day
o Quarterly processing and filing
o U.S. Tax filing
o Spring, Summer, Fall & Winter season sales
o July 4th, Christmas in July
o Back to School
o Olympics
o Halloween We are here
o Election Day
o Singles Day – world’s biggest shopping day, $9B in 12 hours
o Black Friday, Cyber Monday and Holidays
Back to top to repeat again next year!
![Page 7: Webinar: How to Load Test for Your Mobile Peak](https://reader035.fdocuments.net/reader035/viewer/2022070514/5881e3d91a28ab36088b560b/html5/thumbnails/7.jpg)
Mobile Weband Mobile Apps
![Page 8: Webinar: How to Load Test for Your Mobile Peak](https://reader035.fdocuments.net/reader035/viewer/2022070514/5881e3d91a28ab36088b560b/html5/thumbnails/8.jpg)
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 8
Mobile Web
Responsive web site WebSite
• Desktop Users
• Mobile Users
• Same or similar contento Adapts to different
screen sizes
Mobile optimized
• “m-dot” site for mobile
• Custom built for mobile devices
All have html, images, JavaScript, css, plus third-party content
![Page 9: Webinar: How to Load Test for Your Mobile Peak](https://reader035.fdocuments.net/reader035/viewer/2022070514/5881e3d91a28ab36088b560b/html5/thumbnails/9.jpg)
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 9
Native App
• Android – Java
• iOS – Objective C/Swift
• Mobile app frameworko Same source code
Hybrid App
• Mix of native code + html
All have backend calls for login/authentication, account
settings, database calls, transactions, plus third-party SDK
calls
Mobile Apps
WebSite/MobileBackend
![Page 10: Webinar: How to Load Test for Your Mobile Peak](https://reader035.fdocuments.net/reader035/viewer/2022070514/5881e3d91a28ab36088b560b/html5/thumbnails/10.jpg)
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 10
The Need for Load Testing
• Everyone runs load testing on their website, right?o Load and Performance testing is often not done or too late
o Black Friday, Cyber Monday, Holiday failures
• What’s the need for mobile web load testing?o Mobile is growing faster than desktop
o Responsive sites and mobile sites deliver different content to
mobile devices over mobile network
• What about mobile app load testing?o Mobile app back ends serve different content to mobile apps
o Remember Pokemon Go? It wasn’t the app that failed.
![Page 11: Webinar: How to Load Test for Your Mobile Peak](https://reader035.fdocuments.net/reader035/viewer/2022070514/5881e3d91a28ab36088b560b/html5/thumbnails/11.jpg)
Measure Everything
![Page 12: Webinar: How to Load Test for Your Mobile Peak](https://reader035.fdocuments.net/reader035/viewer/2022070514/5881e3d91a28ab36088b560b/html5/thumbnails/12.jpg)
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 12
Measure Real and Virtual Users
• Real Users (RUM) provide deep data about your visitorso No other way to know everything about your users on your site
and in your app
SOASTA mPulse collects and analyzes RUM data from your web
and mobile app users
• Virtual Users tell you how your site or app will do at scaleo You can’t get a million friends to visit your site or app at the same
time!
SOASTA CloudTest lets you easily build and execute load tests at
scales from hundreds to millions of virtual users
![Page 13: Webinar: How to Load Test for Your Mobile Peak](https://reader035.fdocuments.net/reader035/viewer/2022070514/5881e3d91a28ab36088b560b/html5/thumbnails/13.jpg)
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 13
Other Things to Measure with RUM
• Session pathso How are your users going through your site
• Mobile App Performanceo How are your mobile apps performing?
o What are users doing in your mobile apps?
• Revenueo How much money are you making from each user and where?
![Page 14: Webinar: How to Load Test for Your Mobile Peak](https://reader035.fdocuments.net/reader035/viewer/2022070514/5881e3d91a28ab36088b560b/html5/thumbnails/14.jpg)
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 14
Average Performance
![Page 15: Webinar: How to Load Test for Your Mobile Peak](https://reader035.fdocuments.net/reader035/viewer/2022070514/5881e3d91a28ab36088b560b/html5/thumbnails/15.jpg)
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 15
Performance by Device
![Page 16: Webinar: How to Load Test for Your Mobile Peak](https://reader035.fdocuments.net/reader035/viewer/2022070514/5881e3d91a28ab36088b560b/html5/thumbnails/16.jpg)
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 16
Creating a Test Plan – Peak Load
What was the busiest Day, Hour, Minute, Second?
![Page 17: Webinar: How to Load Test for Your Mobile Peak](https://reader035.fdocuments.net/reader035/viewer/2022070514/5881e3d91a28ab36088b560b/html5/thumbnails/17.jpg)
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 17
Creating a Web Test Plan – Page Flows
What were most popular page flows with think times? Which APIs?
![Page 18: Webinar: How to Load Test for Your Mobile Peak](https://reader035.fdocuments.net/reader035/viewer/2022070514/5881e3d91a28ab36088b560b/html5/thumbnails/18.jpg)
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 18
Network Emulation
• Testing to match realistic situationso Users are not all from one location on a fast network
o Packets get delayed, lost, re-ordered
• Apps perform differently as network conditions worseno Reduced throughput
o Timeouts occur, connections are lost
o Transactions take longer, longer sessions
Adding real network modeling to load and performance testing
![Page 19: Webinar: How to Load Test for Your Mobile Peak](https://reader035.fdocuments.net/reader035/viewer/2022070514/5881e3d91a28ab36088b560b/html5/thumbnails/19.jpg)
Test Earlyand Often
![Page 20: Webinar: How to Load Test for Your Mobile Peak](https://reader035.fdocuments.net/reader035/viewer/2022070514/5881e3d91a28ab36088b560b/html5/thumbnails/20.jpg)
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 20
Don’t Wait Til the Last Minute to Test!
![Page 21: Webinar: How to Load Test for Your Mobile Peak](https://reader035.fdocuments.net/reader035/viewer/2022070514/5881e3d91a28ab36088b560b/html5/thumbnails/21.jpg)
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 21
Test Left and Test Right, even into Production
• Performance Testing in developmento Set performance SLAs for new code and test in development,
same as unit and functional tests which code must pass
• Performance Testing in stagingo Higher loads in near-production systems
• Performance Testing in Productiono Full scale load tests to expected load and beyond
o Find weak and stress points and prioritize fixes into project
Performance Testing is Continuous
![Page 22: Webinar: How to Load Test for Your Mobile Peak](https://reader035.fdocuments.net/reader035/viewer/2022070514/5881e3d91a28ab36088b560b/html5/thumbnails/22.jpg)
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 22
Test in Production
Some issues will only be found at scale load on production systems
Don’t forget your third party providers: CDNs, external scripts
![Page 23: Webinar: How to Load Test for Your Mobile Peak](https://reader035.fdocuments.net/reader035/viewer/2022070514/5881e3d91a28ab36088b560b/html5/thumbnails/23.jpg)
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 23
Measure Mobile App Performance Under Load
All mobile apps perform well when there is no load
![Page 24: Webinar: How to Load Test for Your Mobile Peak](https://reader035.fdocuments.net/reader035/viewer/2022070514/5881e3d91a28ab36088b560b/html5/thumbnails/24.jpg)
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 24
Automate Your Testing
Use Continuous Integration tools to automate testing
![Page 25: Webinar: How to Load Test for Your Mobile Peak](https://reader035.fdocuments.net/reader035/viewer/2022070514/5881e3d91a28ab36088b560b/html5/thumbnails/25.jpg)
Architectureand Demo
![Page 26: Webinar: How to Load Test for Your Mobile Peak](https://reader035.fdocuments.net/reader035/viewer/2022070514/5881e3d91a28ab36088b560b/html5/thumbnails/26.jpg)
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 26
Now That the Test is Over
• Analyze results for future areas to addresso Weak or stress areas that aren’t likely this time but …
• Identify third parties that were affected by loado Coordinate with suppliers to address issues
o Align schedules or at least be aware of their changes
Even if the test passed, you’re not done
![Page 27: Webinar: How to Load Test for Your Mobile Peak](https://reader035.fdocuments.net/reader035/viewer/2022070514/5881e3d91a28ab36088b560b/html5/thumbnails/27.jpg)
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 27
Issues Will Happen
• “I am 100% confident that everything we tested will work
just fine.” – Nordstrom Performance Leadero When loads are different than what you modeled, may have
problems
o E.g. At Nordstrom, during a recent anniversary sale, mobile
sign-ins were 9x greater than anticipated
o Used RUM to get to the root cause of the issue
Realize that Issues Will Happen Where You Can’t Predict Them
![Page 28: Webinar: How to Load Test for Your Mobile Peak](https://reader035.fdocuments.net/reader035/viewer/2022070514/5881e3d91a28ab36088b560b/html5/thumbnails/28.jpg)
Q & AResourcesNext Steps
![Page 29: Webinar: How to Load Test for Your Mobile Peak](https://reader035.fdocuments.net/reader035/viewer/2022070514/5881e3d91a28ab36088b560b/html5/thumbnails/29.jpg)
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 29
CloudTest – Load Testing for mobile web, apps
• Load testing from development to productiono Continuous testing at any scale, on-prem or in the cloud
• Real-time analytics with customizable dashboardso Safely test in production with full visibility
• Run tests live or automaticallyo Full hands-on control or hands-off automation
• Full end-to-end viewo Front-end performance to back-end systems
Fast, scalable, affordable load testing for performance optimization
![Page 30: Webinar: How to Load Test for Your Mobile Peak](https://reader035.fdocuments.net/reader035/viewer/2022070514/5881e3d91a28ab36088b560b/html5/thumbnails/30.jpg)
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 30
SOASTA Digital Performance Management Platform
![Page 31: Webinar: How to Load Test for Your Mobile Peak](https://reader035.fdocuments.net/reader035/viewer/2022070514/5881e3d91a28ab36088b560b/html5/thumbnails/31.jpg)
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 31
Questions, Resources & Next Steps
• Holiday Readinesso soasta.com/holiday-readiness
• The Performance Beacono soasta.com/blogo soasta.com/blog/2015-popular-posts-web-performance
• Webinarso soasta.com/webinars
o Load Testing for Peak; RUM-Based Testing
• CloudTest on-Demando soasta.com/cloudtest: SOASTA will build & run tests for you
Resources
![Page 32: Webinar: How to Load Test for Your Mobile Peak](https://reader035.fdocuments.net/reader035/viewer/2022070514/5881e3d91a28ab36088b560b/html5/thumbnails/32.jpg)
©2 016 S OA S TA , A l l r i gh t s r es e rv ed . 32
“The leader in performance analytics.”
www.soasta.com
SOASTA