Cutting-edge Performance Testing on eCommerce Websites

Click here to load reader

  • date post

  • Category


  • view

  • download


Embed Size (px)


Having problems with your website’s performance? Does it take too much time and effort to determine the cause of a particular page’s poor performance? Would you like to find the root cause of client-side issues in an automated way? If you answered yes to any of these questions, then this session is for you. At GSI Commerce, an eBay company, Ron Woody manages a large team of performance engineers working on nearly nearly 100 eCommerce websites. Ron and his team have developed cutting-edge approaches for automating client- and server-side performance testing. Learn the specific approaches Ron’s team uses today for pre-release performance tests, production performance management, and website optimization. Find out the ways they’ve automated cross-browser performance testing—and analysis—to increase productivity and efficiency. Covering these and additional topics Ron shares a toolkit of performance testing ideas and approaches your team can use to ensure optimal application performance and a better user experience.

Transcript of Cutting-edge Performance Testing on eCommerce Websites

  • 1. W11 Performance Testing 5/1/2013 1:45:00 PM Cutting-edge Performance Testing on eCommerce Websites Presented by: Ron Woody GSI Commerce Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 904-278-0524 [email protected]
  • 2. Ron Woody Senior manager of QA performance at GSI Commerce (an eBay company), Ron Woody manages a team of twenty-one performance engineers dedicated to ensuring that web-based performance is within targeted requirements for all client sites and launches. Ron brings more than fifteen years of automation, performance, QA management, and process experience to his current role. He heads the effort to improve performance for all web stores released and supported by GSI while providing performance testing and engineering vision to the company. Ron has served as a consultant to companies such as Comcast, Bloomberg, Vanguard, and Bank of America.
  • 3. 4/16/2013 Automating Client- and ServerClientServerSide Web Performance Testing and Optimization Ron Woody Director of Performance GSI Commerce, an eBay Inc. company |1 Who is GSI Commerce? |2 1
  • 4. 4/16/2013 About GSI Commerce Companies of all sizes have selected GSI Commerce to help deliver an exceptional and engaging eCommerce shopping experience to their customers. |3 GSI Commerce: QA Performance Processes Performance Testing Platform/Product Projects Store Launches Client-Side Testing Server-Side Testing Performance Analysis |4 2
  • 5. 4/16/2013 Business Impact of Performance Performance impacts sales! Poor performance increases the risk of site abandonment Slow performance may impact brand People remember their experience when a site is slow; Users may be reluctant to return Our clients competitors are only a click away |5 Recent Performance Testing Objectives Phase 1: Improve the end-user experience via client-side and server-side testing Phase 2: Increase productivity via Automation Phase 3: Create more realistic client-side test scenarios and enhance our testing further |6 3
  • 6. 4/16/2013 Phase 1 Improve end-user experience via endclientclient- and server-side testing server Client-Side Testing dynaTrace AJAX Edition Improve the End-User Experience Server-Side Testing dynaTrace Development & Test Center Editions Diagnosing server-side bottlenecks End-to-End visibility |7 ClientClient-Side Testing Started with dynaTraces free AJAX Edition More accurate rendering times; greater visibility |8 4
  • 7. 4/16/2013 ServerServer-Side Testing Added dynaTrace Development Team & Test Center Editions |9 Phase 2 Increase Productivity via Automation Server-side Automation LoadRunner with dynaTrace Test Edition dynaTrace AJAX Edition + Test Center Editions provide in-depth views Client-side Automation Use of dynaTrace AJAX APIs to extend our reporting | 10 5
  • 8. 4/16/2013 Benefits of using dynaTrace: Visibility dynaTrace provided visibility we hadnt had before But testing browser use-cases was manual, and very time-consuming Increasing Productivity via Automation LoadRunner / Performance Center QuickTest Professional / Selenium Quality Center / ALM | 11 Solution: Automated Browser Lab Virtualized browser lab to capture the end-user experience with the touch of a button Cutting-edge automated home-grown solution dynaTrace AJAX Edition Selenium Google Page Speed Shunra network emulation tools Real-Time reporting system to show results | 12 6
  • 9. 4/16/2013 GSI Commerces Browser Lab | 13 GSI Commerces Browser Lab | 14 7
  • 10. 4/16/2013 Why dynaTrace? Visibility into Rendering As well as other Web 2.0 challenges Was not available in HTTPWatch IE 7,8,9 support Firebug didnt support IE Now dynaTrace supports Firefox as well | 15 Basic Browser Lab Diagram | 16 8
  • 11. 4/16/2013 GSI Commerces Browser Lab Front-End Front- | 17 A Sample Browser Lab Report | 18 9
  • 12. 4/16/2013 A Sample Browser Lab Report | 19 Using Automation to Save Time and Money With automation, GSI saved hours per test Reducing test time from 20 hours to ~1 hour Nearly a 20x time savings Reproducible results Shortens project time Allows for more iterative testing | 20 10
  • 13. 4/16/2013 Additional Benefits of our Browser Lab Pinpoint problems quickly Problems that were difficult / impossible to see before Full Waterfall / timeline Rendering DOM Manipulations Allows us to be proactive Leads to a faster site Enables clients to make informed decisions based on the results of our testing in QA | 21 Benefits of dynaTrace: Better Collaboration Dev and QA collaboration dynaTrace AJAX Edition sessions shared with Development Now Development sees exactly what Test sees Less back & forth Faster resolution / optimization Dev Teams can now use dynaTrace prior to QA Developed Performance Requirements that include dynaTrace rankings Dev teams now design to these requirements | 22 11
  • 14. 4/16/2013 Phase 3 Create more realistic client-side clienttest scenarios and enhance our testing further Network emulation for more realistic traffic Larger samplings with automation Browser weighting to generate more accurate timings across browsers Automatic removal of outliers | 23 Performance Objectives Revisited (ROI) Phase 1: Improve the end-user experience ROI: We now have visibility into performance issues and how to resolve them quickly and more efficiently thus saving us time and money Phase 2: Increase productivity via Automation ROI: ~20x savings with our Browser Lab solution can now do testing and analysis simultaneously faster time to market; more complete testing Phase 3: Create more realistic test scenarios ROI: More accurate timings through browser weighting and network emulation; increased consistency by reducing noise in our results | 24 12
  • 15. 4/16/2013 Recent additions Leveraging dynaTrace browser agent (AJAX Edition Premium Version) Ability to see end-to-end performance in QA (PurePath) via Performance Center scripts Use of dynaTrace (Production Edition) to become even MORE proactive Ability to find issues that exist in Production (including issues that clients havent reported yet) Determine where users are abandoning our sites Ability to monitor Apache | 25 Whats Next? Browser Lab aspirations Mobile testing Cached vs. Non-Cached pages Film strip Integration with Expand to Continuous Integration Enable Dev to find/fix problems before they reach QA | 26 13
  • 16. 4/16/2013 GSI Commerces Performance Best Practices Performance matters! Define Performance Targets Test both client-side AND server-side performance Cross-browser testing Add server-side testing Tie everything together | 27 GSI Commerces Performance Best Practices Automate! Get Test and Development on the same page Get Proactive Benchmark your site against your competition | 28 14
  • 17. 4/16/2013 QUESTIONS? | 29 15