The benefits of using an APM solution while performance testing
-
Upload
james-smith -
Category
Technology
-
view
3.956 -
download
2
description
Transcript of The benefits of using an APM solution while performance testing
![Page 1: The benefits of using an APM solution while performance testing](https://reader033.fdocuments.net/reader033/viewer/2022061218/54b724464a795903798b484b/html5/thumbnails/1.jpg)
DevOpsGuys
@DevOpsGuys www.devopsguys.com 1
DevOpsGuysThe benefits of using an
APM solution while performance testing
Why Perf testing without APM is like Corona without the Lime…
![Page 2: The benefits of using an APM solution while performance testing](https://reader033.fdocuments.net/reader033/viewer/2022061218/54b724464a795903798b484b/html5/thumbnails/2.jpg)
DevOpsGuys
@DevOpsGuys www.devopsguys.com 2http://www.flickr.com/photos/lendog64/5781379311/
![Page 3: The benefits of using an APM solution while performance testing](https://reader033.fdocuments.net/reader033/viewer/2022061218/54b724464a795903798b484b/html5/thumbnails/3.jpg)
DevOpsGuys
@DevOpsGuys www.devopsguys.com 3
Perf Testing & APM in 30mins
• Why Performance Test? • What is APM anyway? • Benefits of APM when Perf Testing• With some case studies along the
way!• Q&A
![Page 4: The benefits of using an APM solution while performance testing](https://reader033.fdocuments.net/reader033/viewer/2022061218/54b724464a795903798b484b/html5/thumbnails/4.jpg)
DevOpsGuys
@DevOpsGuys www.devopsguys.com 4
Why Performance Test?
1. Ensure that we meet our customer’s expectations of website response time– So that they convert and we make £££
2. Ensure that we have sufficient computing capacity to meet #1 under load (load testing)
3. Understand how the system reacts when things go wrong (stress testing)
![Page 5: The benefits of using an APM solution while performance testing](https://reader033.fdocuments.net/reader033/viewer/2022061218/54b724464a795903798b484b/html5/thumbnails/5.jpg)
DevOpsGuys
@DevOpsGuys www.devopsguys.com 5
![Page 6: The benefits of using an APM solution while performance testing](https://reader033.fdocuments.net/reader033/viewer/2022061218/54b724464a795903798b484b/html5/thumbnails/6.jpg)
DevOpsGuys
@DevOpsGuys www.devopsguys.com 6
APM Defined
“In the fields of information technology and systems management, application performance management (APM), is the monitoring and managing of performance and availability of software applications. APM strives to detect and diagnose application performance problems to maintain an expected level of service. APM is the translation of IT metrics into business meaning (i.e. value).”http://en.wikipedia.org/wiki/Application_Performance_Management
![Page 7: The benefits of using an APM solution while performance testing](https://reader033.fdocuments.net/reader033/viewer/2022061218/54b724464a795903798b484b/html5/thumbnails/7.jpg)
DevOpsGuys
@DevOpsGuys www.devopsguys.com 7
“5 Dimensions of APM*”
1. End user experience monitoring - (Active and passive)
2. Application runtime architecture discovery and modeling
3. User-defined transaction profiling (also called business transaction management)
4. Application component monitoring5. Reporting & Application data analytics
*According to Gartner
![Page 8: The benefits of using an APM solution while performance testing](https://reader033.fdocuments.net/reader033/viewer/2022061218/54b724464a795903798b484b/html5/thumbnails/8.jpg)
DevOpsGuys
@DevOpsGuys www.devopsguys.com
CA
QuestHP
OpTier
Opnet
IBM
Compuware
Oracle
MicrosoftSL
Inetco
NetScout
eG InnovationsProgress
ASG
IderaVisual networks
Info vista
Network Instruments
Knoa
Aternity
Coradient
CorrelsenseNastel
dynaTracePrecise
Arctures Technologies
BMC
Managed Engine
New RelicAppDynamics
2011Riverbed
OpTier
2012
AppNeta
Dell
2013
Gartner APM Magic Quadrant
Legac
y Ven
dors
![Page 9: The benefits of using an APM solution while performance testing](https://reader033.fdocuments.net/reader033/viewer/2022061218/54b724464a795903798b484b/html5/thumbnails/9.jpg)
DevOpsGuys
@DevOpsGuys www.devopsguys.com 9
BENEFITS OF LOAD TESTING WITH APM TOOLS
Never leave home without it…
![Page 10: The benefits of using an APM solution while performance testing](https://reader033.fdocuments.net/reader033/viewer/2022061218/54b724464a795903798b484b/html5/thumbnails/10.jpg)
DevOpsGuys
@DevOpsGuys www.devopsguys.com 10
#1See the Big Picture
(aka Systems Thinking)
![Page 11: The benefits of using an APM solution while performance testing](https://reader033.fdocuments.net/reader033/viewer/2022061218/54b724464a795903798b484b/html5/thumbnails/11.jpg)
DevOpsGuys
@DevOpsGuys www.devopsguys.com 11
Systems Thinking 101
“Systems thinking is a framework for seeing interrelationships rather than things, for seeing patterns rather than static snapshots. It is a set of general principles spanning fields as diverse as physical and social sciences, engineering and management.” - Peter Senge
![Page 12: The benefits of using an APM solution while performance testing](https://reader033.fdocuments.net/reader033/viewer/2022061218/54b724464a795903798b484b/html5/thumbnails/12.jpg)
DevOpsGuys
@DevOpsGuys www.devopsguys.com 12
Command & Control vs Systems Thinking
http://www.systemsthinking.co.uk/2-1.asp
![Page 13: The benefits of using an APM solution while performance testing](https://reader033.fdocuments.net/reader033/viewer/2022061218/54b724464a795903798b484b/html5/thumbnails/13.jpg)
DevOpsGuys
@DevOpsGuys www.devopsguys.com 13
CASE STUDY #12 WEEKS WASTED…
“any improvement not made at the constraint is an illusion”
![Page 14: The benefits of using an APM solution while performance testing](https://reader033.fdocuments.net/reader033/viewer/2022061218/54b724464a795903798b484b/html5/thumbnails/14.jpg)
DevOpsGuys
@DevOpsGuys www.devopsguys.com 14
The Scenario
• E-commerce customer• New website • High data-start times on page load• Poor load testing results• Initial conversation with client• 2 week delay while hosting provider
scaled out the hardware…• Problem still not fixed…
![Page 15: The benefits of using an APM solution while performance testing](https://reader033.fdocuments.net/reader033/viewer/2022061218/54b724464a795903798b484b/html5/thumbnails/15.jpg)
DevOpsGuys
@DevOpsGuys www.devopsguys.com 15
What’s the root cause of the ~1sec Data Start?
Waterfall Diagram of page load times
![Page 16: The benefits of using an APM solution while performance testing](https://reader033.fdocuments.net/reader033/viewer/2022061218/54b724464a795903798b484b/html5/thumbnails/16.jpg)
DevOpsGuys
@DevOpsGuys www.devopsguys.com 16
![Page 17: The benefits of using an APM solution while performance testing](https://reader033.fdocuments.net/reader033/viewer/2022061218/54b724464a795903798b484b/html5/thumbnails/17.jpg)
DevOpsGuys
@DevOpsGuys www.devopsguys.com 17
#2Drill down to the
Detail(Code-level)
![Page 18: The benefits of using an APM solution while performance testing](https://reader033.fdocuments.net/reader033/viewer/2022061218/54b724464a795903798b484b/html5/thumbnails/18.jpg)
DevOpsGuys
@DevOpsGuys www.devopsguys.com 18
Find the slow code!
![Page 19: The benefits of using an APM solution while performance testing](https://reader033.fdocuments.net/reader033/viewer/2022061218/54b724464a795903798b484b/html5/thumbnails/19.jpg)
DevOpsGuys
@DevOpsGuys www.devopsguys.com 19
#3Faster Iteration= Better Value
![Page 20: The benefits of using an APM solution while performance testing](https://reader033.fdocuments.net/reader033/viewer/2022061218/54b724464a795903798b484b/html5/thumbnails/20.jpg)
DevOpsGuys
@DevOpsGuys www.devopsguys.com 20
Maximise Testing Value
Test
AnalyseFix
Deploy
![Page 21: The benefits of using an APM solution while performance testing](https://reader033.fdocuments.net/reader033/viewer/2022061218/54b724464a795903798b484b/html5/thumbnails/21.jpg)
DevOpsGuys
@DevOpsGuys www.devopsguys.com 21
CASE STUDY #156X FASTER IN A DAY…
9am to 9pm. 12hrs. 56x faster!
![Page 22: The benefits of using an APM solution while performance testing](https://reader033.fdocuments.net/reader033/viewer/2022061218/54b724464a795903798b484b/html5/thumbnails/22.jpg)
DevOpsGuys
@DevOpsGuys www.devopsguys.com 22
The Scenario
• New website pre-launch testing• Customer Portal (SaaS solution)• Start Load testing 9am• 4 “Test & Fix” cycles in the day • Final testing completed at 9pm
![Page 23: The benefits of using an APM solution while performance testing](https://reader033.fdocuments.net/reader033/viewer/2022061218/54b724464a795903798b484b/html5/thumbnails/23.jpg)
DevOpsGuys
@DevOpsGuys www.devopsguys.com 23
The End Results = 56x Average
ResponseThroughput
(Req/sec)Initial Testing 3024ms 16/minFinal Result 54ms 59/min
56x faster 3.5x throughput
Before 3024ms After
54ms
![Page 24: The benefits of using an APM solution while performance testing](https://reader033.fdocuments.net/reader033/viewer/2022061218/54b724464a795903798b484b/html5/thumbnails/24.jpg)
DevOpsGuys
@DevOpsGuys www.devopsguys.com 24
Nice flat line!
Before
After
![Page 25: The benefits of using an APM solution while performance testing](https://reader033.fdocuments.net/reader033/viewer/2022061218/54b724464a795903798b484b/html5/thumbnails/25.jpg)
DevOpsGuys
@DevOpsGuys www.devopsguys.com 25
#4Team Work“The Enemy is Performance,
not each other…”
![Page 26: The benefits of using an APM solution while performance testing](https://reader033.fdocuments.net/reader033/viewer/2022061218/54b724464a795903798b484b/html5/thumbnails/26.jpg)
DevOpsGuys
@DevOpsGuys www.devopsguys.com 26
The Blame Game…
Ops DevQA
![Page 27: The benefits of using an APM solution while performance testing](https://reader033.fdocuments.net/reader033/viewer/2022061218/54b724464a795903798b484b/html5/thumbnails/27.jpg)
DevOpsGuys
@DevOpsGuys www.devopsguys.com 27
The Fix-It Game…
Ops
DevQA
![Page 28: The benefits of using an APM solution while performance testing](https://reader033.fdocuments.net/reader033/viewer/2022061218/54b724464a795903798b484b/html5/thumbnails/28.jpg)
DevOpsGuys
@DevOpsGuys www.devopsguys.com 28
![Page 29: The benefits of using an APM solution while performance testing](https://reader033.fdocuments.net/reader033/viewer/2022061218/54b724464a795903798b484b/html5/thumbnails/29.jpg)
DevOpsGuys
@DevOpsGuys www.devopsguys.com 29
Summary
1. See the Big Picture (systems thinking)
2. Drill down to the details3. Faster Iteration = Better Value4. Stop the “Blame Game”– aka “There’s your Problem!”
![Page 30: The benefits of using an APM solution while performance testing](https://reader033.fdocuments.net/reader033/viewer/2022061218/54b724464a795903798b484b/html5/thumbnails/30.jpg)
DevOpsGuys
@DevOpsGuys www.devopsguys.com 30
DevOpsGuys
Thank YouWorkshop Session @
1345End-to-end testing & APM demooperations | availability | performance | scalability | migration | strategy
![Page 31: The benefits of using an APM solution while performance testing](https://reader033.fdocuments.net/reader033/viewer/2022061218/54b724464a795903798b484b/html5/thumbnails/31.jpg)
DevOpsGuys
@DevOpsGuys www.devopsguys.com 31
DevOpsGuys provide development & operations support (DevOps) services
for your online applications.
You run your business. We run your website.
It’s as simple as that.
What We Do
![Page 32: The benefits of using an APM solution while performance testing](https://reader033.fdocuments.net/reader033/viewer/2022061218/54b724464a795903798b484b/html5/thumbnails/32.jpg)
DevOpsGuys
@DevOpsGuys www.devopsguys.com 32
Our service portfolio
• 24 x 7 Application-Centric web operations
• DevOps Transformation & Training
• Continuous Delivery implementation
• Infrastructure Automation & “Infrastructure as Code”
• Application Performance Management
• AppDynamics Implementation & Consultancy
• Web Performance consulting
• Web monitoring• Load Testing• Test Automation and
scripting• JetBrains Product
Implementation & Consultancy
• Octopus Deploy implementation
![Page 33: The benefits of using an APM solution while performance testing](https://reader033.fdocuments.net/reader033/viewer/2022061218/54b724464a795903798b484b/html5/thumbnails/33.jpg)
DevOpsGuys
@DevOpsGuys www.devopsguys.com 33
James Smith (@TheDevMgr)
• 16 years of software delivery experience.
• Proven track record– Pizza Hut– KFC– Dell– RAC– Aviva– Fujitsu Siemens– Whitbread
Steve Thair (@TheOpsMgr)
• 25 years of IT Operations experience.
• Proven track record– Vodafone– Credit Suisse Group– IBM– BNP Paribas– Zurich Financial Services– Totaljobs Group– TSL Education
The Leadership Team
![Page 34: The benefits of using an APM solution while performance testing](https://reader033.fdocuments.net/reader033/viewer/2022061218/54b724464a795903798b484b/html5/thumbnails/34.jpg)
DevOpsGuys
@DevOpsGuys www.devopsguys.com 34
Contact UsVisit us at www.devopsguys.com
Call our team on +44 (0)20 8798 3079
Email our engineers at [email protected]