Load Tests 101! Why and where to start?
-
Upload
aurelien-deleusiere -
Category
Software
-
view
93 -
download
2
Transcript of Load Tests 101! Why and where to start?
![Page 1: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/1.jpg)
(RE)LOAD TESTS 101
WHY AND WHERE TO START?
AURÉLIEN DELEUSIÈRETHE ONLINE COLDFUSION MEETUP
![Page 2: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/2.jpg)
(c)2016 A. Deleusière
WHO AM I?
• Software engineer specializing in web technologies, consulting, development, architecture and troubleshooting
• Founder of Webellian based in France and Poland
• ColdFusion since 1996, Java, PHP (mainly for CMS), AngularJs, ExtJS…
![Page 3: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/3.jpg)
(c)2016 A. Deleusière
WHY LOAD TESTING?
Load Testing
Performance
Resources
Scalability
Robustness
![Page 4: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/4.jpg)
(c)2016 A. Deleusière
WHERE TO START?
• Depends on the architecture!
• Two approaches:
• Define the limit of a system in the absolute
• Validate the capability of a system to resist to an estimated traffic, anticipate spikes link to special events
![Page 5: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/5.jpg)
(c)2016 A. Deleusière
SCENARIO “CALIBRATION”
• have a picture of the “background noise” of the scope tested
• have the best response time to calibrate results analysis
![Page 6: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/6.jpg)
(c)2016 A. Deleusière
SCENARIO “HEALTHINESS”
• have a picture of the profile of resources consumption (JVM Memory and Garbage Collection, System CPU, I/O)
• validate the healthiness of the component before Stress Scenario
![Page 7: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/7.jpg)
(c)2016 A. Deleusière
SCENARIO “STRESS”
• determine the maximum load capacity (MAX) where the target conditions are reach in term of response time and stability
• determine acceptable response time variation (in percentile)MAX
![Page 8: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/8.jpg)
(c)2016 A. Deleusière
SCENARIO “ENDURANCE”
• validate the maximum capacity determined during Stress Scenario
• validate the healthiness of the component under target load
![Page 9: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/9.jpg)
(c)2016 A. Deleusière
SCENARIO “BURST”
• validate the capacity of the component to support a sudden high pressure
• this case of sudden high pressure can happen when you put a server back in the pool, or when a special campaign is starting
![Page 10: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/10.jpg)
(c)2016 A. Deleusière
ABOUT GATLING
• Very high performance testing Engine
• 1 thread ≠ 1 request
• Distributed multi-nodes injectors
• Scala descripted scenarios (≈ DSL)
![Page 11: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/11.jpg)
(c)2016 A. Deleusière
ABOUT GATLING
Question: How much performant?
How many requests per second with a single injector (16 vcpu on AWS)?
![Page 12: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/12.jpg)
(c)2016 A. Deleusière
A SIMPLE EXAMPLE
• Produce a chain “ABCDEFGHIJKLMNOPQRSTUVWXYZ”
• By concatenating ASCII characters from 65 to 90
• Tests done locally:
• MacBook Pro running Virtual Box guest Ubuntu
• ColdFusion 2016u3 (2016.0.03.300466) Developper Edition
• Lucee 5.0.0.254
![Page 13: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/13.jpg)
(c)2016 A. Deleusière
REAL EXAMPLE: A QUOTATION SERVICE
• Service Layer is composed by several Component (S1, S2
…) behind a Load Balancer (LB) in a stateless fashion.
• The Service Layer is behind a Gateway in charge of Authentication, Transformation, Routing and Logging that is also composed by several instances (G1, G2) behind a Load Balancer.
• No database, configuration in memory from configuration (XML, DSL) files.
• What to be tested?...
Serv
ices
LB
Gate
way G3 G2
LB
TRAFFIC
Datacenter 1 Datacenter 2
S1 S3 S5 S7 S2 S4 S6 S8
G1 G4
![Page 14: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/14.jpg)
(c)2016 A. Deleusière
Agents
REAL EXAMPLE: A QUOTATION SERVICE
• Isolate a representative architecture to be tested…
• …in an environment identical to production
• Install the load tests Agent as close as possible to the component to be tested
• Install tools to monitor the resources (JVM memory details, GC, System CPU) like Fusion Reactor or Mission Control
• But you can also do test as a “black box”
Services S1 S2
LB
Gateway G1 G2
LB
TRAFFIC
A1 A2
![Page 15: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/15.jpg)
(c)2016 A. Deleusière
FUSION REACTOR
![Page 16: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/16.jpg)
(c)2016 A. Deleusière
MISSION CONTROL
![Page 17: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/17.jpg)
(c)2016 A. Deleusière
REAL EXAMPLE: A QUOTATION SERVICE
The Load Tests are run against one single component to measure the intrinsic capability of one single instance.
Main objectives:• Measure the maximum capacity of one single component
• Measure the response time at component level
Services S1 S2
LB
Gateway G1 G2
LB
TRAFFIC
A
![Page 18: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/18.jpg)
(c)2016 A. Deleusière
REAL EXAMPLE: A QUOTATION SERVICE
The Load Tests are run against a couple of component behind the Load Balancer.
Main objectives:• Determine the capacity factor between 1 instance and 2
instance that is generally is not linear
• Measure the response time to evaluate the latency introduced by the network devices
Services S1 S2
LB
Gateway G1 G2
LB
TRAFFIC
B
![Page 19: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/19.jpg)
(c)2016 A. Deleusière
REAL EXAMPLE: A QUOTATION SERVICE
The Load Tests are run against the full Service layer through one single Gateway instance.
Main objectives:
• Measure the capacity of one single Gateway instance
• Measure the response time to evaluate the latency introduced by one Gateway instance
Services S1 S2
LB
Gateway G1 G2
LB
TRAFFIC
C
![Page 20: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/20.jpg)
(c)2016 A. Deleusière
REAL EXAMPLE: A QUOTATION SERVICE
The Load Tests are run against the full stack.
Main objectives:• measure the response time with the full stack to validate the
order of magnitude for SLA
If possible validate this result with the full stack.
Services S1 S2
LB
Gateway G1 G2
LB
TRAFFIC
D
![Page 21: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/21.jpg)
(c)2016 A. Deleusière
INITIAL TEST CAMPAIGN
![Page 22: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/22.jpg)
(c)2016 A. Deleusière
INITIAL TEST CAMPAIGN
![Page 23: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/23.jpg)
(c)2016 A. Deleusière
INITIAL TEST CAMPAIGN
![Page 24: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/24.jpg)
(c)2016 A. Deleusière
INITIAL TEST CAMPAIGN
![Page 25: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/25.jpg)
(c)2016 A. Deleusière
INITIAL TEST CAMPAIGN
![Page 26: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/26.jpg)
(c)2016 A. Deleusière
CHRISTMAS 2016 SPIKE CAMPAIGN
![Page 27: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/27.jpg)
(c)2016 A. Deleusière
1. GATHER EXISTING DATA
Experience from 2015:
• brutal ramp up at 6:00am from almost nothing to 2000 rq/min;
• peak around 7:00am around 1850 rq/min;
• plateau between 9:00am and 10:30am around 1500 rq/min;
• then 30% to 40% of sales more than usual during the next weeks.
Let’s focus on the 2000 rq/min
ILLUSTRATIVE
![Page 28: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/28.jpg)
(c)2016 A. Deleusière
2. CHALLENGE BUSINESS EXPECTATIONS
Based on the current trends, the business expects an increase of 30%.
Let’s challenge our system with:• 2600 rq/min• A brutal ramp up from 0
![Page 29: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/29.jpg)
(c)2016 A. Deleusière
3. FORECAST YOUR TRAFFIC
3. 2600 rq / min => 44 rq/sWrong! You have to correct the linear projection
“Feel” the volatility of what is your case.Here for a starting spike it should quite stable => 2x
Let’s challenge our system with:• 88 rq/s• A brutal ramp up from 0
![Page 30: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/30.jpg)
(c)2016 A. Deleusière
4. SIZE YOUR SYSTEM
Original system design
Serv
ices
LB
Gate
way G3 G2
LB
TRAFFIC
Datacenter 1 Datacenter 2
S1 S3 S5 S7 S2 S4 S6 S8
G1 G4
![Page 31: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/31.jpg)
(c)2016 A. Deleusière
SERVICE CAPABILITY
![Page 32: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/32.jpg)
(c)2016 A. Deleusière
SERVICE CAPABILITY
![Page 33: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/33.jpg)
(c)2016 A. Deleusière
SERVICE CAPABILITY
Service 80 rq/s per server
![Page 34: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/34.jpg)
(c)2016 A. Deleusière
GATEWAY CAPABILITY
![Page 35: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/35.jpg)
(c)2016 A. Deleusière
GATEWAY CAPABILITY
![Page 36: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/36.jpg)
(c)2016 A. Deleusière
GATEWAY CAPABILITY
![Page 37: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/37.jpg)
(c)2016 A. Deleusière
GATEWAY CAPABILITY
Gateway 25 rq/s per server
![Page 38: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/38.jpg)
(c)2016 A. Deleusière
CHRISTMAS 2016 SPIKE CAMPAIGN
4. Size your system
Target 88 rq/s
Service: 80 rq/s x 8 = 640 rq/s OKGateway: 25 rq/s x 4 = 100 rq/s OK
But….you have to expect a disaster!Service: 80 rq/s x 4 = 320 rq/s OKGateway: 25 rq/s x 2 = 50 rq/s NOT OK!
Required action: add 4 new nodes in Gateway layer.
Serv
ices
LB
Gate
way G3
LB
TRAFFIC
Datacenter 1
S1 S3 S5 S7
G1 G2
Datacenter 2
S2 S4 S6 S8
G4
![Page 39: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/39.jpg)
(c)2016 A. Deleusière
1. Gather existing data2. Challenge business expectations3. Forecast your traffic4. Size your system5. Seat back in comfort on D-Day and bring the popcorns
![Page 40: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/40.jpg)
(c)2016 A. Deleusière
CHRISTMAS 2016 SPIKE CAMPAIGN
• Measured results
![Page 41: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/41.jpg)
(c)2016 A. Deleusière
CHRISTMAS 2016 SPIKE CAMPAIGN
![Page 42: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/42.jpg)
(c)2016 A. Deleusière
CHRISTMAS 2016 SPIKE CAMPAIGN
Estimated: 2600
![Page 43: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/43.jpg)
(c)2016 A. Deleusière
CHRISTMAS 2016 SPIKE CAMPAIGN
Estimated: 88
![Page 44: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/44.jpg)
(c)2016 A. Deleusière
WHEN LOAD TESTING?
• Usually done on Major version releases
• Often by a specialist consultant
• Should be integrated into your automated tests
![Page 45: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/45.jpg)
(c)2016 A. Deleusière
SPECIAL THANKS
• Mohamadou Mballo, a genius developer that just felt in love with CFML
• Stéphane Landelle father of Gatling for his support and the goodies to give away
• Yamina, my supportive girlfriend, that would be upset if she’s not listed on this page…
…and all of you making CFML so special…
![Page 46: Load Tests 101! Why and where to start?](https://reader036.fdocuments.net/reader036/viewer/2022062503/58719b641a28ab044e8b5bb9/html5/thumbnails/46.jpg)
(c)2016 A. Deleusière
You liked it? Tweet it!@adeleusiere