Automation:xpLor Performance Test

2

Performance Test AutomationObjectives

• Enable developers to run performance tests against any specified xpLor instances to proactively create performance aware culture

• Provide comprehensive reports, analysis, and visualization for xpLor performance test results that are persisted forever

• Create continuous feedback of performance and script breakage against the latest xpLor builds

3

Performance Test AutomationAutomation Types

• Continuous: Runs nightly at 6pm EST against the latest successful build in Bamboo

• Ad hoc: Triggered by developers against a desired xpLor instance

4

ArchitectureGalileo + Fusion + Ansible + JMeter

5

xpLor

6

Ansiblehttp://www.ansibleworks.com/

• GPL licensed software for configuration management, application deployment, and continuous delivery

• Written in Python• Manages servers over SSH – doesn’t require any agents• YAML for reusable system descriptions

https://github.com/blackboard/perf-test-tools/tree/develop/ansible-playbooks/xplor

7

JMeterData + Load Generation

Server poolr6x64o11-pv087, r6x64o11-pv088, r6x64o11-pv089, r6x64o11-pv090, r6x64o11-pv091, r6x64o11-pv092

Data modelhttps://github.com/blackboard/perf-test-tools/tree/develop/jmeter/xplor/config/datamodels

Test scenariohttps://github.com/blackboard/perf-test-tools/tree/develop/jmeter/xplor/config/scenarios

8

Test ResultsReports and Notifications

• Automated analysis + notifications• *Overview statistics• *Cumulative response time rate of change• *Vusers vs. response time – top 5,000 slowest transactions• Transactions details statistics• Continuous trend

* These reports have the ability to view results in categories of all, realtime, or long-running transaction types

9

Test ResultsAutomated Analysis + Email Notifications

• High transaction response time: 90th percentile = 3 seconds

• Low script quality: HTTP 500 = 50 requests; HTTP 400 = 75 requests; failure rate > 1%

• Top 5 Transaction Response Time Degradation with 20% threshold

• 100% failed transaction(s) found• Uneven app load distribution• Crashed JMeter generator

10

Test ResultsAutomated Analysis + Email Notifications

11

Test ResultsOverview Statistics

12

Test ResultsCumulative response time rate of change

13

Test ResultsVusers vs. Response Time – Top 5,000 Slowest Transactions

14

Test ResultsTransaction Details Statistics

15

Test ResultsNavigating to Your Test Results

• From process ID (PID) link in email notification• Galileo “Browse” tab

16

Test ResultsCustomizing Your Test Results Dashboard

17

Test ResultsContinuous Trend

18

Fusion – Task SchedulerTemplate - http://tinyurl.com/perftest-xplor

19

Fusion – Task SchedulerHow to run test against your server

1. Go to Fusion tab on Galileo (http://galileo.pd.local)2. Create “New Fusion Process”

1. Project: jmeter2. Process: My Test

3. Click on the star icon to create a Fusion task from a template

4. Choose the group icon to select “xpLor – Ad hoc Test” template

5. Modify PID variables accordingly (app.server.hosts, app.server.ports, file.server.hosts, file.server.ports, loadtest.duration)

20

Fusion – Task SchedulerHow to run test against your server

xpLor Configuration Requirements:• Set isTesting to true• Set exports.development to false;• Set exports.timestampExpirySeconds to 99999

(We don’t run ping requests in JMeter to keep sessions live)

21

Zabbix MonitoringTemplates

22

Next StepsWhat’s coming?

• Instructional videos for running ad hoc testing• Institute performance aware culture• Shared ownership of JMeter scripts• JMeter training