D I S C I P L I N E D I N N O VAT I O N

AUTOMATED PERFORMANCE TESTING

FOR DESKTOP APPLICATIONS

CIPRIAN BALEA

THE SCARY QUESTION OF APPLICATION

DEVELOPMENT

How does it perform in real life conditions?

Never as first expected!

BENCHMARKING

Ideally: • consumming as little resources as possible without loosing

functionality• the app should not break / slow the system down

What impact does the app have on the system running it?(is the system still reliable)

How does the app perform when running on the host system? (does it keep all functionality)

HOW TO PREDICT THE FUTURE?

Run it multiple times, and average out.

A slingle run doesn’t acurately predict real life usage.

HOW TO GET A CREDIBLE RESULT

Avoid inconsistencies induced by the outside world=> no Internet connection, own webserver & assets

WHAT’S OUR APPLICATION ABOUT

A custom web browser

HARDWARE SETUP

© Copyright 2013 3PILLAR GLOBAL All rights reserved 7

BIG PROBLEM #1: HOW TO KNOW HOW MUCH

THE PAGE TOOK TO LOAD

Small solution:

Based on the HTML content, loading time cannot be computed.(element loading order is not consistent across multiple runs)

BIG PROBLEM #2: HOW TO READ SYSTEM

RESOURCE USAGE WITHOUT 3RD PARY TOOLS

Small solution (a):

BIG PROBLEM #2: HOW TO READ SYSTEM

RESOURCE USAGE WITHOUT 3RD PARY TOOLS

Small solution (b)

BIG PROBLEM #3: HOW TO MANIPULATE THE

APPLICATION’S GUI

Small solution:

WHAT DATA WE COLLECT

Test 1• Webpage load time• CPU usage• Memory usage• Disk I/O

Test 2• Timing a „complex” operation

TEST 1:

READING SYSTEM STATISTICS

HOW IS THE DATA COLLECTED

Not at the start of the game

Not at the end of the game

Throughout the entire game

HOW IS THE DATA COLLECTED

RUNNING THE TEST

One test sequence:

• clear cache

• open the browser

• start reading system data

• load webpage

• close the browser

• save result in an Excel spreadsheet

Full test:

• restore the “clean” snapshot of the test environment

• automatically run the test sequence for 1000 times on the

native browser

• automatically run the the test sequence for 1000 times on

the custom browser

RUNNING THE TEST

TEST 2:

TIMING A „COMPLEX” OPERATION

„COMPLEX” OPERATION

Open the browser

Load a webpage

Bookmark page

Go to homepage

Open bookmark

Wait for page to load

RESULTS

RESULTS

CONCLUSIONS

It can be automated with an initial effor, but the returned value is huge.

Offers very concludent results on what needs to be improved in the application to boost performance.

Performance testing is very important as it can reveal performance flaws.

Is very time & effor consuming to do manually.

Q & A

THANK YOU !