Gabriel carabat a healthy approach for test automation

endava.com

QUALITY. PRODUCTIVITY. INNOVATION.

A healthy approach for test automation

2

Agenda •Introduction

•Define test automation objectives

•The framework approach

•Strategies to run automated tests faster

•Reporting for the execution of automated tests

•Debugging automated tests

•Q & A


3

3

Define test automation objectives

What are the testing types that are valuable to be automated?

●Regression testing

●Browser compatibility testing

What should verify a UI automated test?

●The existence of web elements from web pages

●A specific functionality by handling web elements with

●dynamic data

–Ex: create account, login, search user, update user


4

4

Define test automation objectives

What are the main objectives for automated testing?

●To write automated tests fast

●To write automated tests that are easy to maintain

●To run automated tests fast

●To have a report for each execution of the automated tests

●To debug automated tests easy


5

5

The framework approach

Components of a test automation framework


6

6


Base package

●Class that handle initial requirements in order to start browser/setup for the browser

●sets browser name and version

●sets the OS on which the browser will be used

●sets other capabilities:

–Profile of the browser

–Javascript on/off for the browser

–Accept or not certificates

●Quit browser action, close session

●Setup/tear down actions are implemented once and are used in all tests


7

7


Utility package

●Invest more at the beginning => write tests faster after

●Classes with methods to read and write to different types of files

●properties, .xls, .csv, xml,json files

●Classes with methods to check the database

●setup connection with the DB (ex: Oracle, Sybase, MySql)

●generic methods to insert, select and update different tables from DB

●Classes with methods to generate random/unique input data

●Classes with extra methods handle the interaction with browser(Ex: Selenium Helper)


8

8


Pages package


9

9


Pages package


10

10


Pages package

●Page Object pattern

●Each web page should have web elements mapped in a separate class =>if the UI changes, the fix need only be applied in one place.

●Reduces the amount of duplicated code (web elements)

●Enhance with “loadable” components to make sure that your that all web elements are loaded

●Enhance with “populate” methods

●Should respect a naming convention: (Ex. LoginPage, CreateAccountPage)


11

11


Page data package

●For functional tests that are using dynamic data

●Handle easy data – once you set the input data, you can check this data anytime in your test

●Reduces the amount of duplicated code

●Use “getters” and setters to handle data

●Each page object should have a page data

●Should respect a naming convention(Ex: LoginData, CreateAccountData)


12

12


Tests package

●Unit test style - JUnit, TestNG

●BDD style - JBehave

●Will use classes from base, utility, pages and page data packages

●Allows parametrization(“examples” in JBehave) of the tests => reduce code duplication, increase readability

●Allows tests to be organized in suites

●The main difference between JUnit and TestNG/JBehave is that JUnit is using tests in isolation and TestNG allows to make connections between tests(dependency between tests)


13

13

Reporting for the test execution of automated tests

Reporting with Surefire reports

●Can integrate results from JUnit, TestNG, JBehave tests

●Test classes, test methods, pass, failed, skipped(TestNG and JBehave)

●If a test is failed there's a link to the line of source code for the test

●Time for individual test methods, test classes and overall test suite are listed

●Can be integrated with CI tools (Ex: Jenkins)


14

14

Strategies to run tests faster

Parallelization of your tests

●Use multi-thread tests

●Tests can be run in parallel: test methods and test classes

●Make sure that the tests that you want to run are independent

●Major advantage - decrease time of running test=> early feedback

●Need to consider - a lot of RAM memory will be used

–use of virtual machines on the same machine will not increase the speed of your tests

–if you want to use other machines to increase the memory, make sure that the prerequisites to setup is minimum or none(Selenium Grid vs Jenkins)


15

15


Build test suites wisely

●Different test suites/test packs

●Sanity checks, Smoke tests, Regression tests

●Sanity checks - identify those tests and make a package to check those cases based on the previous experiences

●Schedule tests in cascade – A(pass - go to B, fail – skip all tests), B,C

=> early feedback in case of failure


16

16


Use cloud solutions

●Providers: Saucelabs, Browserstack, Testing Bot

●Less of configuration - setup of virtual machines and devices is taking a few seconds

●Browser compatibility

●Run tests on your local computer, on a computer in your LAN, on a server via VPN

●Mobile browsers - test across official iOS and Android mobile emulators.


17

17

Debugging automated tests

Common errors

●“Element not found” failure - when an element was not found

●Can appear in any step of a test

●Failure at verification level

●Appears only when the expected and the actual results are different

●Other errors – DB connectivity and browser setup

●Errors that occur when using files


18

18


Investigation

●For “Element not found” failure – check the code line in the test to see which is the element that was not found

●Check the page source to see if the element is present (can be done manually, and automatically)

●For verification failures you need only to interpret what caused the failure of the verification

●Adding logs to your steps to see exact data that was used and all the steps performed (LOG4J)

●In case of tests written using BDD style(Given, when, then) you can see the exact step that was failed


19

19


Investigation using screenshots

●Different implementation depending of testing framework used

●The screenshots should be saved in a separate directory and the name of them should follow some conventions:

●Name of the test should contain name of the test class concatenated to a timestamp (if you ran multiple times same test, to be able to have all screenshots, not to override the same screen )

●If it's a crash it's easy to debug the test only by seeing the screenshot

●Only at failure, to avoid checking every screenshot

●Cloud solutions provide videos of your tests


20

Thank you


Gabriel CărăbățTest Automation Lead/CLD Test Automation Practice Lead

[email protected]

+40 763 982 657

en_gcarabat

Gabriel carabat a healthy approach for test automation

Technology

Transcript of Gabriel carabat a healthy approach for test automation