Story-Based Test Automation Using Free Tools @paul_gerrard Paul Gerrard [email protected]...

Story-Based Test Automation Using Free Tools

@paul_g

erra

rd

Paul [email protected]

gerrardconsulting.com

Overview

• Behaviour-Driven Development and similar approaches are still developer-focused– Don’t engage testers as they should– Don’t support higher complexity requirements– Don’t scale to a larger requirements bases

• In this session, we’ll cover:– Acceptance-driven development with free tools– How Story Platform for QA (a sub-set of

Business Story Manager) and Robot Framework can be used to implement this approach.

Intelligent Definition and Assurance Slide 2

Background

• We won’t go into much detail of Behaviour-Driven Development– http://

en.wikipedia.org/wiki/Behavior-driven_development for an intro

• Requirements-test hierarchies:– Go to http://businessstorymanager.com– and watch the videos.


http://en.wikipedia.org/wiki/Behavior-driven_development



http://businessstorymanager.com/

Behaviour-Driven Development using current tools


Story (structured)

Generated Test Code

BDD Tool

Test Fixtures/Test Step Defs.

+

Write the Code + Refactor the

Code

Test Outcomes

Test Toolexecutes tests

System Under Test

BDD and tool support

• A big benefit to developers is that BDD makes the Test-Driven Development approach easier

• Behaviour-Driven Development:– A collaborative approach to specification– Uses Domain–Specific Language in

specifications– Implements Ubiquitous Language

• Currently, tools only support the automated testing part of the BDD regime.


Story Platform QA (BSM subset) Robot Framework


Requirements

Story (structured) Test Cases

Test Export

Keyword Definitions

+

Write the Code+ Refactor the

Code

Test Librarye.g. Webdriverexecutes tests

Test StatusRedAmberGreen

Dictionary

System Under Test

Test Outcomes

Business Story Manager

Business Story Manager and Test Frameworks

Requirement

STORY: Feature Header

Business ScenarioGiven “Search page”When “I find a book” And “I add to basket”Then “basket has my book”

Just enough to clarify understanding

Test ScenarioPre-conditions “Search page”Steps “enter search text”

“click search button”“scroll down to find book”“select book”“view more information to

verify”“select add to basket”

Post Conditions/outcomes “Basket has selected book”

Generated

Story BasedTest Cases

Example: Robot FrameworkTestLog

Selenium (Web) HTTP (web) SSH DB Library Etc.

Manually maintainedKeyword

Definitions 1 per Feature

Keyword Definitions

1 per application

Technical CodePer Environment

TestConfig.

Drivers

System(s) Under Test

How Does it Work?

A walkthrough of the process

using http://sp.qa

http://sp.qa/

The detail for the requirements selected on the left is shown

here.


The feature identifies a capability of the system

described in the requirement


The scenario provides an illustration of a feature in use.

Examples provide broader coverage of the feature.

This test scenario has been edited to provide more detail. These are the automated test

keywords.

NB: The colour coding highlights terms defined in the

dictionary, and data items in scenarios. (The dictionary is available in the full product).

The Lucky Numbers Application


The numbers are submitted and validated and then the user is

asked to confirm the ‘play’.

Lucky numbers features:• Registration• Submit ticket• View my Game Plays• System Status• Change my profile• Reset Password• etc.

Exporting Test Scenarios

We’ve shown just one test scenario here for illustration.

Normally, all test scenarios are exported.


Dropbox asks for confirmation that you want to post the test

case files to your account.


BSM lists all the files that have been posted to your Dropbox

account.

Generated files and resource files


The files generated by BSM have been synchronised with

your Dropbox account

These resource files correspond to your generated test case

files. You maintain your keyword definitions here.

Robot script and resource file


The Robot Script files with test cases are generated by Business

Story Manager

The tester maintains the resource file with the keyword

definitions.

Tests running on a Linux VM that shares Dropbox

System Under Test

Terminal window driving the tests

…and the same test on Windows 7


Results upload service


Use a batch file or shell script to run the tests, then upload the

results to BSM/SP.QA

loadRobotOutput.py Python uploads the output file

Business Story Manager Robot Framework


Requirements

Story (structured) Test Cases

Test Export

Keyword Definitions

+

Write the Code+ Refactor the

Code

Test Librarye.g. Webdriverexecutes tests

Test StatusRedAmberGreen

Dictionary

System Under Test

Test Outcomes

Behaviour-Driven Tooling

• The essential architecture of BDD tools is that1. A tool parses stories and generates test code2. The test code can be used for TDD3. The developer (almost always) writes the step

definitions/fixture code and runs the tests4. The stories, test code and steps are all source-

controlled (in the developers’ library)

• What we have done with SP.QA and Robot is allow non-techies to own this process and align it with stakeholder needs properly.


Summary

• SPQA and Robot Framework combined:– Support requirements of any complexity– Implement keyword-driven test automation– Manage requirements, stories and tests to any

scale

• Acceptance-driven approach:– Traceable from requirements to tests and back– Automated story acceptance and regression

testing– Build automated end-to-end acceptance tests

with reusable keywords.


http://sp.qa

Story Platform for QA

SP.QA provides a subset of Business Story Manager

functionality and is FREE-TO-USE.

http://sp.qa/

Visit SP.QA to access the free Story Platform for QA web

service.

Go to the Image Gallery to see the requirements-test process

in pictures

SP.QA Scope

• What you get– Requirements module– Business Stories and Scenarios– Test Scenarios– Robot Framework Interface

• FREE Service Limits– Limited to 100 business stories – 5 users, 1 application, 1 project

Intelligent Definition and Assurance

BSM Scope

• What you get– Stakeholder and Risk Management modules– Lean Task Management module– Dictionary and Library modules– Role-Based Access Control– All SP.QA Scope

• Limitations– None

• Commercials– £30 per user per month

Slide 34

Story-Based Test Automation Using Free Tools

@paul_g

erra

rd

Paul [email protected]

gerrardconsulting.com

Story-Based Test Automation Using Free Tools @paul_gerrard Paul Gerrard [email protected]...

Documents

Transcript of Story-Based Test Automation Using Free Tools @paul_gerrard Paul Gerrard [email protected]...