Automated Web UI Testing on Steroids - sdm...

Automated Web UI Testing on Steroids

Angel Todorov

www.infragistics.com www.istabg.org


About Infragistics

Worldwide leader in user experience controls

and components targeting jQuery & HTML5,

Windows Forms, Windows Phone, ASP.NET,

Silverlight, WPF, SharePoint, iOS, Android,

Windows UI

Focus on high performance

scenarios for line of business

and data visualizations

Contents

• Background & Motivation

• Good Tests

• Improving JavaScript unit

testing

• Improving scenario testing

• Other Behavior-Driven

Development applications


Background & Motivation

• Execute all unit tests as part of every CI build

• Execute all integration scenarios as part of every

nightly build

• Be able to quickly automate submitted bugs

• Limit misunderstanding between groups and promote

collaboration – everyone contributes to testing


Run tests before check-in – only the

feature

Run tests on CI build trigger – single browser

Run all tests on a nightly build trigger

– all browsers

No single solution !

What makes a good test?

• Easy to write

• Very fast to execute – avg. couple of seconds

• Reliable

• Customer-focused

• Valuable (effective)

• Maintainable

• Part of CI


JavaScript unit testing

• QUnit

• PhantomJS/Awesomium & custom runner script

• Integrates nicely with CI builds

• No interference on the developer machine

• Headless = layout engine + JavaScript engine


Scenario testing: Common challenges

• Test Beds take a long time to create

• Achieving good reuse of test code is not

straightforward (copy / paste)

• Hard to investigate / debug failing tests

Wait a sec…

Writing tests

should be

fun !


Scenario testing: Common challenges – Cont’d

• Different documents not always in sync & up to date

(test code, test plans, etc.)

• Gap between stakeholders & engineers

• Tests are hard to understand and validate by non-

engineers (designers, user experience guys)


Cucumber & BDD

• Behavior driven-development: specialized version of

Test-Driven-Development

• Cucumber: A Ruby tool for running automated

acceptance tests written in a BDD style

• Various implementations for other platforms –

example: SpecFlow (.NET)

• Scenarios grouped in feature files in the Gherkin

format:

Given / When / Then Steps describe a scenario


Cucumber & BDD – example


…Which maps to:


Cucumber & BDD – analyze results (demo)


IG BDD Test Framework

• Ability to generate test beds (HTML & MVC) based

on Given’s

• Out of the box reuse – every step is independent

• Built-in Visual Studio Intellisense:


IG BDD Test Framework – Cont’d

• Feature files => Actual test plan

• Easy for PMs and UX to follow the interactions,

expressed in Gherkin

• Tags and templates bring a lot of semantics (Demo)


IG BDD Test Framework – Cont’d

Test project (component under test)

e.g. Grid Column Moving

IG BDD Test Framework

.feature files

written in Gherkin

(Step Definition

implementations,

templates, tags &

scenario lifecycle

management)

SpecFlow.NET / MSTest / Selenium

WebDriver API

Low-level APIs for

verifications, Assertions,

test-running & reporting


Other Applications of BDD

• Puppet setup:

1 Master, 1 slave running CentOS on Oracle VirtualBox

@generate-puppet-manifest

Scenario: Ensure Apache is provisioned on nodes

Given I enable Apache in the main manifest

When I update my puppet slave nodes

When Changes take effect

Then Apache should be running on my slave nodes

on port 80


Resources

• Selenium Web Driver:

http://seleniumhq.org/docs/03_webdriver.html

• Cucumber: http://cukes.info/

• SpecFlow: http://www.specflow.org/specflownew/

• GhostDriver: https://github.com/detro/ghostdriver

• PhantomJS: http://phantomjs.org

• Qunit: http://qunitjs.com/

• Awesomium: http://awesomium.net/

• Puppet: http://puppetlabs.com/


http://seleniumhq.org/docs/03_webdriver.html

http://cukes.info/

http://www.specflow.org/specflownew/

https://github.com/detro/ghostdriver

http://phantomjs.org/

http://qunitjs.com/

http://awesomium.net/

http://puppetlabs.com/

Angel Todorov Principal Architect

e-mail:

[email protected]


Thank you!


Q & A


Automated Web UI Testing on Steroids - sdm...

Documents

Transcript of Automated Web UI Testing on Steroids - sdm...