Automated testingAistė Stikliūtė @ Vilnius Girls Code 2015-06-17

Me

● Career in software testing

● Testing classes

● Me in the community

Today - Test automation

● Why● What● How (high level + examples)

○ UI (Selenium)○ API (SpecFlow + RestSharp)

● TDD / BDD

Why automate testing?

Can testing be automated?

James Bach:there’s no automated testing, there’s automated checking!

Does it really pay off?● New / legacy system?● Waterfall / Agile?● How difficult to automate?● Team’s discipline?● Personal motivation?● Continuous delivery?

Continuous delivery

software engineering approach -

keep producing valuable softwarethat can be

reliably released anytime

Continuous D. benefits● Accelerated Time to Market

● Building the Right Product

● Improved Productivity and Efficiency

● Reliable Releases

● Improved Product Quality

● Improved Customer Satisfaction

Personal motivationIn between testing and development...

Domain expert

Enjoys coding

Problem solverProblem finder

Usability expert

Has technical skillsHacker

Automated test types

● Functional tests:○ Unit tests○ Integration tests○ End-to-end tests

● Performance tests● Security tests● Usability

What tests can be automated?

* to certain extent!

What cannot / should not?● The “first look”● Functional suitability● Some security tests● Some operations tests● Real user experience tests● Very alternative scenarios (?)

Which are most important?The ones you will use!

● My bet: some of the functional tests

● Unit tests - fastest, find fewest bugs● End-to-end - slowest, find bugs● Integration - in the middle

How to ....automated tests?

Rule #1 - make them visible● Dashboards● Notifications● Attract attention:

○ QAs○ Developers○ Management

● Each commit?● Nightly?● After deployment to different environment?

○ Even PROD (special suite)?

Rule #2 - run tests often

Look ASAP: are they green?

Rule #3 - keep them green● Fix system / tests immediately

○ Unstable tests - (almost) worthless■ Change / refactor tests■ Maybe change the system?■ If nothing else works - automate retrying

○ 1 test keeps failing - test suite (almost) worthless■ Tmp commenting / ignoring may be OK

Rule #4 - include tests in DoD

● Feature not finisheduntil it has automated tests

● Automated tests - team’s responsibility

* DoD - Definition of Done

Rule #5 - automate bugfix tests

I’m baaack! :)

Coverage… ?

* Rodin's The Thinker

GUI / end-to-end tests with Selenium

Selenium

IDE vs. WebDriver

Script ←→ Browser ←→ Click, type, drag&drop, etc.

GUI / end-to-end ?

Selenium WebDriverIWebDriver driver = new FirefoxDriver();

driver.Navigate().GoToUrl("http://www.google.com/");

IWebElement query = driver.FindElement(By.Name("q"));

query.SendKeys("Cheese");

query.Submit();

WebDriverWait wait = new WebDriverWait(driver, TimeSpan.FromSeconds(10));wait.Until( (d) => { return d.Title.ToLower().StartsWith("cheese"); });

driver.Quit();

Page Object patternpublic class LoginPage { public HomePage loginAs(String username, String password) { // ... clever magic happens here } public LoginPage loginAsExpectingError(String username, String password) { // ... failed login here } public String getErrorMessage() { // So we can verify that the correct error is shown }}

API tests with SpecFlow & RestSharp

What is REST API and why test it?● Web architectural style with a set of constraints● Web service APIs that adhere to the constraints - RESTful

Rest API: typically used HTTP methods

Resource GET POST PUT DELETE

Collection URI, such as http://api.example.com/books/

List collection members

Create new entry in the collection

Replace the entire collection

Delete the entire collection

Element URI, such as http://api.example.com/books/book17

Retrieve the member of the collection

Not generally used

Replace the member of the collection

Delete the member of the collection.

SpecFlow. Gherkin language

GIVEN book with ISBN “1-84356-028-3” is in the system

AND it’s available quantity is 9

WHEN I add a book with ISBN “1-84356-028-3”

THEN the book is successfully added to the list

AND available qty. for book with ISBN “1-84356-028-3” is 10

SpecFlow: scenario implementation

SpecFlow: running tests

Test / Business Driven Development

Test-first automation approachTDD:● Developer writes tests, then writes code until tests pass

BDD:● PO / QA / team write tests● Developer implements tests● Developer writes code until tests pass

That’s it!For starters :)