Automated Acceptance Testing and Continuous Delivery

Larry Apke Agile Expert

www.agile-doctor.comlarry@agile-doctor.com

• The ability to consistently and predictably deliver high-quality software that the end user can easily understand and use into a production-like environment every iteration.

• Consistent – every iteration• Predictable – over 90% story completion• High Quality – fully tested with no end user

compromising defects• Usable– the end user knows how to, and can,

efficiently use the functionality

• Consistency problems• Predictability problems• Quality problems• Usability problems

• Sometimes described with different terms:• TDD – Test Driven Development• ATDD – Automated Test Driven Development• BDD – Behavioral Driven Development• Specification by Example• AMDD – Agile Model Driven Development• Executable specifications• EDD - Example Driven Development• AAT - Agile Acceptance Testing

• A collaborative approach to defining requirements and business-oriented functional tests for software products based on capturing and illustrating requirements using realistic examples instead of abstract statements. It is applied in the context of Agile software development methods, in particular Behavior driven development. This approach is particularly successful for managing requirements and functional tests on large-scale projects of significant domain and organisational complexity.

Wikipedia.

• Specifications are written in terms of behavior or by using examples• Given…When…Then (Initial Context…Event…

Result)• Referred to as scenarios • Focuses the conversation at the right level

and becomes something that business and developers can understand

• Scenarios become the framework for automated tests

• The tests that are generated/written from scenarios fail when run

• Code is written / refactored until tests pass and it meets code quality standards

• Write the Story

Feature: CalculatorIn order to avoid silly mistakesAs a math idiotI want to be able to add, subtract, multiply and

divide numbers

• Write the Scenarios

Scenario: 1. Add two numbersGiven I have entered +50 into the calculatorAnd I have entered +70 into the calculatorWhen I press calculateThen the result should be 120 on the screen

• Code and Passing the Tests• Code Coverage• Feature Files

• How long does it take for one line of code that is changed to flow through the delivery chain?

• How long should it take? Some companies release to production multiple times per day – Flickr, Facebook, Twitter, LinkedIn – are held up as examples

• Automated Testing• Every code check in is a release candidate• Broken builds are fixed immediately• There is a deployment pipeline that takes

release candidates from one level to the next ie Dev, QA, UAT, Production

• There are different levels of testing along the pipeline based on increasing confidence

• Highly automated – one button deploy

• Always use version control• Check in everything – “have everything that

can possibly change at any point in the life of the project stored in a controlled manner” – Jez Humble – Continuous Delivery

• Check in to trunk• Manage dependencies and configuration

through automated scripts• Build your environments automatically from

scratch

• Specification By Example - How successful teams deliver the right software

• Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation

• The Cucumber Book: Behaviour-Driven Development for Testers and Developers

• Dan North – Introducing BDD - http://dannorth.net/introducing-bdd/

• Fitnesse - http://fitnesse.org/