Succeeding with Behavior Driven Development (BDD) Testing and Automation
description
Transcript of Succeeding with Behavior Driven Development (BDD) Testing and Automation
Google Confidential and Proprietary
Succeeding with Behavior Driven Development (BDD) Testing and Automation
Seattle Area Software Quality Assurance GroupOct 18, 2012
Alan MyrvoldGoogle
Google Confidential and Proprietary
About me
Feb 2011 - now
2005 - 2011
1998 - 2005
1988 - 1998
GoogleTest Engineer, Ads
MicrosoftSDET in Office Security + Outlook
EntrustTest manager, development manager,security assurance manager
CognosTester, developer, test manager,development manager
Google Confidential and Proprietary
More about me
Google Confidential and Proprietary
Buzzword bingo
ATDD
BDD
Cucumber
Cucumber - JVM
Gherkin
Jasmine
SpecFlow
Google Confidential and Proprietary
Buzzword bingo
ATDD - acceptance test driven development
BDD - behavior driven development
Cucumber - a Ruby tool that supports BDD
Cucumber - JVM - a Java tool that supports BDD
Gherkin - the language used by Cucumber
Jasmine - a javascript tool for BDD
SpecFlow - a .NET tool for BDD
Google Confidential and Proprietary
Cucumber Example
Feature: AdditionIn order to avoid silly mistakesAs a math idiotI want to be told the sum of two numbers
Scenario: Add two numbersGiven I have entered 50 into the calculatorAnd I have entered 70 into the calculatorWhen I press addThen the result should be 120 on the screen
Google Confidential and Proprietary
Given / When / Then
Given - precondition
When - action
Then - assertion on expected result
And - same action as before
Google Confidential and Proprietary
Cucumber step implementation
@When("I have entered (.*) into the calculator")public void enterNumber(int number) {
...}
Google Confidential and Proprietary
Scenario Outline ExampleScenario Outline: Add two numbers
Given I have entered <x> into the calculatorAnd I have entered <y> into the calculatorWhen I press addThen the result should be <z> on the screen
Examples:| x | y | z || 2 | 3 | 5 || 0 | -1 | -1 |
Google Confidential and Proprietary
Why BDD?
Clarifying requirements by example
Demystifying automated tests by using English
Demystifying repeated manual tests by emphasizing why and what to verify.
Google Confidential and Proprietary
Some test types
Google Confidential and Proprietary
How I used BDD at Microsoft
Clarifying requirements in my test plan.
Google Confidential and Proprietary
How we are using BDD at Google
My group, DoubleClick Bid Manager, uses BDD for Java API-level system tests and repeated manual tests.
Other groups use BDD tests for Java WebDriver tests.
We share the same framework, developed internally.
Google Confidential and Proprietary
Tools support + Books
Tools:o Cucumber - Ruby http://cukes.info o Cucumber JVM https://github.com/cucumber/cucumber-jvm o SpecFlow - Binding business requirements to .NET code
http://specflow.org
Bookso The Cucumber Booko The RSpec Booko Cucumber Recipies (beta, scheduled 3/7/2013)
All books from Pragmatic Programmers, http://pragprog.com
Google Confidential and Proprietary
When / Then elsewhere
Mockito … a Java unit testing framework@Mock CalculationEngine engine;Calculator calculator = new Calculator(engine);
when(engine.add(2, 2)).thenReturn(4);calculator.parse("2 + 2 =");assertEquals("4", caclulator.getResult());
Compare the syntax to EasyMock:expect(engine.add(2, 2)).andReturn(4);
Google Confidential and Proprietary
describe / it
Rspec … a Ruby unit testing frameworkdescribe Calculator, "#basics" do it "return 4 for 2+2" do calc = Caculator.new calc.add(2, 2) calc.result.should eq(4) endend
Jasmine … a Javascript unit testing frameworkdescribe("calc", function() { it("2+2 is 4", function() { expect(calc(2, 2).toEqual(4); });});
Google Confidential and Proprietary
Failure mode #1 - Too implementation dependent
BAD
Given I have entered 50 into the calculator
And I have entered 70 into the calculator
When I press addThen the result should be
120 on the screen
BETTER
When I add 50 and 70Then the result is 120
Google Confidential and Proprietary
Failure mode #2 - Programmatic Scripts
BAD
When I set x to 1And while x < 50 And set row x to "empty"
BETTER
Then set the first 50 rows to "empty"
Google Confidential and Proprietary
Failure mode #3 - Too low level
BAD
When I go to the login pageAnd enter "bob" into the
username fieldAnd enter "pass123" into
the password fieldAnd click loginThen I am logged in as
"bob"
BETTER
When I log in as "bob"
Google Confidential and Proprietary
Failure mode #4 - Not exploring interesting cases
BAD
When I add 50 and 70Then the result is 120
BETTER
When I add 50 and 70Then the result is 120
When I add 1e90, 0.1, and -1e90
Then the result is 0.1
Google Confidential and Proprietary
Failure mode #5 - Not using regex
BAD
@When("Set the budget to 100")
public void setBudget(){ ...}
BETTER
@When("Set the budget to (.*)")
public void setBudget(int amount)
{ ...}
Google Confidential and Proprietary
Failure mode #6 - Complex parsing logic
BAD
@When("(.*) the (.*) to (.*)")
public void doAction(String action, String name, String value)
{ if (action.equals("set")
&& name.equals("budget") }
BETTER
@When("Set the budget to (.*)")
public void setBudget(int amount)
{ ...}
Google Confidential and Proprietary
Failure mode #7 - Leaking code details
BAD
When I click the BTN-REFRESH-ALL button
BETTER
When I refresh all
Google Confidential and Proprietary
Failure mode #8 - Bad test architecture
BAD
Calling entry points in code that are fragile, or disappear
BETTER
Using supported public or test API's
Google Confidential and Proprietary
Success tips from me
Use BDD for a small set of tests
Focus on human readability, as a domain expert using the feature would describe a test
Google Confidential and Proprietary
Success tips from The Cucumber Book, by Matt Wayne and Aslak Hellesoy
DAMP beats DRYDAMP: Descriptive and meaningful phrasesDRY: Don’t repeat yourself
Declarative better than imperativeDeclarative: Given I am logged inImperative: Log in as user “Bob”