Google Confidential and Proprietary Succeeding with Behavior Driven Development (BDD) Testing and...
-
Upload
clifton-fry -
Category
Documents
-
view
220 -
download
0
Transcript of Google Confidential and Proprietary Succeeding with Behavior Driven Development (BDD) Testing and...
![Page 1: Google Confidential and Proprietary Succeeding with Behavior Driven Development (BDD) Testing and Automation Seattle Area Software Quality Assurance Group.](https://reader035.fdocuments.net/reader035/viewer/2022070411/56649c935503460f9494f02c/html5/thumbnails/1.jpg)
Google Confidential and Proprietary
Succeeding with Behavior Driven Development (BDD) Testing and Automation
Seattle Area Software Quality Assurance GroupOct 18, 2012
Alan MyrvoldGoogle
![Page 2: Google Confidential and Proprietary Succeeding with Behavior Driven Development (BDD) Testing and Automation Seattle Area Software Quality Assurance Group.](https://reader035.fdocuments.net/reader035/viewer/2022070411/56649c935503460f9494f02c/html5/thumbnails/2.jpg)
Google Confidential and Proprietary
About me
Feb 2011 - now
2005 - 2011
1998 - 2005
1988 - 1998
GoogleTest Engineer, Ads
MicrosoftSDET in Office Security + Outlook
EntrustTest manager, development manager,security assurance manager
CognosTester, developer, test manager,development manager
![Page 3: Google Confidential and Proprietary Succeeding with Behavior Driven Development (BDD) Testing and Automation Seattle Area Software Quality Assurance Group.](https://reader035.fdocuments.net/reader035/viewer/2022070411/56649c935503460f9494f02c/html5/thumbnails/3.jpg)
Google Confidential and Proprietary
More about me
![Page 4: Google Confidential and Proprietary Succeeding with Behavior Driven Development (BDD) Testing and Automation Seattle Area Software Quality Assurance Group.](https://reader035.fdocuments.net/reader035/viewer/2022070411/56649c935503460f9494f02c/html5/thumbnails/4.jpg)
Google Confidential and Proprietary
Buzzword bingo
ATDD
BDD
Cucumber
Cucumber - JVM
Gherkin
Jasmine
SpecFlow
![Page 5: Google Confidential and Proprietary Succeeding with Behavior Driven Development (BDD) Testing and Automation Seattle Area Software Quality Assurance Group.](https://reader035.fdocuments.net/reader035/viewer/2022070411/56649c935503460f9494f02c/html5/thumbnails/5.jpg)
Google Confidential and Proprietary
Buzzword bingo
ATDD - acceptance test driven development
BDD - behavior driven development
Cucumber - a Ruby tool that supports BDD
Cucumber - JVM - a Java tool that supports BDD
Gherkin - the language used by Cucumber
Jasmine - a javascript tool for BDD
SpecFlow - a .NET tool for BDD
![Page 6: Google Confidential and Proprietary Succeeding with Behavior Driven Development (BDD) Testing and Automation Seattle Area Software Quality Assurance Group.](https://reader035.fdocuments.net/reader035/viewer/2022070411/56649c935503460f9494f02c/html5/thumbnails/6.jpg)
Google Confidential and Proprietary
Cucumber Example
Feature: AdditionIn order to avoid silly mistakesAs a math idiotI want to be told the sum of two numbers
Scenario: Add two numbersGiven I have entered 50 into the calculatorAnd I have entered 70 into the calculatorWhen I press addThen the result should be 120 on the screen
![Page 7: Google Confidential and Proprietary Succeeding with Behavior Driven Development (BDD) Testing and Automation Seattle Area Software Quality Assurance Group.](https://reader035.fdocuments.net/reader035/viewer/2022070411/56649c935503460f9494f02c/html5/thumbnails/7.jpg)
Google Confidential and Proprietary
Given / When / Then
Given - precondition
When - action
Then - assertion on expected result
And - same action as before
![Page 8: Google Confidential and Proprietary Succeeding with Behavior Driven Development (BDD) Testing and Automation Seattle Area Software Quality Assurance Group.](https://reader035.fdocuments.net/reader035/viewer/2022070411/56649c935503460f9494f02c/html5/thumbnails/8.jpg)
Google Confidential and Proprietary
Cucumber step implementation
@When("I have entered (.*) into the calculator")public void enterNumber(int number) {
...}
![Page 9: Google Confidential and Proprietary Succeeding with Behavior Driven Development (BDD) Testing and Automation Seattle Area Software Quality Assurance Group.](https://reader035.fdocuments.net/reader035/viewer/2022070411/56649c935503460f9494f02c/html5/thumbnails/9.jpg)
Google Confidential and Proprietary
Scenario Outline ExampleScenario Outline: Add two numbers
Given I have entered <x> into the calculatorAnd I have entered <y> into the calculatorWhen I press addThen the result should be <z> on the screen
Examples:| x | y | z || 2 | 3 | 5 || 0 | -1 | -1 |
![Page 10: Google Confidential and Proprietary Succeeding with Behavior Driven Development (BDD) Testing and Automation Seattle Area Software Quality Assurance Group.](https://reader035.fdocuments.net/reader035/viewer/2022070411/56649c935503460f9494f02c/html5/thumbnails/10.jpg)
Google Confidential and Proprietary
Why BDD?
Clarifying requirements by example
Demystifying automated tests by using English
Demystifying repeated manual tests by emphasizing why and what to verify.
![Page 11: Google Confidential and Proprietary Succeeding with Behavior Driven Development (BDD) Testing and Automation Seattle Area Software Quality Assurance Group.](https://reader035.fdocuments.net/reader035/viewer/2022070411/56649c935503460f9494f02c/html5/thumbnails/11.jpg)
Google Confidential and Proprietary
Some test types
![Page 12: Google Confidential and Proprietary Succeeding with Behavior Driven Development (BDD) Testing and Automation Seattle Area Software Quality Assurance Group.](https://reader035.fdocuments.net/reader035/viewer/2022070411/56649c935503460f9494f02c/html5/thumbnails/12.jpg)
Google Confidential and Proprietary
How I used BDD at Microsoft
Clarifying requirements in my test plan.
![Page 13: Google Confidential and Proprietary Succeeding with Behavior Driven Development (BDD) Testing and Automation Seattle Area Software Quality Assurance Group.](https://reader035.fdocuments.net/reader035/viewer/2022070411/56649c935503460f9494f02c/html5/thumbnails/13.jpg)
Google Confidential and Proprietary
How we are using BDD at Google
My group, DoubleClick Bid Manager, uses BDD for Java API-level system tests and repeated manual tests.
Other groups use BDD tests for Java WebDriver tests.
We share the same framework, developed internally.
![Page 14: Google Confidential and Proprietary Succeeding with Behavior Driven Development (BDD) Testing and Automation Seattle Area Software Quality Assurance Group.](https://reader035.fdocuments.net/reader035/viewer/2022070411/56649c935503460f9494f02c/html5/thumbnails/14.jpg)
Google Confidential and Proprietary
Tools support + Books
Tools:o Cucumber - Ruby http://cukes.info o Cucumber JVM https://github.com/cucumber/cucumber-jvm o SpecFlow - Binding business requirements to .NET code
http://specflow.org
Bookso The Cucumber Booko The RSpec Booko Cucumber Recipies (beta, scheduled 3/7/2013)
All books from Pragmatic Programmers, http://pragprog.com
![Page 15: Google Confidential and Proprietary Succeeding with Behavior Driven Development (BDD) Testing and Automation Seattle Area Software Quality Assurance Group.](https://reader035.fdocuments.net/reader035/viewer/2022070411/56649c935503460f9494f02c/html5/thumbnails/15.jpg)
Google Confidential and Proprietary
When / Then elsewhere
Mockito … a Java unit testing framework@Mock CalculationEngine engine;Calculator calculator = new Calculator(engine);
when(engine.add(2, 2)).thenReturn(4);calculator.parse("2 + 2 =");assertEquals("4", caclulator.getResult());
Compare the syntax to EasyMock:expect(engine.add(2, 2)).andReturn(4);
![Page 16: Google Confidential and Proprietary Succeeding with Behavior Driven Development (BDD) Testing and Automation Seattle Area Software Quality Assurance Group.](https://reader035.fdocuments.net/reader035/viewer/2022070411/56649c935503460f9494f02c/html5/thumbnails/16.jpg)
Google Confidential and Proprietary
describe / it
Rspec … a Ruby unit testing frameworkdescribe Calculator, "#basics" do it "return 4 for 2+2" do calc = Caculator.new calc.add(2, 2) calc.result.should eq(4) endend
Jasmine … a Javascript unit testing frameworkdescribe("calc", function() { it("2+2 is 4", function() { expect(calc(2, 2).toEqual(4); });});
![Page 17: Google Confidential and Proprietary Succeeding with Behavior Driven Development (BDD) Testing and Automation Seattle Area Software Quality Assurance Group.](https://reader035.fdocuments.net/reader035/viewer/2022070411/56649c935503460f9494f02c/html5/thumbnails/17.jpg)
Google Confidential and Proprietary
Failure mode #1 - Too implementation dependent
BAD
Given I have entered 50 into the calculator
And I have entered 70 into the calculator
When I press addThen the result should be
120 on the screen
BETTER
When I add 50 and 70Then the result is 120
![Page 18: Google Confidential and Proprietary Succeeding with Behavior Driven Development (BDD) Testing and Automation Seattle Area Software Quality Assurance Group.](https://reader035.fdocuments.net/reader035/viewer/2022070411/56649c935503460f9494f02c/html5/thumbnails/18.jpg)
Google Confidential and Proprietary
Failure mode #2 - Programmatic Scripts
BAD
When I set x to 1And while x < 50 And set row x to "empty"
BETTER
Then set the first 50 rows to "empty"
![Page 19: Google Confidential and Proprietary Succeeding with Behavior Driven Development (BDD) Testing and Automation Seattle Area Software Quality Assurance Group.](https://reader035.fdocuments.net/reader035/viewer/2022070411/56649c935503460f9494f02c/html5/thumbnails/19.jpg)
Google Confidential and Proprietary
Failure mode #3 - Too low level
BAD
When I go to the login pageAnd enter "bob" into the
username fieldAnd enter "pass123" into
the password fieldAnd click loginThen I am logged in as
"bob"
BETTER
When I log in as "bob"
![Page 20: Google Confidential and Proprietary Succeeding with Behavior Driven Development (BDD) Testing and Automation Seattle Area Software Quality Assurance Group.](https://reader035.fdocuments.net/reader035/viewer/2022070411/56649c935503460f9494f02c/html5/thumbnails/20.jpg)
Google Confidential and Proprietary
Failure mode #4 - Not exploring interesting cases
BAD
When I add 50 and 70Then the result is 120
BETTER
When I add 50 and 70Then the result is 120
When I add 1e90, 0.1, and -1e90
Then the result is 0.1
![Page 21: Google Confidential and Proprietary Succeeding with Behavior Driven Development (BDD) Testing and Automation Seattle Area Software Quality Assurance Group.](https://reader035.fdocuments.net/reader035/viewer/2022070411/56649c935503460f9494f02c/html5/thumbnails/21.jpg)
Google Confidential and Proprietary
Failure mode #5 - Not using regex
BAD
@When("Set the budget to 100")
public void setBudget(){ ...}
BETTER
@When("Set the budget to (.*)")
public void setBudget(int amount)
{ ...}
![Page 22: Google Confidential and Proprietary Succeeding with Behavior Driven Development (BDD) Testing and Automation Seattle Area Software Quality Assurance Group.](https://reader035.fdocuments.net/reader035/viewer/2022070411/56649c935503460f9494f02c/html5/thumbnails/22.jpg)
Google Confidential and Proprietary
Failure mode #6 - Complex parsing logic
BAD
@When("(.*) the (.*) to (.*)")
public void doAction(String action, String name, String value)
{ if (action.equals("set")
&& name.equals("budget") }
BETTER
@When("Set the budget to (.*)")
public void setBudget(int amount)
{ ...}
![Page 23: Google Confidential and Proprietary Succeeding with Behavior Driven Development (BDD) Testing and Automation Seattle Area Software Quality Assurance Group.](https://reader035.fdocuments.net/reader035/viewer/2022070411/56649c935503460f9494f02c/html5/thumbnails/23.jpg)
Google Confidential and Proprietary
Failure mode #7 - Leaking code details
BAD
When I click the BTN-REFRESH-ALL button
BETTER
When I refresh all
![Page 24: Google Confidential and Proprietary Succeeding with Behavior Driven Development (BDD) Testing and Automation Seattle Area Software Quality Assurance Group.](https://reader035.fdocuments.net/reader035/viewer/2022070411/56649c935503460f9494f02c/html5/thumbnails/24.jpg)
Google Confidential and Proprietary
Failure mode #8 - Bad test architecture
BAD
Calling entry points in code that are fragile, or disappear
BETTER
Using supported public or test API's
![Page 25: Google Confidential and Proprietary Succeeding with Behavior Driven Development (BDD) Testing and Automation Seattle Area Software Quality Assurance Group.](https://reader035.fdocuments.net/reader035/viewer/2022070411/56649c935503460f9494f02c/html5/thumbnails/25.jpg)
Google Confidential and Proprietary
Success tips from me
Use BDD for a small set of tests
Focus on human readability, as a domain expert using the feature would describe a test
![Page 26: Google Confidential and Proprietary Succeeding with Behavior Driven Development (BDD) Testing and Automation Seattle Area Software Quality Assurance Group.](https://reader035.fdocuments.net/reader035/viewer/2022070411/56649c935503460f9494f02c/html5/thumbnails/26.jpg)
Google Confidential and Proprietary
Success tips from The Cucumber Book, by Matt Wayne and Aslak Hellesoy
DAMP beats DRYDAMP: Descriptive and meaningful phrasesDRY: Don’t repeat yourself
Declarative better than imperativeDeclarative: Given I am logged inImperative: Log in as user “Bob”