Effective Testing Practices in an Agile Environment
-
Upload
raj-indugula -
Category
Technology
-
view
218 -
download
1
description
Transcript of Effective Testing Practices in an Agile Environment
![Page 1: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/1.jpg)
Effective Testing Practices in an Agile Environment
Raj Indugula [email protected]
www.lithespeed.com
![Page 2: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/2.jpg)
ON-‐DECK AT-‐BAT DONE
What is the problem context?
Can I see it in ac8on?
Concluding thoughts. Any ques8ons?
What are the guiding principles of Agile Tes8ng ?
What are the enabling prac8ces?
![Page 3: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/3.jpg)
Agile Manifesto
Individuals and interactions Processes and tools over
Working software Comprehensive documentation
over
Customer collaboration Contract negotiation over
Responding to change Following a plan over
That is, while there is value in the items on the right, we value the items on the left more.
We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:
http://www.agilemanifesto.org
3
![Page 4: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/4.jpg)
Shared Context drives Quality
Individuals and interactions
Working software
Customer collaboration
Responding to change
4
![Page 5: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/5.jpg)
Quality is a priority
Agile Approach
Maximize value and quality within specified
project constraints.
5
ROI
Customer Value
Sustainability
Quality
5
![Page 6: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/6.jpg)
Integrated Teams & Iterative Delivery Challenges
6
How do we bridge the communication gap between Bus./QA/Dev? How do we ensure that the evolving software does not regress?
![Page 7: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/7.jpg)
ON-‐DECK AT-‐BAT DONE
What is the problem context?
Can I see it in ac8on?
Concluding thoughts. Any ques8ons?
What are the guiding principles of Agile Tes8ng ?
What are the enabling prac8ces?
![Page 8: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/8.jpg)
Move Quality Upstream
• Testing not a trailing activity
• QA leads the Sprint by providing guidance and feedback on the business needs of the emerging product
8
![Page 9: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/9.jpg)
Testing is Continuous, NOT a Phase
http://testobsessed.com/
9
![Page 10: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/10.jpg)
Testing is Collaborative
• Quality is everyone’s problem, not just of the testers
• Testing is the responsibility of the whole
team
10
![Page 11: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/11.jpg)
Tests represent Customer Expectations
• Shared understanding of what it means for a story to be done
11
![Page 12: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/12.jpg)
Quick Feedback
• Faster feedback loops increase Agility – the ability to respond to change
• Test automation provides rapid feedback on how the software is behaving
12
![Page 13: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/13.jpg)
“Leave No Broken Windows”
• Fix bugs as they are found
• The sooner you find a defect, the cheaper it is to fix
13
![Page 14: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/14.jpg)
It isn't Done until it’s…
“Done Done”
Coded Tested Done
14
![Page 15: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/15.jpg)
Test-Driven
Defining tests with the requirements guides development
http://testobsessed.com/
15
![Page 16: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/16.jpg)
ON-‐DECK AT-‐BAT DONE
What is the problem context?
Can I see it in ac8on?
Concluding thoughts. Any ques8ons?
What are the guiding principles of Agile Tes8ng ?
What are the enabling prac8ces?
ATDD
TDD
Other prac8ces
![Page 17: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/17.jpg)
Agile Engineering Practices
17
Agile Engineering practices allow teams to move fast, be flexible and deliver high quality software
Bill Wake, http://www.xp123.com
![Page 18: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/18.jpg)
What are acceptance tests?
• Tests that demonstrate business intent of
system from end user’s point of view
• Black-box testing
18
![Page 19: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/19.jpg)
What is Acceptance Test Driven Development (ATDD)?
A practice in which the whole team collaboratively discusses acceptance criteria, with examples, and then distills them into a set of concrete acceptance tests before development begins. - Elisabeth Hendrickson
19
![Page 20: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/20.jpg)
Key Characteristics
1. Elicit details from the business stakeholder(s) about their expectations
2. Distill acceptance criteria into automatable tests expressed in a natural language
3. Wire the tests to SUT with “glue” code (as part of implementation)
20
![Page 21: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/21.jpg)
21
![Page 22: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/22.jpg)
When to Use
• During Requirements Elaboration “First Cut” once the story is written Card level acceptance criteria
• During a Three Amigos session (Business, QA, and
Dev) Expand the story card acceptance criteria Complete set of test scenarios
22
![Page 23: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/23.jpg)
User Story
As a <type of user>, I can <immediate goal> so that
<business outcome>.
User Role (Who?)
Desired Function (What?) End Result (Why?)
23 3 C’s – CARD, CONVERSATION, CONFIRMATION
![Page 24: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/24.jpg)
Modern Agile Acceptance Model
Conditions of Satisfaction – Broad Terms
Acceptance Criteria – Further Refined
Examples – Actual scenarios or data
Executable Examples – Ready to automate
24
![Page 25: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/25.jpg)
Acceptance – Conditions of Satisfaction
Testing a Registration Function • What constraints should we impose? • Business stakeholders and PO agree that
passwords should not be easy to crack.
Please fill in to register.
Email Address
Password
Register
25
![Page 26: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/26.jpg)
Acceptance – Acceptance Criteria
Acceptance Criteria, the Details The PO works with testers and developers from the team, business stakeholders, users or SMEs to come up with this definition of “not easy to crack”: • Must be at least 8 characters and no more than 12 • Must contain only alphanumerics and the period • Must contain at least one digit • Must contain at least one alpha character
Please fill in to register.
Email Address
Password ????????????
26
![Page 27: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/27.jpg)
Acceptance – Examples Written • Examples, Making it Real • Actual examples are the ultimate way to communicate
requirements
Password Expected Expected Message Comment
abc123 Invalid You password must be at least 8 characters long, and no more than 12 characters long.
abcdefghi Invalid Your password must contain at least one character and one number.
1aaaaaaaaa Valid Why valid?
ajx972dab Valid
27
![Page 28: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/28.jpg)
Acceptance – Testable Examples in Gherkin • Executable Example, Making it Repeatable • Examples that can be executed are the final step
Given the “Unregistered User” user has navigated to the “register” page When entering “newuser” in the “Username” field And entering “abc123” in the “Password” field And entering “abc123” in the “Confirm Password” field And pressing the “Register” button Then the text “Thank you for Registering” should appear on page And the URL should end with “use/accountPage”
28
![Page 29: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/29.jpg)
Gherkin – Another Example
• Plain English with a specific format § Feature § Scenario § Given When Then And But
• Can be used for both manual testing and automation
29
![Page 30: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/30.jpg)
Automated Acceptance Tests
Tool (Cucumber, SpecFlow, FitNesse)
Specification expressed in
common language
“Glue” code that ties specification
to SUT
30
![Page 31: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/31.jpg)
Business Analyst Develop usage scenarios
Developer
Create and maintain acceptance test “glue” code that ties test specification to system under test
Tester Create and execute tests to simulate usage scenarios; Automate regression testing
• Should be an integral part of every iteration/sprint, not just a trailing activity
• Vital not only to prove completeness of a feature in repeatable fashion, but also to prove that software does not regress as it evolves
• Success depends on cooperation & collaboration
Testing Collaboratively
31
![Page 32: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/32.jpg)
Benefits of ATDD
• Improved requirements elicitation
• Consensus between BA/QA/Dev on the story helps prevent bugs & gives clear target for development
• Reuse of Acceptance Scenarios for all phases of testing
• Creates clear examples that everyone understands; discovers some problems before any development
32
![Page 33: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/33.jpg)
ON-‐DECK AT-‐BAT DONE
What is the problem context?
Can I see it in ac8on?
Concluding thoughts. Any ques8ons?
What are the guiding principles of Agile Tes8ng ?
What are the enabling prac8ces?
ATDD
TDD
Other prac8ces
![Page 34: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/34.jpg)
What are unit tests?
• Developer tests that determine whether the smallest piece of testable software in an application is behaving as expected
• White-box testing
34
![Page 35: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/35.jpg)
Structure of a Unit Test
@Test!
public void itAddsOnePlayerToTheGame()!
{!
!Game game = new Game();!
!
!Player al = game.AddPlayer("Al");!
!
!Assert.assertEquals(1, game.numPlayers());!
!Assert.assertTrue(game.isPlaying(al));!
!Assert.assertFalse(game.isPlayable());!
}!
Assert
Arrange
Act
35
…all related to same behavior
…title is a behavior
![Page 36: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/36.jpg)
Tests should…
• Not depend on one another • Not affect one another • Leave the system under test unaltered • Be fast executing • Be executable in any order, number or
combination of one another
36
![Page 37: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/37.jpg)
What is Test Driven Development (TDD)?
1. Write a unit test 1. Use unit testing and mocking framework (JUnit, Mockito etc.)
2. May need to write some skeleton code to make it compile
2. Run it and watch it fail (no code yet)
3. Write code until it passes
4. Repeat
37
Write a new test
Red (Failing Test)
Write Code
Green (Test Passes)
The tests help determine what code you write (“Incremental Design”)
![Page 38: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/38.jpg)
Write a new test Test fails
Integrate Write code
Refactor code, make sure tests pass
Test passes
Developer heartbeat
Red, Green, Refactor
Refactoring is supported by the tests and good unit test coverage makes refactoring safe to do
38
![Page 39: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/39.jpg)
TDD – Benefits
• Better design – less coupling
• Supports change
• Prevents gold-plating
• Prevent bugs
• Discover bugs
39
![Page 40: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/40.jpg)
Test Levels
Isolation Tests
• True unit tests, focusing on behaviors
• Many, and fast
Collaboration Tests
• “End-to-end” tests
• Find bugs in conversations between collaborating modules
40
![Page 41: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/41.jpg)
Test Doubles (Mocks)
Used to isolate unit tests form external dependencies
Why use them?
• Ease of setup • Fast executing • To work with a subsystem that doesn’t exist • Simulate various execution paths of external system
41
![Page 42: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/42.jpg)
Test Automation Pyramid
42
Unit Tests/Component Tests
Cucumber, FitNesse, SpecFlow
xUnit, Mocks
Selenium
Developer-centric (Are we building the code right?)
Business-centric (Are we building the right code?)
Adaptation of Mike Cohn's test automation pyramid
Exploratory Testing
GUI Tests
Acceptance Tests
![Page 43: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/43.jpg)
ATDD/TDD Cycle
43
![Page 44: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/44.jpg)
TDD vs. ATDD Historically ATDD did not come about till well after TDD
ATTD § is a specification and testing practice § Determines what “Done” means for a story § Creates a common understanding and shared language
to discuss features (especially with Gherkin) § Cycle times on the order of hours to days
TDD § is a coding and design practice § Provides a mechanism for exploring the solution space § Creates base of tests to support refactoring § Cycle times on the order of seconds to minutes
44
![Page 45: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/45.jpg)
ON-‐DECK AT-‐BAT DONE
What is the problem context?
Can I see it in ac8on?
Concluding thoughts. Any ques8ons?
What are the guiding principles of Agile Tes8ng ?
What are the enabling prac8ces?
ATDD
TDD
Other prac8ces
![Page 46: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/46.jpg)
Version control Tests with Code
Source Code
Unit/Component Tests Acceptance Tests
Source Control Repository
(Git, Subversion…)
46
![Page 47: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/47.jpg)
Requirements gathering Application Development
Source Control
Continuous Integration (CI) Server
Continuously commit changes
and merge changes from others
Pull changes, Build, run tests
Deploy to higher environment
Poll for changes
Development tasks from requirements
test scenarios from business requirements for acceptance criteria
Acceptance Test Environment - Test harness - Tests (automated + manual)
Execute acceptance tests against the deployed application
QA Environment
Deployable artifacts
Build Automation & Continuous Integration
47
![Page 48: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/48.jpg)
ON-‐DECK AT-‐BAT DONE
What is the problem context?
Can I see it in ac8on?
Concluding thoughts. Any ques8ons?
What are the guiding principles of Agile Tes8ng ?
What are the enabling prac8ces?
![Page 49: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/49.jpg)
Barriers to Adoption
Developer : “I’m a developer, not a tester” Management: “Why does Dev help with testing, when we have QA?”
• Necessitates behavioral change • Quality is not free • Simple, but not easy • Requires discipline • Needs generalizing specialists
49
![Page 50: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/50.jpg)
Bridging the Divide
Shared Understanding
Collaborative Testing Quick Feedback
50
![Page 51: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/51.jpg)
“Before beginning a hunt, it is wise to ask someone what you are looking for before you begin looking for it.” —Winnie the Pooh
51
![Page 52: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/52.jpg)
Thanks!
52
![Page 53: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/53.jpg)
Resources
53
![Page 54: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/54.jpg)
Some Tools (open source)
• Unit testing: JUnit, Mockito, EasyMock
• Acceptance testing: FitNesse Cucumber
• GUI testing: Selenium, Sahi
• Javascript testing: Jasmine, Qunit
• Testing the database: DBUnit, SQLUnit, DBFit
• Performance testing: JUnitPerf, JMeter
• Usability testing: Manual unfortunately
54
![Page 55: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/55.jpg)
Some Tools (open source)
• Unit testing: MSTest, NUnit, Moq
• Acceptance testing: SpecFlow
• GUI testing: Selenium
• Javascript testing: Chutzpa
• Testing the database: Tsqlt, SSDT SQL Test
• Performance testing: JetBrains, VisualStudio 2013, SQLServerProfiler, SQLLoadTest
• Usability testing: Manual unfortunately
55
![Page 56: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/56.jpg)
Agile Software Development
• Extreme Programming Explained - Kent Beck • TDD: A Practical Guide – Dave Astels • Refactoring – Martin Fowler • Agile Estimating and Planning – Mike Cohn • Pragmatic Project Automation – Pragmatic Prog.
56
![Page 57: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/57.jpg)
Acceptance Testing
• Bridging the Communications Gap – Gojko Adzic • Agile Testing – Lisa Crispin and Janet Gregory • The Cucumber Book - Wynne M. and Hellesoy A. Gherkin
https://github.com/aslakhellesoy/cucumber/wiki/gherkin Cucumber
http://cukes.info/ https://github.com/aslakhellesoy/cucumber/wiki/
57
![Page 58: Effective Testing Practices in an Agile Environment](https://reader030.fdocuments.net/reader030/viewer/2022020105/548db799b47959f2248b4a51/html5/thumbnails/58.jpg)
TDD/Refactoring
• TDD: By Example - Kent Beck • TDD: A Practical Guide – Dave Astels § The Art of Agile Development – James Shore • Refactoring – Martin Fowler • Working Effectively with Legacy Code – Mike Feathers • Refactoring to Patterns – Joshua Kerievsky • xprogramming.com, refactoring.com, testdriven.com
58