Post on 22-Jan-2018
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
Test Improvement 4 Agile
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
The Evolution of IT and Testing
Pioneering
Structuring
Op>mizing
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Prototyping Waterfall Agile/Scrum V-‐model Spiral model DEVOPS Itera>ve RAD TDD ATDD BDD W-‐model XP Con>nuous Integra>on/Delivery MBT Exploratory CDT RST RSTM TMap ISEB ISTQB TMap-‐Next Packages SOA Devices Social media Virtualiza>on Cloud Mobile Internet of Things Web Localiza>on Big Data
The Evolution of IT and Testing
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Manifesto for Agile Software Development
We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:
Individuals and interactions over processes and tools Working software over comprehensive documentation
Customer collaboration over contract negotiation Responding to change over following a plan
That is, while there is value in the items on
the right, we value the items on the left more.
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Test Improvement 4 Agile roadmap
Quick wins, good
prac>ces Goals, scope
Interviews, mee>ngs
Assessment
Awareness, commitment,
buy-‐in Implement and evaluate
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Test Improvement 4 Agile roadmap
Cost
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Agile testing maturity levels Forming Norming Performing
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Agile testing maturity levels Forming Norming Performing
Set the basis and the first steps towards working in
an Agile manner
Adopt a process that facilitates the Agile view
on working
Continuously improve the way
you work by living the Agile
way
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Assessment model
Key area Forming Norming Performing 1 Key area 1 1 2 3 4 1 2 3 4 1 2 3 4
2 Key area 2 1 2 3 4 1 2 3 4 1 2 3 4
3 Key area 3 1 2 3 4 1 2 3 4 1 2 3
… …… … … … … … … … … … … … …
n Key area n 1 2 3 1 2 3 1 2 3 4
• Where am I? • Where do I want to be? • How do I get there?
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Test process
• What and when to test? • Finish what you start • Fully integrated • Everybody tests
Forming Norming Performing
TI4Agile
Risk based testing
Testing in and over sprints
Everybody tests
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Keys to success
• Automated tests: unit ß à acceptance • Non-functional testing • Test data • Regression testing • Fit for purpose & context
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Test management
• People management • Generic test approach / strategy • Risk analysis • Release planning • Birds eye view:
Cross teams – sprints – projects
Forming Norming Performing
TI4Agile
Generic process
Support the process
Support the people
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Generic test approach / strategy
Keys to success
Risks Maturity
Skills Product
Size Time
Culture
Ban
dwid
th
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
The wheel of risk
Core
Layer 1
Layer 2
Core
Layer 1
Layer 2
Core
Layer 1
Layer 2
Core
Core
Core
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Planning & Estimation
• Work breakdown & plan for all activities – Preparation, Review, Execution, Retest, Unit test, Bug
fixing • It is a team effort!
Forming Norming Performing
TI4Agile
Know what to plan and estimate
Measurable, small parts
Time is fixed, scope is flexible
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
System
Item Item Item Item
Item Item Item Item
Item Item Item Item
Keys to success
13 8
20
13
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Defect management
• Use one system for all defects and ensure traceability • Decide what and when to log
Forming Norming Performing
TI4Agile
Defects are logged
Not all defects are logged
Focus on working software
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Keys to success
• Take into account: – Co-location – Maturity of team – Number of defects – Organizational influence – Drive for metrics – Size of development – Complexity of development
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Test environment
• Self controlled • Production like • Version controlled • Continuous integration
Forming Norming Performing
TI4Agile
There is an environment
The team is stakeholder
The environment is
flexible
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Keys to success
Virtualization Version control
1
4
9
2
3
6
7
5
8
T1
T2 10
Tags
Trunks
Branches Merges
Discontinued development
branch
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Test automation
• At every level • Important part of the sprint • Risk based • Maintainable
Forming Norming Performing
TI4Agile
Test automation
provides added value
Test automation at different test
levels
Maintainable and continuous
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Keys to success
Maintainable Continuous integration
Testware Test scripts – Test cases – Test data – Test
environment
System Under Test Software – Service – Platform - Infrastructure
Management Reporting
Projects Changes
Patches Fixes
Test Automation
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Regression & E2E testing
• Focus wider than current sprint – Probably wider than the team
• Assign where and when • Business value
Forming Norming Performing
TI4Agile
Regression and E2E in
focus
In and over iteration
Proactively addressed
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Keys to success
• Understandable, visualized, clear E2E information
• Optional: – Hardening Sprint focused on improving the product
• E2E, Regression, Loose ends, Shortcuts, Non-functionals
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
People
Forming Norming Performing
TI4Agile
People are motivated
People embrace
changes in the product
People initiate process changes
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Keys to success
Feedback • State something positive related to the subject • State your criticism objectively • Don’t use the word “but” • State the effect • Suggest an improvement
Thank you for delivering a lot of information in the meeting. I noticed that you were talking a lot, this provided me little room to give my opinion. You might
want to ask others for their input in the future.
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Keys to success
• Know the context • Assist others
• Know your own strengths • Leave your comfort zone
30
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Interaction
• Actively share information • Open and honest • Use meetings for their purpose
– Understandable for all attendees
Forming Norming Performing
TI4Agile
Traceable decisions
Focussed meetings
Ad-hoc, when needed
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Keys to success
Communication = essential • SCRUM-board
32
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Teamwork
• Respect and trust each other • Commitment as a team
– Work towards a team goal • Help each other where possible
– Multi disciplined
Forming Norming Performing
TI4Agile
The team executes a
task
The team is committed
The team is self managing
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Keys to success
• Keep the goals visible
• Team development
Bruce W. Tuckman: Stages of group development
34
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Test profession
• Test skills • Soft skills • Technical skills (drivers, stubs, logging, …) • Share test knowledge
– Within the discipline, over teams – Within team, over disciplines
Forming Norming Performing
TI4Agile
Proficient in testing
Focus on soft and technical
skills
Profession includes more
than just testing
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Key to success
• Charters, SBTM, … • Exploratory testing
– Simultaneously learn, plan, design, execute and report
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Stakeholder commitment
• Deliver acceptance criteria and share the context • Recognise the value of the team and of each role • Prioritise backlog, risks and defects • Create an effective environment • Participate in acceptance • Allow freedom of choice
Forming Norming Performing
TI4Agile
Stakeholders start projects
Stakeholders help projects
Stakeholders participate in
projects
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
The path to commitment
Contact Awareness
Understanding
Posi2ve percep2on
Adop2on
Ins2tu2onali-‐za2on
Internaliza2on
Level of com
mitm
ent
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Assessment model
Key area Forming Norming Performing 1 Stakeholder commitment 1 2 3 4 1 2 3 4 1 2 3
2 Planning & Estimation 1 2 3 4 1 2 3 4 1 2 3 4
3 People 1 2 3 4 1 2 3 4 1 2 3
4 Interaction 1 2 3 4 1 2 3 4 1 2 3 4
5 Teamwork 1 2 3 1 2 3 4 1 2 3 4
6 Test process 1 2 3 1 2 3 4 1 2 3
7 Test management 1 2 3 4 1 2 3 1 2 3
8 Test profession 1 2 3 4 1 2 3 4 1 2 3
9 Test automation 1 2 3 1 2 3 4 1 2 3 4
10 Regression & E2E testing 1 2 3 4 1 2 3 1 2 3 4
11 Defect management 1 2 3 1 2 3 4 1 2 3
12 Test environment 1 2 3 1 2 3 1 2 3 4
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Transitioning from traditional to agile Process Waterfall
Development Iterative &
Incremental Agile
Development
Measure of Success
Management Culture
Requirements & Design
Coding & Implementation
Test & Quality Assurance
Planning & Scheduling
Conformance to plan
Response to change, working code
Command & control
Leadership collaborative, empowerment of teams
Big & upfront documentation
Continuous / emergent / just in time elaboration
Code all features in parallel. Test later
Code & unit test, deliver serially
Big, planned / test late Continuous & concurrent testing starting early
detailed / fixed scope, estimate time & resources
Two-level plan / fix date, estimate scope
(Scaling Software Agility: Best Practices for large Organisations – DeanLeffingwell 2007)
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
Key area Forming Norming Performing 1 Stakeholder commitment 1 2 3 4 1 2 3 4 1 2 3
2 Planning & Estimation 1 2 3 4 1 2 3 4 1 2 3 4
3 People 1 2 3 4 1 2 3 4 1 2 3
4 Interaction 1 2 3 4 1 2 3 4 1 2 3 4
5 Teamwork 1 2 3 1 2 3 4 1 2 3 4
6 Test process 1 2 3 1 2 3 4 1 2 3
7 Test management 1 2 3 4 1 2 3 1 2 3
8 Test profession 1 2 3 4 1 2 3 4 1 2 3
9 Test automation 1 2 3 1 2 3 4 1 2 3 4
10 Regression & E2E testing 1 2 3 4 1 2 3 1 2 3 4
11 Defect management 1 2 3 1 2 3 4 1 2 3
12 Test environment 1 2 3 1 2 3 1 2 3 4
Questions?
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
Test Improvement 4 Agile
Ruud Teunissen Senior Test Consultant
Polteq Test Services BV ruud.teunissen@polteq.com – @RuudTeunissen
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
01. Stakeholder commitment – Forming
1. The principal stakeholder is defined and known by the testers
2. Stakeholders deliver the committed resources
3. Stakeholders actively acquire information on the progress of the project
4. Stakeholders are willing to adapt their way of working to the test process
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
01. Stakeholder commitment – Norming
1 The stakeholders provide a mandated representative (PO) to the team
2 The stakeholders define business value and provide a prioritized product backlog
3 The stakeholders define acceptance criteria for the items on the product backlog
4 Stakeholders attend the review meeting
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
01. Stakeholder commitment – Performing
1 The product owner is actively participating in the team activities
2 Stakeholders trust and value the team(s)
3 Stakeholders share responsibility for the quality of the product
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
02. Planning & Estimation – Forming
1. Each test activity is planned (prepare - define - execute)
2. Test levels are defined and overlap is minimized (UT – ST - AT)
3. A technique for test effort estimation is applied
4. Planning is agreed with the stakeholders
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
02. Planning & Estimation – Norming 1 Estimation is used to estimate team effort (cross
functional)
2 The planning meeting is used to elaborate on user stories
3 Testing work is broken down into measurable, small tasks (preferably 8hrs or less)
4 Burn-down (or up) charts are used to keep track of progress
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
02. Planning & Estimation – Performing 1 Scope is adjusted when estimation seems to be
incorrect, since time boxes are fixed
2 Changes on user stories are fit into the planning wherever possible
3 The team(s) decide what is delivered in an iteration
4 Team velocity is used in planning
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
03. People – Forming
1. People are well trained and/or experienced in their functions
2. People are willing to put in extra effort when needed (commitment)
3. People can explain their value in the project context
4. People take full responsibility for their work
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
03. People – Norming
1 People understand SCRUM terminology and know the purpose of the different SCRUM meetings
2 SCRUM master keeps track of the process
3 SCRUM master removes roadblocks outside the team
4 People have a positive attitude towards change
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
03. People – Performing
1 People proactively provide feedback
2 People know how to handle feedback and use the feedback to improve
3 People are able to help with tasks outside their main area of expertise (T-shaping)
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
04. Interaction – Forming
1. It is possible to trace back points of action, agreements and decisions
2. Progress, product quality and risks are communicated
3. Meetings are only attended by relevant people
4. Tooling to support the necessary communication within the project is supplied
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
04. Interaction – Norming
1 The stand-up meeting is used to actively monitor progress and adjust the planning accordingly
2 The planning meeting is used to discuss and identify risks
3 The retrospective deals with positive points and improvement suggestions
4 Documentation reflects the current state of the product
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
04. Interaction – Performing
1 People use ad-hoc meetings to obtain or communicate relevant information
2 Testers share their test ideas as soon as possible
3 Documentation is fit for purpose (just enough)
4 Good practices are shared over the teams (and projects)
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
05. Teamwork – Forming
1. The team is co-located
2. Each team member is involved in important decisions
3. The team understands its goal
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
05. Teamwork – Norming
1 Testing is an integrated task of a multi disciplinary team
2 The team commits to the scope in a sprint
3 SCRUM coaching is provided to improve the teamwork
4 The team contains all skills necessary to deliver a working product
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
05. Teamwork – Performing
1 Team members know and value each other
2 Specialists share their expertise to improve the knowledge in the team
3 The team is responsible for a sufficient quality product
4 Team members can take on different roles instead of being limited to their function
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
06. Test process – Forming
1. The effort for test tasks is based on product risks
2. Test design techniques are applied when creating test cases
3. Testing considers different quality characteristics
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
06. Test process – Norming
1 Non-functionals are addressed in and over sprints
2 Pairing is used to improve the test quality
3 The Definition of Done includes items related to testing
4 Developers use simple techniques for unit testing
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
06. Test process – Performing
1 Testing debt is monitored and managed
2 User acceptance is fully integrated in the process
3 All team members have a responsibility for the test process
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
07. Test management – Forming
1. Test management allocates the proper test staff
2. A generic test approach is available
3. Test management provides birds eye view on quality and risks in the project
4. Test management contributes to the release planning by adding a quality view to high level estimation
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
07. Test management – Norming
1 Test management initiates risk identification at product backlog level
2 Test management supports the product owner(s) to identify dependencies between user stories prior to finalizing the release planning
3 Test management supports the SCRUM master in removal of test roadblocks
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
07. Test management - Performing
1 Test management facilitates test meetings to share the testing pitfalls and lessons learned over teams (and projects)
2 Test management provides mentoring to the teams to improve the quality of testing
3 Test management increases awareness and understanding of Agile testing within the organization
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
08. Test profession – Forming
1. Testers have received specific test training and/or have sufficient experience in the field of structured testing
2. Testers can explain the rationale behind chosen techniques that have been applied
3. Test functions are part of the organizations career development
4. Testers can explain their added value
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
08. Test profession – Norming
1 Testers have received training(s) regarding soft skills
2 Technical activities are part of the testing job (e.g. working with drivers/stubs, code reviews)
3 Testers provide support to testing activities of other team members
4 Testers use exploratory testing in addition to using structured techniques
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
08. Test profession – Performing
1 Testers are proficient in at least one programming language
2 Testers are proactively gathering relevant information to be able to execute their tasks and to minimize waiting time
3 Testers strive to continuously improve their testing
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
09. Test automation – Forming
1. The organization acknowledges the importance of test tooling and allocates sufficient budget
2. Test cases are automatable by describing them using ‘inputs - actions - expected results’
3. Decisions on what to automate are based on risks and return on investment
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
09. Test automation – Norming
1 Automated unit testing is part of the development process
2 Tests are automated to save time for manual testing
3 Test cases are designed for automation
4 Test automation is seen and treated as software development
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
09. Test automation – Performing
1 Automation is modular and maintainable
2 Requirements are set up to be executable
3 Teams are allowed to try different tooling to reach their own optimal setting
4 Test automation is used to support continuous integration
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
10. Regression & E2E testing – Forming
1. There is a basic strategy for regression testing
2. There is a basic strategy for E2E testing
3. Test sets are maintained and updated
4. Responsibilities for E2E and regression testing are clear
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
10. Regression & E2E testing – Norming
1 Regression testing is part of every iteration
2 Regression and E2E testing is applied over multiple iterations and when needed over multiple teams and multiple projects
3 The user stories contain information on relevant E2E functionality
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
10. Regression & E2E testing – Performing
1 The team is aware of the value of and need for E2E and regression testing
2 All testing is used to address current business value
3 Regression test sets can be easily scoped to address specific areas of functionality
4 E2E test possibilities are utilized continuously
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
11. Defect management – Forming
1. All persons involved in logging and/or tracking defects use the same defect tracking tool
2. Defects have a common set of characteristics (e.g. unique id, description, status)
3. All raised defects are followed up
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
11. Defect management – Norming
1 There are simple criteria for deciding when to log a defect
2 Defects that can not be fixed within the sprint have to be added to the product backlog
3 Defects, either logged or not logged, are communicated with the complete team
4 Impact of defects on the product, but also on the sprint needs to be taken into account
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
11. Defect management – Performing
1 The team strives to fix defects as soon as possible (<1 day)
2 The person that logged the defect helps to solve the defect
3 Defects are logged with the minimal set of information needed to follow up the defects
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
12. Test environment – Forming
1. The environment is managed (setup, maintenance, version management, etc.)
2. The environment is sufficiently representative for the test
3. The data used in the environment can easily be refreshed
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
12. Test environment – Norming
1 The test environment is always available to the team
2 Changes to the environment are communicated to the involved teams
3 The version management system used for the environment supports branching
logo van Flair 17-‐12-‐2010 Polteq logo_RGB.png
R G B
35 30 96 Blauw
232 62 38 Rood
108 174 68 Groen
12. Test environment – Performing
1 Testing takes place on all relevant environments
2 The environment supports the use of continuous integration
3 The team manages its own environments
4 A mechanism to efficiently support testing in different configurations is in place (e.g. virtualization)