Testing: Heaven or HellOpenWest 2014

About Me

What is a test?

● Automated?● Code?

Why Testing?

● Find bugs● Feel confident in the code● Product problems● Technology problems

What is a test?

● Agile Testing Quadrants– Brian Marick

– Agile Testing● Lisa Crispin● Janet Gregory

Quadrant 1

● Unit Tests● Component Tests

Quadrant 1 Hell

● User service rewrite– Focus on business

– Written long after code

– Filled with dependencies

Quadrant 1 Hell

● JavaScript Testing– Tests complete if they pass

– Manual tests

Quadrant 1 Heaven

● Revenue Dashboard– Focus on unit

– Write with code

Quadrant 1 Heaven

● Clipboard functionality– Isolate dependencies

– Write tests to fail

Benefits

● Find technical bugs early● Refactoring● Documentation● Design● Quick feedback

Quadrant 2

● Functional Tests● Examples● Story Tests● Prototypes● Simulations

Quadrant 2 Hell

● Documentation Search– Make it up as you go

– Don't worry about the problem being solved

Quadrant 2 Hell

● Documentation Search– Make it up as you go

– Don't worry about the problem being solved

Quadrant 2 Heaven

● Prototypes– Know what you need to do

– Ask for examples

Quadrant 2 Heaven

● Clipboard refactoring– Focus on solving the problem

– Test as the end user

Benefits

● Universally understood● Focuses on customer● Defines done-ness

Quadrant 3

● Exploratory Tests● Scenarios● Usability Testing● User Acceptance Testing● Alpha/Beta

Quadrant 3 Hell

● Test each piece separately– Never use the product as a user would

Quadrant 3 Hell?

● Angry customers● Negative feedback at sprint review

Quadrant 3 Heaven

● Customer feedback● Suggestions at sprint review● Work as a user

Benefits

● Determines next steps● Explore the system as a whole● Negative feedback can be positive

Quadrant 4

● Performance Testing● Load Testing● Security Testing● “-ility” Testing

– Reliability

– Maintainability

– Compatibility

Quadrant 4 Hell

Quadrant 4 Hell

● Google Apps Invitations– Don't think about volume

● Load testing on dev environment– Test load in a different configuration

Quadrant 4 Heaven

● Third-party security testing● Production environment for testing

Benefits

● Addresses soft requirements● Think about technology at scale● Test boundaries of technology

What is a test?

● Anything that attempts to disprove your assumptions

Testing Heaven or Hell

● Match tests to assumptions

Questions?