#LFMF: Tales of Test Automation Gone Wrong
27
#LFMF Learn from My Fail: Tales of Test Automa6on Gone Wrong (Plus a Li*le Bit about How to Do It Right) (and others who shall remain anonymous even if they don’t deserve it) Elisabeth Hendrickson Quality Tree So7ware, Inc. @testobsessed
-
Upload
elisabeth-hendrickson -
Category
Technology
-
view
2.813 -
download
5
description
This is a "Best Of" list of my (and others) failures over the years in attempting to adopt test automation. In it, you'll find 13 categories of What Not to Do. Presented at Turku Agile Day 2012 (#tad012).
Transcript of #LFMF: Tales of Test Automation Gone Wrong
#LFMF
Learn from My Fail:
Tales of Test Automa6on Gone Wrong
(Plus a Li*le Bit about How to Do It Right)
(and others who shall remain anonymous even if they don’t deserve it)!
Elisabeth Hendrickson Quality Tree So7ware, Inc.
@testobsessed
Copyright ©
2012 Elisabeth Hen
drickson
, Quality Tree So7
ware, Inc.
Fail Pa;ern #0: “No 6me to automate” No automaFon means rapid growth in the
manual tesFng burden. Or untested so7ware.
Copyright ©
2012 Elisabeth Hen
drickson
, Quality Tree So7
ware, Inc.
Velocity!
Copyright ©
2012 Elisabeth Hen
drickson
, Quality Tree So7
ware, Inc.
Fail Pa;ern #1: Automa6on for the sake
of automa6on
If you really want a mess, use a count of automated tests as a measure of
progress.
Copyright ©
2012 Elisabeth Hen
drickson
, Quality Tree So7
ware, Inc.
Image of prinFng office from hJp://www.shorpy.com/node/12652
Copyright ©
2012 Elisabeth Hen
drickson
, Quality Tree So7
ware, Inc.
Fail Pa;ern #2: All automa6on is end-‐to-‐end through the GUI
End-‐to-‐end tests are slow and fragile. Some are good.
More are not beJer.
Copyright ©
2012 Elisabeth Hen
drickson
, Quality Tree So7
ware, Inc.
Flooded basement image courtesy of carlpenergy Showerhead image courtesy of deusx Images from Flickr licensed through CreaFve Commons AJribuFon
Copyright ©
2012 Elisabeth Hen
drickson
, Quality Tree So7
ware, Inc.
Fail Pa;ern #3: Dispropor6onate
Amount of Time Spent Fixing Broken Tests Just when you get one working again, another
breaks.
Copyright ©
2012 Elisabeth Hen
drickson
, Quality Tree So7
ware, Inc.
Image courtesy of cdrussorusso From hJp://www.flickr.com/photos/23516192@N08/2631474033/ Licensed Under CreaFve Commons AJribuFon
Copyright ©
2012 Elisabeth Hen
drickson
, Quality Tree So7
ware, Inc.
Fail Pa;ern #4: Fragile Tests
They pass, they fail, but there seems to be no
correlaFon between the test results and the actual so7ware behavior.
Copyright ©
2012 Elisabeth Hen
drickson
, Quality Tree So7
ware, Inc. Hey, the
build is red.!
Yeah, it’s the 2nd Tuesday with a full moon this
year.!
Copyright ©
2012 Elisabeth Hen
drickson
, Quality Tree So7
ware, Inc.
Fail Pa;ern #5: Inter-‐dependent Tests
When one fails, they all fail together.
Copyright ©
2012 Elisabeth Hen
drickson
, Quality Tree So7
ware, Inc.
Run #1!
Run #2!
Run #3!
Run #4!
Copyright ©
2012 Elisabeth Hen
drickson
, Quality Tree So7
ware, Inc.
Fail Pa;ern #6: Useless Tests
The tests can never fail, so they yield no informaFon.
Copyright ©
2012 Elisabeth Hen
drickson
, Quality Tree So7
ware, Inc. …
assert True
… But rarely. More common: tests that
verify the software
does what the software does.!
Yes, I have actually seen this in tests.!
Copyright ©
2012 Elisabeth Hen
drickson
, Quality Tree So7
ware, Inc.
Fail Pa;ern #7: Bad Tests
Obfuscated intent, massive duplicaFon, and Fght coupling between the interface and the
tests makes for an unmaintainable mess.
Copyright ©
2012 Elisabeth Hen
drickson
, Quality Tree So7
ware, Inc.
Test Login
Input Text first_name fred
Input Text last_name flintstone
Input Text user_name fflint
Input Text password p@ssw0rd
Click Submit
Click Logout
Click Login
Input Text username fflint
Input Text password badpassword
Click Submit
Input Text ...
WAT?!?!
Copyright ©
2012 Elisabeth Hen
drickson
, Quality Tree So7
ware, Inc.
Fail Pa;ern #8: Bad Code
Smelly test automaFon code that violates SOLID
principles is as much of a problem as smelly producFon code.
Copyright ©
2012 Elisabeth Hen
drickson
, Quality Tree So7
ware, Inc.
Copyright ©
2012 Elisabeth Hen
drickson
, Quality Tree So7
ware, Inc.
Fail Pa;ern #9: Only specialists
automate
They might be good at automaFon, but they’re fighFng an uphill baJle.
Copyright ©
2012 Elisabeth Hen
drickson
, Quality Tree So7
ware, Inc.
Fail Pa;ern #10: Forcing devout non-‐
coders to write automa6on
If they lack either interest or apFtude to automate, the
results will be bad.
Copyright ©
2012 Elisabeth Hen
drickson
, Quality Tree So7
ware, Inc.
Fail Pa;ern #11: Tests That Don’t Get Run
Someone writes them, no one runs them. So when they suffer bitrot, no one can hear them scream.
Copyright ©
2012 Elisabeth Hen
drickson
, Quality Tree So7
ware, Inc.
Photo by crabchick hJp://www.flickr.com/photos/crabchick/2627370039/ Licensed under CreaFve Commons AJribuFon
Unrun !tests!become !creaky,!dusty,!sad.!
Copyright ©
2012 Elisabeth Hen
drickson
, Quality Tree So7
ware, Inc.
Fail Pa;ern #12: Failing Tests Ignored
They run.
They fail.
No one cares.
Copyright ©
2012 Elisabeth Hen
drickson
, Quality Tree So7
ware, Inc.
Image of the 1906 San Francisco Earthquake a7ermath from hJp://www.shorpy.com/node/9866
Copyright ©
2012 Elisabeth Hen
drickson
, Quality Tree So7
ware, Inc. #LFMF
Automate important expectations!Treat test code with the same level of care as production code!
Run all the automated tests in the CI build (no manual kickoff)!Fix failing tests immediately!Foster a “whole team tests” culture!
Copyright ©
2012 Elisabeth Hen
drickson
, Quality Tree So7
ware, Inc.
Many thanks to …!@hackyourtv!Juha Rantanen!Jyrki Pulliainen!Jarkko Kailanto!Teema Frisk!
