Common Objections to TDD (and their refutations)
-
Upload
seb-rose -
Category
Technology
-
view
4.257 -
download
3
description
Transcript of Common Objections to TDD (and their refutations)
IBM Software Group
© 2009 IBM Corporation
Common Objections to TDD
(and their refutations)
Seb Rose
Twitter: @sebrose
Blog: claysnow.blogspot.com
E-mail: [email protected]
Phone: 01721 788178
IBM Software Group | Rational software
Innovation for a smarter planet2
Agenda
Motivation
Education
Inappropriate project
Cultural
Time pressures
Unclear benefits
IBM Software Group | Rational software
Innovation for a smarter planet3
Refutations. Really?
re·fute
/rɪˈfyut/ Show Spelled
verb (used with object), re·fut·ed, re·fut·ing.
1. to prove to be false or erroneous, as an opinion or charge.
2. to prove (a person) to be in error.
IBM Software Group | Rational software
Innovation for a smarter planet4
TDD – in one slide
Test First predates XP and Agile
TDD has been around for a whileOne of XP’s core practicesWidely referencedStill quite controversial
I am not going to describe TDD in any detailRed/Green/Refactor (and variants) ‘T’ for Test has unfortunate connotationsFirst ‘D’ definitely stands for DrivenSecond ‘D’ may stand for Development or Design
IBM Software Group | Rational software
Innovation for a smarter planet5
IBM Software Group | Rational software
Innovation for a smarter planet6
Survey
Simple survey ran for 3 monthsJan – Mar 2012260 unique respondents896 objections
“What reasons are there not to practice TDD” “The question wreaks of elitism - it should instead be ‘what reasons are
there to practice TDD’”.
Free text made for a wide range of responsesHarder to analyseSplit into 18 types, grouped into 5 categories
IBM Software Group | Rational software
Innovation for a smarter planet7
Results
EducationTimeBenefitsProjectCultural
IBM Software Group | Rational software
Innovation for a smarter planet8
Breakdown
Education (223)Lack of Practice, Lack of Investment
Project (194)Domain, Legacy, Environment, Tooling
Cultural (179)Management, Team, Fanaticism, Demarcation, Egotism
Time (174)Slowness of development, maintenance and execution
Benefits (120)Lack of Proof & Experience, Alternatives
IBM Software Group | Rational software
Innovation for a smarter planet9
Interconnectedness of all things
Classification is subjectiveMany responses could be categorised several ways
Respondents had different points of viewTheir own objectionsObjections they had heard others express
Further analysis required
Follow-up survey(s)
IBM Software Group | Rational software
Innovation for a smarter planet10
IBM Software Group | Rational software
Innovation for a smarter planet11
The required techniques of effective reasoning
are pretty formal, but as long as programming is
done by people that don't master them, the
software crisis will remain with us and will be
considered an incurable disease. And you know
what incurable diseases do: they invite the
quacks and charlatans in, who in this case take
the form of Software Engineering gurus.
- Dijkstra (ACM Turing Award Lecture 1972)
Software Crisis!
IBM Software Group | Rational software
Innovation for a smarter planet12
Fanaticism and Fluff
The steps of FDD are simple:
1. Take a tiny piece of fluff from the plate and put it on your head, holding your head quite still to ensure that the fluff does not fall off your hair.
2. Write a line of code.
3. Say, “I am the Fluff Lord, within the Dominion Of The Fluffists.”
4. Repeat.
That’s it.
Seriously, that’s all there is to it.
IBM Software Group | Rational software
Innovation for a smarter planet13
Agenda
Motivation
Education
Inappropriate project
Cultural
Time pressures
Unclear benefits
Wrap up
IBM Software Group | Rational software
Innovation for a smarter planet14
Hard to learn
“TDD is hard.”
Yes, it is
Any technique is hard to masterFundamental changes are hard to adopt
10,000 hours of practice
Did you stop trying to ride a bicycle when you fell off?
IBM Software Group | Rational software
Innovation for a smarter planet15
Where to start
“Sheer lack of knowledge on how to approach it appropriately. There are too many bowling examples and not enough practicality.”
How do you usually learn something new?
Unit Testing skills are foundational
Coding Dojo to practice TDDhttp://cyber-dojo.comBetter used as a group, but still useful solo
IBM Software Group | Rational software
Innovation for a smarter planet16
Tests before code
“The compiler complains if I write tests before the code”
First step to get to green is to make the test compileOnly then can you move on to make the test pass
IBM Software Group | Rational software
Innovation for a smarter planet17
Tests will be buggy
“Your test code is just your production code, written from the other end (i.e. just as complex and likely to have bugs)”
No process is infallible “To err is human …”
Safety in numbersPair programmingPeer review
IBM Software Group | Rational software
Innovation for a smarter planet18
Unfamiliar architectural style
“How will we ever know what’s going on if we can't see all of the code at once?”
TDD tends to lead to small, concise implementationsLow couplingHigh cohesion
“Proliferation” of interfaces
Literate-ish programming
IBM Software Group | Rational software
Innovation for a smarter planet19
No design improvement
“My observations of code from TDD-based projects show no significant improvement in architecture, security, code style, testability, etc. over other projects built with testing in mind.”
TDD ensures testabilityTendency for smaller decoupled composition
Other design concerns (e.g. security, performance etc.) not addressed by TDD
IBM Software Group | Rational software
Innovation for a smarter planet20
Learning several things at once
“In a new technology, it's too difficult to learn how to TDD as well as how to master that technology.“
Even if you are experienced with TDD, it changes your perspective on the new technology IdiomsTooling
Kent Beck suggests reimplementing xUnit in the new language
IBM Software Group | Rational software
Innovation for a smarter planet21
Discipline
“TDD is a discipline and a work habit. It's very difficult to establish the habit.”
Habits are hard to form, but also hard to break.
Nothing works better than positive feedback
There are many self-help guides available Switch – Chip & Dan HeathDrive – Dan Pink
IBM Software Group | Rational software
Innovation for a smarter planet22
Agenda
Motivation
Education
Inappropriate project
Cultural
Time pressures
Unclear benefits
Wrap up
IBM Software Group | Rational software
Innovation for a smarter planet23
Too simple
"This code is too simple to need a test“
Then the tests will be simple too
What’s simple to you may be opaque to others
As the code evolves, tests can help keep it simple
IBM Software Group | Rational software
Innovation for a smarter planet24
Just a spike
“Because you're prototyping an idea and it's much faster to spike without tests. If you do end up using the code then you can write tests and refactor.”
Use the right tool – TDD is not mandatory
Discipline to ensure spike does not mutate “Write one to throw away”TDD the spike anyway!
IBM Software Group | Rational software
Innovation for a smarter planet25
Not useful for all aspects of design
“I don't think that deriving a design by writing tests is a useful practice. Tests by themselves cannot cover many aspects of a design (designing for concurrency and performance in particular by writing tests is something that I've never seen anyone do).”
TDD is not the only tool in the toolboxProbably wrong tool for performance testingCan be applied to concurrency testing (but not recommended)
IBM Software Group | Rational software
Innovation for a smarter planet26
Not useful for functional/declarative programming
“Because it is largely adequate in imperative programming and not when you go functional and declarative - when code reads like specification.”
Orthogonal.
Absence of side effects makes testing MORE effective.
Discuss!
IBM Software Group | Rational software
Innovation for a smarter planet27
Legacy code
“Difficulty starting with legacy code. A simple change done through TDD can take orders of magnitude longer due to a need to redesign toward testability. ”
Working Effectively With Legacy Code – Feathers
Small, conservative steps, eventually tame fear and doubt
Consider less intrusive approaches initiallye.g. TextTest – texttest.org
IBM Software Group | Rational software
Innovation for a smarter planet28
Insufficient tooling
“There is no unit-test framework for the language I'm using, and I don't have time/inclination to develop one myself.”
There probably is.
You might not need one “TDD in C” – Olve Maudal
IBM Software Group | Rational software
Innovation for a smarter planet29
GUI
“I do a lot of UI code and don't really know how to properly do that with TDD.”
“Subcutaneous” testing
Presenter First patternBased on MVC/MVP patternsVery shallow view, with no business logicStateless presenter orchestrates interaction between view and model
IBM Software Group | Rational software
Innovation for a smarter planet30
Excessive coupling to data
“A test expresses a unit of change as data that fits the needed computation. Creating that data is harder than writing the program that accomplishes the change.”
Unit tests preferably utilise ‘small’ amounts of data
For domains where this is not possibleYou will need large amounts of data irrespective of test approachBootstrap process with constructed dataContinue with captured data
IBM Software Group | Rational software
Innovation for a smarter planet31
Agenda
Motivation
Education
Inappropriate project
Cultural
Time pressures
Unclear benefits
Wrap up
IBM Software Group | Rational software
Innovation for a smarter planet32
Management antipathy
“Management want results NOW; very much willing to clean up small oversights later. 'Close is good enough' attitude.”
Management are generally result focused
Previous bad experiences affect appetite for changeBoth ways!
Start smallNeed to demonstrate benefit to ‘bottom line’
IBM Software Group | Rational software
Innovation for a smarter planet33
Team resistance
“Teammates are not prepared. They do not have enough knowledge or experience in unit testing.”
“No one tells me how to program!”
Changing behaviours is hard “Fearless Change” has many patterns for introducing change
IBM Software Group | Rational software
Innovation for a smarter planet34
I never make mistakes
“I already know what the code needs to do and it's low risk. I don't need a test for it.”
“My first design idea is always perfectly good.”
“My code always works the first time.”
Do people really believe this?
Even if they do, is everyone that ever touches the code going to be so talented?
IBM Software Group | Rational software
Innovation for a smarter planet35
Testing is for testers
“We don't need TDD, we've got a QA department. They'll find the bugs for us.”
TDD is not just about testingDrives designRefactoring/regression safety net
Write tests to explore EVERY defect you find
Living documentation Helps future developers understand intended behaviours
IBM Software Group | Rational software
Innovation for a smarter planet36
All TDD-ers are fanatics
“It has become a cult. Its advocates have made it antithetical to ‘Individuals and interactions over processes and tools’. It is evangelized through coercion and browbeating -- necessarily, as it isn't compelling on it's own.”
“Someone needs to tell unclebob that he might be right, but he's part of the problem - to non-modern coders (which felt like the majority last time I looked) he comes across like a raving loony.”
IBM Software Group | Rational software
Innovation for a smarter planet37
Fluff (continued)
If at any time you even THINK about writing a line of code before putting fluff on your head, then you’ve to delete all your code, shake all the fluff from your head onto the plate and start all over again.
I have been practicing FDD for 12 years now; sometimes in the office, I have so much fluff on my head that my boss thinks I’m a hay-stack, only made of fluff. A sort-of fluff-stack, if you will.
But one thing is beyond doubt: the code I write, when I’m in this fluff-zone, is the most high-quality code that anyone has ever seen.
IBM Software Group | Rational software
Innovation for a smarter planet38
Agenda
Motivation
Education
Inappropriate project
Cultural
Time pressures
Unclear benefits
Wrap up
IBM Software Group | Rational software
Innovation for a smarter planet39
More code to write
“Takes too much time to write test.”
How will your code get tested?Test team?Customers & Stack Trace Driven Development?
How much time will be spent figuring out how to make changes later?
What proportion of your time do you actually spend coding?
IBM Software Group | Rational software
Innovation for a smarter planet40
More code to maintain
“If I change something it will break a whole bunch of tests that I will have to fix and it will be more work for me in the end than just verifying my changes manually.”
It is USEFUL to know when behaviour changes
Foundational Unit Testing skills reduce brittleness next 6 slides
IBM Software Group | Rational software
Innovation for a smarter planet41
Testability
Testability needs to be designed in TDD ensures code is testable
Code with hidden dependencies is hard to test Dependency Injection/Inversion
Pass dependencies into code under test Write factories that permit injection of test doubles
Interfaces should be cohesive Wide interfaces encourage unnecessary coupling
Avoid globals, singletons etc.
Retro-fitting unit tests is hard Take small steps Introduce a ‘seam’ – c.f. Working Effectively with Legacy Code
IBM Software Group | Rational software
Innovation for a smarter planet42
Necessity Test observable behaviour
Don’t modify encapsulation to aid testing If a behaviour isn’t observable through the public interface what is it for?
Don’t slavishly write one test per method Test behaviours Some methods may not need any dedicated tests Methods that implement useful behaviours may need many tests
Choose test variants carefully Edge conditions Invalid inputs Multiple invocations Error signalling
IBM Software Group | Rational software
Innovation for a smarter planet43
Granularity
Test a SINGLE observable behaviour It is tempting to combine related behaviours in a single test – DON’T
… even if EXACTLY the same steps are needed
public void shouldSortTwoStringsAndReportCorrectSize() {
SortedSet<String> animals = new TreeSet<String>();
animals.add(“Zebra”);
animals.add(“Anteater”);
assertEquals(2, animals.size());
assertEquals(“Anteater”, animals.first());
assertEquals(“Zebra”, animals.last());
}
IBM Software Group | Rational software
Innovation for a smarter planet44
Understandability Name tests to describe the behaviour under test
Describe nature of the test Is it checking that preconditions are enforced? Is a dependency going to signal an error?
Long names are fine – you only type them once Be precise
shouldReturnCorrectValue is not a good name for a test shouldReturnCorrectSumOfTwoIntegersWithoutOverflow should_return_correct_sum_of_two_integers_without_overflow
When a test fails you want to know WHAT WENT WRONG You don’t want to reverse engineer the test You don’t want to run smaller tests to isolate the failure
IBM Software Group | Rational software
Innovation for a smarter planet45
Maintainability
Unit Tests should be written to same quality as Production code Tests will be maintained and read just as often as production code Code is communication to other developers not just a compiler
Organise tests into cohesive suites
Refactor tests to avoid duplication Use suites to perform common set up/tear down operations Extract common code into methods Extract common functionality into classes
Remove redundant tests
IBM Software Group | Rational software
Innovation for a smarter planet46
Reiteration: 5 -ities
Testability
Necessity
Granularity
Understandability
Maintainability
MUNGT ?
TeNGUM?
IBM Software Group | Rational software
Innovation for a smarter planet47
Mock-based tests rot
“Unit-test mock/stub assumptions rots”
Use collaboration and contract tests – J.B. RainsbergerCollaboration tests make assumptions about the contract; contract tests
try to justify those assumptionsA stub in a collaboration test must correspond to an expected result in a
contract testAn expectation in a collaboration test must correspond to an action in a
contract test
IBM Software Group | Rational software
Innovation for a smarter planet48
That library is third party
“API layers above and below you don't provide adequate mocks. Testing against the ‘real thing’ is hard/impossible.”
“Don't mock types you don't own” – Joe Walnes
Write adapters that provide a domain specific API
Write acceptance tests that verify the library’s behaviourRun whenever adopting a new version
IBM Software Group | Rational software
Innovation for a smarter planet49
Slow execution
“Running the tests takes too long.”
For TDD to work tests need to (build and) execute in seconds
Some environments need careful configurationC/C++Rails
Are they really Unit Tests? (See next slide)
IBM Software Group | Rational software
Innovation for a smarter planet50
A test is not a unit test if:
It talks to the database
It communicates across the network
It touches the file system
It can’t run at the same time as other unit tests
You have to do special things to your environment (such as editing config files) to run it
(Michael Feathers’ blog, 2005)
IBM Software Group | Rational software
Innovation for a smarter planet51
Agenda
Motivation
Education
Inappropriate project
Cultural
Time pressures
Unclear benefits
Wrap up
IBM Software Group | Rational software
Innovation for a smarter planet52
Absence of research
“No scientific proof of it actually having any benefit (compared to just code reviews, pair programming or etc.) for the same amount of time”
“It's not practical for the kind of work I do. By which I mean, it cannot be conclusively shown to provide tangible benefits that outweigh the perceived costs.”
“In an industry where todays must have offering is tomorrows dustbin liner, it's often prudent to wait before trying the latest snake oil!”
IBM Software Group | Rational software
Innovation for a smarter planet53
Absence of research (cont.)
“We found that test-first students on average wrote more tests and, in turn, students who wrote more tests tended to be more productive. We also observed that the minimum quality increased linearly with the number of programmer tests, independent of the development strategy employed.”
On the Effectiveness of Test-first Approach to ProgrammingProceedings of the IEEE Transactions on Software Engineering 2005
Lots of other papers are referenced from:http://bradapp.blogspot.co.uk/2009/08/studies-on-effectiveness-of-tdd.html
IBM Software Group | Rational software
Innovation for a smarter planet54
Our process is good enough already
“My team doesn't practice it. It is difficult to change existing practices if they seem to work reasonably well.”
“I've personally been a part of four 'large' software projects where active pursuit of a test suite was in play. None of the four projects ever launched into production. The people I worked with were smart and highly ambitious, yet they (we) failed to launch.
“I've personally been a part of dozens of software projects where active pursuit of a test suite was NOT in play. Most every one of those projects launched into production in a timely manner.”
IBM Software Group | Rational software
Innovation for a smarter planet55
Test after
“It's easier to write the code first and the test after.”
“Writing tests after code is just as good.”
“Writing tests before code doesn't make sense.”
TestabilityRework needed if code not testable
DisciplineWill you really write the tests later?
IBM Software Group | Rational software
Innovation for a smarter planet56
BDD
“Why practice TDD when you can practice BDD?”
BDD extends TDD
Involves stakeholders, not just developersEven harder to implement.
IBM Software Group | Rational software
Innovation for a smarter planet57
ATDD, BDD, GOOS
Many shades of outside-in, test-first development
IBM Software Group | Rational software
Innovation for a smarter planet58
“And check that nothing happens that shouldn’t happen and everything else happens that should happen for all variations of this scenario and all possible states of the ATM and all possible states of the customer’s account and all possible states of the rest of the database and all possible states of the system as a whole, and anything happening in the cloud that should not matter but might matter.”
Given the account is in credit
And the card is valid
And the dispenser contains cash
When the customer requests cash
Then check that the account is debited
And check that cash is dispensed
And check that the card is returned
None of these are enough either
IBM Software Group | Rational software
Innovation for a smarter planet59
Manual testing
“It is much faster to just think about the proper implementation, write the code and manually test it.”
Until you need to do it again
… and again
Manual scriptsError proneTime consumingExpensive
IBM Software Group | Rational software
Innovation for a smarter planet60
Agenda
Motivation
Education
Inappropriate project
Cultural
Time pressures
Unclear benefits
Wrap up
IBM Software Group | Rational software
Innovation for a smarter planet61
Not sufficient, but necessary?
TDD is not enough. Consider:AcceptanceUsabilityPerformance / StressExploratory
Systems have been, and continue to be, successfully delivered without TDD.But they have also been delivered without designs, documentation, or…. testing of any kind.
IBM Software Group | Rational software
Innovation for a smarter planet62
Conclusions
There are many impediments to adopting TDD
Benefits are not universally accepted
TDD is not a “Silver Bullet”
IBM Software Group | Rational software
Innovation for a smarter planet63
Essential Reading
Test Driven Development by Example – Kent Beck
Growing Object Oriented Software Guided By Tests – Steve Freeman/Nat Pryce
Working With Legacy Code – Michael Feathers
Fearless Change – Mary Lynn Manns/Linda Rising
JUnit Recipes – J.B. Rainsberger
IBM Software Group | Rational software
Innovation for a smarter planet64
Other references
Fluff Driven Development: http://www.threeriversinstitute.org/blog/?p=594
Presenter First: http://atomicobject.com/files/PresenterFirstAgile2006.pdf
TDD in C: http://www.pvv.org/~oma/TDDinC_Smidig2007.pdf
On the effectiveness of Test-first approach to Programming:
http://nparc.cisti-icist.nrc-cnrc.gc.ca/npsi/ctrl?action=rtdoc&an=5763742&article=0&lang=en&ext=pdf