Who will guard the guards themselves? How to trust your automation and avoid deceit
-
Upload
bas-dijkstra -
Category
Software
-
view
1.641 -
download
0
Transcript of Who will guard the guards themselves? How to trust your automation and avoid deceit
![Page 1: Who will guard the guards themselves? How to trust your automation and avoid deceit](https://reader034.fdocuments.net/reader034/viewer/2022050614/5a649f8a7f8b9a27568b7ab9/html5/thumbnails/1.jpg)
Who will guard the
guards themselves?
How to trust your automation and avoid deceit
Bas Dijkstra
www.ontestautomation.com
@_basdijkstra
![Page 2: Who will guard the guards themselves? How to trust your automation and avoid deceit](https://reader034.fdocuments.net/reader034/viewer/2022050614/5a649f8a7f8b9a27568b7ab9/html5/thumbnails/2.jpg)
_I build test automation solutions
_I teach test automation, service virtualization
_I write www.ontestautomation.com
TechBeacon, StickyMinds, …
![Page 3: Who will guard the guards themselves? How to trust your automation and avoid deceit](https://reader034.fdocuments.net/reader034/viewer/2022050614/5a649f8a7f8b9a27568b7ab9/html5/thumbnails/3.jpg)
Continuous Testing
DEV ??? PROD
testing testing testing
testing
![Page 4: Who will guard the guards themselves? How to trust your automation and avoid deceit](https://reader034.fdocuments.net/reader034/viewer/2022050614/5a649f8a7f8b9a27568b7ab9/html5/thumbnails/4.jpg)
![Page 5: Who will guard the guards themselves? How to trust your automation and avoid deceit](https://reader034.fdocuments.net/reader034/viewer/2022050614/5a649f8a7f8b9a27568b7ab9/html5/thumbnails/5.jpg)
“Sed quis
custodiet ipsos
custodes?”
But who will guard the guards themselves?
--- Juvenal, Satire VI, lines 347–348 ---
![Page 6: Who will guard the guards themselves? How to trust your automation and avoid deceit](https://reader034.fdocuments.net/reader034/viewer/2022050614/5a649f8a7f8b9a27568b7ab9/html5/thumbnails/6.jpg)
Confidence“Our product is
able to perform
function X”
Control Trust
“I control the
execution of X”
“I trust our
product to be able
to perform X”
![Page 7: Who will guard the guards themselves? How to trust your automation and avoid deceit](https://reader034.fdocuments.net/reader034/viewer/2022050614/5a649f8a7f8b9a27568b7ab9/html5/thumbnails/7.jpg)
I want to be confident about the ability of our
product to perform function X
Automated tests are used to build trust in our
product performing function X correctly
I need to be able to trust my automated tests
around function X
![Page 8: Who will guard the guards themselves? How to trust your automation and avoid deceit](https://reader034.fdocuments.net/reader034/viewer/2022050614/5a649f8a7f8b9a27568b7ab9/html5/thumbnails/8.jpg)
“I’m not upset that
you lied to me,
I’m upset that from
now on I can’t
believe you”
--- Friedrich Nietzsche ---
![Page 9: Who will guard the guards themselves? How to trust your automation and avoid deceit](https://reader034.fdocuments.net/reader034/viewer/2022050614/5a649f8a7f8b9a27568b7ab9/html5/thumbnails/9.jpg)
False positives
Persistent > fix it or fire it
Intermittent > ‘flaky tests’
https://youtu.be/XnkWkrbzMh0
Richard Bradshaw – Selenium Conf Berlin
![Page 10: Who will guard the guards themselves? How to trust your automation and avoid deceit](https://reader034.fdocuments.net/reader034/viewer/2022050614/5a649f8a7f8b9a27568b7ab9/html5/thumbnails/10.jpg)
![Page 11: Who will guard the guards themselves? How to trust your automation and avoid deceit](https://reader034.fdocuments.net/reader034/viewer/2022050614/5a649f8a7f8b9a27568b7ab9/html5/thumbnails/11.jpg)
![Page 12: Who will guard the guards themselves? How to trust your automation and avoid deceit](https://reader034.fdocuments.net/reader034/viewer/2022050614/5a649f8a7f8b9a27568b7ab9/html5/thumbnails/12.jpg)
![Page 13: Who will guard the guards themselves? How to trust your automation and avoid deceit](https://reader034.fdocuments.net/reader034/viewer/2022050614/5a649f8a7f8b9a27568b7ab9/html5/thumbnails/13.jpg)
![Page 14: Who will guard the guards themselves? How to trust your automation and avoid deceit](https://reader034.fdocuments.net/reader034/viewer/2022050614/5a649f8a7f8b9a27568b7ab9/html5/thumbnails/14.jpg)
![Page 15: Who will guard the guards themselves? How to trust your automation and avoid deceit](https://reader034.fdocuments.net/reader034/viewer/2022050614/5a649f8a7f8b9a27568b7ab9/html5/thumbnails/15.jpg)
![Page 16: Who will guard the guards themselves? How to trust your automation and avoid deceit](https://reader034.fdocuments.net/reader034/viewer/2022050614/5a649f8a7f8b9a27568b7ab9/html5/thumbnails/16.jpg)
False negatives
![Page 17: Who will guard the guards themselves? How to trust your automation and avoid deceit](https://reader034.fdocuments.net/reader034/viewer/2022050614/5a649f8a7f8b9a27568b7ab9/html5/thumbnails/17.jpg)
False negative: exhibit A
![Page 18: Who will guard the guards themselves? How to trust your automation and avoid deceit](https://reader034.fdocuments.net/reader034/viewer/2022050614/5a649f8a7f8b9a27568b7ab9/html5/thumbnails/18.jpg)
100%
code coverage!
False negative: exhibit B
still
![Page 19: Who will guard the guards themselves? How to trust your automation and avoid deceit](https://reader034.fdocuments.net/reader034/viewer/2022050614/5a649f8a7f8b9a27568b7ab9/html5/thumbnails/19.jpg)
If you want to trust your tests,
start building tests you can trust on
![Page 20: Who will guard the guards themselves? How to trust your automation and avoid deceit](https://reader034.fdocuments.net/reader034/viewer/2022050614/5a649f8a7f8b9a27568b7ab9/html5/thumbnails/20.jpg)
![Page 21: Who will guard the guards themselves? How to trust your automation and avoid deceit](https://reader034.fdocuments.net/reader034/viewer/2022050614/5a649f8a7f8b9a27568b7ab9/html5/thumbnails/21.jpg)
Keep your tests fresh
Do code review on your test code
Test your tests periodically
Review your suite from time to time
Don’t be afraid to throw away tests
![Page 22: Who will guard the guards themselves? How to trust your automation and avoid deceit](https://reader034.fdocuments.net/reader034/viewer/2022050614/5a649f8a7f8b9a27568b7ab9/html5/thumbnails/22.jpg)
Tools to help you
keep your tests fresh
![Page 23: Who will guard the guards themselves? How to trust your automation and avoid deceit](https://reader034.fdocuments.net/reader034/viewer/2022050614/5a649f8a7f8b9a27568b7ab9/html5/thumbnails/23.jpg)
Mutation testing
http://pitest.org/
(Java)
![Page 24: Who will guard the guards themselves? How to trust your automation and avoid deceit](https://reader034.fdocuments.net/reader034/viewer/2022050614/5a649f8a7f8b9a27568b7ab9/html5/thumbnails/24.jpg)
Property-based testing
https://github.com/pholser/junit-quickcheck
(Java)
http://blog.jessitron.com/2013/04/property-based-testing-what-is-it.html
![Page 25: Who will guard the guards themselves? How to trust your automation and avoid deceit](https://reader034.fdocuments.net/reader034/viewer/2022050614/5a649f8a7f8b9a27568b7ab9/html5/thumbnails/25.jpg)
![Page 26: Who will guard the guards themselves? How to trust your automation and avoid deceit](https://reader034.fdocuments.net/reader034/viewer/2022050614/5a649f8a7f8b9a27568b7ab9/html5/thumbnails/26.jpg)
_Email: [email protected]
_Blog: www.ontestautomation.com
_LinkedIn: www.linkedin.com/in/basdijkstra
_Twitter: @_basdijkstra