Frank Cohen - Are We Ready For Cloud Testing - EuroSTAR 2010
Mark Hrynczak Atlassian - EuroSTAR Conference · 2019. 7. 25. · Atlassian $ Testing ≠ Quality...
Transcript of Mark Hrynczak Atlassian - EuroSTAR Conference · 2019. 7. 25. · Atlassian $ Testing ≠ Quality...
![Page 1: Mark Hrynczak Atlassian - EuroSTAR Conference · 2019. 7. 25. · Atlassian $ Testing ≠ Quality ... on your users’ actual experience Tactics to reduce/replace testing. Listen](https://reader035.fdocuments.net/reader035/viewer/2022071109/5fe4cf4895f99e6f6b7ec769/html5/thumbnails/1.jpg)
Do less testing
Mark Hrynczak
Atlassian
![Page 2: Mark Hrynczak Atlassian - EuroSTAR Conference · 2019. 7. 25. · Atlassian $ Testing ≠ Quality ... on your users’ actual experience Tactics to reduce/replace testing. Listen](https://reader035.fdocuments.net/reader035/viewer/2022071109/5fe4cf4895f99e6f6b7ec769/html5/thumbnails/2.jpg)
![Page 3: Mark Hrynczak Atlassian - EuroSTAR Conference · 2019. 7. 25. · Atlassian $ Testing ≠ Quality ... on your users’ actual experience Tactics to reduce/replace testing. Listen](https://reader035.fdocuments.net/reader035/viewer/2022071109/5fe4cf4895f99e6f6b7ec769/html5/thumbnails/3.jpg)
$
![Page 4: Mark Hrynczak Atlassian - EuroSTAR Conference · 2019. 7. 25. · Atlassian $ Testing ≠ Quality ... on your users’ actual experience Tactics to reduce/replace testing. Listen](https://reader035.fdocuments.net/reader035/viewer/2022071109/5fe4cf4895f99e6f6b7ec769/html5/thumbnails/4.jpg)
Testing ≠ Quality
![Page 5: Mark Hrynczak Atlassian - EuroSTAR Conference · 2019. 7. 25. · Atlassian $ Testing ≠ Quality ... on your users’ actual experience Tactics to reduce/replace testing. Listen](https://reader035.fdocuments.net/reader035/viewer/2022071109/5fe4cf4895f99e6f6b7ec769/html5/thumbnails/5.jpg)
“Effective software teams are all alike; every dysfunctional team is dysfunctional in its own way”
L. TOLSTOY
![Page 6: Mark Hrynczak Atlassian - EuroSTAR Conference · 2019. 7. 25. · Atlassian $ Testing ≠ Quality ... on your users’ actual experience Tactics to reduce/replace testing. Listen](https://reader035.fdocuments.net/reader035/viewer/2022071109/5fe4cf4895f99e6f6b7ec769/html5/thumbnails/6.jpg)
ProcessDeveloperRequirements Quality Software
Dysfunctional software team
![Page 7: Mark Hrynczak Atlassian - EuroSTAR Conference · 2019. 7. 25. · Atlassian $ Testing ≠ Quality ... on your users’ actual experience Tactics to reduce/replace testing. Listen](https://reader035.fdocuments.net/reader035/viewer/2022071109/5fe4cf4895f99e6f6b7ec769/html5/thumbnails/7.jpg)
ProcessDeveloperRequirements Low-qualitySoftware
Dysfunctional software team
![Page 8: Mark Hrynczak Atlassian - EuroSTAR Conference · 2019. 7. 25. · Atlassian $ Testing ≠ Quality ... on your users’ actual experience Tactics to reduce/replace testing. Listen](https://reader035.fdocuments.net/reader035/viewer/2022071109/5fe4cf4895f99e6f6b7ec769/html5/thumbnails/8.jpg)
ProcessDeveloperRequirements
Testing Phase
Quality Software
Dysfunctional software team
![Page 9: Mark Hrynczak Atlassian - EuroSTAR Conference · 2019. 7. 25. · Atlassian $ Testing ≠ Quality ... on your users’ actual experience Tactics to reduce/replace testing. Listen](https://reader035.fdocuments.net/reader035/viewer/2022071109/5fe4cf4895f99e6f6b7ec769/html5/thumbnails/9.jpg)
ProcessDeveloperRequirements
Testing Phase
Quality Software
Dysfunctional software team
![Page 10: Mark Hrynczak Atlassian - EuroSTAR Conference · 2019. 7. 25. · Atlassian $ Testing ≠ Quality ... on your users’ actual experience Tactics to reduce/replace testing. Listen](https://reader035.fdocuments.net/reader035/viewer/2022071109/5fe4cf4895f99e6f6b7ec769/html5/thumbnails/10.jpg)
ProcessDeveloperRequirements
Testing Phase
Quality Software
Dysfunctional software team
![Page 11: Mark Hrynczak Atlassian - EuroSTAR Conference · 2019. 7. 25. · Atlassian $ Testing ≠ Quality ... on your users’ actual experience Tactics to reduce/replace testing. Listen](https://reader035.fdocuments.net/reader035/viewer/2022071109/5fe4cf4895f99e6f6b7ec769/html5/thumbnails/11.jpg)
ProcessDeveloperRequirements
Testing Phase
Quality Software
Dysfunctional software team
![Page 12: Mark Hrynczak Atlassian - EuroSTAR Conference · 2019. 7. 25. · Atlassian $ Testing ≠ Quality ... on your users’ actual experience Tactics to reduce/replace testing. Listen](https://reader035.fdocuments.net/reader035/viewer/2022071109/5fe4cf4895f99e6f6b7ec769/html5/thumbnails/12.jpg)
ProcessDeveloperRequirements
Testing Phase
Quality Software
Dysfunctional software team
![Page 13: Mark Hrynczak Atlassian - EuroSTAR Conference · 2019. 7. 25. · Atlassian $ Testing ≠ Quality ... on your users’ actual experience Tactics to reduce/replace testing. Listen](https://reader035.fdocuments.net/reader035/viewer/2022071109/5fe4cf4895f99e6f6b7ec769/html5/thumbnails/13.jpg)
Conflict in roles
TesterWrites the test code
DeveloperWrites the functional code
![Page 14: Mark Hrynczak Atlassian - EuroSTAR Conference · 2019. 7. 25. · Atlassian $ Testing ≠ Quality ... on your users’ actual experience Tactics to reduce/replace testing. Listen](https://reader035.fdocuments.net/reader035/viewer/2022071109/5fe4cf4895f99e6f6b7ec769/html5/thumbnails/14.jpg)
Conflict in aims
TesterWants to block release
DeveloperWants to release
![Page 15: Mark Hrynczak Atlassian - EuroSTAR Conference · 2019. 7. 25. · Atlassian $ Testing ≠ Quality ... on your users’ actual experience Tactics to reduce/replace testing. Listen](https://reader035.fdocuments.net/reader035/viewer/2022071109/5fe4cf4895f99e6f6b7ec769/html5/thumbnails/15.jpg)
Conflict in attitude
TesterCares about quality
DeveloperDoes not care about quality
![Page 16: Mark Hrynczak Atlassian - EuroSTAR Conference · 2019. 7. 25. · Atlassian $ Testing ≠ Quality ... on your users’ actual experience Tactics to reduce/replace testing. Listen](https://reader035.fdocuments.net/reader035/viewer/2022071109/5fe4cf4895f99e6f6b7ec769/html5/thumbnails/16.jpg)
Conflict in perception
TesterValue is implicit and unmeasurable
DeveloperDelivers explicit value to the business
![Page 17: Mark Hrynczak Atlassian - EuroSTAR Conference · 2019. 7. 25. · Atlassian $ Testing ≠ Quality ... on your users’ actual experience Tactics to reduce/replace testing. Listen](https://reader035.fdocuments.net/reader035/viewer/2022071109/5fe4cf4895f99e6f6b7ec769/html5/thumbnails/17.jpg)
Process Quality SoftwareDeveloperRequirements
Dysfunctional software team
![Page 18: Mark Hrynczak Atlassian - EuroSTAR Conference · 2019. 7. 25. · Atlassian $ Testing ≠ Quality ... on your users’ actual experience Tactics to reduce/replace testing. Listen](https://reader035.fdocuments.net/reader035/viewer/2022071109/5fe4cf4895f99e6f6b7ec769/html5/thumbnails/18.jpg)
High-qualityProcess
Quality SoftwareDeveloperRequirements
Dysfunctional software team
![Page 19: Mark Hrynczak Atlassian - EuroSTAR Conference · 2019. 7. 25. · Atlassian $ Testing ≠ Quality ... on your users’ actual experience Tactics to reduce/replace testing. Listen](https://reader035.fdocuments.net/reader035/viewer/2022071109/5fe4cf4895f99e6f6b7ec769/html5/thumbnails/19.jpg)
High-qualityProcess
Quality SoftwareRequirements High-qualityDeveloper
Dysfunctional software team
![Page 20: Mark Hrynczak Atlassian - EuroSTAR Conference · 2019. 7. 25. · Atlassian $ Testing ≠ Quality ... on your users’ actual experience Tactics to reduce/replace testing. Listen](https://reader035.fdocuments.net/reader035/viewer/2022071109/5fe4cf4895f99e6f6b7ec769/html5/thumbnails/20.jpg)
High-qualityProcess
Quality SoftwareHigh-qualityDeveloper
High-qualityRequirements
Effective software team
![Page 21: Mark Hrynczak Atlassian - EuroSTAR Conference · 2019. 7. 25. · Atlassian $ Testing ≠ Quality ... on your users’ actual experience Tactics to reduce/replace testing. Listen](https://reader035.fdocuments.net/reader035/viewer/2022071109/5fe4cf4895f99e6f6b7ec769/html5/thumbnails/21.jpg)
Testing ≠ Quality
![Page 22: Mark Hrynczak Atlassian - EuroSTAR Conference · 2019. 7. 25. · Atlassian $ Testing ≠ Quality ... on your users’ actual experience Tactics to reduce/replace testing. Listen](https://reader035.fdocuments.net/reader035/viewer/2022071109/5fe4cf4895f99e6f6b7ec769/html5/thumbnails/22.jpg)
Do less testing
![Page 23: Mark Hrynczak Atlassian - EuroSTAR Conference · 2019. 7. 25. · Atlassian $ Testing ≠ Quality ... on your users’ actual experience Tactics to reduce/replace testing. Listen](https://reader035.fdocuments.net/reader035/viewer/2022071109/5fe4cf4895f99e6f6b7ec769/html5/thumbnails/23.jpg)
What is testing for?What are the ongoing costs?
![Page 24: Mark Hrynczak Atlassian - EuroSTAR Conference · 2019. 7. 25. · Atlassian $ Testing ≠ Quality ... on your users’ actual experience Tactics to reduce/replace testing. Listen](https://reader035.fdocuments.net/reader035/viewer/2022071109/5fe4cf4895f99e6f6b7ec769/html5/thumbnails/24.jpg)
Automated testingIs it safe to release?
Manual testingCan we find bugs?
![Page 25: Mark Hrynczak Atlassian - EuroSTAR Conference · 2019. 7. 25. · Atlassian $ Testing ≠ Quality ... on your users’ actual experience Tactics to reduce/replace testing. Listen](https://reader035.fdocuments.net/reader035/viewer/2022071109/5fe4cf4895f99e6f6b7ec769/html5/thumbnails/25.jpg)
Automated testing is not cost free
(Virtual) Hardware Time to run
Framework & implementation Ongoing maintenance
![Page 26: Mark Hrynczak Atlassian - EuroSTAR Conference · 2019. 7. 25. · Atlassian $ Testing ≠ Quality ... on your users’ actual experience Tactics to reduce/replace testing. Listen](https://reader035.fdocuments.net/reader035/viewer/2022071109/5fe4cf4895f99e6f6b7ec769/html5/thumbnails/26.jpg)
Automated testing is not cost free
(Virtual) Hardware Time to run
Framework & implementation Ongoing maintenance
![Page 27: Mark Hrynczak Atlassian - EuroSTAR Conference · 2019. 7. 25. · Atlassian $ Testing ≠ Quality ... on your users’ actual experience Tactics to reduce/replace testing. Listen](https://reader035.fdocuments.net/reader035/viewer/2022071109/5fe4cf4895f99e6f6b7ec769/html5/thumbnails/27.jpg)
Acceptable testing:
Several days of manual testing to find bugs
Absolute reliance on fully automated pipeline
Reducing time between releases from:
Months to weeks
Weeks to days
Days to hours
Every commit
Journey to continuous deployment
![Page 28: Mark Hrynczak Atlassian - EuroSTAR Conference · 2019. 7. 25. · Atlassian $ Testing ≠ Quality ... on your users’ actual experience Tactics to reduce/replace testing. Listen](https://reader035.fdocuments.net/reader035/viewer/2022071109/5fe4cf4895f99e6f6b7ec769/html5/thumbnails/28.jpg)
How to do less testing?
![Page 29: Mark Hrynczak Atlassian - EuroSTAR Conference · 2019. 7. 25. · Atlassian $ Testing ≠ Quality ... on your users’ actual experience Tactics to reduce/replace testing. Listen](https://reader035.fdocuments.net/reader035/viewer/2022071109/5fe4cf4895f99e6f6b7ec769/html5/thumbnails/29.jpg)
How to spend less time doing testing?
![Page 30: Mark Hrynczak Atlassian - EuroSTAR Conference · 2019. 7. 25. · Atlassian $ Testing ≠ Quality ... on your users’ actual experience Tactics to reduce/replace testing. Listen](https://reader035.fdocuments.net/reader035/viewer/2022071109/5fe4cf4895f99e6f6b7ec769/html5/thumbnails/30.jpg)
Prevent
Identify potential problems
before coding
Mitigate
Reduce the impact of
problems to end users
Listen
Adjust the quality bar based
on your users’ actual
experience
Tactics to reduce/replace testing
![Page 31: Mark Hrynczak Atlassian - EuroSTAR Conference · 2019. 7. 25. · Atlassian $ Testing ≠ Quality ... on your users’ actual experience Tactics to reduce/replace testing. Listen](https://reader035.fdocuments.net/reader035/viewer/2022071109/5fe4cf4895f99e6f6b7ec769/html5/thumbnails/31.jpg)
Listen
• Realistic and consistent bug
policy
• Impact-driven priorities
• Don’t sweat the small stuff
Mitigate
• Monitoring
• Staged deployments with
automatic rollbacks
• Feature flags
Prevent
• Kickoffs to identify edge cases
• Paired development
• Train your developers in quality
thinking
Examples
![Page 32: Mark Hrynczak Atlassian - EuroSTAR Conference · 2019. 7. 25. · Atlassian $ Testing ≠ Quality ... on your users’ actual experience Tactics to reduce/replace testing. Listen](https://reader035.fdocuments.net/reader035/viewer/2022071109/5fe4cf4895f99e6f6b7ec769/html5/thumbnails/32.jpg)
AutomateMachines tell you that deployment is safe
TrustDevelopers are allies with the same goals
MeasureTesting has costs, especially time-related
ReplaceAlternative ways to engineer quality software
Takeaways MindsetGoal is quality software, not testing activity