Automation Pyramid - Make it right!
-
Upload
sachin-natu -
Category
Software
-
view
200 -
download
1
description
Transcript of Automation Pyramid - Make it right!
![Page 1: Automation Pyramid - Make it right!](https://reader033.fdocuments.net/reader033/viewer/2022042601/5485ebf3b4af9f6e0d8b4f77/html5/thumbnails/1.jpg)
Achieving the Right Testing Pyramid
![Page 2: Automation Pyramid - Make it right!](https://reader033.fdocuments.net/reader033/viewer/2022042601/5485ebf3b4af9f6e0d8b4f77/html5/thumbnails/2.jpg)
Background
5%
95% No Automation
Automation Coverage
![Page 3: Automation Pyramid - Make it right!](https://reader033.fdocuments.net/reader033/viewer/2022042601/5485ebf3b4af9f6e0d8b4f77/html5/thumbnails/3.jpg)
Background
1.5 Months4.5 Months
0 1 2 3 4 5 6 7
Months
Typical Release Cycle
Development Regression
![Page 4: Automation Pyramid - Make it right!](https://reader033.fdocuments.net/reader033/viewer/2022042601/5485ebf3b4af9f6e0d8b4f77/html5/thumbnails/4.jpg)
Background
![Page 5: Automation Pyramid - Make it right!](https://reader033.fdocuments.net/reader033/viewer/2022042601/5485ebf3b4af9f6e0d8b4f77/html5/thumbnails/5.jpg)
Critical Issues reported from Production
0
2
4
6
8
10
12
Critical Data Issues Cosmetic Localization
Issues
![Page 6: Automation Pyramid - Make it right!](https://reader033.fdocuments.net/reader033/viewer/2022042601/5485ebf3b4af9f6e0d8b4f77/html5/thumbnails/6.jpg)
Background
![Page 7: Automation Pyramid - Make it right!](https://reader033.fdocuments.net/reader033/viewer/2022042601/5485ebf3b4af9f6e0d8b4f77/html5/thumbnails/7.jpg)
After Automation Introduced
Selenium Tests 45%
Unit & Integration Tests 20%
No Automation 35%
Automation Coverage
![Page 8: Automation Pyramid - Make it right!](https://reader033.fdocuments.net/reader033/viewer/2022042601/5485ebf3b4af9f6e0d8b4f77/html5/thumbnails/8.jpg)
After Automation Introduced
2.5 Months 3 Weeks
0 0.5 1 1.5 2 2.5 3 3.5
Months
Typical Release Cycle
Development Regression
![Page 9: Automation Pyramid - Make it right!](https://reader033.fdocuments.net/reader033/viewer/2022042601/5485ebf3b4af9f6e0d8b4f77/html5/thumbnails/9.jpg)
Critical Issues reported from Production
0
2
4
6
8
10
12
Critical Data Issues Cosmetic Localization
After automation
Before automation
![Page 10: Automation Pyramid - Make it right!](https://reader033.fdocuments.net/reader033/viewer/2022042601/5485ebf3b4af9f6e0d8b4f77/html5/thumbnails/10.jpg)
State of the Product
Tightly coupled with UI
Browser Dependent Data tests
False fails
Difficult to pinpoint real culprit
Difficult to maintain and time consuming as test are fragile
Lagging with current release development
Image Courtesy: Alex Yakyma
![Page 11: Automation Pyramid - Make it right!](https://reader033.fdocuments.net/reader033/viewer/2022042601/5485ebf3b4af9f6e0d8b4f77/html5/thumbnails/11.jpg)
State of the Product
Manual Checking
End- to End Selenium Tests 70%
Integration Tests20%
Unit Tests10%
![Page 12: Automation Pyramid - Make it right!](https://reader033.fdocuments.net/reader033/viewer/2022042601/5485ebf3b4af9f6e0d8b4f77/html5/thumbnails/12.jpg)
Analysis of Bugs Report
10%
90% Data Failures
Bug Report Analysis
UI Navigation Failures
![Page 13: Automation Pyramid - Make it right!](https://reader033.fdocuments.net/reader033/viewer/2022042601/5485ebf3b4af9f6e0d8b4f77/html5/thumbnails/13.jpg)
Our transition story…
• We decided to "Test things right , at right place on right time" which would result in "right testing pyramid“
![Page 14: Automation Pyramid - Make it right!](https://reader033.fdocuments.net/reader033/viewer/2022042601/5485ebf3b4af9f6e0d8b4f77/html5/thumbnails/14.jpg)
Our transition story…
Application UI
Service Layer
DAO Layer
Automated UI Tests
Tightly Coupled
Fragile
Lag with current development
Automate One Level Below UI
UI Independent
Robust & Stable
With current dev cycle
Fast Execution
![Page 15: Automation Pyramid - Make it right!](https://reader033.fdocuments.net/reader033/viewer/2022042601/5485ebf3b4af9f6e0d8b4f77/html5/thumbnails/15.jpg)
Introduced BDD with Cucumber
![Page 16: Automation Pyramid - Make it right!](https://reader033.fdocuments.net/reader033/viewer/2022042601/5485ebf3b4af9f6e0d8b4f77/html5/thumbnails/16.jpg)
Current Status
Selenium Tests 10%
Workflow Tests: 40%
No Automation 20%
Unit & Integration Tests 30%
Test Automation Status
![Page 17: Automation Pyramid - Make it right!](https://reader033.fdocuments.net/reader033/viewer/2022042601/5485ebf3b4af9f6e0d8b4f77/html5/thumbnails/17.jpg)
After Right Testing Pyramid
1 Week2.75 Months
0 0.05 0.1 0.15 0.2 0.25 0.3
0 0.5 1 1.5 2 2.5 3 3.5
Months
Typical Release Cycle
Regression Time Development
![Page 18: Automation Pyramid - Make it right!](https://reader033.fdocuments.net/reader033/viewer/2022042601/5485ebf3b4af9f6e0d8b4f77/html5/thumbnails/18.jpg)
Pain points
• Learning curve for the team
• Difficult to implement in case where no API is exposed or tight coupling with UI
• Heavy dependency on pre-populated database
![Page 19: Automation Pyramid - Make it right!](https://reader033.fdocuments.net/reader033/viewer/2022042601/5485ebf3b4af9f6e0d8b4f77/html5/thumbnails/19.jpg)
Looking ahead
• Expose RESTful APIs for all upcoming features
• Move BDD specs to lower layers
• Moving away from baseline database