Journey to 1000 tests a day
-
Upload
teknologika -
Category
Technology
-
view
1.794 -
download
2
description
Transcript of Journey to 1000 tests a day
![Page 1: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/1.jpg)
Journey to 1000 Tests a DayBruce McLeod
Tuesday, 15 November 11
![Page 2: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/2.jpg)
DISCLAIMER
The views and content presented in this presentationare my own and do not represent the views of any of my employers, past or present.
All content is copyright 2011 Bruce McLeod
Creative commons attribution for images is given at the end of this presentation
Tuesday, 15 November 11
![Page 3: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/3.jpg)
Tuesday, 15 November 11
![Page 4: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/4.jpg)
Every journey has a first stepMy introduction to Test Automation
Tuesday, 15 November 11
![Page 5: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/5.jpg)
Record and playback
Tuesday, 15 November 11
![Page 6: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/6.jpg)
Huge gains compared to manual execution
• Win32 desktop application
• 5 different clients (95, 98 Beta, NT4, NT 351 and Citrix)
• 5 days work for 3 people could be done in 4 hours
• 97% efficiency improvement
Tuesday, 15 November 11
![Page 7: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/7.jpg)
But there were problems
• Tests were not maintainable
• Spent days in maintenance and development
• They were brittle and fragile
• And when I left for my next job ...
Tuesday, 15 November 11
![Page 8: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/8.jpg)
Tuesday, 15 November 11
![Page 9: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/9.jpg)
Tuesday, 15 November 11
![Page 10: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/10.jpg)
KindergartenThe early manual coding years
Tuesday, 15 November 11
![Page 11: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/11.jpg)
Manual coding beginsTuesday, 15 November 11
![Page 12: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/12.jpg)
A 2001 test automation recipe
Ingredients1 eager client 1 test application14 test automators
Tuesday, 15 November 11
![Page 13: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/13.jpg)
A 2001 test automation recipe
MethodStep 1 : Write an automation “framework”Step 2 : Create a gui mapStep 3 : Write a testStep 4 : Repeat step 4 another 200 times
Tuesday, 15 November 11
![Page 14: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/14.jpg)
Lessons learnt
• A Successful project• 200 tests automated in 6 months
• Team was a real dev. team with all the same issues• Versioning• Merging and branching• Coding standards• Individual differences in style and design
Tuesday, 15 November 11
![Page 15: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/15.jpg)
Tuesday, 15 November 11
![Page 16: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/16.jpg)
ChildhoodThe innocent years of keyword testing
Tuesday, 15 November 11
![Page 17: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/17.jpg)
What if we remove the need to code?
Tuesday, 15 November 11
![Page 18: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/18.jpg)
Spreadsheets !!!Tuesday, 15 November 11
![Page 19: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/19.jpg)
Keyword driven testing
Object Action DataTextfield (domain) Enter text <domain>
Textfield (username) Enter text <username>
Textfield (password) Enter text <password>Button (login) Click One left click
lObject - Action - data syntax
lAutomation libraries with excel front end
Tuesday, 15 November 11
![Page 20: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/20.jpg)
Keyword driven testing• Advantages• Reduction in coding effort• “Anyone” can write tests!!
• Challenges• “Anyone” can write tests!!• Each implementation is custom• Maintenance effort moves from code into ...
Tuesday, 15 November 11
![Page 21: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/21.jpg)
… Lots of Spreadsheets !!!Tuesday, 15 November 11
![Page 22: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/22.jpg)
Tuesday, 15 November 11
![Page 23: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/23.jpg)
AdolescenceEmbracing open source
Tuesday, 15 November 11
![Page 24: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/24.jpg)
Manual coding returnsTuesday, 15 November 11
![Page 25: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/25.jpg)
Until …
Tuesday, 15 November 11
![Page 26: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/26.jpg)
Tuesday, 15 November 11
![Page 27: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/27.jpg)
Unexpected benefits
• We could now use 1st generation developer tools• Visual Studio 2005• Version control• C# and .net instead of VBA
Tuesday, 15 November 11
![Page 28: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/28.jpg)
Unexpected benefits
• Testers were now true peers to developers• Same tools• Same development language• Same recruitment process• Object oriented design
Tuesday, 15 November 11
![Page 29: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/29.jpg)
Tuesday, 15 November 11
![Page 30: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/30.jpg)
A discovery & epiphany!The “braidy tester” blog posts
Tuesday, 15 November 11
![Page 31: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/31.jpg)
The “sparkle” stack• Used to test Microsoft Expression• Key principles• Test cases written in customer view• Verification and data separated from tests• Application internals re-created in test code
Tuesday, 15 November 11
![Page 32: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/32.jpg)
Tuesday, 15 November 11
![Page 33: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/33.jpg)
How can I execute 1000 tests a day ?
Tuesday, 15 November 11
![Page 34: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/34.jpg)
Design goals
• Maintainability• Robustness• Performance• Maintainability• Rapid test development
Tuesday, 15 November 11
![Page 35: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/35.jpg)
Maintainability
Tuesday, 15 November 11
![Page 36: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/36.jpg)
Devs change stuffIt’s their job !
Tuesday, 15 November 11
![Page 37: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/37.jpg)
Each action should occur onceand once only
Tuesday, 15 November 11
![Page 38: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/38.jpg)
Each control should be mapped onceand once only
Tuesday, 15 November 11
![Page 39: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/39.jpg)
Abstract the engineYou shouldn’t know what’s “under the bonnet”
Tuesday, 15 November 11
![Page 40: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/40.jpg)
Robustness
Tuesday, 15 November 11
![Page 41: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/41.jpg)
Tests should fail, not your tools.
Tuesday, 15 November 11
![Page 42: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/42.jpg)
Embrace regex
Tuesday, 15 November 11
![Page 43: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/43.jpg)
Leverage your vendor, and the community
Tuesday, 15 November 11
![Page 44: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/44.jpg)
Performance
Tuesday, 15 November 11
![Page 45: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/45.jpg)
CPU should be your bottleneck
Tuesday, 15 November 11
![Page 46: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/46.jpg)
Never wait !Wait, don’t sleep
Tuesday, 15 November 11
![Page 47: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/47.jpg)
Scale-out
Tuesday, 15 November 11
![Page 48: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/48.jpg)
Rapid test development
Tuesday, 15 November 11
![Page 49: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/49.jpg)
K.I.S.S
Tuesday, 15 November 11
![Page 50: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/50.jpg)
Cheat !• Leverage an IDE
•Refactoring• Use code generation•Write less code• Use design patterns
Tuesday, 15 November 11
![Page 51: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/51.jpg)
What about the future?
Tuesday, 15 November 11
![Page 52: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/52.jpg)
Tuesday, 15 November 11
![Page 53: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/53.jpg)
Open source frameworks will consolidateDriven by adoption of webdriver
Tuesday, 15 November 11
![Page 54: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/54.jpg)
More focus on mobile automationiOS, Android, Windows Phone
Tuesday, 15 November 11
![Page 55: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/55.jpg)
Behaviour driven developmentWill become mainstream
Tuesday, 15 November 11
![Page 56: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/56.jpg)
Given I am a logged in userWhen I have a valid credit cardAnd I have 1 click purchasing enabledThen I should see the message “your purchase was successful”And I should be re-directed to the home page.
Tuesday, 15 November 11
![Page 57: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/57.jpg)
Tuesday, 15 November 11
![Page 58: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/58.jpg)
Tuesday, 15 November 11
![Page 59: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/59.jpg)
Automation is development.Hire, plan, develop and execute with that in mind.
Tuesday, 15 November 11
![Page 60: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/60.jpg)
Tuesday, 15 November 11
![Page 61: Journey to 1000 tests a day](https://reader037.fdocuments.net/reader037/viewer/2022110118/5556d9d4d8b42ac60c8b45c0/html5/thumbnails/61.jpg)
Image attribution• Creative commons attribution credit is required for the
following users on flickr• seattlemunicipalarchives• mathias-erhart• jaffathecake• 28096801• loty• brilliantmichael
Tuesday, 15 November 11