DT4Session6/9/163:00PM
WhereIsTestinDevOps?
Presentedby:
NikhilKaul
SmartBearSoftware
Broughttoyouby:
350CorporateWay,Suite400,OrangePark,FL32073888---268---8770··[email protected]://www.techwell.com/
NikhilKaulSmartBearSoftwareNikhilKaulisproductmanager,testingproductsatSmartBearSoftware,theleaderinsoftwarequalitytoolsfortheconnectedworld.NikhilwaspreviouslyatDigité,anapplicationlifecyclemanagementsolutionsprovider,wherehegainedinsightintothesoftwaredevelopmentandtestingmarket.Nikhilisveryinvolvedindiscussionstakingplaceindevelopmentandtestingcommunities.HismostrecentthoughtscanbefoundinForbesandCIO.com.Inhisfreetime,Nikhilenjoysrunning,hiking,andreading.FollowhimonTwitter@kaulnikhil.
4
On our plate today
1
Industry trends
2
What these trends mean for testers
3
Practices that can help facilitate the transformation
7
The way apps are being built is changing
• Cross-platform dev becoming easier
— Microsoft buys Xamarin
— Windows 10
• Universal Windows Apps
• Write once & deploy it across platforms
— Apple making Swift open source
8
Web continues to get more sophisticated
Approval for plugins
Closing APIs exposing
local machines
Technology giving an
access
Responsive UI
Web Components CSS implementation varies
9
Web and mobile worlds colliding
‒ Cordova
‒ PhoneGap
‒ Open web tech
Mobile
‒ HTML
‒ JavaScript
‒ CSS
Web
React, React Native, Combined JSX, getUserMedia , Geolocation
10
Browsers & app stores are becoming locked
Approval time for plugins and apps increasing
Closing APIs exposing
local machines
Technology giving an
access
11
More to test in less time
Cross platform apps
Changes in web & mobile
High quality
Reusability critical
12
Testing in a DevOps world involves deciding
Test Reporting
Test Reduction
Test Design
Where to start
13
Testing in a DevOps world involves deciding
Test Reporting
Test Reduction
Test Design
Where to start
15
Testing in a DevOps world involves deciding
Test Reporting
Test Reduction
Test Design
Where to start
18
Reuse those API tests across SDLC
• API functional test
• API load test
• GUI functional test
• API security test
• Monitoring
Time reduced
by 80%
19
The feedback from UI tests is more exhasutive
Browser: Chrome
End-to-end testing
x x x x
HTML5, Angular JS Network Service/API/Database
21
Create one page object for each page
Home Page of Amazon
Result Search Page
Product Details Page
Confirmation Page
22
Abstraction solves some of the problems
Page Object Class 1
Test
Page Object Class 2
Abstract control identification
Test steps
Abstract control identification
Abstract control actions
Abstract control actions
23
The brittle code now looks like
Magic Strings Locating the control
Abstracting the set text action for the control
Using the abstracted set text action
25
Use PairWise testing to cut down tests
I(0,1)
B (0,1)
U(0,1)
App
B I U
1 1
1 0
0 1
0 0
B I U
1 0
0 0
1 1
0 1
B I U
1 1
1 0
0 1
0 0
B I U
1 1 1
1 0 0
0 1 0
0 0 1
1
2
3
Final Set
27
Numbers often don’t share the right picture
• 300 tests
• Each test has .5% failure rate
• Pass rate for each tests: 99.5%
• Pass rate for the suite: (99.5%)^300 = 22.23%
28
Use statistics for measuring test stability
Sample Size (N)
• 50
P
• 29/50 = .58
1-p
• 21/50
Desired confidence (α)
• 95% confidence
29
Takeaways
1
The role QA is becoming more important
2
Start small and build a modular test design
4
Reuse tests across software development lifecycle
3
API testing is key: Don’t forget about the GUI
Top Related