Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and...
Transcript of Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and...
![Page 1: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/1.jpg)
© 2016 by© 2016 by .
Wolfgang Platz, Founder & CPO
Testing in a DevOps World
![Page 2: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/2.jpg)
awards
Gartner MQLeader
© 2016 by
founded
2007
locations
AustriaGermanySwitzerlandUnited KingdomNetherlandsUSAAustraliaNew ZealandSingapore
support
TSIA 2014Staff Excellence
97% Renewal Rates
Forrester WaveStrong Performer
customers330+ large enterprise customers
![Page 3: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/3.jpg)
calls us a Leader
“Tricentis delivers innovative tools that have provided strong results in driving automation success”.
� “Market-leading, model-based test planning and automation”
� “Market-leading test case design and planning with strong analytic and risk coverage model”
© 2016 by
“It should be considered by enterprises that have struggled to make test automation work, and by those seeking to support agile continuousautomation practices.”
� “Integrated synthetic test data management, service testing and support fortesting analytic applications”
![Page 4: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/4.jpg)
The Future of Testing is …
© 2016 by
![Page 5: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/5.jpg)
… but we can take it!
© 2016 by
![Page 6: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/6.jpg)
Do the right things – and do things right.
Continuous Testing is a must.1
2
© 2016 by
Do the right things – and do things right.
Manual Testers become Automation Specialists.
2
3
![Page 7: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/7.jpg)
The Digital Disruption …
© 2016 by
“Businesses must continuously exploit digital technologies to both create new sources of customer value and increase operational agility in service of customers.”
Forrester, March 2014, “The Future Of Business Is Digital”
![Page 8: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/8.jpg)
… happens at a breathtaking pace
© 2016 by
Source: Tricentis customer, retail industry
![Page 9: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/9.jpg)
Bi-modal business leads to a Bi-modal IT
Systems of
Innovation
Cha
nge
Governance
+ -
Mode 1
Mode 2
Agile
Gartner‘s Perspective:
© 2016 by
Systems of
Record
Cha
nge
Governance
- +
Mode 1
traditional
Agile
![Page 10: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/10.jpg)
Systems of
Engagement
Cha
nge
+Mobile is on the
rise
Big Data and Cloud also
Technology
2x budget growth
over systems of record
BudgetEverybody talks about …
© 2016 by
Systems of
Record
Cha
nge
-Packaged is on
the rise
Sources: Forrester, The Midyear Global Tech Market Outlook For 2015 To 2016; Gartner, Forecast Alert: IT Spending, Worldwide, 2Q15
… you can’t make money without of enterprise IT budgets
80%
![Page 11: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/11.jpg)
If the trend continues …
© 2016 by
Budget allocation of Systems of Record vs. Systems of Innovation; Enterprises, prognosis based on 3% IT growth and 3x growth of Innovation over Record
… unicorns will still be rare animals.
![Page 12: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/12.jpg)
The importance of Systems of Record will not go away.
© 2016 by
Modern development and testing needs to fit all types of systems.
![Page 13: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/13.jpg)
Agile
Waterfall
Others
Use of Software Development Methodologies, 2006 - 2013
?
?
© 2016 by
Agile
Agile development has taken over 2/3 of software development projects. Has it plateaued? Will it replace waterfall completely?
![Page 14: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/14.jpg)
“Agile development for our Systems of Record and for quarterly releases? Really?
© 2016 by
Can’t they just stay waterfall?”
SVP QA, major US bank
![Page 15: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/15.jpg)
How it all began …
© 2016 by
How it all began …
![Page 16: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/16.jpg)
You remember the 80% Nightmare?
01/01/2015 31/05/2015myMajorTask80%From ToTask Name
© 2016 by
80% ready … and it took ages from there to get things done …
![Page 17: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/17.jpg)
The law – 80:20 rule
… we implement 80%
© 2016 by
With 20% of effort …
implement 80% of functionality.
![Page 18: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/18.jpg)
Split tasks!
01/01/2015 31/05/2015myMajorTask
100%
From ToTask Name
mySubTask1 01/01/2015 31/01/2015
© 2016 by
100%
Do not accept anything but 0% or 100%!
mySubTask1 01/01/2015 31/01/2015
mySubTask2 01/02/2015 28/02/2015 100%mySubTask3 01/03/2015 31/03/2015 0%…
![Page 19: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/19.jpg)
Why agile development? – let’s stay waterfall!
In waterfall, customers see the value very late.
What if requirements change?
© 2016 by
t
50% of time/effort10% of value
![Page 20: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/20.jpg)
The law – 80:20 rule
… we can provide 80%
of value.
© 2016 by
With 20% of effort …
of value.
![Page 21: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/21.jpg)
Agile Development!
© 2016 by
t
+
![Page 22: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/22.jpg)
In agile, value is delivered early.
50% of time/effort75% of value
Agile Development!
© 2016 by
t
![Page 23: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/23.jpg)
Agility at different speeds
Systems of
Innovation
Cha
nge
Governance
+ -
Mode 1
Mode 2
Agile
Our perspective:
More
Agile
© 2016 by
Systems of
Record
Cha
nge
Governance
- +
Mode 1
traditional
Agile
Less
Agile
![Page 24: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/24.jpg)
The Core Essentials of Agile
Ups
trea
mEpic/User Story
� Speed - Short iterations (sprints)� Communication - daily meetings (stand-ups)� Ownership - product owners� Feedback - early and frequent� Transparency - burn-down/burn-up
Core essentials
5
© 2016 by
Code/Config
� Transparency - burn-down/burn-up
Dow
nstr
eam
Delivery
� Continuous Integration (CI)
� Continuous Testing (CT)� Continuous Delivery (CD)
Core essentials
3
![Page 25: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/25.jpg)
“Amazon deploys every 11.6 seconds.”
© 2016 by
“Amazon deploys every 11.6 seconds.”
Velocity 2011, May 2011: Jon Jenkins, "Velocity Culture"
![Page 26: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/26.jpg)
Agility at different speedsStaging
area Production
Code/ Config
Epics/ User stories
© 2016 by
Release planning
Release Start
Fully tested release candidates hit the staging area
Deployment
GoLive
Upstream Downstream
Start EndSprint
![Page 27: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/27.jpg)
How often will you deploy?Staging
area Production
Code/ Config
Epics/ User stories
?
© 2016 by
Release planning
Release Start
Fully tested release candidates hit the staging area
Deployment
GoLive
Upstream Downstream
Start EndSprint
?
![Page 28: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/28.jpg)
The better metric (than deploys per day):
© 2016 by
Lead time = How long does it take from code committed to code successfully in production?
![Page 29: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/29.jpg)
Agile Dev Operationsmeets
Dev Ops
© 2016 by
“DevOps is all about Continuous Testing!”http://devops.com/2015/02/02/devops-continuous-testing/
![Page 30: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/30.jpg)
How to get there …
Continuous Testing is a must.1
© 2016 by
How to get there …
![Page 31: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/31.jpg)
What customers are asking1 …
“Are we testing too much?”
“Despite usage of so many automation tools, our cost and cycle time of testing
© 2016 by
is not reducing significantly. Why?”
1 … Source: Accenture customer survey, 2015
“We expect testing teams to provide useful insights and feedback whether we can go productive. What do numbers of test cases tell us?”
![Page 32: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/32.jpg)
Customer Survey1
18% Everybody
© 2016 by
1 … Customer analysis Tricentis
18%average test automation
degree
Everybodywants to invest in test automation
![Page 33: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/33.jpg)
© 2016 by
Automate tests.Do things right.
![Page 34: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/34.jpg)
40%average risk coverage
achieved in comprehensive enterprise test portfolios1
67%average level of
redundancy in enterprise test portfolios1
© 2016 by
1 … Source: Customer projects, Tricentis
enterprise test portfolios1 test portfolios1
2 out of 3 test cases never yield defects.
60% of risk is secured through
hope & pray.
![Page 35: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/35.jpg)
37.9%… only …
© 2016 by
Can you believe it?
37.9%… want to improve test
case quality
![Page 36: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/36.jpg)
Do the mess for less?
© 2016 by
Automate tests.Do things right.
![Page 37: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/37.jpg)
Do the right things. Have the right test cases.Test less.
© 2016 by
Do things right. Automate tests.Run faster.
![Page 38: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/38.jpg)
Do the right things – Risk Based Testing
© 2016 by
Do the right things – Risk Based Testing
![Page 39: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/39.jpg)
Today’s reporting is based on numbers of test cases.
Version 7.5 42,278 10,086 910
© 2016 by
What does it tell us?Is 53,274 the right number of test cases?
Can I go productive?
![Page 40: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/40.jpg)
The law – (almost) 80:20 again
… we cover 80% of our
business risk.
© 2016 by
With 30% of test cases …
business risk.
![Page 41: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/41.jpg)
66% 9% 15% 10%Core Bank
Risk Coverage [%]
© 2016 by
We need to introduce a new currency in testing.
Risk Coverage
![Page 42: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/42.jpg)
Do things right - Automate
© 2016 by
Do things right - Automate
![Page 43: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/43.jpg)
80 %Manual Tests Manual Testers
Manual testing is dominant.
© 2016 by 43
Automated UI Tests
API Tests
20 %Automated Tests
Automation Engineers
Today
Test automation requiresprogrammers.
![Page 44: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/44.jpg)
Today, test automation is adevelopers discipline.
© 2016 by
![Page 45: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/45.jpg)
So many tools you need to learn as a TA Developer …
© 2016 by
![Page 46: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/46.jpg)
You sometimes feel like this?
© 2016 by
![Page 47: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/47.jpg)
Remaining Manual Tests (Exploratory)
Automated UI Tests80 %
Manual Tests
< 15 %
Manual Tests
© 2016 by
UI
API
Today Target state(required)
API Tests
Orchestrated Service Virtualization
47
20 %Automated Tests
> 85 %
Test Automation
![Page 48: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/48.jpg)
Set-up
Maintenance
Efficiency of Test Access
Task
4x
6x
FactorAPI
testing
25%
16%
UI test automation
100%
100%
© 2016 by
Runtime
Timing
6x
100+ x<1%
progressive
100%
regressive
Future Test Automation will focus on API testing.
![Page 49: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/49.jpg)
Remaining Manual Tests (Exploratory)
Automated UI Tests80 %
Manual Tests
< 15 %
Manual Tests
© 2016 by
UI
API
Today Target state(required)
API Tests
Orchestrated Service Virtualization
49
20 %Automated Tests
> 85 %
Test Automation
![Page 50: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/50.jpg)
With the unprecedentedneed for automation,will today’s manual testersall become developers?
© 2016 by
all become developers?
![Page 51: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/51.jpg)
Exploratory Tests Manual Testers
API Tests
Automated UI Tests
Automation Specialists(former Manual Testers) 10Some of them will.
How about all the
© 2016 by 51
Orchestrated Service Virtualization
API Tests
Automation Engineers 1
:
How about all the others?
![Page 52: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/52.jpg)
Logical Name
Enter Vehicle Data
Make
Engine Performance [kW]
Number of Seats
Fuel
List Price [$]
Annual Mileage
Usage
Year of Construction
License Plate Number
Create Vehicle Insurance
Value
Audi
200
5
Petrol
35.000
10.000
Private
{Year}
B-CD 123
Action
Input
Input
Input
Input
Input
Input
Input
Input
Input
Model-BasedBrowser
Script-BasedBrowser
TestEasy to read. No programming skills.
TestEasy maintenance for UI & API tests.
TestMaximum reuse across technologies.
CodeSynchronization Behavior
CodeRecovery Handling
CodeData Driven Framework
1
2
3
© 2016 by
Enter Product Data
Next
Enter Insurant Data
Verify Quote Details
Next
Start Date
Insurance Sum [$]
Payment Option
Price List
Insurance Sum
Gross Premium [$]
{CLICK}
{CLICK}
01/03/2016
7Mio
Yearly
1.535,22
Input
Input
Input
Input
Input
Verify
Select
Select
Business Information Do you see the business information?
TestRobust against changes.
Maximum reuse across technologies. Data Driven Framework
CodeKeyword Driven Framework
3
4
Assert. AreEqual(" 1.535,22",driver.FindElement(By.XPath("//table[@id=' PriceList']/tbody/tr[ 10]/td[ 2]/b")).GetAttribute(" value"));
![Page 53: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/53.jpg)
Logical Name
Enter Vehicle Data
Make
Engine Performance [kW]
Number of Seats
Fuel
List Price [$]
Annual Mileage
Usage
Year of Construction
License Plate Number
Create Vehicle Insurance
Value
Audi
200
5
Petrol
35.000
10.000
Private
{Year}
B-CD 123
ActionAPI Test Case
Webservice
Logical Name
Enter Vehicle Data
Make
Engine Performance [kW]
Number of Seats
Fuel
List Price [$]
Annual Mileage
Usage
Year of Construction
License Plate Number
Create Vehicle Insurance
Value
Audi
200
5
Petrol
35.000
10.000
Private
{Year}
B-CD 123
Action
Input
Input
Input
Input
Input
Input
Input
Input
Input
UI Test CaseBrowser
© 2016 by
Enter Product Data
Enter Insurant Data
Verify Quote Details
Start Date
Insurance Sum [$]
Payment Option
Price List
Insurance Sum
Gross Premium [$]
01/03/2016
7Mio
Yearly
1.535,22
Business Information
Enter Product Data
Next
Enter Insurant Data
Verify Quote Details
Next
Start Date
Insurance Sum [$]
Payment Option
Price List
Insurance Sum
Gross Premium [$]
{CLICK}
{CLICK}
01/03/2016
7Mio
Yearly
1.535,22
Input
Input
Input
Input
Input
Verify
Select
Select
Business Information
Manual Testers become Automation Specialists
![Page 54: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/54.jpg)
Remaining Manual Tests (Exploratory)
Automated UI Tests80 %
Manual Tests
How much time have we got?
© 2016 by 54
UI
API
Today Target state(required)
API Tests
Orchestrated Service Virtualization
20 %Automated Tests
3 years from now
![Page 55: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/55.jpg)
Remaining Manual Tests (Exploratory)
Automated UI Tests80 %
Manual Tests
How much time have we got?
© 2016 by 55
UI
API
Today
API Tests
Orchestrated Service Virtualization
20 %Automated Tests
3 years from now
6 years from now
![Page 56: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/56.jpg)
Tosca - process perspective
© 2016 by
![Page 57: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/57.jpg)
© 2016 by
Tosca is a comprehensive and fully integrated suite.
![Page 58: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/58.jpg)
Do the right things – and do things right.
Continuous Testing is a must.1
2
© 2016 by
Do the right things – and do things right.
Manual Testers become Automation Specialists.
2
3
![Page 59: Testing in a DevOps World - Sogeti.nl in a DevOps World-Wolfgang Platz... · automation work, and by those seeking to support agile continuous automation practices.” “Integrated](https://reader035.fdocuments.net/reader035/viewer/2022062912/5dd0fc4dd6be591ccb63a991/html5/thumbnails/59.jpg)
Thank You for your attention
Wolfgang PlatzStay in touch!
© 2016 by
Email: [email protected]
Follow us on Twitter: @tricentis
59