Tricentis Continuous Testing Platform - KMS SolutionsTricentis Continuous Testing Platform Mike...
Transcript of Tricentis Continuous Testing Platform - KMS SolutionsTricentis Continuous Testing Platform Mike...
1
Tricentis Continuous Testing Platform
Mike Schlabinger, VP Global Sales Enablement
4
What is Continuous Testing?
Continuous Testing is the process of executing automated tests as part of
the software delivery pipeline in order to obtain feedback on the business
risks associated with a software release as rapidly as possible.
• Right Feedback, Right Time, for the Right Stakeholder
• Can happen at any point in the SDLC
5
Cloud-based performance labs
at the fingertips of every
developer and tester for on-
demand load testing.
DistributedLoad Testing
Resilient regression testing
across any architecture or
application stack at the speed
of change.
AutomatedContinuous Testing
Scalable, in-sprint test
management for open source
test automation, exploratory
testing and BDD.
Agile Dev Testing
Software Testing Reinvented for Agile and DevOps
Tricentis Continuous Testing Platform
6
Cloud-based performance
testing labs on-demand
DistributedLoad Testing
Resilient regression testing at
the speed of change
AutomatedContinuous Testing
Open source testing and agile
test management
Agile Dev Testing
Software Testing Reinvented for Agile and DevOps
Tricentis Continuous Testing Platform
7
Cloud-based performance
testing labs on-demand
DistributedLoad Testing
Resilient regression testing at
the speed of change
AutomatedContinuous Testing
Open source testing and agile
test management
Agile Dev Testing
Cross Project and Portfolio Visibility
Tricentis Analytics
Tricentis Continuous Testing Platform
8
DistributedLoad Testing
AutomatedContinuous Testing
Agile Dev Testing
2xRisk Coverage
30-40%Cost Reduction
2-3x Release Speed
10xPerformance
Compared to other testing tools
Tricentis Continuous Testing Platform
9
Tricentis AnalyticsMetrics
Joint Analytics & Reporting
Cross-Project Reporting
CustomDashboard
AnalyticsDashboard
ReportingDatabase
ToscaProjects
qTestProjects
10
Integration Roadmap
Risk Based Prioritization
Test Case DesignOptimize
AutomatedContinuous Testing
UI Automation
API TestingAutomate
Active Test Data Management
Test Driven Service VirtualizationProvide
Exploratory Testing
Manage
Test Management
Manual Testing integrate
2
Presented at Tricentis Accelerate Conference in Vienna
12.0, Oct 2018
12.1, Q1 2019 Public availability
1 2
12.2, Q2 2019
3
Public availability
Risk-based Testing and Test Case Design shared qTest and Tosca
Unified Dashboard
1Analytics
Agile Dev Testing
Risk Based Prioritization
Test Case DesignOptimize
3
Active Test Data Management
4
13.0, Q3/Q4 2019
4
Public availability
Active TDM shared qTest and Tosca
Provide
Internal, Subject to Change
11
Expansive Technology SupportO
PE
N
SO
UR
CE
RPG
TE
CH
NO
LO
GY
SU
PP
OR
TD
EV
OP
SS
UP
PO
RT
12
Software Testing Reinvented for DevOps
Make open source tools more productive• Central, agile test management
• Orchestrate execution across tools
• Consolidated analytics and reporting
13
Software Testing Reinvented for DevOps
Language and legacytechnology support
• Single interface for multiple languages
• Model-Based Test Automation for rapid creation, management and execution
• Consolidated analytics and reporting
RPG
14
Software Testing Reinvented for DevOps
Industry’s broadest support for Packaged Applications
• Business focused interface for ease of use
• Model-Based Test Automation for rapid creation, management and execution
• Consolidated analytics and reporting
15
Software Testing Reinvented for DevOps
Complete support for SAP
• Single, business interface for all SAP UIs
• Model-Based Test Automation for rapid creation, management and execution
• Resilient end-to-end tests across technologies
SAP WinGUI SAP Fiori
17
Software Testing Reinvented for DevOps
Agile Dev Testing
• Agile Test Management by providing manual and automated test traceability to Jira issues, bugs, and releases in real-time
• Shift Quality Left with open source test automation management, test script scheduling, and support for BDD
• Predictive Test Analytics to gauge release readiness through executive dashboards & portfolio level reporting
18
Software Testing Reinvented for DevOps
AutomatedContinuous Testing
• Continuous Testing with model-based test automation and support for open source tools throughout the DevTest process
• Central Platform with seamless integration into DevOps toolchains for developers, testers and business users
• High Speed Releases by optimizing and orchestrating end-to-end tests across teams, projects and applications
19
Software Testing Reinvented for DevOps
Distributed LoadTesting
• Easy Test Creation with ability to create tests in open source and proprietary tools for performance testers of all skill levels
• Flexible, Massive Scale by leveraging the cloud through shared or private AWS and Azure integrations that fits into your DevOps pipeline
• Rich Bottleneck Analysis that highlights areas of risk for quick resolution of issues
23
Integrate
Prioritize
Design
Plan Code & Test Design Version Build Test in Sandbox Deploy to Stage
Monitor & RPA
Provision
Integrate
Virtualize
Automate
Configure
Integrate
Explore
Manage
Integrate
Distribute
Execute
Report,
Automate & Integrate
PLAN DESIGN BUILD CONTINUOUS TESTING DEPLOY MONITOR
24
A Sprint in a Tester’s Life
Refinement Sprint
Epic / User Stories created
User Stories (de-)refined
Acceptance criteria (ATDD/BDD) aligned
Model based SV – Test
Model-based API Test
Model based UI Test, Web Desktop & Mobile
Test Data Management
Load Test
Exploratory Testing
As UI-User I want to retrieve customer data via json service (Middleware)
REPORTINGRisk & TCD
Provide Test Artifacts as “Self-Service“ in & across teams
DEVs
QA
PO
GUI Load Test
As a User I want to view customer data (UI)
26
About Tricentis
30
Top Analyst Recognitions
“It should be considered by enterprises that have struggled to make test automation work, and by those seeking to support agilecontinuous automation practices.”
“Tricentis‘ Tosca Testsuite makesautomation easy with model-basedautomation. Via Tosca Testsuite, Tricentisprovides top test automation andoptimization design capabilities, test assetreuse and combined automation.“
31
Dec 2018
Jan 2011
Aug 2014
Dec 2015
Nov 2017
32
Dec 2018
Jan 2011
Aug 2014
Dec 2015
Nov 2017
“It should be considered by enterprises that have struggled to make test automation work, and by those seeking to support agile continuous automation practices.”
33
#44
Awards & Accolades
34
Agile testing for core insurance system,
4.5M lines of code
19K tests → 3.5K tests(-82%)
46% → 93%(2x)
4 weeks → 3 days(9x)
2K tests → 0.6K tests(-70%)
Continuous testing for online banking system
30% → 85%(2.8x)
120 hours → 30 min (240x)
Continuous Testing through a Digital TCoE,
all core systems
6 weeks → 5 days(8.4x)
12K tests → 4.2K tests(-65%)
43% → 92%(2.2x)
35
Bringing standardization to 600+ agile teams with
30+ automation tools
Providing cutting edge digital experiences for
65 million active users
Doubled QA licenses of qTest in 2nd year to
support global teams in Japan and California
Launched PlayStation Store by centralizing QA results
with integrating to Jira
Standardized all hardware and digital
app testing across qTest
More than 400+ test automation agents
connected for a single view of application health
Centralized reporting across 30+ automation tools and frameworks
Clear visibility into manual to automated progress for executives to see QA value
Moving from Legacy QC/ALM to modern agile
practices
Scalable users across multiple NetApp divisions will millions of test results from home grown test
automation
Best of breed approach for agile transform using Jira
Software + qTest
Migrated 25 large scale projects including test cases,
plans, linked traceability
36
Transformed TCOE from Legacy ALM to modern
CI/CD model
Scaling extreme test automation for API, SDK,
UI, & performance
Providing digital customer experiences across web, and mobile
Compliance through centralizing QA results
and integrating with Jira
Scaled open source test automation from hundreds
to thousands
Decreased average regression testing time from 2 days to 2 hours
Reduce test maintenance time through trend
analysis
Substantially reduced critical production defects
from 15 avg to 0
Increased test automation to 83%
Scaled monthly test activity to 153 platforms
tested, 422 builds & 300,000 test runs
Scaled number of test runs executed from 45 to
7,000 per minute
Reduced a test run reporting from hours to
minutes
100%
ManualTesting
0%
AutomatedTesting
TestingPresent
Customer Case Study
There’s a way to do it better. Find it!
Project Charter
11Manual
Testers
4755Manual
Test Cases
?Unknown
Coverage
10Weeks
Execution
Risk Coverage Optimization Test Data Management
48Hours
Execution
Single Agent
8Hours
Execution
Multiple Agents
Customer Case Study
11Manual
Testers
4755Manual
Test Cases
?Unknown
Coverage
10Weeks
Execution
89%Business Risk
Coverage
1193Manual
Test Cases
37%Business Risk
5Weeks
Execution
2.5Weeks
Execution
92%Automated
Test Cases
53%Business Risk
Coverage
Smoke Testing
34Minutes
Execution
Smoke Testing
Test Case Count 7%
Automation (UI & API) & OSV Continuous Integration & Distributed Execution
75% Redundancy 50% Effort Test Data
64% API Tests
8Hours
Execution
Multiple Agents
Customer Case Study
72%Critical Defect Reduction
Production Defects
89%Business Risk
Coverage
1193Manual
Test Cases
5Weeks
Execution
2.5Weeks
Execution
92%Automated
Test Cases
34Minutes
Execution
Smoke Testing
53%Business Risk
Coverage
Smoke Testing
Test Case Count 7%
Test Case Design & Exploratory Testing
64% API Tests
Risk Coverage Optimization Test Data Management Automation (UI & API) & OSV Continuous Integration & Distributed Execution
SAPWithin five months a team of three implemented the coreregression suite, including 333 business scenarios across various business functions such as Financial Processes (Accounts Payable, Accounts Receivable, General Ledger), Procurement, Controlling, HR, Project related Processes, and automatic Invoice Processing, covering 34,237 test actions covering 6,734 verification points. The total execution of the SAP automated regression suite takes less than 24 hours; this would take 518 hours to execute manually. By running the automation suite the total effort is reduced by 96%.
RMS Case Study
42
Questions?