Post on 24-Apr-2018
Main Author - Navaneetha Kowdle (Associate Director)
Co – Author - Akshay Mullick (Consultant)
Cognizant Technologies
An Automated approach towards Dev-Ops based continuous Regression Testing for SAP Landscapes
2
Abstract
SAP implementations can be extremely complex and test repositories, built over a time,can be voluminous
The constant need for changes impact existing functionality continuously and QA ingeneral & Regression testing in particular are key activities to ensure nothing is broken
Many of the world’s largest implementers of SAP are continuously looking to improve theeffectiveness of an already mature QA organization.
Regression Testing, although a key activity, takes time and effort for each release and QAorganizations are looking at making this effective and agile
Focused testing using an impact based approach can help reduce RT scope whilstmaintaining efficacy
A customized Dev-Ops solution for Regression Testing SAP systems can dramaticallyreduce time and effort
This presentation presents a successful implementation of a dev-ops based continuousRegression Testing approach using Impact analysis
3
Covered in Two Phases
Phase 1 - Impact Analysis - Tool Selection & implementation
Phase 2 - Continuous RT – adaption of Shift Left approach using DevOps
An Automated approach towards Dev-Ops based continuous Regression Testing for SAP Landscapes
Phase 1 - Erstwhile approach Vs Impact based approach
Optional Scenarios fragile(dependency on skills of individuals)
Baseline TestingDriven by business critical scenarioscovering core processes
Understanding the changed objectsLimiting Test Scope
Normal RT Scoping : Impact based RT Scoping :
“Gap Identification” will identify that our RT Capability misses certain test cases and triggers creation of those
before RT cycle
RegressionTest Scope“Baseline”
Why BPCA (Impact Analysis Tool) for SAP
BPCA is a part of SAP-SolMan Suite
Provided a means to make use of existing Test repository to build a blueprint, thus providing a 1-1 mapping.
BPCA is capable of understanding existing test assets’ coverage (TBoMs)
More granular than other tools compared, thus providing finer impact (Subroutine/Form level).
Features such as configurable Test Scope Optimization / Multiplicity parameter to define extent of testing
Advantageous in building customized solution for continuous testing
Implementing BPCA based RT– Key Steps
Business Blueprint
Set up of business process hierarchy Business Requirements TBOM Creation (Static/Semi-
Dynamic/Dynamic)
CHARM
Change Document information Transport Information
Business Process Change Analyzer
Impact Analysis Test Scope Optimization Multiplicity Parameter
Connection Setup
Test Planning &Setup
TAO/UFT Scripts Re-structuring Requirements Mapping Test Lab Creation
Test Execution
Impacted Automated Test Scenario Execution Impacted Manual Test Scenario Execution
Defect Tracking
Defect raised and tracked for failed scenarios
Pain Points encountered during BPCA implementation
Effort for TBoM creation Manual Also TAO with Association, PFA Consolidation, link to HP ALM
Requirements module OSS Notes for enabling TBoM creation (auto and manual) – guinea pig
approach TAO scenarios restructuring – PFA consolidation etc to Need to review HP ALM hierarchical structure in order to align with Blueprint /
BPCA needs Technical Connection Setup with HP ALM Approach for auto TBoM creation for UFT scenarios Connections between SAP SolMan and Managed Systems
Regression testing – Impact based approach
BPCA & TSO
S7D4713S7D4714
S7D4716
S7D4717
S7D4712
QA Environment
HP Quality Center
Developer implements the change and release the transport
Change and Transport information flows to CHARM
BPCA & TSO Executed with Transports collected from CHARM
Test Lab created of impacted Scenarios after TSO from BPCA
Automated Test Scenarios selected and executed in Pre-PRD
Pre-Prd Environment
S7D4716
S7D4716
S7D4716
S7D4716
Phase 2 -Taking it Further – Pioneering work Increasing Test Coverage Continuous RT – adaption of Shift Left approach using DevOps
-
H/HH/H
Unit / Component / Assembly Test
Build
Functional Design
Technical Design
Integration TestCycle 1 / 2
Regression testing ensures that a
change or fix has not caused faults in unchanged parts of
the system. User acceptance testing ensures that changes
meet the business
requirements.
Integration testing ensures that the e2e process as described
in the design documents function
as described.
“The V Model”
Gate 0
Gate 1
New or changed requirements are brought to production by projects. Fixes
to incidents and problems are brought to production by
application maintenance.
Gate 2A
Gate 3
Gate 2B
Business Requirements
User Acceptance Test
Regression Test
Focus on testing „new“ functionality
…limits speed and flexibility
LightRegression
Test-
H/HH/H
Unit / Component /
Assembly Test
Build
User Acceptance Test
Functional Design
Regression Test
Technical Design
BusinessRequirements
Integration TestCycle 1 / 2
Gate 2B
Gate 0
Gate 1 Gate 2A
Gate 3
…will enable Fast Track Deploymentsnot compromising on resiliencefinally resulting into Continuous Deployment…
“Shift Left”
Focus on resilience
Manual Process
Manual Process
Manual Process
Manual Process
Continuous Automated Regression Testing
BPCA & TSO
S7D4713S7D4714
S7D4716
S7D4717
S7D4712
QA Environment
HP Quality CenterS7D4716
Test Data Solution
Automated Inbuilt
Mechanism
Auto scenarios
Automated BPCA
ExecutionAutomated
Test Lab Creation
Automated Test
Selection
Automated Test
Execution
Benefits seen post implementation
Test Coverage
Gaps identified which are not covered under Baseline Test Scenarios.
Gap Fulfillment increased test coverage by approximately 15% in year 1 .
Cost
Save cost by engaging less number of testing resources due to targeted testing.
Reduced Regressing Cycle time saved Testing Cost Early defect detection/correction reduced
debug/retest effort significantly
Accelerated Approach
Shortened overall Release Cycle window by approximately 3 working days
Defects are caught earlier in Release Cycle.
Tangible Benefits
Up to 60% reduction in testing effort.
Brought in 25% more agility which helps in faster “Time to Market”.
14
Author Biography
Navneeth Kowdle is an Associate Director with the Process & Qualityconsulting division ( a part of Cognizant Business Consulting group). In thiscapacity , his role entails overall management of PQC practice in theCommunication ,Media and Technology space, in addition topersonally getting engaged with customers in providing consultancy servicesin niche areas of Quality Assurance such as DevOps, Business ProcessTesting , QA transformation strategy etc
Akshay Mullick is working as an Consultant with the Process & Qualityconsulting division ( a part of Cognizant Business Consulting group. His role isto act as a bridge between business and technology for client and proposesolutions which addresses their problem statement. His area of expertise isAgile, Dev-Ops, Change Management, Tools & Automation around Impactbased testing
Main Author
Co - Author