Centralized test automation framework implementation
-
Upload
bharathi-krishnamurthi -
Category
Technology
-
view
4.104 -
download
4
description
Transcript of Centralized test automation framework implementation
IBM Confidential
Centralized Test Automation using Hybrid Model
1
Bharathi Easwaran
AgendaAgenda
2
Automation Decision Making Challenges of Automation Benefits of automation Components of Automation testing Reasons for Automation failure Successful Automation Testing Automation Decision making Process Automation Guidelines for Testers Identify Test Cases to automate Hybrid Framework Approach Elements of a Framework
Factors Decides Automation TestingFactors Decides Automation Testing
3
Test
Au
tom
ati
on
Test
Au
tom
ati
on
Challenges in Test AutomationChallenges in Test Automation
4
Benefits of AutomationBenefits of Automation
AS ALPS | Apr 10, 2023 | IBM Confidential 5
Components of Automation TestingComponents of Automation Testing
6
Automation Frame Work :
Automation Framework is a process of Structuring and
Organizing Scripts to increase the level of reusability
and readability of scripts.
Tool :
Is a mechanism used to simplify and speed the effort
required for testing.
Regression Suite :
A collection of Regression scenarios in the form of
automated build
7
While Talking about automation decision making, we have to list out the reasons for automation failure. This activity will help you to create a quality in automation testing.
Automation Failure Leads to..
Effective Test Execution can’t be continued Unable to achieve the defined ROI Decrease the defect finding Rate Dissatisfaction with Client
Reasons Why Automation Testing Might Fail ?
Successful Successful AutomationAutomation TestingTesting
8
9
Test Automation at ProjectTest Automation at Project
Requirement Analysis Tool Compatibility
• Products Functionality Study
• Record a Script with the available tool for core functional flow as per business
• Run the recorded scripts multiple times with different test base
• Validate the compatibility between tool and the application
• 80% - 90% of the test objects should be able to identified by the tool
Automation Suite
creation
Delivery
Select Framework • Setup the
Automation Environment
• Create automation suites
• Debug the suites
• Validate the suites
• Automation framework
• Regression Suites
• User Manual • Documents
for Process
FrameworkCompatibility
• Project duration• Project Size• Test data
Management• Business
requirement • Automation
Budget
Create a POC
Select Framework
Automation Guidelines for TestersAutomation Guidelines for Testers
10
Modularize test scripts for multiple execution combinations. Identify and abstract common functions used across multiple
test scripts. Structure scripts with minimal dependencies to ensure scripts
can run unattended even when multiple failures occur.
Decouple complex business function testing from navigation, limit-testing, and other simple verification and validation activities.
Abstract and decouple test data from the test scripts.
follow the consistency in Process. Maintain Logs and Documents. Analyze and evaluate the Failures.
Guidelines to Identify Test Cases to automatedGuidelines to Identify Test Cases to automated
11
Validation Yes/No
Accurate Expected Results
Clear Functional Requirement
Consistent Functional Behavior
Dependence on other systems
Core functional component
Test Data Availability and Connectivity
Each regression test case/scenario should be validated based on the Checklist.
IBM Confidential
Centralized Test Automation usingCentralized Test Automation usingHybrid Model Hybrid Model
12
13
Hybrid-Automation FrameworkAutomation Framework :A framework can be a wrapper around complex internal architecture which makes end user to interact with system easily. It also define guidelines and set standards for all phases of ADLC(automation development life cycle).
Hybrid-Framework :The hybrid Test automation framework is what most frameworks evolve into over time and multiple projects. The most successful automation frameworks generally accommodate both Keyword driven as well as Data driven This allows data driven scripts to take advantage of the powerful libraries and utilities that usually accompany a keyword driven architecture. The framework utilities can make the data driven scripts more compact and less prone to failure than they otherwise would have been.
Driver Script
Test Scripts 1
Test Scripts 2
Test Scripts 3
Function Library Data
Hybrid Automation Framework - Diagram
Centralized Automation FrameworkCentralized Automation Framework
14
Automation Script encapsulation :
Function Libraries, Scripts all are encapsulated components, Users will be able to access only the Scripts related to their testing user get prevented from accessing the entire framework scripts.
Systematic Approach: Defines a standard procedures, processes, best practices and guidelines for test automation implementation. A tester can’t create a script on this own style or way for the project test automation.
Quality of automation : The Framework will be evaluated in different levels to object quality in automation.
Framework Design: Hybrid model has been used to design the centralized automation framework.
15
Project 1
Technology -Web
Project 2
Technology Web
New Project 3
with
New Technology (SAP)
Framework need to be upgraded to the New SAP Technology.
Web Function Library
VB Function Library
Java Function Library
Recovery System
General Function Library
Environment
Centralized Framework
Start Up Scripts
Framework ready to implement test automation
Framework ready to implement test automation
Note: Assume that web technology is implemented in framework and “sap” is a new technology that’s need to be
implemented. So Project 1 and project2 can be implemented immediately and project 3 needs an up-gradation in the framework.
Example diagram – Extendable to New technology using Centralized Framework
Features of Centralized Automation FrameworkFeatures of Centralized Automation Framework
Design of the Framework is independent of “Functionality & Technology”
Minimum level of maintenance and framework change requirement
Framework should provide user friendly Regression Suite
Systematic Approach in automation implementation
On the shelf Product – Ready for the current market requirement.
16
Framework Benefits ComparisonFramework Benefits Comparison
Benefits Customized Automation Scripts
Automation Framework(Generic)
Centralized Automation Framework
Cost of Implementation
Low Medium High
Extendable to New technology
No No Yes
Functional Independence
No Yes Yes
Technology independence
No No Yes
Script Maintenance Required
High Medium Low
User Friendly No Yes Yes
Encapsulated Scripts
No Yes Yes
Reliable and Stable No Yes Yes
ROI Low Medium High
Reusability Low Limited High
17
18
Architecture of Centralized Automation Framework Architecture of Centralized Automation Framework
Application
Driver Sheet
Main DataSheet
ResultSheet
Sub Script
IF Flag Y/N
No
Yes
Perform Validation
Driver Script
IF Flag Y/N
Yes
No
Main Script
IF Flag Y/N
No
Yes
Perform Validation
SubSuite
Yes
No
Sub Suite Folder
Regression Suite
Automation Framework Start
Application
Driver Sheet
Main DataSheet
ResultSheet
Sub Script
IF Flag Y/N
No
Yes
Perform Validation
Driver Script
IF Flag Y/N
Yes
No
Main Script
IF Flag Y/N
No
Yes
Perform Validation
SubSuite
Yes
No
Sub Suite Folder
Regression Suite
Automation Framework Start
19
Framework - Work Flow DesignFramework - Work Flow Design
19
Web
App Type
SAP
Object Class
Web Button
Web Edit Box
Operation
Checkpoint
Function
Operation Type
Std.CheckPoint
TxtCheckPoint
User Defined Function Call
Framework design will be independent of functionality and technology
Framework need to be upgraded for new technology and Automation Suites to be upgraded for new functionally, but design remains constant
Application type, Object Class, Object Name, Operation and Operation Type are the deciding factor of the automation flow of execution
Your Data Drives the Flow not the framework
Framework will process the inputs and perform the operation in the application, update the result back to the suite,
If your data is incorrect then the suite will fail
20
High Level Framework Design High Level Framework Design
Driver Sheet :
The Starting Point of Framework and will contain all the information about Regression Suite( like (Path, Suite Name, Module Name and Flag, etc )
Driver Script :
Will check the Flag status of All the Regression Suite in the Driver sheet
Flag = Yes ( Suite need to be executed)
Flag = No ( Suite need not to execute)
Main Script :
It will locate the regression suite and get data from the Main datasheet
It will read all the rows in the datasheet, 1 to n rows,
before process a row, it will check the Flag Status( Yes/No)
Main Datasheet :
Each Row will contain technical data, test data and a flag
One line of Script will be converted into one row in Datasheet
Sub Script and Sub Data Sheet :
We can call Reusable Suites inside your Reg. Suites
Function Library : Set of Reusable Functions
ResultSheet : Customized results for user s better understanding
Driver Script Driver Sheet
Main Script Main Data Sheet
Sub Script Sub Data Sheet
n
n
Frame work Automation Suite
1. Driver Sheet2. Driver Script3. Main Script 4. Sub Script5. Function Library6. Recovery
scenarios
1. Main Data Sheet
2. Object Repository
3. Reusable Suite
4. Result Sheet
21
Steps to Create the Automation suiteSteps to Create the Automation suite
Step 1: Create a Folder for your Automation Suite (Ex. “SupportRequest”)
Step 2: Record the Flow which you need to automate and Save the Flow Script
Step 3: Save the Local Object Repository as Shared Obj. Repository in the Folder (Automation Suite Folder)
Step 4: Use the “Data Generator” to Convert your Flow into Datasheet and save the same In the automation Folder
Step 5: Save a Result Sheet Template
Create a Folder for Regression Suite
Record the scenarios and Save
Save as the Object Repository as “ObjRep.tsr” in Regression Suite Folder
Use “Data Generator” to Convert the Recorded
scripts into Datasheet. Save as the Datasheet as “DataSheet.xls” in
Regression Suite Folder
Insert a Result template in Regression Folder
“Results.xls”
Ready for Execution
22
Driver SheetDriver SheetApplication ModuleName Path Dependencies Flag
CRM Login C:\\Shell Automation\\CRM\\Login Yes
CRM Transactio C:\\Shell Automation\\CRM\\Transaction Yes
CRM Opportunity C:\\Shell Automation\\CRM\\Opportunity NO
SAP SupportRequest C:\\Shell Automation\\SAP\\SupportRequest Yes
Driver sheet is a Starting point of the framework architecture
Driver Sheet is an repository to store all the automation suite informations
Path and Flag are mandatory fields
We can free to add more fields for user reference. ( Like tester name, comments )
23
Flow ScriptFlow Script
23
Flow Script is the script which you recorded to automate
Later Data Generator will covert your flow script into data in a form which the framework can
understand.
24
Main Data SheetMain Data Sheet
Main Data Sheet will have a huge amount of data, they are Flow Data, technical and test data, Test case information's
Flow data : Windows Name, Object Name and Object Class etc.,
Test Data : Account Id, Customer Id, Login Details and URL etc.,
Technical Data : Operation, Operation Type, Parameters, Flag & Comments
Test case Info(Optional ) : Test case Id, Test Description etc.,
Your Flow Scripts will be converted into Main Data and the Same will be saved in Excel
Single Line of Flow Script will be converted into data structure in Main Data Sheet
Main Data Sheet is in a table form, so easy for the end user to understand the flow and execute/ maintain the same,
Main Data Sheet is like a “keyword view” of your flow script
25
Main Data SheetMain Data Sheet
IBM Confidential
Elements of a FrameworkElements of a Framework
26
Common ComponentsCommon Components
Below are the components of theAutomation Framework:
Data Tables Environment Libraries Object repository Results Runtime Scripts Recovery scenario Documents Tools
27
Tools
Data Tables EnvironmentObject
Repository
Scripts
LibrariesRecovery scenario
• Runtime• Documents
Framework
Results
Data TablesData Tables
Data table leads to Parameterization Repository for Test Data Designed for User Interaction Data table can be an Excel Document, Ms-Access, Database Framework will Design the Data table Structure
and mapping
28
Username Password
User1 pwd1
User2 pwd2
User3 pwd3
EnvironmentEnvironment
29
LibrariesLibraries
Time and resource can be saved using reusable functions.
Decomposing the scripts into functions, will increate the level of script Understandably.
Creating functions will reduce the dependencies within the scripts.
Scripts maintenance cost will considerably will come down.
Support Future requirements.
.30
ApplicationDriver Scripts
Function Library
Function library
Collection of Reusable operations in the form of Predefined Structure
Object Repositories / Object MapsObject Repositories / Object Maps
The Storage band for Application Objects
It Provides the interaction between scripts and
application
The object repository is shared and module-specific.
While Execution the Object repository will help the
script to Identify the Objects in the application
Repository will have list of objects along with
configured properties and their values
Object Repository will have enough no.of properties
to uniquely identify an Object
Shared object repository should have the extn :
ABC.tsr (Test Shared Repository)
Module-specific repository should have a extn :
ABV.mtr (Module Test Repository)
31
Object Repository
Object1
Property Name
Property Name
Property Value
Property Value
Object2
Property Name
Property Name
Property Value
Property Value
Structure of Object RepositoryStructure of Object Repository
Tools, Scripts, Recovery scenario & ResultsTools, Scripts, Recovery scenario & ResultsTools : This folder contains macro files. Using these files will reduce time for writing scripts, converting files, and shortcut to generate code. Scripts : This folder contains local test scripts; this folder will be used while development and maintenance stage. Once script gets delivered it will upload to centralize location Test Director or Quality Centre.Recovery scenario : This folder can be used as recovery scenario function library file. Also, through QTP particular scenario, converted file is kept under this folder and associates the test script.Results : The current execution status will be updated after every run. All the Script failures, test status, recovery actions will be logged in Result.
32
QTP Settings
Library Files
Recovery Scenario
s
Object Configurat
ion
Environment
Open Script
Test Script
Run Script
Test Script
Script Failure
Test ScriptResume Executio
n
Execution Completed
Update Result
Close QTP
Thank YouThank You
AS ALPS | Apr 10, 2023 | IBM Confidential 33