Automatic test automationAutomatic test automationWhat we do, how far we got, and where do we want to go
2
AgendaAgenda
Our original target
SAP Tools or non SAP tools for testautomation
Fully automatic test of business processes
How far did we manage
Fully automatic test of output management
Perspectives
3
AgendaAgenda
Our original target
SAP Tools or non SAP tools for testautomation
Fully automatic test of business processes
How far did we manage
Fully automatic test of output management
Perspectives
4
Targets: TestautomationTargets: Testautomation
Testautomation of the essential business processes in order to confirm the correct functionality of the software
to simulate the essential business processes covered by the software to apply fully automatic tests of very complex business processes to significantly reduce staff required for the manual test Shorter lead times, optimizing the use of available testing ressources Precise repeatability of test scenarios across multiple releases
Testresult reliabilty Increased product stability and quality Increase the reliabilty of the test performance Increasing the test coverage level
Documentation of test results Automatic generation of test results documentation
5
AgendaAgenda
Our original target
SAP Tools or non SAP tools for testautomation
Fully automatic test of business processes
How far did we manage
Fully automatic test of output management
Perspectives
6
How far did we manageHow far did we manage
Testautomation of the essential business processes in order to confirm the correct functionality of the software
to simulate the essential business processes covered by the software to apply fully automatic tests of very complex business processes to significantly reduce staff required for the manual test Shorter lead times, optimizing the use of available testing ressources Precise repeatability of test scenarios across multiple releases
Testresult reliabilty Increased product stability and quality Increase the reliabilty of the test performance Increasing the test coverage level
Documentation of test results Automatic generation of test results documentation
7
How far did we manageHow far did we manage- the essentials- the essentials
We had to find solutions to these problems: Business processes do not pay attention to the input methods of the
software- Some of these input methods are not supported by eCatt ( IDOC and
batch processing)
In order to achieve fully automatic regression tests we must find a way to produce test data dynamically without human interference
In order to achieve absolute test result reliability, we had to find a way to remove factors which would reduce the sharpness of the test processes
The complicated and complex business processes consist of a large number of steps which need to be performed in a controlled sequence and to exchange parameters in a controlled and secure manner
8
How far did we manageHow far did we manageinput methods input methods
Examples of test processes with various input methods
Dialogue – „normal“ online user transaction – easily supported by eCatt
Batchprocess – Dunning ( occasionally with a user interface to type in the information to be processed – but the processing itself is batch and asynchronous
Creation and processing of input files
Creation and processing of IDOC files
Automatic database inspection checks
9
How far did we manageHow far did we manageDynamic supply of test dataDynamic supply of test data
Some „trivial“ test data problems
Which date do I need for this dunning test proces? Which date is it in two months? How do I know if this person has already been „inserted“ into the system How do I know, which date it is in 20 days – also if it needs to be a bank day
10
How far did we manageHow far did we managetest result reliabilitytest result reliability
How to reach test reliability:
You need to know everything about the test data How old it is Which attributes it has Each and every attribute Each and every connection or link to other data elements
11
How far did we manageHow far did we managecomplex business processescomplex business processes
The means of referenced eCatt scripts does not offer a configurative flexibility
The referenced eCatt scripts would not be the right means for the input methods which can not be processed with eCatt
12
How far did we manageHow far did we manageRequirements - summaryRequirements - summary
If you want to control your test process, you must create and process all required data yourself. You cannot rely on the quality of data already in the system.
You need to be able to produce valid test data automatically and supply this test data also automatically to your test process.
You need to be able to service different kind of input methods in order to be able to offer test of entire business processes.
If you want to provide a fully automatic test service you must focus on relatively low maintenance.
13
How far did we manageHow far did we manageSolution – head linesSolution – head lines
Our test processes generate and, where required, process their test data themselves.
Our test processes support different input methods.
The steps of our test processes are linked together and exchange parameters through configurations (rather simple table entries).
Due to the configuration ability we concentrate our maintenance on changes in the software products which we test.
14
How far did we manageHow far did we manageSolution – head linesSolution – head lines
We developed additional SAP functions in order to embed the SAP eCatt standard functions into a solution which would fit our requirements.
- The addtitionals provide several test configuration functions
- Test process definition
- Test steps parameter exchange
- IDOC configuration
- Setup of test files
- Etc Test performing functions
- Start up and scheduling of test processes (not available in the eCatt environment)
- Utilities to start up the test steps in the correct sequence Test result displaying functions
- Various views on the results of the performed tests
15
AgendaAgenda
Our original target
SAP Tools or non SAP tools for testautomation
Fully automatic test of business processes
How far did we manage
Fully automatic test of output management
Perspectives
16
Fully automatic test of business processesFully automatic test of business processesBasic exampleBasic example
One testprocess contains several test steps
The steps of the testprocess are linked together through table entries
Each test step produces parameters
These parameters can be picked and processed up by succeeding steps
The test steps are also supplied with test data from the standard eCatt test data container
17
Fully automatic test of business processesFully automatic test of business processesTestprocesses – Monitoring overviewTestprocesses – Monitoring overview
18
Fully automatic test of business processesFully automatic test of business processestestprocess – test stepstestprocess – test steps
19
Fully automatic test of business processesFully automatic test of business processestestprocess – test stepstestprocess – test steps
20
Fully automatic test of business processesFully automatic test of business processestestprocess – test parameters and valuestestprocess – test parameters and values
21
FullyFully automatic test of business processesautomatic test of business processesThe parameters are picked up by suceeding stepsThe parameters are picked up by suceeding steps
22
Fully automatic test of business processesFully automatic test of business processesSome examples – screen shotsSome examples – screen shots
To minimize maintenance efforts, it is very important, to create the test scripts as modular as possible. For example, there is the business partner dialogue; in oscare ®, there are 39 different roles for bp’s and according to most of them, the dialogue is different, mainly concerning the number of tab strips, which ones are to process and their order.
Hereinafter two examples are shown by screenshots, at first a school and then an employer are created.
Fully automatic test of business processesFully automatic test of business processesCreation of a school as a business partnerCreation of a school as a business partner
Fully automatic test of business processesFully automatic test of business processesCreation of a school as a business partnerCreation of a school as a business partner
Fully automatic test of business processesFully automatic test of business processesCreation of a school as a business partnerCreation of a school as a business partner
Fully automatic test of business processesFully automatic test of business processesCreation of a school as a business partnerCreation of a school as a business partner
Fully automatic test of business processesFully automatic test of business processesCreation of a school as a business partnerCreation of a school as a business partner
Fully automatic test of business processesFully automatic test of business processesCreation of an employer as a business partnerCreation of an employer as a business partner
Fully automatic test of business processesFully automatic test of business processesCreation of an employer as a business partnerCreation of an employer as a business partner
Fully automatic test of business processesFully automatic test of business processesCreation of an employer as a business partnerCreation of an employer as a business partner
Fully automatic test of business processesFully automatic test of business processesCreation of an employer as a business partnerCreation of an employer as a business partner
Fully automatic test of business processesFully automatic test of business processesCreation of an employer as a business partnerCreation of an employer as a business partner
Fully automatic test of business processesFully automatic test of business processesCreation of an employer as a business partnerCreation of an employer as a business partner
Fully automatic test of business processesFully automatic test of business processesCreation of an employer as a business partnerCreation of an employer as a business partner
Fully automatic test of business processesFully automatic test of business processesCreation of an employer as a business partnerCreation of an employer as a business partner
Fully automatic test of business processesFully automatic test of business processesCreation of an employer as a business partnerCreation of an employer as a business partner
Fully automatic test of business processesFully automatic test of business processesCreation of an employer as a business partnerCreation of an employer as a business partner
38
Fully automatic test of business processesFully automatic test of business processesSome examples – screen shotsSome examples – screen shots
To cover all different requirements with one script, it is neccessary, to encapsulate every tab strip within one single script. All of those scripts are referred within a so called ‚masterscript‘. The testdatacontainer is used, to manage the process by different flags, like for example ‚I_DO_PARTNERINFO‘. If the flag is set, the relevant tab strip is processed, otherwise not.
Moreover, the same tab strips can have different positions, depending on the role. To handle that fact, a table had to be developed, where the information is stored, which tab strip can be found at what place (according to the role), what is it‘s name and what is it‘s ID.
39
Fully automatic test of business processesFully automatic test of business processesSome examples – screen shotsSome examples – screen shots
Below all entries for the ‚Partnerinfo‘-tab are shown:
40
Fully automatic test of business processesFully automatic test of business processesSome examples – screen shotsSome examples – screen shots
Here the protocol entry, that was originated while choosing the ‚Partnerinfo‘-tab strip during runtime, when name and id were determined:
41
AgendaAgenda
Our original target
SAP Tools or non SAP tools for testautomation
Fully automatic test of business processes
How far did we manage
Fully automatic test of output management
Perspectives
42
Fully automatic test of output management Fully automatic test of output management What was the problem and how it was solvedWhat was the problem and how it was solved
A precise and reliable test of the correct generation and formation of RDI and XSF files is rather problematic.
The requirements describing how these fields supposedly should be populated were defined and stored in testautomation.
Functions to examine the correct population of the fields of these output files were developed.
Most of the required business processes had already been test automated.
Eventually we just had to add the output function and connect them to the field inspection functions.
43
Fully automatic test of output management Fully automatic test of output management Two kinds of field inspectionTwo kinds of field inspection
Mainly in order to avoid loosing control of the automated output inspection checks we split the checks up in two categories.
Appearance checks – does the output field actually occur in the output stream?
Content checks – does the output field hold exactly the expected value?
44
Appearance checksAppearance checksRDIRDI
Target: Do all required fields occurr? Did unexpected fields occurr?
How it works: We test automate the business process The steps of the business process
populate the data fields, which should eventually show up in the output streams.
The OPM field documentation of the data streams is transferred to a simple R/3 table
The appearance checks are appended to the business and print part process
OPM checks
Business
process
Printing part
45
Appearance checksAppearance checksExamplesExamples
Comparison with the OPM documentation in order to check that no unexpected fields occurred
Selective check in order to inspect if these three fields occurred.
46
Content ChecksContent ChecksHow it worksHow it works
The content check contains two components
Extraction of the expected field content Configuration table entries set up exactly from which teststeps, exactly which parameter
the expected content of this output field holds. The function that extracts the expected field content reads these configuration entries
and picks out the parameter values.
Comparison - expected with actual output stream fields This function compares the expected field content with the actual stream fields.
47
Content CheckContent Check
OPM checks
Businessprocess
Print part
48
Content ChecksContent Checks
Extraction of expected values
49
Content chekcsContent chekcs
Comparison – expected - actual
50
AgendaAgenda
Our original target
SAP Tools or non SAP tools for testautomation
Fully automatic test of business processes
How far did we manage
Fully automatic test of output management
Perspectives
51
JAVA
ABAP
ORACLEFORMS
ORACLE
JSF
WebDynpro Visual
ComposerWD4J
JSP
WD4A
CRM2007WebClient
UI
BSP
CompositionEnvironment
Composition Tools
AS Java
WebDynpro
SAPGUIWebDyn
pro
Webbrowser
NWBC
Portal
PerspectivesWhere do we go from here
52
AgendaAgenda
Our original target
SAP Tools or non SAP tools for testautomation
Fully automatic test of business processes
How far did we manage
Fully automatic test of output management
Perspectives
53
SAP-EcattSAP native free to use toolVery integrated automation capabilities and interfaces
No serious support of non SAP GUI DialoguesTosca
Acquisition and / or licensing costs
„Videocam“-Technology Films everything also WEB dialogues (understands nothing)No SAP-Integration No SAP InterfacesVery limited (if at all available) test result checks
Compuware und MercuryHigh priced and licensing costs
Films everything also WEB dialogues (understands nothing )
No SAP-IntegrationNo SAP Interfaces
Very limited (if at all available) test result checks
SAP or non SAP tools for testautomation
54
No non SAP product provides a professional SAP integration
With these products you won‘t be able to conduct
Regression tests without manual processing of test data - (this is already the first knock-out)
A detailed query or inspection of the SAP Database content, in order to evaluate the result of yout test operation. (this is how it got the second knock-out)
supply an exact repeatability without sacrifizing the test reliability
If you want to try it anyway, it would require extreme ressources, furthermore the question: "Why don‘t we do the whole test manually anyway" would have to be answered
SAP or non SAP tools for testautomation
55
eCATT or different SAP tools for testautomation?eCATT or different SAP tools for testautomation?
Supported protocols : SAP GUI WebDynpro Java Web, SAP-Web (Portal) Win32 .NET Terminal (3270, 5250) PowerBuilder Oracle Forms, Oracle Applications, Siebel,
PeopleSoft MS Office, VB, Visual Age
License costs : depends
SAP Quick Test Professional by HP
Supported protocols: SAP GUI WebDynpro Java WebDynpro ABAP (only selected objects) SAP Web Services
License costs: none
SAP eCATT
Top Related