Generic API Test tool By Moshe Sapir Almog Masika.
-
Upload
nora-wilkins -
Category
Documents
-
view
215 -
download
0
Transcript of Generic API Test tool By Moshe Sapir Almog Masika.
![Page 1: Generic API Test tool By Moshe Sapir Almog Masika.](https://reader036.fdocuments.net/reader036/viewer/2022082820/56649ee95503460f94bfad0a/html5/thumbnails/1.jpg)
Generic API Test tool
By
Moshe Sapir
Almog Masika
![Page 2: Generic API Test tool By Moshe Sapir Almog Masika.](https://reader036.fdocuments.net/reader036/viewer/2022082820/56649ee95503460f94bfad0a/html5/thumbnails/2.jpg)
Agenda
Testing practice in the industry The Automatic API test tool concept Attributes supplied to the developer Test coverage Design and implementation Tool demonstration
![Page 3: Generic API Test tool By Moshe Sapir Almog Masika.](https://reader036.fdocuments.net/reader036/viewer/2022082820/56649ee95503460f94bfad0a/html5/thumbnails/3.jpg)
Testing practice in the industry
Testing software module requires deep understanding of the module’s inner logic and its interfaces’ semantics
Currently, dedicated tests are written to each software component
Usually the tests do not try to retrieve functional information from the module’s code
Changes in the software logic are resulted in changes in the test automation logic
![Page 4: Generic API Test tool By Moshe Sapir Almog Masika.](https://reader036.fdocuments.net/reader036/viewer/2022082820/56649ee95503460f94bfad0a/html5/thumbnails/4.jpg)
Automatic API Test Tool Concept
Enable the developer to add metadata to the modules code
Use .NET attributes to represent the metadata
The metadata should contain information which will enable automatic testing of the module
Create automatic test tool that given the module and attributes can generate tests
Generic API Test Tool
Tested Module
Tested Code
Test Results
![Page 5: Generic API Test tool By Moshe Sapir Almog Masika.](https://reader036.fdocuments.net/reader036/viewer/2022082820/56649ee95503460f94bfad0a/html5/thumbnails/5.jpg)
Attributes supplied to the developer – Function level attributes
Input-Output Assertion:– Assertion is defined by three expressions
<P, Q, L>
– If P is true before the function is run, then Q is true after the function is run, or one of the exceptions in L is thrown during execution
– For double Sqrt(double x) the developer will define: P: x > 0 Q: ret ^ 2 = x L = {DivisionByZero }
![Page 6: Generic API Test tool By Moshe Sapir Almog Masika.](https://reader036.fdocuments.net/reader036/viewer/2022082820/56649ee95503460f94bfad0a/html5/thumbnails/6.jpg)
Attributes supplied to the developer – Function level attributes (cont)
Input Parameters attribute– Provide set of interesting assignments to the
methods parameters– For double tan(double x) the developer will
define the following assignments: X = pi X = 0 X = pi/2 X = -1234.45*pi
![Page 7: Generic API Test tool By Moshe Sapir Almog Masika.](https://reader036.fdocuments.net/reader036/viewer/2022082820/56649ee95503460f94bfad0a/html5/thumbnails/7.jpg)
Attributes supplied to the developer – Class level attributes
Class Instances– Provide set of interesting instances of the class– For class Int32 the developer will define the following
instances: 0 1 -1 2^31 (-2^31) + 1 12345 -780
![Page 8: Generic API Test tool By Moshe Sapir Almog Masika.](https://reader036.fdocuments.net/reader036/viewer/2022082820/56649ee95503460f94bfad0a/html5/thumbnails/8.jpg)
Attributes supplied to the developer – Class level attributes (cont)
Class Invariants– Define set of Boolean expressions that should be true after and
before executing any member function on an instance of the class– For Account class:
class Account {string Owner;int Balance;
}
The developer will define: Owner != string.Empty Balance >= 0
![Page 9: Generic API Test tool By Moshe Sapir Almog Masika.](https://reader036.fdocuments.net/reader036/viewer/2022082820/56649ee95503460f94bfad0a/html5/thumbnails/9.jpg)
Test Coverage – Method level coverage
General test– Run the method as many times as possible, using
parameters supplied by the developer. If no parameters were supplied, try to generate the them
Input – Output Assertion test– For each <P, Q, L> defined on the method, try to find an
assignment that satisfies P, run the method and validate Q and L.
![Page 10: Generic API Test tool By Moshe Sapir Almog Masika.](https://reader036.fdocuments.net/reader036/viewer/2022082820/56649ee95503460f94bfad0a/html5/thumbnails/10.jpg)
Test Coverage – Assembly level coverage
Function call composition– Try to build a call graph of the functions in
the assembly and generate the call paths. Validate input-output assertions and class invariants during the calls:
For the classes:A, B, C, D And the functions:
M1: A x B -> A M2: B x C -> D M3: A x D -> A Build the following graph:
A B C D
M1 M2
M3
![Page 11: Generic API Test tool By Moshe Sapir Almog Masika.](https://reader036.fdocuments.net/reader036/viewer/2022082820/56649ee95503460f94bfad0a/html5/thumbnails/11.jpg)
Design and implementation – Functional decomposition
Test Attributes- Attributes that are available to the assembly developer.
User Interface- Retrieve from the user the input required to run the test and to present him the test results.
Test Engine- Responsible to run the tests and produce the test results.
Results Reporting: Responsible to save the results in persistent file and redirect them to the graphic user interface.
Tester User Interface
Test Engine Result Reporting
Test Attributes
![Page 12: Generic API Test tool By Moshe Sapir Almog Masika.](https://reader036.fdocuments.net/reader036/viewer/2022082820/56649ee95503460f94bfad0a/html5/thumbnails/12.jpg)
Design and implementation – Functional decomposition
Data flow between the different components:
User Interface
Result ReportingTest Engine
Test Parmeters
Test Results
Processed Results
![Page 13: Generic API Test tool By Moshe Sapir Almog Masika.](https://reader036.fdocuments.net/reader036/viewer/2022082820/56649ee95503460f94bfad0a/html5/thumbnails/13.jpg)
Test Engine Design: The Algorithm design pattern
+Run()
-TestedMethod
FunctionRunner
TestMatrix
+Generate() : TestMatrix
MatrixGenerator
+ExecuteTest()
-TestParameters-MatGenerator : MatrixGenerator-FuncRunner : FunctionRunner
TestExecuter
-TestedMethod
MethodTester-TestedClass
ClassTester
-TestedAssembly
AssemblyTester
GeneralMethodTester
InputOutputAssertionsTester
ConstCorrectnessTester
1
1
1
1 -Generates1
*
-Consume
1 *
ObjectBehaviorTester FunctionCompositionTester
+Execute(in TestParameter)
-ExecutersList-TestFinishedEvent
TestEngine
1
![Page 14: Generic API Test tool By Moshe Sapir Almog Masika.](https://reader036.fdocuments.net/reader036/viewer/2022082820/56649ee95503460f94bfad0a/html5/thumbnails/14.jpg)
Test Attributes Hierarchy
System.Attribute
MethodAttribute
ClassAttribute
-Q-P-L
InputOutputAssertion
-AssingmenstList
InputParameters
-IsConst : bool
Const
-InvariantList
Invariants
-MethodCallSequence
ObjectBehavioure
-InterestingInstances
Instances
![Page 15: Generic API Test tool By Moshe Sapir Almog Masika.](https://reader036.fdocuments.net/reader036/viewer/2022082820/56649ee95503460f94bfad0a/html5/thumbnails/15.jpg)
Design and implementation – User Interface Structure
-TestEngine
CmdLineParser
-TestEngine-TestResultsBox-AssemblyTreeView
MainWindow
AssmeblyTreeViewTestResultBox
TestParametersDialog
1
1
1
1
1 *
TestEngine
1
1
1
1
User Interface Component
Test Engine Component
![Page 16: Generic API Test tool By Moshe Sapir Almog Masika.](https://reader036.fdocuments.net/reader036/viewer/2022082820/56649ee95503460f94bfad0a/html5/thumbnails/16.jpg)
Test execution sequence diagram
MainWindow InputOutputAssertionTesterTestEngine
ExecuteTest
ExcelReporterUIReporter
HandleTestResults(TestResult)
HandleTestResults(TestResult)
UpdateUI
UpdateFile
MatrixGenerator
Generate
FunctionRunner
Run
Execute(TespParams)
SetTestParams(TesParams)
![Page 17: Generic API Test tool By Moshe Sapir Almog Masika.](https://reader036.fdocuments.net/reader036/viewer/2022082820/56649ee95503460f94bfad0a/html5/thumbnails/17.jpg)
Demonstration
. . .
![Page 18: Generic API Test tool By Moshe Sapir Almog Masika.](https://reader036.fdocuments.net/reader036/viewer/2022082820/56649ee95503460f94bfad0a/html5/thumbnails/18.jpg)
Conclusions
.NET attributes can be used to keep information that will enable automatic testing of software
The technique we have shown is limited to platforms that allow introduction of Meta data in the binary code (.NET, Java)
The Syntax of defining expressive test attributes can be error prone (mainly due to the use of strings as function identifiers)
The ability of such tools to verify large scale software is still to be investigated. Currently is seems can be used as first tire testing (BVT).