A Model Driven Tool for Automated System Level Testing of Middleware Turker Keskinpala Abhishek...
-
Upload
jamir-saltonstall -
Category
Documents
-
view
215 -
download
0
Transcript of A Model Driven Tool for Automated System Level Testing of Middleware Turker Keskinpala Abhishek...
![Page 1: A Model Driven Tool for Automated System Level Testing of Middleware Turker Keskinpala Abhishek Dubey Steve Nordstrom Ted Bapty Sandeep Neema Institute.](https://reader036.fdocuments.net/reader036/viewer/2022062511/5518c533550346881f8b57c3/html5/thumbnails/1.jpg)
A Model Driven Tool for Automated System Level
Testing of MiddlewareTurker KeskinpalaAbhishek DubeySteve Nordstrom
Ted BaptySandeep Neema
Institute for Software Integrated SystemsVanderbilt University
Nashville, TN USA
![Page 2: A Model Driven Tool for Automated System Level Testing of Middleware Turker Keskinpala Abhishek Dubey Steve Nordstrom Ted Bapty Sandeep Neema Institute.](https://reader036.fdocuments.net/reader036/viewer/2022062511/5518c533550346881f8b57c3/html5/thumbnails/2.jpg)
STV'06 WorkshopMarch 30, 2006
Outline
Motivation and Challenges Model Driven Approach Tool Structure Test Series Definition Modeling
Language Example Test Series Definition Conclusions and Ongoing Work
![Page 3: A Model Driven Tool for Automated System Level Testing of Middleware Turker Keskinpala Abhishek Dubey Steve Nordstrom Ted Bapty Sandeep Neema Institute.](https://reader036.fdocuments.net/reader036/viewer/2022062511/5518c533550346881f8b57c3/html5/thumbnails/3.jpg)
STV'06 WorkshopMarch 30, 2006
Motivation and Challenges Many middleware framework configurations are
generally represented by XML Configuring functional operation of components Component deployment in the cluster
Frameworks are complicated and need to be tested Application topologies Parameter variations
Need for many test configurations
![Page 4: A Model Driven Tool for Automated System Level Testing of Middleware Turker Keskinpala Abhishek Dubey Steve Nordstrom Ted Bapty Sandeep Neema Institute.](https://reader036.fdocuments.net/reader036/viewer/2022062511/5518c533550346881f8b57c3/html5/thumbnails/4.jpg)
STV'06 WorkshopMarch 30, 2006
Motivation and Challenges
Hand writing test configurations Cumbersome Error prone Inefficient
Big configuration space of control and deployment parameters of applications
Response to changes in hardware resources or other criteria Difficult to scale up manually written XML
configurations
Usual outcome: Under tested software
![Page 5: A Model Driven Tool for Automated System Level Testing of Middleware Turker Keskinpala Abhishek Dubey Steve Nordstrom Ted Bapty Sandeep Neema Institute.](https://reader036.fdocuments.net/reader036/viewer/2022062511/5518c533550346881f8b57c3/html5/thumbnails/5.jpg)
STV'06 WorkshopMarch 30, 2006
Goals
Testing large sets of system configurations Simple powerful specifications Tool to take specifications to instances
Configure series of tests from the specifications
Help finding “corner cases” Potential functional errors Potential performance problems Unexpected system interactions
![Page 6: A Model Driven Tool for Automated System Level Testing of Middleware Turker Keskinpala Abhishek Dubey Steve Nordstrom Ted Bapty Sandeep Neema Institute.](https://reader036.fdocuments.net/reader036/viewer/2022062511/5518c533550346881f8b57c3/html5/thumbnails/6.jpg)
STV'06 WorkshopMarch 30, 2006
Model Driven Approach
Domain Specific Modeling Use domain specific models to encapsulate relevant
parts of the system properties
Provide multiple view points Decouple types, resources, dataflow and
deployment aspects
Generative modeling Vary range of behavioral and structural system
parameters
System generation Automatically generate configurations from higher
level models
![Page 7: A Model Driven Tool for Automated System Level Testing of Middleware Turker Keskinpala Abhishek Dubey Steve Nordstrom Ted Bapty Sandeep Neema Institute.](https://reader036.fdocuments.net/reader036/viewer/2022062511/5518c533550346881f8b57c3/html5/thumbnails/7.jpg)
STV'06 WorkshopMarch 30, 2006
Tool Structure
Test Series Definition Models
![Page 8: A Model Driven Tool for Automated System Level Testing of Middleware Turker Keskinpala Abhishek Dubey Steve Nordstrom Ted Bapty Sandeep Neema Institute.](https://reader036.fdocuments.net/reader036/viewer/2022062511/5518c533550346881f8b57c3/html5/thumbnails/8.jpg)
STV'06 WorkshopMarch 30, 2006
Test Series Definition Define tests using a high level modeling
language called Test Series Definition Modeling Language (TSDML) Define application types Form a template test case from those types Define dataflow association between applications
by connection rules Define deployment associations for applications Define the variance on the system size and
structure Define variance on the application parameters Define available resources
Automatically create series of test configuration models by replicating and adapting the template test case
![Page 9: A Model Driven Tool for Automated System Level Testing of Middleware Turker Keskinpala Abhishek Dubey Steve Nordstrom Ted Bapty Sandeep Neema Institute.](https://reader036.fdocuments.net/reader036/viewer/2022062511/5518c533550346881f8b57c3/html5/thumbnails/9.jpg)
STV'06 WorkshopMarch 30, 2006
Tool Structure
Test Series Definition Models
Generator (1-*)
Test Configuration Models
![Page 10: A Model Driven Tool for Automated System Level Testing of Middleware Turker Keskinpala Abhishek Dubey Steve Nordstrom Ted Bapty Sandeep Neema Institute.](https://reader036.fdocuments.net/reader036/viewer/2022062511/5518c533550346881f8b57c3/html5/thumbnails/10.jpg)
STV'06 WorkshopMarch 30, 2006
Test Configuration Models
Form the configuration layer Each test configuration model capture:
Specific dataflow scenario Specific deployment scenario Configuration Schema
Represents single system representation to be tested
Automatically generate single configuration file per configuration model
Test driver scripts can also be generated
![Page 11: A Model Driven Tool for Automated System Level Testing of Middleware Turker Keskinpala Abhishek Dubey Steve Nordstrom Ted Bapty Sandeep Neema Institute.](https://reader036.fdocuments.net/reader036/viewer/2022062511/5518c533550346881f8b57c3/html5/thumbnails/11.jpg)
STV'06 WorkshopMarch 30, 2006
Tool Structure
Test Series Definition Models
Generator (1-*)
Test Configuration Models
XML Configs Test Scripts
Generator (1)
Application Runtime (System Under Test)
Test E
xecutio
n &
V
alida
tion
En
gin
eApplications
![Page 12: A Model Driven Tool for Automated System Level Testing of Middleware Turker Keskinpala Abhishek Dubey Steve Nordstrom Ted Bapty Sandeep Neema Institute.](https://reader036.fdocuments.net/reader036/viewer/2022062511/5518c533550346881f8b57c3/html5/thumbnails/12.jpg)
STV'06 WorkshopMarch 30, 2006
System Under Test
XDAQ Framework High performance distributed middleware
framework developed at CERN Will be used for Compact Muon Selenoid
(CMS) experiment Run-time support for clustered applications Predefined functions to configure and
control the applications and their parameters
XML is used for configuration of XDAQ and its applications
![Page 13: A Model Driven Tool for Automated System Level Testing of Middleware Turker Keskinpala Abhishek Dubey Steve Nordstrom Ted Bapty Sandeep Neema Institute.](https://reader036.fdocuments.net/reader036/viewer/2022062511/5518c533550346881f8b57c3/html5/thumbnails/13.jpg)
STV'06 WorkshopMarch 30, 2006
Test Series Definition Modeling Language (TSDML)
Parameterized model Varying system structure
(size) Iterators: e.g. i,j,k = {start,
step, stop} Replicators: e.g. r1=n*i,
r2=m*j Connectors: Boolean
function of iterator, and replicator
If src=(i,r1) and dst=(j,r2) C=f(src,dst) C=2*src==3*dst
i
r1
App1
j
App2
r2
Connector
![Page 14: A Model Driven Tool for Automated System Level Testing of Middleware Turker Keskinpala Abhishek Dubey Steve Nordstrom Ted Bapty Sandeep Neema Institute.](https://reader036.fdocuments.net/reader036/viewer/2022062511/5518c533550346881f8b57c3/html5/thumbnails/14.jpg)
STV'06 WorkshopMarch 30, 2006
Iterator-Replicator Example
Assume i = 1:1:2, j= 1:1:4, r1 = 2, r2 = 4
r2
i
r1
App1
j
App2
Test Case i j #App1 #App2
1 1 1 2 4
2 1 2 2 8
3 1 3 2 12
4 1 4 2 16
5 2 1 4 4
6 2 2 4 8
7 2 3 4 12
8 2 4 4 16
![Page 15: A Model Driven Tool for Automated System Level Testing of Middleware Turker Keskinpala Abhishek Dubey Steve Nordstrom Ted Bapty Sandeep Neema Institute.](https://reader036.fdocuments.net/reader036/viewer/2022062511/5518c533550346881f8b57c3/html5/thumbnails/15.jpg)
STV'06 WorkshopMarch 30, 2006
Connector Examplei = 1:1:2, j= 1:1:4,
r1 = 2, r2 = 3CR = src=floor((dst-1)/(j*r2/i*r1)) + 1
i
r1
App1
i
App2
r2
Test Case i j #App1 #App2
1 1 1 2 4
2 1 2 2 8
3 1 3 2 12
4 1 4 2 16
5 2 1 4 4
6 2 2 4 8
7 2 3 4 12
8 2 4 4 16Connector
![Page 16: A Model Driven Tool for Automated System Level Testing of Middleware Turker Keskinpala Abhishek Dubey Steve Nordstrom Ted Bapty Sandeep Neema Institute.](https://reader036.fdocuments.net/reader036/viewer/2022062511/5518c533550346881f8b57c3/html5/thumbnails/16.jpg)
STV'06 WorkshopMarch 30, 2006
Connector Example (ctd…)
App1
App2
App1
App2
App2
App2
(1)
(2)
(1)
d
s
t
s
r
c
Rule:
src=floor((dst-1)/(j*r2/i*r1)) + 1
(2)
(3)
(4)
1=floor(0/2) + 1 => TRUE1=floor(1/2) + 1 => TRUE1=floor(2/2) + 1 => FALSE1=floor(3/2) + 1 => FALSE
![Page 17: A Model Driven Tool for Automated System Level Testing of Middleware Turker Keskinpala Abhishek Dubey Steve Nordstrom Ted Bapty Sandeep Neema Institute.](https://reader036.fdocuments.net/reader036/viewer/2022062511/5518c533550346881f8b57c3/html5/thumbnails/17.jpg)
STV'06 WorkshopMarch 30, 2006
Test Series Definition Modeling Language (TSDML)
Varying application parameters with parameter generator Has a function attribute Function of iterator Function attribute may be:
Internal function e.g. result = src * 256 where src = i
Matlab script e.g. lut = [10 20 55 88 400 800
1025 4490 7342]; result = lut[src]; Internal lookup table
![Page 18: A Model Driven Tool for Automated System Level Testing of Middleware Turker Keskinpala Abhishek Dubey Steve Nordstrom Ted Bapty Sandeep Neema Institute.](https://reader036.fdocuments.net/reader036/viewer/2022062511/5518c533550346881f8b57c3/html5/thumbnails/18.jpg)
STV'06 WorkshopMarch 30, 2006
Example TSDML
![Page 19: A Model Driven Tool for Automated System Level Testing of Middleware Turker Keskinpala Abhishek Dubey Steve Nordstrom Ted Bapty Sandeep Neema Institute.](https://reader036.fdocuments.net/reader036/viewer/2022062511/5518c533550346881f8b57c3/html5/thumbnails/19.jpg)
STV'06 WorkshopMarch 30, 2006
Example TSDML
2*i
1:1:8 1:1:1
1:1:1
2*i16*
i
1*i
![Page 20: A Model Driven Tool for Automated System Level Testing of Middleware Turker Keskinpala Abhishek Dubey Steve Nordstrom Ted Bapty Sandeep Neema Institute.](https://reader036.fdocuments.net/reader036/viewer/2022062511/5518c533550346881f8b57c3/html5/thumbnails/20.jpg)
STV'06 WorkshopMarch 30, 2006
Conclusion
Domain specific modeling for capturing relevant system parts
Vary system structure and application parameters from high level model
Generate series of test configurations from a single high level model
Generate the configuration representation
Mitigate the complexity of creating many system test configurations
![Page 21: A Model Driven Tool for Automated System Level Testing of Middleware Turker Keskinpala Abhishek Dubey Steve Nordstrom Ted Bapty Sandeep Neema Institute.](https://reader036.fdocuments.net/reader036/viewer/2022062511/5518c533550346881f8b57c3/html5/thumbnails/21.jpg)
STV'06 WorkshopMarch 30, 2006
Ongoing Work
Add another layer on top of TSDML System model
Application behavior Target framework
Analyze the models Assess important parts of the system
to exercise and test
![Page 22: A Model Driven Tool for Automated System Level Testing of Middleware Turker Keskinpala Abhishek Dubey Steve Nordstrom Ted Bapty Sandeep Neema Institute.](https://reader036.fdocuments.net/reader036/viewer/2022062511/5518c533550346881f8b57c3/html5/thumbnails/22.jpg)
STV'06 WorkshopMarch 30, 2006
Thank you!
![Page 23: A Model Driven Tool for Automated System Level Testing of Middleware Turker Keskinpala Abhishek Dubey Steve Nordstrom Ted Bapty Sandeep Neema Institute.](https://reader036.fdocuments.net/reader036/viewer/2022062511/5518c533550346881f8b57c3/html5/thumbnails/23.jpg)
A Model Driven Tool for Automated System Level
Testing of MiddlewareTurker KeskinpalaAbhishek DubeySteve Nordstrom
Ted BaptySandeep Neema
Institute for Software Integrated SystemsVanderbilt University
Nashville, TN USA