Multi-Paradigm Models as Source for Automatic Test Construction

Victor Kuliamin

ISP RAS, Moscow

Why Multiple Models?

Requirements

Functionality ReliabilityEfficiency Usability

0,000

0,200

0,400

0,600

0,800

1,000

1,200

1,400

1,600

1,800

2,000

1 2 3 5 10 15 20

f1()

f2()

accept()

read()

modifyAll()

Testing

?

Modeling Techniques

OperationalCan be executed by virtual machine

ContractPre- and postconditions, data integrity constraints

History-basedConstraints on possible traces

AlgebraicEquivalence between different execution histories

(C)(E)FSM, LTS, PN, CSP, ASMSDL, LOTOS, Lustre, VDM, Murphi, Simulink

Z, B, ADL, JML, Eiffel, VDM, RSLLarch-C++

TL, MSC

Larch, ML, OBJ

Tasks of Testing

Software under Test

Evaluate Correctness

Organize Bundle of

Test Inputs

Construct

Single Test Input

Evaluate Testing Quality

Test Results

Transform Test Inputs and Responses

Gather Responses

Modeling Techniques Comparison

Operational Contract History-based Algebraic

Behavior Evaluation Closeness to Requirements

Low-level Coverage

High-level Coverage

Test Sequence Construction

Single Input Construction

Scalability

Concurrency

Comparison Results

There is no the best technique No one technique is good for everything May be a mix of different approaches can

fit more needs?

UniTesK Technology

Model-based testing technology

Developed in 2000 – 2002 in

ISP RAS

UniTesK Solutions

Contract specifications of behavior

FSM and LTS testing models

Contract Specifications

Preconditions and postconditionsof interface operations andasynchronous reactions

Data integrity constraints

Close to requirements Suitable for oracle generation Provide low-level coverage

criteria

Functional Requirements

Contract Specifications

FSM and LTS Testing Models

Define states and admissible inputactions

More abstract than originalspecifications

Guarantee some low-level coverage

Suitable for test sequence construction

Provide high-level coverage criteria

Contract Specifications

!Coverage Requirements

?

!

?!

!

?

?

?

Relation between Models

states

parameters operation domain

1

2

3coverage goals

Whole Picture I

Software under Test

Model of Behavior Testing Model

Coverage Model

Test OracleTest Sequence Construction

Whole Picture II

Software under Test Model of Behavior Testing Model

Coverage Model

Operation

Data

Event

Operation

Operation

pre post

pre post

Eventpre post

invariantsData model

Operation

StateCalculation

Scenario method

Scenario method

Tool Demo

Set of Integers – Scenario I

0

1 2

312

7

5

States of behavior model

3

5

States of FSM model

Mapping Abstract Call to Specific

1

2

3

current state

parameters

states

Set of Integers – Scenario II

0

1 2

312

7

5

States of FSM model = States of behavior model

Failure

{ -2147483648, 2147483647 }

Add ( -715827883 ) / false

References

1. V. Kuliamin, A. Petrenko, N. Pakoulin, I. Bourdonov, and A. Kossatchev. Integration of Functional and Timed Testing of Real-time and Concurrent Systems. Proc. of PSI 2003. LNCS, Springer-Verlag, 2003.

2. V. Kuliamin, A. Petrenko, I. Bourdonov, and A. Kossatchev. UniTesK Test Suite Architecture. Proc. of FME 2002. LNCS 2391, pp. 77-88, Springer-Verlag, 2002.

3. A. K. Petrenko, I. B. Bourdonov, A. S. Kossatchev, V. V. Kuliamin. Experiences in using testing tools and technology in real-life applications. Proceedings of SETT’01, India, Pune, 2001

4. I. B. Bourdonov, A. S. Kossatchev, V. V. Kuliamin. Using Finite State Machines in Program Testing. "Programmirovanije", 2000, No. 2 (in Russian). Programming and Computer Software, Vol. 26, No. 2, 2000, pp. 61-73 (English version)

5. I. Bourdonov, A. Kossatchev, A. Petrenko, and D. Galter. KVEST: Automated Generation of Test Suites from Formal Specifications. Proceedings of World Congress of Formal Methods, Toulouse, France, LNCS, No. 1708, 1999, pp. 608-621

6. http://www.ispras.ru/groups/rv/rv.html

Contact

Victor V. Kuliamin

E-mail: kuliamin@ispras.ru

109004, B. Kommunisticheskaya, 25

Moscow, Russia

Web: http://www.ispras.ru/groups/rv/rv.html

Phone: 007-095-9125317

Fax: 007-095-9121524

Thank you!