Model Based Testing

Model Based Testing Aparna Vijaya & Beaulah

Vineela P

Supervisor: Antonio Cicchetti

April 22, 2023 2

Presentation Outline

• Model Driven Development

• Model Based Testing (MBT)

• MBT Tools Survey

• Supporting the automatic test case and oracle generation using system models– Aparna

• Traceability in MBT– Beaulah

April 22, 2023 3

Motivation Significance of Model Driven

Development in Software Engineering- Emerging trend in software industry- Handle system complexity - Deviation from traditional development- Support requirements change- Support for highly distributed systems - Improve consistency - Provide quality solutions - Improve the productivity

April 22, 2023 4

Models and Modeling• What is a Model ?

A Model is an abstract representation of the system• Modeling Languages

- UML - xtUML - SysML

April 22, 2023 5

Generalized approach for MDD

• Shift focus from writing code to developing models

• User works on a platform independent model

• Selects the specific target platform

• Tool generates the code.

“Design once, run anywhere”

www.cs.tufts.edu/comp/180/teams/t4/site/MDD_all2.ppt

April 22, 2023

Model Based Testing – Overview• “Model-based testing is black box testing process

for derivation of test cases from a model that describe functional aspects of the system under test and executing those test cases.”

6

is a partial description of

Model System

Abstract Tests Executable Tests

can be run against

are abstract versions of

can be derived from

April 22, 2023 7 Practical Model Based Testing- A Tools Approach, Mark Utting & Bruno Legeard

• Five Main Steps

1) Model the SUT

2) Generate abstract tests from the model.

3) Concretize the abstract tests to make them executable

4) Execute the tests on the SUT and assign verdicts.

5) Analyze the test results.

Model Based Testing Approach

April 22, 2023 8

Pros & Cons of MBT

• Improved quality of the product

• Better communication using models

• Eases updating of test suites for changed requirements

• Early exposure of ambiguities in specification and design

• Generate variety of test suites from the same model by different test selection criteria.

• Complexity of the models

April 22, 2023 9

Tool Survey Tool Name Type Modeling

LanguageUsability Automatic

generation of test cases

Export models to other formats

Model Simulation

Platform

UPPAAL Academic UML Yes No Can export to XML file

Yes Windows

CowSuite Research UML Yes Yes - Yes Windows

JUMBL Academic TML (Markov Chain Models)

No Yes - No Windows /Unix

Enterprise Architect

Commercial UML Yes Only for Modeling and does not support model based testing

Spec Explorer Research SPEC # and ASML

Yes Yes Can export to XML file

No Windows

Model Junit Academic Java No Yes Can export to XML file

No Windows

April 22, 2023 10

Tool Survey cond … Tool Name Functionality Reliability Usability Efficiency Maintainability Portability Licensing and

pricing

UPPAAL Suitability: supports MBTInteroperability: models nor testcases cannot be interchanged between tools

Fault tolerance: Only if the model is created and simulated without any errors testing is possible. Each phase is completely depended on previous phase

Yes, graphical user interface and drag and drop approach

Time, effort and resource used : less

Analyzability: Yes Changeability: YesTestability: Yes

Installability: easy

Open source and academic tool

CowSuite Suitability: supports MBTInteroperability: models nor test cases cannot be interchanged between tools






JUMBL Suitability: supports MBTInteroperability: models nor testcases cannot be interchanged between tools

No, a new language called TTL has to be mastered for modeling

Time, effort and resource used : High

Analyzability: No Changeability: YesTestability: Yes

Installability: time consuming


April 22, 2023 11

Tool Survey cond … Tool Name

Functionality Reliability Usability Efficiency Maintainability Portability Licensing and pricing

Enterprise Architect

Suitability: Supports only modeling and not MBT





Commercial tool

Spec Explorer

Interoperability :Databases parts of states, transitions and test suites can be exported to an XML file but not the State machine diagram as whole. So interoperability is not achieved in this tool.

Fault Tolerance: Tool does not support this. Recoverability: If the application crashes it can be recovered as data is stored locally.

Easy to operate with good GUI. But application does not support lower versions Microsoft word 2007.

An efficient MBT tool supporting all phases of MBT – modeling to result comparison

Easy to analyze the application behavior.

Installability: Simple and Easy to install instructions are provided.

Open Source

Model J Unit

There is a provision for user to add his new test generation algorithms.

Recoverability: If the application crashes it can be recovered as data is stored locally.

GUI that is downloaded from internet is not working

Supports all phases of MBT

Since GUI is not working we could not analyze the entire application.

Installability: Easy

Open source

April 22, 2023 12

Problem Statement

“To design a test generator that addresses the whole test design problem from choosing input values and generating sequence of operation calls to generate executable tests that include verdict information”

April 22, 2023 13

Model based testing at Ericsson

• Spirent testing tool

• Qtronic

• Bridgepoint

April 22, 2023 14

Existing Approaches• Combinatorial Design Paradigm

- What is it all about?

- Pros and Cons of this approach

David M. Cohen, Siddhartha R. Dalal, Jesse Parelius, & Gardner C. Patton, "The Combinatorial Design Approach to Automatic Test Generation”

April 22, 2023 15

Existing Approaches contd… • Usecase approach

Clementine Nebut, Franck Fleurey, Yves Le Traon & Jean-marc Jezequel, “Automatic Test Generation: A Use Case Driven Approach”

April 22, 2023 16

Existing Approaches contd… Leirios Test Generator

- Main principles

- Approach in LEIRIOS tool

C. Grandpierre, B. Legeard, F. Peureux , F. Bouquet , “A Test Generation Solution to Automate Software Testing”

April 22, 2023 17

Overview of technologies used in proposed approaches

• Tefkat

• Metamodel

• MOFScript

• TTCN 3

• Ecore model

• XMI format

• State Diagrams

April 22, 2023 18

Proposed Approach (1)• From Class diagrams

Keith Duddy, Anna Gerber, Michael Lawley, Kerry Raymond & Jim Steel, Model Transformation: A declarative, reusable patterns approach

Source Model

(Class Diagram)

Source Model

(Class Diagram)

Tefkat Tefkat

Source Metamodel Source Metamodel

MOF script MOF script

Target Text

(XMI format)

Target Text

(XMI format)

State Diagram State Diagram

Generate Test

(TTCN 3)

Generate Test

(TTCN 3)

Execute tests Execute tests Test Result Test Result

Adaptor

(Ecore model)

Adaptor

(Ecore model)

Traversed transitionsTraversed transitions

Data-flow testing techniquesData-flow testing techniques

April 22, 2023 19

Proposed Approach (2)• From Sequence diagrams

C Doungsa-ard, K Dahal, and A Hossain, “AI Based Framework for Automatic Test Data Generation”

AI Framework

AI framework for determining test path

AI framework for determining test path

Determine coverage for testDetermine coverage for test

Generate tests

(TTCN3)

Generate tests

(TTCN3)

Execute tests in test harness Execute tests in test harness

Test Result Test Result

Sequence Diagram

(Bridgepoint)

Sequence Diagram

(Bridgepoint)

XMI snippet

(Bridgepoint)

XMI snippet

(Bridgepoint)

Control Flow GraphControl Flow Graph

April 22, 2023 20

Analysis & Future work

• Why MOFScript generation in method one.

• Class diagrams with pre and post conditions of the particular state

• Automatic test case generation can be done as a background process while the user is creating models could also be a research area

April 22, 2023 21

Problem statement

“Achieving traceability between the test cases generated and requirements in a model based testing (MBT) scenario”

April 22, 2023 22

Quick glance about traceability • Ensure interrelation between

different artifacts involved software engineering

• Enable to perform quick and easy impact analysis

• Enhance the product quality

Specifications 1. RQ1 2. RQ23. RQ3

DESIGNREQUIREMENTS

FuncX(){Int x;

}

Func y(){Double y;

}

CODE TEST CASES

Test case 1 Test case 2 Test case 3

SDLC Phase - Artifacts

Specification - requirements

Design - design models

Implementation - code( functions, libraries)

Validation - test case

April 22, 2023 23

Requirements Traceability in MBT

Would enable • Relevance of the test case generated from model • How a modification in the model would effect the

generated test case • Provides the effective coverage

April 22, 2023 24

State of the Art for Traceability in MBTRM TOOLS

ReqTracer

DOORS

Requisite Pro DOORS

Requirement Management Tool

Model J Unit DOORKEEPER 4 Eclipse

Model Based Testing Tool

Connector API

April 22, 2023 25

Drawbacks & AlternativeDrawbacks - • Semantic variation between source artifact and target

artifact • Need for an RM tool to support modeling notations of

MBT tool • Integration problems Alternative - • Need to have traceability with in MBT tool • Benefits of traceability through model

transformations with in MDD tools

April 22, 2023 26

Test cases with state chart diagram

April 22, 2023 27

Requirements trace model

• A new requirement trace model is introduced

• Provided as input MBT tool

• All the requirements are represented in trace model and linked to the corresponding system functions

-RQ1-RQ2

Requirements

+alarm on()+alarm off ()+set alarm ()

System Application«traces»

April 22, 2023 28

How this approach works • A transition to new state in state chart diagram is

done by an action or function • New state corresponds to state of system which is

desired to be => in new state system might has fulfilled a requirement

• Requirement identifiers are tagged with corresponding method in trace model

• By generating test cases using state chart diagram and linking trace model, traceability is achieved with MBT tool

April 22, 2023 29

Conclusion – Future work

• This would avoid problems with semantics while linking requirements between different tools

• Not only with state chart diagrams, we can have traceability model linked with other test generation models

Model Based Testing

Documents

Transcript of Model Based Testing