USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING...
Transcript of USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING...
![Page 1: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/1.jpg)
Rachid Kherrazi 6-4-2014 Grenoble
USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES
![Page 2: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/2.jpg)
PEOPLE INVOLVED
• Rachid Kherrazi
– Senior Consultant @ Nspyre
– Domain
• Process and Product Improvement
• RAMS (Reliability Availability Maintenance and Safety)
• Model Based Testing and Model Driven Engineering
• Also : Arjan van Der Meer & Marc Hamilton
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014 2
![Page 3: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/3.jpg)
ABOUT NSPYRE
MARKET SEGMENTS
• High Tech
• Traffic & Infra
• Industry
AREA’S OF EXCELLENCE
(Model Based) Systems Engineering /Model Driven
Engineering / Model Based Testing /Industrial Automation /
Simulation / Big Data / Mobile Solutions
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014 3
![Page 4: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/4.jpg)
CONTENTS
1) MDSD Introduction
2) MDE with ASD:Suite
3) MBT with MS Spec Explorer
4) ASDSpec
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014 4
![Page 5: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/5.jpg)
V-MODEL AND MODEL DRIVEN SYSTEM DEVELOPMENT
Requirements
Analysis
System Functional
Analysis
Design Synthesis
SW Analysis &
Design
SW Implementation & Unit
Test
Module Integration
& Test
(Sub-)System
Integration &Test
Model/
Requir
em
ents
Reposi
tory
Model Driven System
Development
System Acceptance
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014 5
![Page 6: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/6.jpg)
V-MODEL AND MODEL DRIVEN SYSTEM DEVELOPMENT
Requirements
Analysis
System Functional
Analysis
Design Synthesis
SW Analysis &
Design
SW Implementation & Unit
Test
Module Integration
& Test
(Sub-)System
Integration &Test
Model/
Requir
em
ents
Reposi
tory
Model Based System
Engineering
System Acceptance
Some of related technology
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014 6
![Page 7: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/7.jpg)
V-MODEL AND MODEL DRIVEN SYSTEM DEVELOPMENT
Requirements
Analysis
System Functional
Analysis
Design Synthesis
SW Analysis &
Design
SW Implementation & Unit
Test
Module Integration
& Test
(Sub-)System
Integration &Test
Model/
Requir
em
ents
Reposi
tory
Model Driven
Engineering
System Acceptance
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014
Some of related technology
7
![Page 8: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/8.jpg)
V-MODEL AND MODEL DRIVEN SYSTEM DEVELOPMENT
Requirements
Analysis
System Functional
Analysis
Design Synthesis
SW Analysis &
Design
SW Implementation & Unit
Test
Module Integration
& Test
(Sub-)System
Integration &Test
Model/
Requir
em
ents
Reposi
tory
System Acceptance
Model Based Testing
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014
Some of related technology
8
![Page 9: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/9.jpg)
Integration Design
Coding
Requirements System test
Test case specification
(design of logical test case)
(reduction of number of test cases by application of test techniques
Test execution
(reporting)
Test case generation
(design of physical test case)
(selection of input values and calculation of expected results)
PAGE 9
V-MODEL FOR TESTING
3 main steps in test process
Module
Subsystem
System
9
![Page 10: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/10.jpg)
MBT IS THE AUTOMATION OF TEST CASE GENERATION
Manual Automatic
Test specification
Test execution
Test generation
Model
base
d t
est
ing
Tra
dit
ional
manual
test
ing
Tra
dit
ional
auto
mati
c t
est
ing
Manual
Scripting
Testing skills + Scripting skills + Modeling skills
Manual
Modeling
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014 10
![Page 11: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/11.jpg)
PERCEIVED BENEFITS
• Increased productivity (increased automation)
• Better test script maintenance
• Improved product reliability (new type of bugs, Increased test coverage)
• Agility ( Easily react to new feature changes, Reusability of test semantics, Early test engagement, Drive quality upstream)
• Increased employee satisfaction (challenging, new horizon, fun)
• Question: How can we improve MBT e.g. increase further the productivity?
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014 11
![Page 12: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/12.jpg)
MBT IS THE AUTOMATION OF TEST CASE GENERATION
Manual Automatic
Test specification
Test execution
Test generation
Model
base
d t
est
ing
Tra
dit
ional
manual
test
ing
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014 12
Testing skills
Manual
Modeling
MBT n
ext
Semi-automatic
Modeling
Manual
Scripting
+ Scripting skills Modeling skills
Tra
dit
ional
auto
mati
c t
est
ing
![Page 13: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/13.jpg)
IDEA: USING BEST OF BOTH WORLDS
• Automatic generation of (partial) Spec Explorer test model from existing ASD interface model
Benefits
• Less effort for model creation
– reuse of existing work
• Testing of complete system including
– Legacy code
– External components
– Data combination testing
– Interaction testing
• Results: High Quality, Reduced cost
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014
Statistical Dynamic
ASDspec
13
![Page 14: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/14.jpg)
USE CASE: CONTAINER TERMINAL
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014
• Multiple components
• Components need to interact to function
• Controller needed to coordinate interaction
14
![Page 15: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/15.jpg)
Method
OVERVIEW OF PROJECT STAGES
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014 15
Use case
/SUT
Goal
Technique
Tool
Results
Stage 1
Container Terminal (CT)
Development of control
software for the CT
Model Driven Engineering +
hand written sw
ASD:Suite
Create design Models +
interface models in ASD
Generate code
Productivity (code
generation) c.t. trad. dev
-
Testing complete system,
Interaction, data, external
code
Stage 2
Container Terminal (CT)
Verification of the
developed control software
of the CT (generated +
Hand written)
Model based Testing
Spec Explorer
Create Test Model in Spec
Explorer, generate test
suite
Test Productivity
Modeling skills,
complexity costs
Stage 3
Container Terminal (CT)
Verification of the developed control
software of the CT (generated + Hand
written)
Model based Testing
ASDspec & Spec Explorer
Generate Test Model from existing ASD
interface Model , complete Test Model ,
generate test suite
+ Productivity (code generation, partial test
generation)
Testing complete system, Interaction, data,
external code
(benefits only in case of existing of ASD
Models)
![Page 16: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/16.jpg)
CONTENTS
1) MDSD Introduction
2) MDE with ASD:Suite
3) MBT with MS Spec Explorer
4) ASDSpec
Company
Tool
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014 16
![Page 17: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/17.jpg)
ASD (ANALYTICAL SOFTWARE DESIGN)
• Model Driven Engineering (code generation)
• Component Based Development
• Models are verified mathematically at design
time (formal methods)
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014 17
![Page 18: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/18.jpg)
COMPONENT BASED DEVELOPMENT ASD-STYLE
Code Generation 2014 18
Client
A
B C
Interface A
Interface B Interface C
Client
component
ASD
component
Used
component
Implemented
service
Used
service
Component: Decomposition
Specification
Design
Verification
Generation
Interface model
specifies
behaviour
Design model
implements
behaviour
For each component a interface mode and design model are created in ASD
![Page 19: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/19.jpg)
ASD: WORKFLOW
• Designer defines behavior in component models
• ASD:suite verifies models using model checking
• ASD:suite generates implementation code
19 Code Generation 2014
But some shortcoming
• No support for
Legacy/External code
• No support for indirect
component interaction
• Limited data
interaction
Testing is needed.
![Page 20: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/20.jpg)
RESULTS: ASD-CONTAINER TERMINAL
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014 20
Higher overall productivity compared to traditional development.
Main remaining problems:
- Testing interaction and complete system.
- Debugging of third party library (some bugs found in manual written
code, legacy code)
Productivity
![Page 21: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/21.jpg)
CONTENTS
1) MDSD Introduction
2) MDE with ASD:Suite
3) MBT with MS Spec Explorer
4) ASDSpec
Company
Tool
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014 21
![Page 22: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/22.jpg)
MBT WITH SPEC EXPLORER
Explore &
Analyze
Remodel
Generate
C# Model (or other .Net
Language)
Model Graph
Test Suite Execute
Nunit
http://www.nunit.com/
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014 22
![Page 23: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/23.jpg)
MODELING IN SPEC EXPLORER [TypeBinding("CraneComponent")]
class Crane
{
public Cranestates Cranestatevar;
internal ICrane_NIModel ICrane_NIimpl;
internal ICrane ICraneimpl;
[Rule(Action = "new CraneComponent(craneNr)",
ModeTransition = "crane->crane0")]
Crane(int craneNr)
{
ICraneimpl = new ICraneCranecomponent(this);
craneList.Add(this);
}
State declaration
Initialization
Used components
Spec Explorer
represents models as
annotated C# code Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014 23
![Page 24: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/24.jpg)
MODELING IN SPEC EXPLORER
[Rule(Action = "this.PlaceContainer(height)", ModeTransition = "crane4->crane4")]
public void ICrane_PlaceContainer(GripperHeightEnum height)
{
switch (basecomponent.Cranestatevar) {
case Cranestates.Positioned: {
basecomponent.Cranestatevar
= Cranestates.PlacingContainer;
basecomponent.ICrane_NIimpl.PlaceContainer(height);
return;
}
default:{
Condition.IsTrue(false);
throw new InvalidOperationException();
}
};
}
Every model method
represents an action, its
conditions and its effects.
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014 24
![Page 25: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/25.jpg)
MBT-VISUALIZATION
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014
Completed models can
be explored to
discover all possible
behaviour
25
![Page 26: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/26.jpg)
MBT- VISUALIZATION
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014
Explored model
contains all possible
action sequences
26
![Page 27: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/27.jpg)
TEST GENERATION
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014
Spec Explorer
creates test
sequences to cover
explored behaviour
27
![Page 28: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/28.jpg)
GENERATED TEST SCRIPTS
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014
Nunit test scripts are
used to implement
and execute tests
28
![Page 29: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/29.jpg)
RESULTS: SPEC EXPLORER-CONTAINER TERMINAL
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014 29
Significantly less effort than traditional automated testing
• Model Based Testing + Software Analysis
• Support of data combination testing
• Support of Model composition, incremental
However some remaining problems:
• Modeling still comparatively expensive
• Modeling effort, complexity and skills (experienced tester needed)
![Page 30: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/30.jpg)
HOW TO COMBINE BOTH BENEFITS
ASD generates verified
software components
Spec Explorer generates
automated software tests
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014
Complications
• Modeling needed
• MBT skills needed
Complications
• Legacy code
• External code
• Component
Interaction
• Limited support of
data Client
A
B C
Test
script
A
B C
Test
model
30
![Page 31: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/31.jpg)
CONTENTS
1) MDSD Introduction
2) MDE with ASD:Suite
3) MBT with MS Spec Explorer
4) ASDSpec
Company
Tool
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014 31
![Page 32: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/32.jpg)
OUR SOLUTION: ASDSPEC
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014
ASDSpec generates Test Model (push button) from existing ASD
interface models
Tester need to refine manually the generated model (add behavior,
data, slicing,…)
32
![Page 33: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/33.jpg)
COMBINING THE BEST OF BOTH WORLDS
Automatic generation of partial Spec Explorer test model from existing ASD model
(push button)
Benefits
• Effortless model creation
– reuse of existing work
• Testing of complete system including
– Legacy code
– External components
– Data combination testing
– Interaction testing
Results:
– High Quality, Reduced cost
– Adds dynamic testing to ASD
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014
Statistical Dynamic
ASDspec
33
![Page 34: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/34.jpg)
ASDSPEC: WORKFLOW
ASDspec
ASD
Interface
Model
Spec
Explorer
Model
Cord
Script
Spec
Explorer
Test
Cases
Reuse existing ASD interface models to generate
Spec Explorer MBT models automatically
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014 34
![Page 35: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/35.jpg)
ASDSPEC: DATA AND MODEL COMPOSITION
ASDspec
ASD
Interface
Model
Spec
Explorer
Model
Cord
Script
Spec
Explorer
Test
Cases
Add data
interaction
Add model
composition
Generated code can be extended with any Spec
Explorer feature
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014 35
![Page 36: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/36.jpg)
GENERAL MDSD ARCHITECTURE
Model
Reverse
Engineering
Model
Repository
Model Code,
Doc, etc.
Model
Editor(s)
Model
Checker
Model Transformation
M2M M2T
Metamodel(s)
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014 37
![Page 37: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/37.jpg)
ASDSPEC ARCHITECTURE
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014
ASDSpec
makes use of
existing MDSD
architecture
Model
Reverse
Engineering
Model
Repository
Spec
Explorer
Model
SpecExplorer
Cord
Model
Editor(s)
Model
Checker
Model Transformation
M2M M2T
Metamodel(s)
ASD
Interface
models
38
![Page 38: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/38.jpg)
ASDSPEC: TECHNOLOGIES
QVTO ASD
XML
SE
model
Cord
script
Acceleo
Acceleo
ASD
EMF
SE
EMF
Cord
EMF
QVTO
QVTO
ASDSpec
Model transformations bridge gap between ASD
and Spec Explorer
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014 39
![Page 39: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/39.jpg)
RESULTS: ASDSPEC- CONTAINER TERMINAL
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014 40
Spec Explorer ASD + ASDSpec
Approach Generate test suite Generate test model
Techniques MBT MBT
Effort/complexity Medium Low
Test cases 89 93
Perceived effectiveness Medium High
Bugs Some bugs in hand written code
/HAL
All known bus found
• Low effort model creation
- Easy reuse of existing work
• Testing of complete system
- Legacy code, data interaction
• High Quality, Cost Reduction
![Page 40: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/40.jpg)
TRADITIONAL SW DEVELOPMENT TRADITIONAL MANUAL TESTING
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014
Design spec
Hand written code
Detail design
External code
Interface spec
Legacy code
Integration testing
System testing
Unit testing
Manual
Test suite
41
Summary
![Page 41: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/41.jpg)
TRADITIONAL SW DEVELOPMENT TRADITIONAL AUTOMATED TESTING
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014
Design spec
Hand written code
Detail design
External code
Interface spec
Legacy code
Integration testing
System testing
Unit testing
Test
executi
on
Tool
Manual
scripted
Test suite
42
Summary
![Page 42: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/42.jpg)
ASD BASED SW DEVELOPMENT TRADITIONAL AUTOMATED TESTING
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014
Design spec
Design model
Generated code Hand written code
Interface model
External code
Interface spec
Legacy code
Integration testing
System testing
Unit testing
Manual
scripted
Test suite
Test
executi
on
Tool
43
Summary
![Page 43: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/43.jpg)
ASD BASED SW DEVELOPMENT MODEL BASED TESTING
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014
Design spec
Design model
Generated code
Manual Test
model
Interface model
External code
Interface spec
Config,
data
Legacy code
Integration testing
System testing
Unit testing
Generated
Test suite
Manual
scripted
Test suite
Test
executi
on
Tool
MBT
tool
Hand written code
44
Summary
![Page 44: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/44.jpg)
ASD BASED SW DEVELOPMENT MODEL BASED TESTING+ASDSPEC
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014
Design spec
Design model
Generated code
Generated
Test model
Interface model
External code
Interface spec
Config,
data
Legacy code
Integration testing
System testing
Unit testing
Generated
Test suite
Manual
scripted
Test suite
Test
executi
on
Tool
MBT
tool
Hand written code
45
Summary
![Page 45: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/45.jpg)
TOOL STATUS
• Tool currently in prototype phase
• as an Eclipse plugin
• For now available on request
• Future steps
– Support data handling/configuration aspects in
ASDSpec, instead of relying on manual additions in
Spec Explorer,
– Supporting other MBT tools.
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014 46
![Page 46: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/46.jpg)
CONCLUSIONS
• MDE and MBT technologies have matured a lot in the
latest years
• It is a matter of time….. Evolution…..
Nineth Workshop on Model-Based Testing April 6, 2014 ETAPS 2014 47
![Page 47: USING FORMAL SPECIFICATIONS TO SUPPORT MODEL BASED TESTING ...mbt-workshop.org/papers/mbt2014-02.pdf · MODEL BASED TESTING ASDSPEC: A TOOL COMBINING THE BEST OF TWO TECHNIQUES .](https://reader033.fdocuments.net/reader033/viewer/2022060423/5f19bc06194def25f7251447/html5/thumbnails/47.jpg)
Using Formal Specifications to
Support Model Based Testing
ASDSpec: A Tool Combining the
Best of Two Techniques
ASD
Static Dynamic
ASDspec
Static Dynamic
ASDspec