Verification & Validation

• Verification– from Latin veritas meaning truth.– Building the product right.

• Validation– from Latin Valere meaning to be worth.– Building the right product.

Tools for V & V• Must be able to support

– modelling

– simulation

– state-space search

– testing

• Must be scalable

• Must be able to operate on different platforms– Unix or Linux

– Windows NT

– Windows 95,98

Example: Telelogic Tau• Modelling

– MSC(Message Sequence Chart)• consists of entities and events occurring in time

between those entities.

– SDL(Specification and Description Language)• consists of a system defining the boundaries of the

overall task.

• the system consists of components called blocks which are connected by channels.

• the blocks can consist of blocks and/or processes.

• the processes are the actors of the system and are series of actions, changes or functions that bring about a particular result.

– TTCN(Tree and Tabular Combined Notation)• A tree contained in a table.

• Tree is composed of texts having different indentations. Texts having the same indentation have the same height in the tree.

• Each text is preceded by “?” (incoming event) or “!”(outgoing event).

– State Space Coverage• A graph that shows all the different combinations of

inputs and outputs possible and their resulting transitions.

• It can be used to check for deadlocks and unspecified receptions.

• Simulation– Tau contains a simulator that allows the interactive

inspection and checking of the system’s behavior.

• State-space search– a bit-state or a random walk can be done with the

validator in Tau. These will explore the state space of the SDL system to find the most possible run-time errors.

• Testing– TTCN link– Autolink

Case study: Egg timer system

• An egg timer which boils an egg for the amount of time specified by the user.

SDLSystem Diagram

• A system defines a set of blocks and a set of channels through which interaction occurs between the environment and the system.

• Signals, signal lists and types are also defined.

SDL• Specification and Description Language

– Initially built for communicating systems– Divides the specification in

• Environment

• System(s)SD T rw /a/insite/users/isales/sdl/E ggB oiler/egg_ boiler.sdt

rw /a/insite/users/isales/sdl/E ggB oiler/

A nalysis M odel

U sed F iles

SD L System Structu re

E ggB oiler [unconnected]

D ecT im e rw D ecT im e.spdD isplaying rw D isplaying.sprI n terfacing rw I n terfacing.sprT im ing rw T im ing.spr

E g g T im er R W E ggB oilerC ontroller.sbk

E g g T im erS y stem R W E ggT im erSystem .ssy

T T C N T est Specification

O ther D ocum ents

The Environment Border

Into the blocks

• Blocks are composed of processes or of other blocks.

• Each block eventually contains processes.

• The processes communicate with the environment of the block via signal routes.

Signals and Signal lists

• Signals are the means through which processes interact.

• Signals to and from the environment are declared at the system level.

• Signal lists group signals that are used in the same grouping at several places.

SDL• A Whole system

SD T rw /a/insite/users/isales/sdl/E ggB oiler/egg_ boiler.sdt

rw /a/insite/users/isales/sdl/E ggB oiler/

A nalysis M odel

U sed F iles

SD L System Structu re

E ggB oiler [unconnected]

D ecT im e rw D ecT im e.spdD isplaying rw D isplaying.sprI n terfacing rw I n terfacing.sprT im ing rw T im ing.spr

E g g T im er R W E ggB oilerC ontroller.sbk

E g g T im erS y stem R W E ggT im erSystem .ssy

T T C N T est Specification

O ther D ocum ents

SystemBlock

Process

Procedure

Inside an SDL System

S ys tem E g g T im erS ystem 1(1)

S IG N A L O neM inuteB tn, S tartB tn , R esetS to p B tn;

S IG N A LL IS T B utto ns = O neM inuteB tn, S tartB tn , R esetS to p B tn;

S IG N A L D isp lay(M inuteD ig it, S eco nd D ig it1, S eco nd D ig it2), B ell;

S IG N A LL IS T To U ser = D isp lay, B ell;

S IG N A L S tartB o iling , S to p B o iling ;

S Y N TY P E M inuteD ig it = Integ er C O N S TA N TS 0:5E N D S Y N TY P E ;

S Y N TY P E S eco nd D ig it1 = Integ er C O N S TA N TS 0:5E N D S Y N TY P E ;

S Y N TY P E S eco nd D ig it2 = Integ er C O N S TA N TS 0:9E N D S Y N TY P E ;

E g g T im er

E g g B o iler

U serInterface(To U ser)

(B utto ns )

InternalS tartB o iling ,S to p B o iling

Channel (and channel name)System name

Signals (Messages)Declarations

Signalist (set of messages)

Newtype definitions

SDL System Structure

E g g T im er

E g g B o iler

U serInterface(To U ser)

(B utto ns )

InternalS tartB o iling ,S to p B o iling

Signals are shown in square brackets,separated by a comma

System blocks

Channel name

Signallists are declared like signals,but enclosed in parenthesis

Environment of the system

The EggTimer SDL Block

B lo ck E g g T im er 1(1)

S IG N A L IncM inute, S tartT im er, R esetT im er;

S IG N A L D isp layT im e(M inuteD ig it,S eco nd D ig it1,S eco nd D ig it2), R ing TheB ell;

D ecT im e

Interfac ing (1,1)

T im ing (1,1) D isp laying (1,1)

U serInterface

InterfaceIn

(B utto ns )

U serInterface

To T im er

IncM inute,S tartT im er,R esetT im er

In terfaceO ut

D isp lay,B ell

In ternalTo B o iler

S tartB o iling ,S to p B o iling

To D isp lay

D isp layT im e,R ing TheB ell

Connection points (system channel names)

Procedure declaration Process declarationsGeneral declarations (signals, types, vars)

routes

Signals

The Egg-Timer SDL Block Structure

B lo ck E ggT im er 1(1)

Interfac ing (1,1)

D ecT im e

Tim ing (1,1) D isp laying (1,1)

U serInterface

InterfaceIn(B utto ns )

U serInterface

To T im erIncM inute,S tartT im er,R esetT im er

InterfaceO ut

D isp lay,B ell

InternalTo B o iler

S tartB o iling ,S to p B o iling

To D isp lay

D isp layT im e,R ing TheB ell

ProcessesAt least 1 instance and at most1 instance of this process

Procedure Reference

Displaying sends to the environmentand receives from Timingetc...

SDL Process exampleP ro cess T im ing 1(1)

D C L m M inuteD ig it := 0, s1 S eco nd D ig it1 := 0, s2 S eco nd D ig it2 := 0;

T im er O neS eco nd ;

S etting T im e

IncM inute

m < 5

m := m + 1

D isp layT im e(m ,s1,s2)

S etting T im e

S tartT im er

S et(N O W + 1,O neS eco nd )

S tartB o iling

T im ing

O neS eco nd

D ecT im e(m ,s1,s2)

D isp layT im e(m ,s1,s2)

m = 0 ands1 = 0 and

s2 = 0

S to p B o iling

R ing TheB ell

S etting T im e

S et(N O W + 1,O neS eco nd )

-

R esetT im er

m := 0;s1 := 0;s2 := 0;

R eset(O neS eco nd )

S etting T im e

R esetT im er

m := 0;s1 := 0;s2 := 0;

S etting T im e

true

false

true false

State

Initial stateInitial transition

Declarations

Input

Output

If

Procedure call

Task

Timer Input(or timeout)

SDL Declaration example

D C L m M inuteD ig it := 0, s1 S eco nd D ig it1 := 0, s2 S eco nd D ig it2 := 0;

T im er O neS eco nd ;

To say you’re declaring something

Identifier(instance name)

Instance type

InitializationTimer declaration

Timer Identifier

SDL Transition exampleS etting T im e

IncM inute

m < 5

m := m + 1

D isp layT im e(m ,s1,s2)

S etting T im e

true

false

FROM

WHEN

ACTIONS

SEND

TO

Local variableParameters to send

SDL Tasks

m := m + 1

S et(N O W + 1,O neS eco nd )

D ecT im e(m ,s1,s2)

m := 0;s1 := 0;s2 := 0;

m := 0;s1 := 0;s2 := 0;

Incrementing or attributing a value

Making a procedure call

Setting the timer

Set([NOW+]<Duration>, <TimerName>)<variable> := <expression>[; <another task]

SDL Procedures(and operators)

;F PA R IN /O U T m M inuteD ig it, IN /O U T s1 S eco nd D ig it1, IN /O U T s2 S eco nd D ig it2;

P ro ced ure D ecT im e 1(1)

s2 = 0

s1 = 0

m = 0

s1 := 5;s2 := 9;

m := m -1;

s2 := 9;s1 := s1 - 1;

s2 := s2 - 1;

true

true

truefalse

false

false

Formal Parameters

Param modifiername

and type

ProcedureStart

Procedure return

If first ‘seconds’digit is 0 thengo down, elsego left

Modifiers:IN - Parameter is an Input parameterOUT - Parameter is an Output parameterIN/OUT - Parameter is both IN and OUT

SDL Message HandlingEach process

•has one and only one message queue•sends (by default) a message to the closest receiver (if the process itself is capable of receiving such message, it will send to itself)•the default sending procedure can be modified by

•VIA:

•TO:

<Message_name> VIA <channel_name>

<Message_name> TO <process_id>

T im ing (1,1)

To T im erIncM inute,S tartT im er,R esetT im er

To D isp lay

D isp layT im e,R ing TheB ell

SDL Message Handling

This process queue has four possible messages•IncMinute•StartTimer•ResetTimer•OneSecond

The first three are external messages where OneSecond is an internal message, indicating when the clock has decreased one second.

There could be several other messages comingfrom other processes or other blocks

Simulation

• Once a complete SDL system has been designed, the Tau tool allows us to simulate the system i.e. to inspect and check its behavior.

• After having generated an executable simulator, a simulator user interface can be started. It looks like the following:

Validation

• The validator is used to explore the state space to find errors or inconsistencies in the system.

• It can also be used to verify the system against requirements by running system level MSCs in the validator.

• An executable validator must be generated and the validator user interface can then be started.

Principles of Validation

i) Design– User-centred view

• scenarios

• use-cases

• MSCs

• functional, blackbox, providing a service (or NOT), end to end

– guided execution in SDL high-level design– refined design, re-run validation scenarios

Next i:Tr. Start

SDL Guided execution• Using the Navigator in a SDL Validator

tool.Up 1

Next 1:Signal:

OneMinuteBtn

Next 2:Signal: StartBtn

Next 2:Signal:

ResetStopBtnNext 1:Tr. Start

...

Next 1:...

Input: DisplayTimeSender: Timing:1Parameter(s):1,0,0

By clicking on theNext, we go derivinga scenario by followingthe state-space tree

SDL Guided Execution(in MSC form)

M S C Valid ato rTrace

env_ 0

Interfac ing

Interfac ing _ 1

T im ing

T im ing _ 2

D isp laying

D isp laying _ 3

Id le

S etting T im e

D isp laying

O neM inuteB tn

IncM inute

S etting T im er

D isp layT im e

1, 0, 0

S etting T im e

D isp lay

1, 0, 0

D isp laying

Valid ato r traceg enerated b yS D T Valid ato r 2.3

Functional MSC for the previous EggTimer scenario

M S C Validato rTrace

E nviro nm ent E g g T im erS ys tem

O neM inuteB tn

D isp lay

1, 0, 0

Valid ato r traceg enerated b yS D T Valid ato r 2.3

And executing this MSC in the Validator Tool,by clicking on “Verify MSC” and selectingthe proper file, we obtain (as the system has been correctly specified) 1 report

1 MSC VerificationMSC ValidatorTraceverifiedDepth: 7

Principles of Validationii) Test

• functional user interface tests in TTCN

S1

S2 S3

c ab

ab

+Test NameEvent a Event b pass

Tree and TabularCombined Notation

•TTCN standard for formal test case descriptions•Facilitate sharing of tests•Eliminates Ambiguities•UNIX-based TTCN Workbench toolkit•TTCN used for X.25, ISDN, CCS7, and many other protocols•All PDU parameters specified•All tests are formally assigned verdicts

Principles of Validation

ii) Test (cont’d)

• basic syntax & semantics of TTCN (TREE and TABLE Combined Notation)– Test Case Dynamic Behaviour

Test Case Name: Group:Purpose:Default:

Behavior Constr. VerLabel Description Reference dict Remarks

A!X A?Y A!J A?K Pass

A?Z Fail Wrong Resp.

A?Otherwise Inc. Test body not reached

Extended Comments:

Principles of Verification

i) Design– State-space search

Process Chart

Transition Chart

State Space

-transitions

reduced state space

Principles of Verification

i) Design (cont’d)– Possible outcomes

• Unspecified reception

• queue overflow

• deadlock

• infinite livelock (no real progress)

– Random walk

Principles of Verification

ii) Test– Conformance Testing (TTCN)

• preamble

• test step

• verification sequence

• postamble

Principles of VerificationStructure of a Transition Test Case

for Conformance Testing

PREAMBLE: Initializationto Preamble State

TEST BODY: Application ofTest Stimulus and Confirmation of

System Response

VERIFICATION: Confirmationof Post-Transition State

POSTAMBLE: Reset to StableState for Next Test

Principles of VerificationRepresent S1 S3 S4a/X b/Y

As a sequence of user actions and system responses, starting in state 1 and finishing in state 4.

In TTCN

+Preamble(S1) !a ?X !b ?Y +Verify(S4) PASS * FAIL * Inconclusive

Industrial Strength Tool Support

• Value added– MSC coverage and test coverage.– auto test generation.– reverse engineering can be done to a certain

extent.– provides graphics editor(MSC & SDL).– provides editor for TTCN(ITEX).