Logical Program Specification and Testing Prof. Dr. Holger Schlingloff Humboldt-Universität zu...

Logical Program Specification and Testing

Prof. Dr. Holger SchlingloffHumboldt-Universität zu Berlin

andFraunhofer FIRST, Berlin, Germany

21.11.2009SEFM 2009, © Prof. Dr. Holger Schlingloff 2/90

Plan for Today

• Terms and definitions• Propositional

modelling• Testing problems• Topics in testing• First-order

formalisms, Z• Functional (unit)

testing and test selection

• Structural testing and test coverage

• Model-based test generation


Specification of Systems

• One of the first activities in software development requirements specification modelling module

decomposition implementation integration deployment

• Often neglected, mostly only informal

• Specification „a posteriori“results in quality problems, acceptance problems

• Both implementation and test rely on the specification

Requirements

Specification

Implementation

ImplementationModel

TestingModel

Test CasesTesting


Specification-based Testing

• Constructing the test suite from the specification as opposed to constructing it from the

implementation (code-based testing)

code-based testing cannot detect missing requirements

specification-based testing cannot detect additional (unspecified) features

specified implemented

tested

Behaviour


VVT: Validation, Verification, Testing

Trying to answer different questions• Testing: Did we build the software right?• Validation: Did we build the right software?• Verification: Can we show that the software is correct?

• Dijkstra: “Testing can only show the presence of bugs, not their absence.”

• Hoare (attributed): “Beware of this program. I haven’t tried it yet, I only proved its correctness.”

Consider two airplanes: One brand new, with verified but untested software. The other one with software which is thoroughly tested but not verified. Which one would you enter?


Implementation and Testing

• “Programmer” and “Tester” are fundamentally different roles programmer wants to show correctness of his

creation tester has the task to find errors, i.e. testing is

successful if it uncovers deficiencies

• “Programmer” and “Tester” are essentially similar roles programmer creates executable artefacts

(programs) from specifications tester creates executable artefacts (test suites) from

specifications


Implementation and Testing

•V-Model: constructive and analytical part of software life cycle

Requirements Analysis

Module Design

Unit Design & Implementation

Module Integration

System IntegrationArchitecture Design

System Deployment

Requirements Testing

Module Testing

Unit Testing

Integration Testing

System TestingDesign Testing

Acceptance Testing


Topics in Testing

• What is the specification, what is the SUT?• Which interfaces to the SUT are needed? (Harness)• What is the testing objective? (Purpose, Conditions)• How are test cases derived? (Test case generation)• How can the verdict be assigned? (Test oracle)• How to write down test cases? (Testing languages)• When is a test suite sufficient for the objective? (Test

strategy)• How are test cases executed? (Testing environment)• When to stop testing? (Test coverage)• How to reuse test results for subsequent activities?

(Regression testing)


Classification of Testing (1)

• According to system life-cycle or structure analysis, design, implementation, integration,

deployment module/unit, component, system, user

• According to class of SUT operating system, middleware, driver, library,

application, GUI, web-service, embedded software, ...

• According to testing method / test selection method static or dynamic, structural or functional, control- or

data oriented, single or regression test, ...


Classification of Testing (2)

• According to testing purpose or objective functional testing, acceptance test, usability test,

load test, interoperability test, safety test, …

• According to available information and specification method Black-Box, White-Box, Grey-Box UML-/model based, contract/requirement based,

style guide based, formal specifications, …

• According to tool use and degree of automation: manual or automatic (scripted) test execution, test

case generation, test evaluation, management and documentation


Levels of Testing

• User level: requirements test, rapid prototyping, usability test, installation and configuration test, load and stress test

• System level: system test, design test, module interaction test, acceptance test, back-to-back-test, GUI testing, performance and robustness test

• Module level: module test, integration test, communication test, data flow test, data integrity test, cause-effect test

• Unit level: unit test, logic test, equivalence class test, boundary value test, control flow test, loop test


Testing Tools

• Like any other software engineering activity, the effectiveness and efficiency of testing highly depends on the tools deployed

• www.testingfaqs.org lists more than 500 (!) tools in the following categories unit test tools test drivers, test suite management test implementation, static analysis test design tools, test coverage monitors load and performance testing GUI test drivers defect tracking systems, bundled suites


Testing Languages

Several languages are being used for writing down test cases .doc (esp. for manual test execution) MS Excel or .txt, .csv etc. (tabular notation) csh, .bat (command-line SUT) Perl, Python, AWK, Tcl, … C, C++ language of the SUT TSL, TestML, ATML TTCN-3 …


Specification Languages

• Modelling formalisms transition systems, Petri nets, SDL, Message Sequence

Charts,… UML (state machines, other diagrams) Simulink/Stateflow

• Logical specifications propositional and first-order logic, set theory, Z object oriented logics (B, OCL, OZ) temporal logics (LTL, CTL*, TLA) programming logics (JML, ACSL, ASM/ASML, Spec#)

• Algebraic and process algebraic specifications CSP, CCS Lotos, Larch, ACP, π-calculus CASL

combinations & translations!


First-Order Predicate Logic

• In contrast to propositional logic, first-order logic adds structure to basic propositions quantification on (infinite) domains

•New syntactic elements let R be a set of relation symbols, where

each pR has an arity nN0 let V be a denumerable set of (first-order or

individual) variables an atomic formula is p(x1,…,xn), where pR

is n-ary and (x1,…,xn)Vn.


FOL: Syntax

•Syntax of first-order logicFOL ::= R (Vn) | | (FOL FOL) | V

FOL•Of course, x = ¬x ¬•Examples

xxy ¬p(x) xy (p(x,y) p(y,x)) (xy p(x,y) yx p(x,y))

Which of the formulas are valid / satisfiable / unsatisfiable?


Typed FOL

• Often, types/sorts are used to differentiate domains

• Signature =(D, F, R), where D is a (finite) set of domains F is a set of function symbols, where each fF has an

arity nN0 and a type DDn+1

- 0-ary functions are called constants R is a set of relation symbols, where each pR has an

arity nN0 and a type DDn

- unary relations are called predicates- propositions can be seen as 0-ary relations

• Remark: domains and types are for ease of use only (can be simulated in an untyped setting by additional predicates)


Terms and Formulas

• Let again V be a (denumerable) set of (first-order) variables, where each variable has a type DD (written as x:D)(for any type, there is an unlimited supply of variables of that type)

• The notions Term and Atomic Formula AtF are defined recursively: each variable of type D is a term of type D if f is an n-ary function symbol of type (D1,…Dn,Dn+1) and t1, …, tn

are terms of type D1, …, Dn, then f(t1,…,tn) is a term of type Dn+1 if p is an n-ary relation symbol of type (D1,…Dn) and t1, …, tn are

terms of type D1, …, Dn, then p(t1,…,tn) is an atomic formula

• Revised syntax of first-order logicFOL ::= AtF | | (FOL FOL) | V:D FOL


Examples

x:Boy y:Girl loves(x,y)x:Human y:Human (needs(x,y) loves(y,x))x,y:Int equals(plus(x,y), plus(y,x))x:Int ¬equals(zero(), succ(x))• …


FOL: Models

• (We give the typed semantics only)

•First-Order Model let universe U be some nonempty set, and

let DU U for every DD interpretation I: assignment F ↦ Un+1

R↦ Un

valuation V: assignment V↦ Uinterpretations and valuations must respect typing

model M: (U,I,V)


FOL: Semantics

• Given a model M: (U,I,V), the value tM of term t (of type D) can be defined inductively if t=xV, then tM=V(x) if t=f(t1,…,tn) , then tM=I(f)(t1

M,…,tnM)

• Likewise, the validation relation ⊨ between model M and formula M ⊨ p(t1,…,tn) if (t1

M,…,tnM)I(p)

M ⊭ ; M ⊨ () if M ⊨ implies M ⊨ M ⊨x if M‘ ⊨ for some M‘ which differs at most

in V(x) from M

• Validity and satisfiability defined as in the propositional case


FOL=

•Equality is not definable in FOL•First order logic with equality contains an

additional (binary) relation = which is always interpreted as equality of domain elements

•Written in infix notation, i.e. (x=y) for =(x,y)•Axioms

(x=x) reflexivity

(x=y (y=z x=z)) transitivity

(x=y y=x) symmetry

(x=y ( (y:=x))) substitution


Example Revisited

• Let’s specify the triangle problem in FOL=

• Signature =(D, F, R), where D={int, triangletype}, where

triangletype = {equilateral, isosceles, scalene, noTriangle, error}

F={0,200,+,triange}, where triangle : int3 triangletype R={<}

• Specification: (x<1 v y<1 v z<1 triangle(x,y,z)=error) (200<x v 200<y v 200<z triangle(x,y,z)=error) (x+y<z v x+z<y v x+y<z triangle(x,y,z)=noTriangle) (x=y ^ y=z triangle(x,y,z)=equilateral) (x=y ^ ~y=z v x=z ^ ~y=z v y=z ^ ~x=z

triangle(x,y,z)=isosceles) (~x=y ^ ~y=z ^ ~x=z triangle(x,y,z)=scalene)


Remarks

•A clean requirements analysis eases the formal specification considerably

•First-order logic is well-suited to formalize most mathematical problems

•Again, a formal specification can be used for interactively deriving an implementation, or interactively deriving and evaluating test cases

- we’ll come back to this later


Exercise

•Give a FOL= specification of the “integer square root” function!


The DIM Function

Which function does the following code compute?• 30+((m mod 2) xor (m div 8))-n*(n==2)

• if m==2 then 28else if m<7 and even(m) or m>7 and odd(m) then 30 else 31

• if m==2 then 28else if m in {4,6,9,11} then 30 else 31

• array DiM=[31,28,31,30,31,30,31,31,30,31,30,31]return DiM[m]


Another Example: nextDate

• A “date” consists of three integers: month, date, year

• nextDate takes a date and returns the date of the following day checks whether input date is valid (according to

Gregorian calender, no Feb 31st etc.) return value respects leap years etc.

• Two sources of complexity input domain check rules of leap years


Rules for Leap Years

• One year is approximately 365.2422 days

• Julius Caesar: every fourth year is a leap year• Pope Gregory in 1582 reformed Caesarian rules• Year is leap year iff divisible by 4 but not by 100, or

if divisible by 400. Thus 1600, 2000, 2004 and 2008 are leap years, but 1700, 1900 and 2100 are not.

• Leap seconds due to slow-down of earth rotation…

Task: specify the nextDate function


Break!

21.11.2009SEFM 2009, © Prof. Dr. Holger Schlingloff 31/9020.5.2008

Z

• A specification language based on FOL= and set theory emerged as a notation for formal reasoning developed during the 70’s and 80’s international ISO standard 2002 tools freely available

• Specification structured into schemas allows comprehensible description allows iterative refinement

• Strong typing type-checking tools


Set theory

• Comprehension scheme {x: T|(x) ● expr(x)}

- expr(x) is an expression of type D involving variable x of type T

- The set of all values of expr(x) (in DU) where the value of x (in TU) satisfies (x)

{x: T|(x)} stands for {x: T|(x) ● x}

• Set operations y{x: T|(x) ● expr(x)} stands for

x:T ((x) y=expr(x)) M1M2 stands for x(xM1xM2) etc.

• Power set operator M1ℙM2 if M1M2 (but: set variables not available in

FOL!)

21.11.2009SEFM 2009, © Prof. Dr. Holger Schlingloff 33/9020.5.2008 Slide H. Schlingloff, Logical Specification

Z: Logic

• Properties described in FOL (Q x:T|(x) • (x))

- [quantifer][variable]:[type]|[constraint]•[predicate]

(x:T| • ) stands for x:T ( ∧ ) (x:T| • ) stands for x:T ( )

• Z schemes: name, signature and formulas


Z semantics

• Every Z scheme defines a set of (first-order) models M: (U,I,V) („each model being a function from names defined by the specification to values that those names are permitted to have by the constraints imposed on them in the specification“) U contains a domain for each type in the scheme

(named and unnamed types), such that the set constraints are satisfied- e.g. ℙM is the set of all subsets of M- e.g. ℤ is the set of integers

I is an interpretation of function and relation symbols- built-in functions are interpreted as expected

V is a first-order variable valuation, such that all specification formulae are satisfied- note: type names cannot be used as variables!


Example

defines the set of models


Z: Sets

•specifications are operations upon sets set by extension: s == {1, 2, 7}, t == {a, b} set by comprehension: {x: s | x is even} power set: ℙt = {, {a}, {b}, {a,b}} cartesian product: s × t = {(1,a), (1,b),

(2,a), ...}

• types are maximal sets each value in specification has exactly one

type built-in type - ℤ


Example

day:day = {x: N | (0<x ^ x<32)}

month:month = {1,2,3,4,5,6,7,8,9,10,11,12)

year:x : year | x>1581


Z: Relations

• relationships between objects

•composition of compatible relations

• functions lambda notation: (λ declaration|

constraint•result)


Z: Schemas - State Changes

•description of operations

•delta abbreviation


Z Example

•Specification of phonebook basic types

list of numbers


•change of state: Delta-Notation

•specifies extended models unprimed variables: current state primed variables: next state


•what if record is already present?


•alternative state change

•schema composition


The Z standard

• International standard 2002

•Defines standard operations sets, powersets tuples, products, sequences functions, relations numbers

•Markup languages LaTeX, ASCII


Z: References and Tools

• Z notation J. Mike Spivey (1992).

The Z Notation: a reference manual Jim Davies and Jim Woodcock (1996).

Using Z: Specification, Refinement and Proof

• Community Z tools (CZT) editing and type-checking Z specifications czt.sourceforge.net

• Smartesting Leirios Test Generator (LTG) derivation of test cases from Z and B schemas


Summary: Z

• Basic building blocks: Z schemes declarations (signature) predicates (formulas constraining variable values) well-defined semantics via predicate logic

• High expressiveness by set theory and logic• Modularity (but no object orientation)• Well-suited for program verification

• Not so well-suited for refinement (transformational program development) and/or test generation


Unit Testing

• Often considered “the” testing often the first analytical step after coding first execution of actual system parts often done by programmer herself

• The SUT consists of: procedures, functions (in imperative languages) modules, units, classes, interfaces (in oo programs) blocks (in model-based development)

• Same or similar development environment as for SUT itself (compiler, linker, platform, …)


Unit Testing Methodology

• Each unit is tested independently of all the others no external influences or disturbances fault localisation is usually no problem nesting is allowed, poses additional difficulties

• Unit is linked with testing program setting of SUT environment (variables) by testing

program invocation of SUT functions with appropriate

parameters evaluation of result by comparison of variables


Unit Testing Goals

• Exhibition of faults in the program text wrong calculations, wrong operations incorrect output values, incorrect parameter treatment missing paths, missing cases, missing exception

handling duplicate cases, spurious outputs, redundant code wrong loops, wrong boundary values, wrong pointer

arithmetic timing and synchronisation problems (difficult!) incorrect (error) messages

• Non-functional properties (efficiency, usability) are of secondary importance


An Example

Beispiel: Yoonsik Cheon, University of Texas at El Paso, www.cs.utep.edu/~cheon/cs3331/notes/unit-testing.ppt

public final class IMath { /* * Returns an integer approximation * to the square root of x. */ public static int isqrt(int x) { int guess = 1; while (guess * guess < x) { guess++; } return guess; }}

/** A class to test the class IMath. */public class IMathTestNoJUnit { /** Runs the tests. */ public static void main(String[] args) { printTestResult(0); printTestResult(1); printTestResult(2); printTestResult(3); printTestResult(4); printTestResult(7); printTestResult(9); printTestResult(100); } private static void printTestResult(int arg) { System.out.print(“isqrt(“ + arg + “) ==> “); System.out.println(IMath.isqrt(arg)); }}


Discussion

•What is the output of the test?

•What advantage (if any) does this method have compared to manual testing

•Which types of errors can be found, which can’t be found this way?

•What problems do you see in this procedure?

•What could be improved?


JUnit

• Controlled test execution and evaluation

• Public domain• Integrated into

IDEs (e.g. Eclipse)

• Supports testing by developers

• Eliminates tedious work in writing tests

import junit.framework.*;public class IMathTest extends TestCase { public void testIsqrt() { assertEquals(0, IMath.isqrt(0)); assertEquals(1, IMath.isqrt(1)); … assertEquals(10, IMath.isqrt(100)); } public static Test suite() { return new TestSuite(IMathTest.class); }public static void main (String[] args) { junit.textui.TestRunner.run(suite()); }}


JUnit Criticism

• Advantages automated, repeatable test execution test suites linked to program code, for each class a

test class full flexibility e.g. for test data import, access to

internal (public) automated evaluation of assertions possibility to test exception handling integration with IDE

• Disadvantages no test case selection no test oracle


Unit Test Criticism

• Writing Unit Test Cases is a work which is closely related to the implementation and to the code, often done by implementers “programmers know their code best”, but also tend

to overlook their own errors (no redundancy) “why bother with a specification if the code is

available”, but also the user perspective may be lost “unit tests can help in debugging”, but they may not

demonstrate the correctness of the SUT “background information available”, but

undocumented assumptions may also have other impacts as well


Test Selection

• For complexity reasons it is often not possible to test all possible inputs to a function (unit) Isqrt takes 32-bit integer 1010 values (month, day, year) 12*31*700=260.000

combinations

• Test selection problem: given an upper size to the test suite, which subset of all possible test cases has the highest error-uncovering probability? equivalence classes boundary values decision tables


Equivalence Class Method

• 1st step: partition the inputdomain into a finite numberof equivalence classes(w.r.t. potential errors) e.g. equilateral triangle, i.e. three equal positive

integers e.g. [-maxint,-1] [0] [1,3] [4, maxint]

• 2nd step: choose one representative from each class e.g. (2,2,2) e.g. -3, 0, 2, 7

• 3rd step: combine representatives into test cases only feasible combinations


How to build equivalence classes

• Look at the domains for input and output• For each parameter there are valid and invalid

classes enumerations: contained / not contained different computation paths: for each path a valid,

plus one invalid class outputs which must be calculated differently: one

class for each case input preconditions: one valid and one invalid class

for each input

• Split a class if you have reasons to assume that the elements are treated differently


How to combine representative values

• Complete: cartesian product of representative classes usually too many test cases, combinatorial explosion

• Heuristic: Choose according to the following strategy test cases which cover many previously uncovered cases test cases covering exactly one invalid class pair-wise combination of values

ex: (2,2,2) and (2,2,3), (-7,1,2), (5,“a”,2)

Äq1 Äq2 Äq3 …

Par1 Value1.1Value1.2

Par2 Value2.1 …

…

ParN


Example

•Which are the test cases arising by the equivalence class method?

•Which cases are not covered?

public final class IMath {

public static int idiv (int x, y) { /* Returns the integer quotient of the two input values */

/* idiv (x,y) = z. (z*y <= x ^ z*(y+1) > x) */ }}


Discussion

• Pro systematic procedure reasonable number of test cases well-suited for “small” functions with pre- and post-

conditions

• Con selection of test cases by heuristics interaction between parameter values neglected with complex parameter sets many equivalence

classes

• Improvement boundary value analysis: for each parameter in each

test case, find satisfying and unsatisfying assignments


Equivalence classes for young snakes


Structural Testing

• Resorting to the structure of the model or program program text, UML state diagram, …

• Test coverage = number of reached goals / number of set goals e.g. 100% of all statements are executed at least once e.g. 30% of all loops are traversed at least three times

• control flow oriented starting point: control flow graph of a model or program test case: path through the graph

• data flow oriented starting point: access of variables in a model or program test case: pair of writing and reading


control flow graph

void countChar (int& vocNumber, int& totalNumber){char chr;cin>> chr;while ((chr >=À´) && (chr <=`Z´) && (totalNumber <INT_MAX)){

totalNumber +=1;if ((chr ==À´)||

(chr ==È´) || (chr ==Ì´) || (chr ==Ò´) || (chr ==Ù´)){

vocNumber +=1;}cin>> chr

}} Ex. taken from Liggesmeyer (f) / Balzert

start

n1

n2

n4

n5

n6

end

n3


Coverage

•Criteria statement coverage branch coverage condition coverage

- simple condition coverage

- multiple condition coverage

- minimal condition coverage

path coverage


Statement Coverage

• C0-Test• Each program

statement must be executed in at least one test case

• Example: (A,1) yields path (start,n1,n2,n3,n4,n5,n6,n2,end)

• Edge (n4,n6) is not traversed!

start

n1

n2

n4

n5

n6

end

n3





}}


Critique Statement Coverage

• Often „complete statement coverage“ is the absolutely minimal criterium for the construction of a test suite in theory it is an undecidable problem whether a

certain statement is reachable at all!• Percentage measured in number of reached /

number of all program statement; (desirable: 100%)

• E.g. in DO-178B (above level C)• Often used, easy to calculate• Weak criterion (18% discovered errors)• E.g. (x>5) for (x>=5) not discovered


Branch Coverage

•C1-Test each edge between two

nodes must be traversed at least once

e.g. (A,B,1) yields path (start,n1,n2,n3,n4,n5,n6,n2,

n3,n4,n6,n2,end)

•Coverage: Percentage of traversed edges

start

n1

n2

n4

n5

n6

ende

n3


Critique Branch Coverage

• Subsumes statement coverage• Still, loops are insufficiently tested (e.g. only

once)• Each branching condition must be true in one and

false in another test case • Edges can be weighted to correlate the coverage

with the number of test cases

• Well suited to find logical errors, not so well for data errors

• Easy to implement with tool support (code annotation)


Exercise / Homework

int Fact (int N){if (N<0){

fac=-1}else if (N==0 || N==1){

fac=1}else {

fac=N; K=N-1;while(k<>0){

fac=fac*k;k=k-1

}}Fact=fac;

}

Start

Stop

N<0

k<>0

N==0 || N==1fac=-1

fac=1

Fact=fac

fac=N

k=N-1

fac=fac*Kk=k-1

(a)

(c)

(b)

(d)

(e)

(f)

(g)

(h)

(j)

(o)

(p)

(l)(k)

(m)

(n)

j n

j n


Condition Coverage

• Decisions in the program text

• Several variants: simple condition coverage

(C2): each atomic conditon must be true at least once and false at least once

multiple condition coverage (C3 or C2(M)): all combinations of atomic conditions

minimal condition coverage: each decision in the flow graph





}}


simple condition coverage

• Each atomic conditon must be true at least once and false at least once e.g. (p & q || r) yields six test cases

• Condition coverage is often combined with branch coverage , so called condition/decision coverage

• Problem: could be compiler dependent! (incomplete evaluation of conditions)

• Problem: how to control the program flow such that it yields the required conditions (dependent variables)

• Problem: total condition might be always the same


Multiple Condition Coverage

•All variations of atomic conditions e.g. (p & q || r) yields eight test cases

•Total decision is guaranteed to vary

•Exponentielly many possibilities

•Problem: possible dependencies of variables!(e.g. (chr==`A´)||(chr==`E´)) can not both be true)

•Not a feasible coverage criterion!


Minimal Condition Coverage

• Evaluation with respect to the structure of the formula (each subformula true and false)

• compound decisions will be evaluated compoundly• problem: ((A&&B)||C) is covered by (www) and (fff),

but not tested satisfactorily• Modification: additional proof that each atomic

decision is relevant (e.g. by one positive and one negative test case)

• In combination with branch coverage used for flight critical software (MC/DC); most important coverage criterion to date


Path coverage

• Each path through the control flow graph• In general there are infinitely many paths!

(Coverage?)• Even if loops are restricted: „very many“• Structured path coverage: equivalence

classes of paths (similar to boundary values) each loop executed no time, one time, more

than two times (boundary or interior condition)

• Additionally manual constructed test cases• Tool support?


Coverage Tools


Break!


Model-Based Testing

•A technology where tests are generated from finite state machines UML state machine diagrams

- potentially with class diagrams and/or OCL constraints

UML sequence or activity diagrams Simulink diagrams timed or hybrid automata


Description of Systems

• Finite automata have been known since the 1960’s Moore / Mealy: used to describe relations between

words (input sequence is transformed into output sequence)

Rabin / Scott: used to describe sets of words (accepting / non-accepting states)

• Can be used to describe the control flow of any system states are (equivalence classes of) configurations of the

SUT transitions are actions (external or internal) changing

the state


Labelled Transition Systems

• “finite automaton without accepting states”• Formally: (S,L,T,s0)

S: finite or countable nonempty set of states L: finite set of labels, L transition relation T S (L {}) S s0 initial state

• Run=finite sequence {(silisi+1)iN}, where s0 is the initial state and (silisi+1) T

• Trace = sequence of observable actions (labels) of a run

• Undirected communication! Actions just “happen”!


Example: A Light Switch

•Can be switched up and down

•May internally switch off

•Cf. windshield wiper example

off low high

up up

up

dndndn


Test Generation

Remarks: each “box” can be manual or automated if everything is automated, only the tools are tested

Requirements: SUT must accept inputs from test driver SUT must provide recognisable outputs for test driver SUT must be resettable by test driver SUT must be deterministic

Spec(FSM)

Test Generator

SUT

TestSuite

Test Driver/ Evaluator

TestResults

Code Generator


UML StateCharts

•can be seen as LTS with hierarchy parallelism inheritance


Parallelism

•What is the meaning of parallelism in the specification? structural: must be implemented in parallel engineering: may be implemented in any

order pragmatic: will be implemented according

to the tool’s scheduling strategy


Test Generation from StateCharts

• ATG (“automated test generator”): Add-on to the UML-tool Rhapsody by ILogix / IBM

• First, the model is translated into C++ by the Rhapsody code generator

• Then, inputs and outputs to the model / SUT are identified

• Then, ATG constructs test cases from the generated code according to certain coverage goals all states all transitions MC/DC


A “Real-Life” Example

• A safety protocol for industrial automation


ParTeG – Parallel Test Generation

• http://parteg.sourceforge.net/


What have we learned?

• Terms and definitions• Propositional modelling• Testing problems

• Topics in testing• First-order formalisms, Z• Functional (unit) testing

test selection

• Structural testing test coverage

• Model-based test generation


That’s It!

Logical Program Specification and Testing Prof. Dr. Holger Schlingloff Humboldt-Universität zu...

Documents

Transcript of Logical Program Specification and Testing Prof. Dr. Holger Schlingloff Humboldt-Universität zu...