Object-Oriented Testing meets TTCN-3 · 2013-01-15 · Object-Oriented Testing meets TTCN-3 TTCN-3...

OOT-T3C © 2006 M. Winter

OO-Testing meets TTCN-3 1

Prof. Dr. Mario Winter FH Köln

Object-Oriented Testing meets TTCN-3

TTCN-3 User Conference 2006The Evolution of TTCN-3

31 May- 2 June 2006, Berlin, Germany

Slide 2 Prof. Dr. Mario Winter, FH KölnTTCN3UC Berlin 2006


Why do we test?

Why is testing OO-SW difficult?Specification and testing of OO-SW

The UML Testing ProfileTTCN-3 cases for OO

Man (imperfect)

God (perfect)

Software(faulty)

Specification(correct (?))

Program(misconduct)

Testcase(temptation)





Overview

• Why is testing object-oriented software difficult?• Specification and testing of object-oriented software• The UML Testing Profile • TTCN-3 cases for OO



which fails observably

Failure

Why do we test?

• Errare humanum est! (to err is human)• Even with formal specification and

program generation ...– Spec could be wrong– Generator doesn‘t work as expected– Platform doesn‘t work as expected– ...

• Testing checks that the Software– ... does what it should do and ...– ... doesn‘t what it shouldn‘t do

• But always remember that ...– ... „testing can only show the presence

of faults (i.e. bugs), not their absence“!!! [E.W. Djikstra]

Fault

in some program

Error

of a person







TestLevel

When do we Test?

Test cases based on work artifacts (documents)

Requirements

Construction

Analysis

Architecture

Design

Coding

Unit Test

Integration Test

System Test

AcceptanceTest

Valid

ation

&

Verif

icatio

n



Analysis and design

Implementation andexecution

Evaluation of test exit criteria

Post testing activities

Start

End

Planning and control

Test process[ISTQB Certified Tester – Foundation Level Syllabus 2005]



How do we Test?

• Static Testing– No executables necessary– Automated source code analysis– Manual reviews and inspections

• Dynamic Testing– Executables necessary– Test cases with input and expected behaviour/output– Test harness for unit and integration testing needed

Processor

Test ObjectObservationStimulation







Black box testing vs. white box testing

Input parameter/state

Specification needed!

Output parameter/state

Black Box Testing

Test object

Input parameter/state

Specification and realization (sources)needed!

Output parameter/state

White Box Testing

Test object





Test driver

Test output

Compare &report

Test case 1 Test case 2 ... Test case m

Why is dynamic testing difficult?



Stubs, Mocks

Stub 1

Stub 2

...

Stub n

Run time environment, analysis tools, simulators, monitors

Test object

Point of Control

Point ofObservation

Processor





Why is object-oriented testing even more difficult?

• Object-Orientation = “Normal programming” + Encapsulation + Inheritance + Polymorphism + Late Binding

• Encapsulation– Bundling of data with the operations that manipulate that data– Classes often hide encapsulated information by presenting only public interfaces

• Inheritance– Forms new classes using already defined classes– Generalization: is-a relationship, captures hierarchical relationship between classes

• Polymorphism – Ability of object (references) belonging to different classes to respond to calls of

operations of the same name, each one according to a class-specific behavior– Programmer (and program) doe not have to know the exact classes of objects in

advance, so behavior can be implemented in different ways• Late binding (aka. dynamic binding)

– At run time, the machine (“system”) decides which method to execute– Methods searched upwards the inheritance hierarchy





Example: Inheritance or Generalization?Superclass

Subclass

Rectangle

- side1- side2

+ setSide1()+ setSide2()+ getArea()

Square

- side1- side2


SomeClass

- ...

+ ...

11setSide1(10)

: SomeClass myR:Rectanglefoo()

setSide2(20)

A := getArea()

A = 200OK!

setSide1(10)

: SomeClass myR : Squarefoo()

setSide2(20)

A := getArea()

A = 100?A = 200?OOPS!

{ inv: side1 = side2 }

myR





Hierarchical Inheritance Testing (HIT)

Create new test casesExecute new test cases


Repeat test cases of superclass

White-box test cases


New defined operations

Complement test casesExecute test cases of superclass and complementedtest cases

Redefined operations(Pure generalization)

Execute test cases of superclassInherited operations

Black box test casesOperations of subclass



Where are we now?

• Why is testing object-oriented software difficult?• Specification and testing of object-oriented software• The UML Testing Profile • TTCN-3 cases for OO





Input parameters/State

Specification needed!

Output parameters/State

Black Box Testing

Class Under Test (CUT)

Black box testing of classes

• Goal – Validation of the class interface, i.e. its public operations (and members)

• Value1. Conformance of the class realization w.r.t. its specification2. Robustness of the class

• Needs – Assertions, i.e. pre- and postconditions for each public operation of the class under test

(CUT) together with its invariant• Result

– Reusable and extendable class test cases which validate the classes interface conformance and its robustness





How to specify interfaces? Design by contract (B. Meyer)

• Client uses suppliers (public) operations – Precondition = clients price– Postcondition = suppliers obligation– Invariant = “eternal” assumptions

• Contract– Iff the client satisfies the precondition, then the

supplier guarantees the postcondition!– Client and supplier satisfy their invariants

before and after each (public) operation execution

• A contract is a prescription, not only a description!

Contract

Client Supplier

Inva

riant

Inva

riant

If you promise to call me with the precondition satisfied thenI, in return, promise to deliver a final state in which the

postcondition (and my invariant) is satisfied







Specification of interfaces with assertions

• Precondition (of an operation)– States the properties that must hold whenever the operation is called.– Refers to input-parameters of the operation und the state of the component

• Postcondition (of an operation)– States the properties that the operation guarantees when it returns (assuming its

precondition was satisfied) – Refers to output-parameters of the operation und the state of the component

• Both preconditions and postconditions describe properties of individual operations –but often there are more general properties

• Invariant (of the class)– Expresses global properties of all instances of a class, which must be preserved by all

operations– Refers to the state / instant variables of the class– Must be satisfied before and after each execution of each (public) operation





Example: BoundedStack (Stack with bounded capacity)

2 ← size1

MAXSIZE

class BoundedStackState preserving operations

size():integer; // Number of elementsMAXSIZE(): integer; // Maximum count of elementstop():Object; // Pointer to topmost element

State changing operationsBoundedStack(maxSize: integer);// Constructor~BoundedStack(); // Destructorpush(element: Object); // Stack element on toppop(); // Removes topmost element

class BoundedStackState preserving operations

size():integer; // Number of elementsMAXSIZE(): integer; // Maximum count of elementstop():Object; // Pointer to topmost element

State changing operationsBoundedStack(maxSize: integer);// Constructor~BoundedStack(); // Destructorpush(element: Object); // Stack element on toppop(); // Removes topmost element

push(element: Object)3 ← size21pop()

top():Object









Contract of BoundedStack class BoundedStack {

/** invariant@ self.size() >= 0 AND self.size() <= self.MAXSIZE() */

public BoundedStack (Integer maxSize){/** pre@ maxSize > 0 *//** post@ self.MAXSIZE() = maxSize@pre */}public void push (Object item) throws FullStackException {/** pre@ self.size() < self.MAXSIZE() *//** post@ self.size() = self.size()@pre+1 AND self.top() = item@pre */}public Object top () throws EmptyStackException {/** pre@ self.size() > 0 *//** post@ return != null */}public void pop () throws EmptyStackException {/** pre@ self.size() > 0 *//** post@ self.size() = self.size()@pre - 1 */}public Collection all () {/** pre@ true *//** post@ (self.size() > 0 implies return.size() = self.size()) AND

(self.size() = 0 implies return = null)*/}

}

class BoundedStack {

/** invariant@ self.size() >= 0 AND self.size() <= self.MAXSIZE() */

public BoundedStack (Integer maxSize){/** pre@ maxSize > 0 *//** post@ self.MAXSIZE() = maxSize@pre */}public void push (Object item) throws FullStackException {/** pre@ self.size() < self.MAXSIZE() *//** post@ self.size() = self.size()@pre+1 AND self.top() = item@pre */}public Object top () throws EmptyStackException {/** pre@ self.size() > 0 *//** post@ return != null */}public void pop () throws EmptyStackException {/** pre@ self.size() > 0 *//** post@ self.size() = self.size()@pre - 1 */}public Collection all () {/** pre@ true *//** post@ (self.size() > 0 implies return.size() = self.size()) AND

(self.size() = 0 implies return = null)*/}

}



Contract-based test cases for BoundedStack

Context ExtendedStack A B C D E F invariant@ this.size() >= 0 AND TRUE TRUE TRUE TRUE TRUE TRUE invariant@ this.size() <= this.MAXSIZE TRUE TRUE TRUE TRUE TRUE TRUE ExtendedStack() pre@ true TRUE ExtendedStack()post@ this.size() = 0 AND TRUE ExtendedStack()post@ this.MAXSIZE = maxSize TRUE push() pre@ this.size() < this.MAXSIZE TRUE TRUE push() post@ this.size()=this.size()@pre+1 - TRUE top() pre@ this.size() > 0 FALSE TRUE TRUE top() post@ return != null - - TRUE pop() pre@ this.size() > 0 FALSE TRUE TRUE TRUE pop() @post this.size() = this.size()@pre – 1

- - - TRUE

all() pre@ true TRUE TRUE TRUE TRUE TRUE TRUE all() post@ this.size() > 0 AND - - - - FALSE TRUE all() post@ return.size() = this.size() OR - - - - dc TRUE all() post@ not this.size() > 0 AND - - - - TRUE FALSE all() post@ return = null - - - - TRUE dc

Context BoundedStackinvariant@ self.size() >= 0 AND

self.size() <= self.MAXSIZE()BoundedStack() pre@ maxSize > 0BoundedStack()post@ self.size() = 0 AND

self.MAXSIZE() = maxSizepush() pre@ self.size() < self.MAXSIZE()push() post@ self.size()=self.size()@pre+1top() pre@ self.size() > 0top() post@ return != nullpop() pre@ self.size() > 0pop() @post self.size() = self.size()@pre – 1all() pre@ trueall() post@ self.size() > 0 AND

( return.size() = self.size() ORnot self.size() > 0 ANDreturn = null )

Don‘t CareDon‘t Care

Conformance testing: precondition satisfiedRobustness testing: precondition not satisfiedTest oracle: postcondition satisfied

Conformance testing: precondition satisfiedRobustness testing: precondition not satisfiedTest oracle: postcondition satisfied







State-based specification for BoundedStack

empty filled fullpush

push[ size < MAX-1]pop[ size > 1 ] push

pop[ size = 1 ]pop

push[ size = MAX-1]

~Stapel

Stapel

final

initaltop top

Three (five) states:empty: size() = 0;filled: 0 < size() < MAXSIZE();full: size() = MAXSIZE();initial: before construction;final: after destruction;


3 ← size21

MAXSIZE





Life cycle-based test cases (aka. state based)

filled

emptypush

filledpush

full

push

empty

pop

filledpop

filledpop

fullpush

final

~Stapel

initialStapel

filledtop

fulltop

errortop

error

~Stapel

error

~Stapel

errorpop

empty filled fullpush

push[ size < MAX-1]pop[ size > 1 ] push

pop[ size = 1 ]pop

push[ size = MAX-1]

~Stapel

Stapel

final

initaltop top



3 ← size21

MAXSIZE







Testing operation calls and late binding

Am1(){myB.m2();}

B1m2(){...}

B2m2(){...}

m2(aB: B){...}

A2A1

myBB

Operating object (as „client"), parameter object und server object classes:How many different combinations of classes and states?Operating object (as „client"), parameter object und server object classes:How many different combinations of classes and states?

OpUTOpUT

CallCall

Parameter classParameter classClient classClient class

Server classServer class

Two states SA1 and SA2Two states SA1 and SA2

Three States SB1, SB2 and SB3

Three States SB1, SB2 and SB3

(3 C * 2 S) * (3 C * 3 S) * (3 C * 3 S) = 486 possibilities!(3 C * 2 S) * (3 C * 3 S) * (3 C * 3 S) = 486 possibilities!





Idea: Not all combinations, but all pairwise combinations• Orthogonal arrays (OA)• 2 x 3 classes with 2 resp. 3 states!• 6 factors with cardinality 2 resp 3

Am1(){myB.m2();}

B1m2(){...}

B2m2(){...}

m2(aB: B){...}

A2A1

myBB

A = 1, A1 = 2, A2 = 3, ZA1 = 1, ZA2 = 2B = 1, B1 = 2, B2 = 3, ZB1 = 1, ZB2 = 2, ZB3 = 3

A = 1, A1 = 2, A2 = 3, ZA1 = 1, ZA2 = 2B = 1, B1 = 2, B2 = 3, ZB1 = 1, ZB2 = 2, ZB3 = 3

OA18(21x37)

Nr ClientState

ClientClass

ParameterClass

Paramete-State

ServerClass

ServerState

1 1 1 1 1 1 1 1 1

2 1 1 2 2 2 2 2 2

3 1 1 3 3 3 3 3 3

4 1 2 1 1 2 2 3 3

5 1 2 2 2 3 3 1 1

6 1 2 3 3 1 1 2 2

7 1 3 1 2 1 3 2 3

8 1 3 2 3 2 1 3 1

9 1 3 3 1 3 2 1 2

10 2 1 1 3 3 2 2 1

11 2 1 2 1 1 3 3 2

12 2 1 3 2 2 1 1 3

13 2 2 1 2 3 1 3 2

14 2 2 2 3 1 2 1 3

15 2 2 3 1 2 3 2 1

16 2 3 1 3 2 3 1 2

17 2 3 2 1 3 1 2 3

18 2 3 3 2 1 2 3 1







Create and execute test cases

• Testscripts – TTCN-3 – Parameterized – Executable

:A {ZA1} :B {ZB1}myB

:B {ZB1}aB

A = 1, A1 = 2, A2 = 3, ZA1 = 1, ZA2 = 2B = 1, B1 = 2, B2 = 3, ZB1 = 1, ZB2 = 2, ZB3 = 3A = 1, A1 = 2, A2 = 3, ZA1 = 1, ZA2 = 2B = 1, B1 = 2, B2 = 3, ZB1 = 1, ZB2 = 2, ZB3 = 3

module Test of Class A {modulepar {

B_Type theB_Par;B_Type anotherB_Par;

}external const float TestExecutionTime;control {var verdicttype testCaseVerdict := none; testCaseVerdict :=

execute(connectmyB(theB) ,TestExecutionTime);if (testCaseVerdict == pass) {

testCaseVerdict := execute(m1(anotherB) ,TestExecutionTime);

}}

}

module Test of Class A {modulepar {

B_Type theB_Par;B_Type anotherB_Par;

}external const float TestExecutionTime;control {var verdicttype testCaseVerdict := none; testCaseVerdict :=

execute(connectmyB(theB) ,TestExecutionTime);if (testCaseVerdict == pass) {

testCaseVerdict := execute(m1(anotherB) ,TestExecutionTime);

}}

}

Nr ClientState

ClientClass

ParameterClass

Paramete-State

ServerClass

ServerState

1 1 1 1 1 1 1 1 1





Specification and testing OO-SW

All messages, all nodes, allbranches, ..., all paths

UML sequence diagr.

All actions, all transitions, n-Paths, ..., all paths

UML activity diagramDiagram with formalsemantics

All messages, all linksUML communication diagram

Structure based

All functionsAPIInformal (natural language)All clausesObject-Z, VDMFormal theory

Normal flow, all alternate flows, ..., all flows

UML use case diagram

Informal (natural language)Functionbased

(declarative/ operational)

All messages, all nodes, all branches, ..., all paths

SDL MSC,UML sequence diagram

Message basedInteraction

based(operational)

All states, all transitions, n-Paths, ..., all paths

UML state chartDiagram with formal semantics

State based (operational)

All conditions, MC/DC, ...UML-OCLFormal languageContract based

(declarative)

Method / Coverage CriteriaConcrete SyntaxTypeSpecification







Where are we now?

• Why is testing object-oriented software difficult?• Specification and testing of object-oriented software• The UML Testing Profile• TTCN-3 cases for OO



Example: Automated Teller Machine (ATM)

System Under Test(SUT)ATM

Main Test Component

(MTC)

Bank User

Parallel TestComponent

(PTC)

Bank ServerUser

InterfaceNetworkInterface

Test Coordination

Test System

Why is testing OO-SW difficult? Specification and testing of OO-SW


NetworkInterface

UserInterface

Bank ServerBank User at ATM

Ina Schieferdecker: The UML 2.0 Testing Profile. Proc. TAV 19, Erlangen, 2003





ATM system test scenario

Menu selection

Verify Amount& write ec-card

Take ec-card

Card readerEjectmoney

Take

& Eject ec-card

amount

emptyMoney

Input



Identification



UTP: ATM system test class diagram

ATM«import»

ATMTest

«testSuite»ATMSuite

-verdict : Verdict-amount : IMoney-targetBank : SwiftId-targetAccount : String-sourceAccount : String

«testCase» +validWiring() : Verdict«testCase» +invalidPIN() : Verdict«testCase» -authorizeCard() : Verdict

*

-accounts«testComponent»BankEmulator

IBank

«interface»IAccount

-pinOk : Boolean-enteredPIN : String-message : String-t1 : Timer

«testComponent »HWEmulator

hwComIATM

IHardware

Test packageTest packageTest componentTest component

MiscellaneousMiscellaneousTest suite

with test casesTest suite

with test cases







UTP: ATM system test case behavioral specificationsd invalidPIN

HardwareEmulator

«sut»atm

storeCardData(current)

display(”Enter PIN”)

isPinCorrect(invalidPIN)

«validationAction»pass

BankEmulator

{readOnly} Integer invalidPIN; { current.isPinCorrect(invalidPIN) == false }

isPinCorrect(invalidPIN)

display(”Invalid PIN”)

display(”Enter PIN again”)

isPinCorrect : falseisPinCorrect : false

t1(2.0)

t1

{0 .. 3}

SUT and Test Component Lifelines

SUT and Test Component Lifelines

StimulusStimulus

ObservationObservation

Timing annotations

Timing annotations

Arbitrated verdict

Arbitrated verdict

Data partitionData partition





Where are we now?

• Why is testing object-oriented software difficult? • Specification and testing of object-oriented software• The UML Testing Profile • TTCN-3 cases for OO





TTCN-3 – The standard


The UML Testing Profile TTCN-3 cases for OO

Presentation Format n

Graphical Format

Tabular Format

Other Types & Values 2

IDL

ASN.1 Types & Values

Other Types & Values n

Presentation Format nUML Testing Profile

…

C++

msc mi_synch1_conc1

mtc ISAP1 MSAP2

TTCN-3CoreLanguage

:testcase myTestcase () runs on MTCType system TSIType{ mydefault := activate (OtherwiseFail);

verdict.set(pass);:connect(PTC_ISAP1:CP_ISAP1,mtc:CP_ISAP1);:map(PTC_ISAP1:ISAP1, system:TSI_ISAP1);:PTC_ISAP1.start(func_PTC_ISAP1());PTC_MSAP2.start(func_PTC_MSAP2());Synchronization(); all component.done;log(”Correct Termination”);

}:



Eclipse Test & Performance Tools Platform (TPTP)

• Object oriented testing with TTCN-3• Open source top level project of the Eclipse foundation• Includes the HYADES TTCN-3 tool set

– TTCN-3 Core Language Editor– TTCN-3 Compiler

• TPTP is divided into four projects. – TPTP Platform – Monitoring Tools– Testing Tools– Tracing and Profiling Tools

• But– TTCN-3 integration with least common denominator principle – Many powerful TTCN-3 concepts were left out!

http://www.eclipse.org/tptp







Using C/C++ with TTCN-3

• Draft ETSI ES 201 873-10 Methods for Testing and Specification (MTS); The Testing and Test Control Notation version Part 10: Using C/C++ with TTCN-3 (lead author: Jens Grabowski)





What did we reach with TTCN-3?

• OO-Testing = Message Sequences = TTCN-3• Excellent notation for black-box test scripts

– Modular– Reusable

• Excellent tool support for TTCN-3 specification and execution– Commercial and open source– Different domains (communication, automotive, web, ...) well covered







What is still missing regarding OO-Testing?

• Subtyping for TTCN-3 structures– Data structures– Modules– Test Cases– ...

• Full UML testing profile binding– Subclassing (i.e. generalization) – Tool Support w.r.t. full UML support

• Java Binding („Using Java with TTCN-3“)

• More wishes?



Rectangle

- side1- side2


Rectangle

- side1- side2


Square

- side1- side2


Square

- side1- side2




The Future: MDA and MBT

UML(EDOC)

SystemsourceArchitecture

UTP/TTCN-3

Testcode

UTP/TTCN-3



ModelDrivenArchitecture

ModelBasedTesting





Summary

• Why is testing object-oriented software difficult?– Inheritance– Polymorphism, late binding

• Specification and testing of object-oriented software– Unified Modeling Language– Testing techniques

• The UML Testing Profile– Black box testing notation– ATM case study

• TTCN-3 cases for OO– Eclipse TPTP– Using C/C++ with TTCN-3



Program (Object-Oriented)

Testcase(TTCN-3)

Object-Oriented Testing meets TTCN-3 · 2013-01-15 · Object-Oriented Testing meets TTCN-3 TTCN-3...

Documents

Transcript of Object-Oriented Testing meets TTCN-3 · 2013-01-15 · Object-Oriented Testing meets TTCN-3 TTCN-3...