Writing a Test Strategy

7/24/2019 Writing a Test Strategy

1/32

Writing a Test Strategy

Tor Stlhane


2/32

Why a testing strategyWe need a testing strategy to help System and software testers plus Test-and-

Evaluation sta to determine the overalltest strategy for development ormodication of a software intensive system

Project staeholders ! customers and seniormanagement ! approve the test strategy Testers and system and software analysis to

determine

Test o"jectives #ualication re$uirements %erication and validation criteria


3/32

Testing strategy concepts

We will discuss the following concepts& Purpose of a test strategy

Testing focus

'ontents of a test strategy Software integrity levels

Test o"jectives and priorities


4/32

Purpose of a test strategy !(

The test strategy is important to )"tain consensus on test goals and

o"jectives from staeholders ! e*g*

management+ developers+ testers+customers and users ,ange epectations right from the start .e sure that we are heading in the right

direction /dentify the type of tests to "e

conducted at all test levels


5/32

Purpose of a test strategy !0

When we write a test strategy is important toremem"er that&

Whatever we do some ind of test strategywill emerge* Thus+ we might as well specifythe one we thin is "est

1 documented strategy is the mosteective way to get an early agreement ongoals and o"jectives

We need to address& 2uman factors ! usa"ility /nteropera"ility+ ecept for stand-alone

systems


6/32

Testing focus

)ur focus will depend on which staeholder weare considering at the moment&

3sers ! acceptance test and operational tests

1nalysis ! systems test+ $ualication tests

4esigner ! integration tests

Programmer ! unit tests

The main point is that we need to defne thestakeholder frst ! then the tests to "e run*


7/32

'ontents of a test strategy !(

The following is a list of what can "especied in a test strategy* 5ot all ofit is needed in all cases ! only usewhat is necessary*

Project plan+ riss and activities 6elevant regulations ! depending of

application area 6e$uired processes and standards Supporting guide lines


8/32

'ontents of a test strategy !0

Staeholders ! e*g* users+ testers+maintainers ! and their o"jectives

5ecessary resources ! people+

computers Test levels and phases

Test environment

'ompletion criteria for each phase 6e$uired documentation and review

method for each document


9/32

Software integrity level

There are several ways to denesoftware integrity levels* When wechoose an integrity level this willstrongly in7uence the way we do

testing*We will loo at three denitions of

integrity levels&

/EEE (8(0 ! general software /S) 09090 ! automotive software /E' 9(:8; ! general safety critical

software


10/32

/EEE (8(0 ! generalsoftware

ow ! some functions have noticea"leeect on system performance "utcreates only inconveniences


11/32

%?v 1ctivities

V&V activityDevelopmentrequirements

level

Designlevel

Implementationlevel

Test level

SW Integrity Level 4 3 2 1 4 3 2 1 4 3 2 1 4 3 2 1

Acceptance test eecution ! ! !

Acceptance test plan ! ! !

Inter"ace analysis ! ! ! ! ! ! ! ! !

#anagement an$ revie%support

! ! ! ! ! ! ! !

#anagement revie% o"V&V

! ! ! ! ! ! ! ! ! ! !


12/32

/S) 09090 ! automotivesoftware

The 1S/> level ! 1+ .+ ' or 4 ! is theoutcome of the com"ination of threefactors&

S ! Severity* 2ow dangerous is aevent

E ! Pro"a"ility* 2ow liely is the event

' ! 'ontrolla"ility* 2ow easy theevent to control if it occurs


13/32

@inding the 1S/> level

Severity Pro"a"ility 1 2 3

S1

'1 (# (# (#

'2 (# (# (#

'3 (# (# A

'4 (# A )

S2

'1 (# (# (#

'2 (# (# A

'3 (# A )

'4 A )

S3

'1 (# (# A

'2 (# A )

'3 A )

'4 ) D


14/32

,ethods for software

integration testingMethods and Measures According

to req.ASIL

A B C D

1 Requirements based test 9.4.4 ++ ++ ++ ++

2 External interface test 9.4.4 + ++ ++ ++

3 Fault injection test 9.4.4 + + ++ ++

4 Error uessin test 9.4.4 + + ++ ++


15/32

,ethods for software unittesting

Methods and Measures Accordingto req.

ASIL

A B C D

1 Functional tests !.4.2 "ee table !.2

2 "tructural co#erae !.4.2 "ee table !.3

3 Resource usae measurement !.4.2 + + + ++

4 $ac%&to&bac% test bet'eensimulation model and code( ifa))licable

!.4.2 + + ++ ++


16/32

Safetyintegrity

level

High demand or continuous mode ofoperation

*robabilit, of a danerous failure )er-our

4

1/&9

tooo out for anything that is& 'omple ! large+ intricate or convoluted

5ew ! no past history in this product

'hanged ! anything that has "een tampered

with or HimprovedI 3pstream dependency ! a failure here will

cascade through the system

4ownstream dependency ! sensitive to failure inthe rest of the system

'ritical ! a failure here will cause seriousdamage


29/32

.achAs generic ris list ! 0 Precise ! must meet the re$uirements eactly

Popular ! will "e used a lot

Strategic ! of special importance to the usersor customers

Third-party ! developed outside the project 4istri"uted ! spread out over time or space

"ut still re$uired to wor together

.uggy ! nown to have a lot of pro"lems

6ecent failure ! has a recent history offailures*


30/32

Test and system level ! (


31/32

Test and system level ! 0

@rom the diagram on the previous slidewe see that we can test on the Electronics level ! e*g* 4oor1ctuator

sends the right signal State B signal level ! e*g* door is closed

i 4oorState'losed >ogical level ! e*g* the door remain

closed as long as the speed is non-Gero Safety level ! e*g* the door remain

closed as long as the train is moving


32/32

1cnowledgement

The rst part of this presentation ismainly taen from Kregory T* 4aichAspresentation H4ening a Software

Testing StrategyI+ =8 1pril 0880*

Writing a Test Strategy

Documents

Transcript of Writing a Test Strategy