Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

50
Software Testing Copyright © André Murbach, 2005 1  Software Testing – An Overview & Some Practical Hints Dr. André Murbach [email protected]

Transcript of Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

Page 1: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 1/50

Software Testing Copyright © André Murbach, 20051

 

Software Testing – An Overview

& Some Practical Hints

Dr. André Murbach

[email protected]

Page 2: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 2/50

Software Testing Copyright © André Murbach, 20052

Overview

What Is Testing?

Definition

Why Testing?

Testing & Defects

Testing & Quality

How To Do Testing?

Test Cases & Co.

Testing Activities

Economics of TestingPsychology of Testing

Defect Management

Types of Tests

Testing Techniques

Test Stages

A Minimal Test Process

Page 3: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 3/50

Software Testing Copyright © André Murbach, 20053

What Is Testing?

Definition A:

The purpose of testing is to discover errors. Testing is theprocess of trying to discover every conceivable fault orweakness in a work product. (Glenford Myers, The Art of 

Software Testing, 1979)

-> A single aspect: find problems

-> Serious effort / really try hard to find them

Page 4: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 4/50Software Testing Copyright © André Murbach, 20054

What Is Testing?

Definition B:

The purpose of software testing is to give increasedconfidence in those areas of the product that work and todocument issues with those areas of the product that do not

work. (Mark Fewster and Dorothy Graham, Software Test Automation, 1999)

-> Constructive: Confidence that the product is good enough

Know the remaining problems

Page 5: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 5/50Software Testing Copyright © André Murbach, 20055

What Is Testing?

Tasks:

Try to find problems – fault-directed 

Measure / estimate quality – conformance-directed  

– Quality factors like correctness, robustness, ...

Results: (Justified) confidence that software satisfies expectations

Problems known and documented

Disputed:

Static / dynamic testing

„Hard“ / „soft“ requirements & problems

Page 6: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 6/50

Software Testing Copyright © André Murbach, 20056

"Is the Moon an Enemy Rocket?"

A system able to detect and identify flying objects. Objects notidentifiable as own airplanes are enemy targets.

The system identified the moon as an enemy target. Was

there a software defect?

"Soft" / user perspective: yes

"Hard" / developer perspective: no - specifications are

satisfied

Page 7: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 7/50

Software Testing Copyright © André Murbach, 20057

Why Testing?

Testing & Defects

Testing & Quality

Page 8: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 8/50

Software Testing Copyright © André Murbach, 20058

Why Does Software Have Defects?

Errare humanum est

Software is man made

Errors are inevitable

Contributing factors:

Time pressure

Misunderstandings, communication problems

Size & complexity of software systems

Defects may remain in the SW

Software defects may be difficult to detect

Buggy software is common / accepted by the market(at present)

Page 9: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 9/50

Software Testing Copyright © André Murbach, 20059

Cost of Defects

Very expensive examples:

Therac-25 accidents killing several people

Ariane 5 explosion

Typical case:

Increased support & maintenance costs

Customer dissatisfaction

– Costs difficult to predict

– Long term consequences

Page 10: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 10/50

Software Testing Copyright © André Murbach, 200510

Defect Cost During Development

Costs already during development

The longer a defect remainsundetected, the moreexpensive it becomes

Increased development effort

Schedule slips

Investments in software quality

practices reduce those costs– Detect & correct

problems early

Page 11: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 11/50

Software Testing Copyright © André Murbach, 200511

The Solution?

The most effective remedies:

Highly qualified developers

Reviews

Testing

But why does tested software still have defects?

Unqualified testing?

Tests may be erroneous themselves?

The testing was not exhaustive?

Page 12: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 12/50

Software Testing Copyright © André Murbach, 200512

Why Does Tested Software Still Have Defects?

Untested code segments

Customer applies instructions in a different sequence thanexecuted during testing

Customer applies combination of inputs not applied duringtesting

Execution environment (HW, OS, other applications) differsfrom the environment used during testing

Page 13: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 13/50

Software Testing Copyright © André Murbach, 200513

Exhaustive Testing Is Not Feasible

Example: two lines on a screen

A display with 1024 x 768 = 786'432 pixels

Number of ways to draw a line = 786'432^2 6 x 10^11

Two lines: 36 x 10^22

Age of the universe 5 x 10^16 s

Page 14: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 14/50

Software Testing Copyright © André Murbach, 200514

Testing & Quality

Relationships between Testing and Software Quality

Measuring quality by testing

-> Ready to ship? (release decision)

Improving quality by testing

Testing-in quality?

Page 15: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 15/50

Software Testing Copyright © André Murbach, 200515

SW Quality (Product Quality)

External Quality vs. Internal Quality

External quality: Reliability, correctness, usability, ...

-> visible to the user

Internal quality: Understandability, maintainability, ...

-> visible to the developer

Quality factors / quality dimensions

Page 16: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 16/50

Software Testing Copyright © André Murbach, 200516

Why Testing? – Summary

Find & eliminate defects, improve quality

"Form follows failure"

Document issues

Estimate / measure quality

Ready to ship?

Reduce cost

Contractual obligations, industry standards

Page 17: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 17/50

Software Testing Copyright © André Murbach, 200517

How Is Testing Done?

Page 18: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 18/50

Software Testing Copyright © André Murbach, 200518

Specifying Tests – Some Terms

Test condition

Condition to cover by a test case

Example: „illegal value for input A“

Test case

Preconditions

Inputs to apply

Expected outcomes

Test procedure

Instructions to the tester

Leads to the execution of test cases

Page 19: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 19/50

Software Testing Copyright © André Murbach, 200519

Testing Activities (1)

Planning

Breakdown into topics / areas to test

Define goals / confidentiality levels for each area Think of methods to apply (testing techniques)

Effort estimation & schedule

Infrastructure

Test Design

Select testing techniques Identify the conditions to cover (by applying the techniques)

Find test cases covering the conditions

Create test specifications (test procedures, test scripts, testdata)

Page 20: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 20/50

Software Testing Copyright © André Murbach, 200520

Testing Activities (2)

Test Execution

Execute the tests

Record the results, report defects

Feedback on test specifications

May need to be repeated many times (test cycles)

Debugging related

Make failures reproduceable

Verify fixes (re-testing)

Page 21: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 21/50

Software Testing Copyright © André Murbach, 200521

Economics of Testing

Page 22: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 22/50

Software Testing Copyright © André Murbach, 200522

When To Stop Testing?

Effort optimization

Determining test effort is an economical decision – not atechnical one.

Risk / cost / time considerations

Risk / cost of defects; cost of testing; time to market

Test Effort

Bene fi t

Page 23: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 23/50

Software Testing Copyright © André Murbach, 200523

Test Prioritization

Optimize test execution by prioritizing tests based on risk

Prioritize tests so that whenever you stop testing youhave done the best testing in the time available.

Page 24: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 24/50

Software Testing Copyright © André Murbach, 200524

Psychology of Testing

 

Page 25: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 25/50

Software Testing Copyright © André Murbach, 200525

The Testing Paradoxon

Paradoxical goals:

Testing tries to find defects

Testing tries to demonstrate the absence of defects

Paradoxical approach to second goal:

Looking for something you don't really want to find

It is sometimes difficult to follow both goals in parallel.

Page 26: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 26/50

Software Testing Copyright © André Murbach, 200526

Analogy: Scientific Experiments

Test - Experiment

Requirements - Theory

Software under test - Reality

Scientific theories can't be proven. They just gain increasing

confidence as they withstand attempts to falsify them.

A good test is similar to a scientific experiment that tries to

confirm or falsify a hypothesis.

Page 27: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 27/50

Software Testing Copyright © André Murbach, 200527

A Tester's Mindset

Combination of "destructive creativity" and analytical systematicity

Try to break, try to falsify, be nasty

Be systematic - ensure coverage, objectivity, reproduceability

User perspective

Complementing the developper perspective

See yourself as representative of the user community

Be demanding / critical

Required expertise & knowledge

Business / product / domain knowledge

Testing experience

Testing methodology

Page 28: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 28/50

Software Testing Copyright © André Murbach, 200528

How To Measure Good Quality?

Measuring poor quality is relatively easy

Measuring good quality is difficult

-> Justified confidence in the quality of the software requiresconfidence in the quality of the testing.

SW Quality

Test Quality

few defects

very few defects few defects

many defects+

+

-

--

--

++ +

+

Page 29: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 29/50

Software Testing Copyright © André Murbach, 200529

A Constructive "Defect Culture"

Defects are bad news:

Debugging instead of implementing new features

Additional work and time needed

Nobody likes to be accused of having done bad work

Deal with defects in a constructive way:

Accept & expect errors / defects– Errors are natural – plan for them

– Focus on the problem to be solved – not on finding the guilty

– Learn from errors / defects Establish an efficient defect management

– Bug tracking tool

– Enforce fixing & verifying defects

– Triage / change control board

Page 30: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 30/50

Software Testing Copyright © André Murbach, 200530

Types of Tests

Page 31: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 31/50

Software Testing Copyright © André Murbach, 200531

Testing Techniques

Behavioral

Functional

StaticDynamic

Non-functional

Structural

DataFlow

ControlFlow

Defs/UsesCriteria

StatementCoverage

BranchCoverage Condition

Coverage

Equivalence Partitioning

Boundary Analysis

State Transition Techniques

Usability

Performance

Reliability

etc.etc.

etc.

etc.

Reviews

Inspections

StaticAnalysis

etc.

FormalVerification

Page 32: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 32/50

Software Testing Copyright © André Murbach, 200532

What Is a Testing Technique?

A procedure for selecting test cases

Systematic, less dependent on individual test designer

Page 33: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 33/50

Software Testing Copyright © André Murbach, 200533

Equivalence Partitioning

Partition the inputs (or outputs) into areas which areconsidered equivalent (e.g., same qualitative behavior)

Assumption: if one value works, all will work

Take one value from each equivalence class

Page 34: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 34/50

Software Testing Copyright © André Murbach, 200534

Domain Testing

„Bugs lurk in corners and congregate at boundaries.“

Determine test points for each partition boundary – ON pointsand OFF points

Page 35: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 35/50

Software Testing Copyright © André Murbach, 200535

State Transition Testing

Based on a model of the expected behavior

Possible coverage criteria:

Exercise every transition

Exercise every transition pair

Invalid transitions: for each state, apply each invalidevent

...

On / RedButton / 

„woom!“

Off / 

OFF ON

Page 36: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 36/50

Software Testing Copyright © André Murbach, 200536

Statement Coverage / Branch Coverage

Statement coverage: Exercise every statement in the sourcecode

Branch coverage: Exercise every branch / decision outcome

Read A, B

If (A < 0) then

Print „A negative“

endIf

If (B < 0) then

Print „B negative“

endIf

Read

Print

Print

A

B

Page 37: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 37/50

Software Testing Copyright © André Murbach, 200537

A Selection Of Testing Techniques

Functional Tests

Equivalence

partitioning Boundary analysis

Domain analysis

Cause-effect analysis State-based tests

Use case testing

Structural Tests

Statement coverage

Branch coverage

Condition coverage

Data flow based tests

Other Techniques

Error guessing

Exploratory / ad hoctesting

Risk-based testing

Page 38: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 38/50

Software Testing Copyright © André Murbach, 200538

Testing Throughout The Lifecycle

Test Stages ( Testing Techniques):

Unit test / module test

Integration test

System test

Acceptance test

Alpha test

Beta test / field test

Regression test

Smoke test / sanity check

related to the „V-Model“

tests at system level

Page 39: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 39/50

Software Testing Copyright © André Murbach, 200539

The V-Model

Page 40: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 40/50

Software Testing Copyright © André Murbach, 200540

System Test – Indispensable

System test finds defects not detectable at lower levels

The complete system exhibits behaviors not present / 

testable at lower levels

Severity of defects not detectable by system test typically low

Defects not easily visible at system level are not soproblematic to users

-> Defects found at system level are those that are really

relevant to the users

Other tests at system level (smoke test, beta test, etc.) areless thorough / systematic

Page 41: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 41/50

Software Testing Copyright © André Murbach, 200541

A Minimal Test Process

Page 42: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 42/50

Software Testing Copyright © André Murbach, 200542

Introducing a Minimal Test Process

Suggesting a test process that is simple yet effective. Might beuseful if there isn't already anything better in place.

Overview:

A single test stage: system test

Task: Test planning & test design Fault model-based – enumerate potential problems &

cover them by tests

Task: Test execution Apply test cycles on a succession of release candidates 

Defect management; change & version control; release

decision

Page 43: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 43/50

Software Testing Copyright © André Murbach, 200543

Task: Test Planning & Test Design

Test design based on a fault model

Idea: Enumerate potential faults, write test procedures tocover them

The primary challenge: Cover the most critical faults „What is not tested will not work!“ (Murphy, Murbach &

Küpfer, 2002 ;-)

Identifying everything that should be tested is moreimportant than applying sophisticated testingtechniques

Page 44: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 44/50

Software Testing Copyright © André Murbach, 200544

Proposal: Mind Map As Fault Model

Explore the application – refine the mind map

Identify types of potential faults – what could go wrong?

Page 45: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 45/50

Software Testing Copyright © André Murbach, 200545

Test Design Based On Fault Model

1. Define fault model

What faults are conceivable? What could possibly go wrong?

– An item might be as simple as: „feature XY notworking for typical values“

Which faults do I want to test for? Base your choice on risk / criticality.

Goal: Not to miss any important faults

2. Select test models & techniques – or find your own „naive“ approach

Such that they cover the fault model

3. Develop / use test models & apply techniques

-> Test conditions

4. Write test procedures

Page 46: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 46/50

Software Testing Copyright © André Murbach, 200546

Test Design – Writing Test Procedures

To write the test procedures:

To specify the tests to execute, find a format you feel

comfortable with Be short & precise

Always consider maintainability

Example:

Description Details

Feature X in mode Y.- Goto mode Y, set a=0, - Green light goes on

select b such that ... - ...

T k T E i

Page 47: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 47/50

Software Testing Copyright © André Murbach, 200547

Task: Test Execution

1st full test cycle near code complete

Goal: Gain overview over the pending problems

Development & testing continues in parallel

When you have a defined version without any known defectsplanned to be fixed – declare it a release candidate (RC)

RC is the software (including the known bugs) to beshipped provided no new high-severity defects are found

Typically, there is a succession of RC's (RC1, RC2, ...)

Each time a non-acceptable defect is found, a newversion is produced -> RC(n+1)

A complete test cycle passed on RCn without any non-acceptable defects -> Declare RCn the release version

P C i & T A i i i

Page 48: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 48/50

Software Testing Copyright © André Murbach, 200548

Process – Construction & Test Activities

RC2 RC3RC1

full cycle

Req. Analysis – Design – Implementation

Test DesignTest Cycles

& Retesting

Test Planning

Release

T t E ti S Hi t

Page 49: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 49/50

Software Testing Copyright © André Murbach, 200549

Test Execution – Some Hints

Triage on found defects

Test manager & project manager decide on which

defects to fix – and which not

Be aware of the cost and risks of defect fixes late in the project

New defects may appear (newly introduced or previouslymasked)

-> Additional test cycles may be needed

Fix defects as early as possible

Plan for sufficient test cycles

Summary & Complement:Key Testing Principles & Influences

Page 50: Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

8/4/2019 Software Testing – An Overview & Some Practical Hints - Dr. André Murbach

http://slidepdf.com/reader/full/software-testing-an-overview-some-practical-hints-dr-andre-murbach 50/50

Software Testing Copyright © André Murbach, 200550

Key Testing Principles & Influences

Six Key Testing Principles

Complete testing is notpossible

Testing is creative and

difficult An important reason for

testing is to prevent errors

Testing is risk-based Testing must be planned

Testing requiresindependence

Key Influences

Individual creativity andinsight

Application knowledge and

understanding Testing experience

Testing methodology

Effort and resources

Source: B. Hetzel, The Complete Guide to Software Testing, 1988