Software TestingTesting Process

Agenda

• Determining the test methodology

• Planning the tests

2

The Testing Process

3

• Planning and Design for testing is done in all phases of software development.

Determine the test methodology

Planning the tests

Designing the tests

Performing the tests

Test Methodology Determination

• Main issues to handle before deciding the methodology What Quality Standard is needed? Which testing strategy should be followed?

• Quality Standard Depends mainly on the characteristics of the project Examples

Hospital Patient Bed Monitor Software Software to handle feedback

Criteria for selecting the right standard Nature of the issue Magnitude (severity) of the damages expected in case of failure

4

Software Failure Damage Types

• Damages to the users1. Danger to human life2. Affect organizations functions3. Affects functioning of the main component4. Affects essential functions (but replacement is available)5. Affects functioning of software packages6. Affects functioning of a main component but does not affect the

entire system7. Inconvenience to the user

• Damages to the developer Financial Loss Non-quantitative damages

5

Testing Strategy

• Issues include Type of testing to use

Big Bang or Incremental If Incremental

Top-Down Bottom-Up

Which part of the testing plan should use White-Box Testing

Which part of the testing plan should use Automated Testing

6

Planning the Tests

• Tests to be planned Unit tests Integration tests System tests

• Issues to consider before test plan What to Test? Which sources to use for test cases? Who performs the tests? Where to perform the tests? When to terminate the tests?

7

What to Test?

• Straightforward Plan Unit test for all individual units (or modules) Integration tests for all integrations of the units One complete test for the system

• Issues with straightforward Plan Should we unit test all modules – even if they are reused? Should we test simple units / modules?

• Factors to decide Which modules to be unit tested? Which integration to be tested? Assign priority to the test cases.

8

Rating for Priority - Factors

• Two main factors Damage Severity Level

See the damage types discussed before

Software Risk Level Module / Application Issues

Magnitude Complexity Percentage of Original vs. Reused Software

Programmer Issues Qualification Experience Availability Self-Evaluation

9

Rating for Priority - Formula

10

C = A + BC = (k x A) + (m x B)C = A x B

C is the combined priorityA is the severity of damageB is the risk severityk and m are constants

Sources for Test Cases

• Two main sources Real-life test cases Synthetic test cases

• Issues Use one or both sources How many from each Characteristics of test cases

• More on this in the next lecture

11

Who performs the Tests?

• Determined by the planning stage Integration and unit tests are performed by the

development team System test performed by an independent testing team

(internal or external) More than one testing team for a large project

• If the organization is small (no testing team available) Test by another development team Outsourcing

12

Where to perform the Tests?

• Unit and Integration Test Development Site

• System Test Location Choices Developer Site Customer Site Consultant Site

If performed on the developer site or consultant site – simulate the environment as on the customer site (for acceptance)

13

When to stop testing?

• Five Routes (Paths) Available

• Route A: Completed Implementation Entire test plan is carried out Perfection Approach Disregard the budget and timeline

• Route B: Mathematical Models Application Estimate the % of undetected errors (rate of error detection) Stop when we go below the rate Model may not represent the real-world situation

14

When to stop testing?

• Route C: Error Seeding Errors are hidden inside the tested software Stop when % of undetected seeded errors reach a predefined

level Additional work to add / hide the errors Based on past experience

• Route D: Dual Independent Testing Team Two teams implement the testing process independently Work Assignment

Read and complete the problem on page 226-227 of the Book (Galin)

• Route E: Resource Exhaustion Terminate when budget is over or time is finished

15