Software Testing Strategies

TCS2411 Software Engineering 1

Software Testing Strategies

“How do you organise your software tests?”


Lecture ObjectivesTo describe the process of planning

for software testingTo understand the different levels of

testing that can be doneTo describe the different approaches

to debugging software


Software Testing StrategyThe approach towards software testingA set of tests that is planned and

conducted to ensure successful construction of software

Guideline or “road map” of testing activities - planning, design, execution, data collection, evaluation


Examples of Software Testing Strategies

Testing begins at module level and works “outward” towards integration of the entire computer-based system

Different testing techniques at different points in time

Testing is conducted by developer and an independent test group

Testing and debugging are different activities, but debugging is accommodated


A Software Testing Strategy

SystemEngineering Requirements Design Code

UnitTest

IntegrationTest

ValidationTest

SystemTest


Software Testing Organization

Developer : unit testing Ensure each functions performs according to

its design Sometimes conduct integration testing

Independent testing group (ITG) Report to SQA team

Developer and ITG work closely to ensure thorough test is well conducted

Developer must be present during testing to correct uncovered errors.


Software Testing Steps

High-order tests

Integration test

Unit test

Testingdirection

code

design

requirements

Testingdirection

code

design

requirements


Unit TestingVerification of the smallest unit in

software - the moduleWhite box orientedCan be conducted in parallel for

multiple modulesUnit test cases designed based on

source code - examine likely causes for errors

Test environment includes driver and/or stubs


Unit Test Environment

Driver

Module tobe tested

StubStub …..


Unit Testing Considerations

Interface - tested to ensure that information properly flows into and out of the unit

Local data structures - examined to ensure that data stored temporarily maintains its integrity in an algorithm’s execution

Boundary conditions - tested to ensure that module operates properly at boundaries


Unit Testing Considerations (Continued)

Independent paths - to ensure all statements in a module executed at least once

Error-handling paths clear and correct messages/descriptions correct processing of errors not interrupted by system sufficient information to locate cause of error


Unit Test Environment

Driver

Interface Local data structures Boundary conditionsIndependent pathError Handling path

Test cases

Module to be tested

Stub Stub

•Act as main program•Accept test data•Past to the component to be tested•Print results

•Replace module that are subordinate (called by) component to be tested•Use subordinates module interface•Do minimal data manipulation•Print verification entry•Return control to the module

Notes :•Driver and stub are software that must be written.•Both are not delivered with final product

RESULTS


Integration TestingSystematic technique for constructing

software while testing to uncover errors associated with interfacing

Take unit-tested modules and build the program structure based on the design

Nonincremental (“big bang”) - all modules combined in advanced chaos!

Incremental integration - part by part, e.g. top-down or bottom-up


Top-Down IntegrationBeginning with main module, modules are

integrated by moving downward through the control hierarchy

Subordinate modules are added into the structure either depth-first or breadth-first

Verifies major control or decision points early in testing

Problem when lower level processes needed to test upper levels


Top-Down Integration Diagram

M1

M2 M3 M4

M5 M6

M8

M7


Top-Down Integration Process

Beginning at main module, stubs are replaced one at a time with actual modules

Tests are conducted as each module is integrated

On the completion of each set of tests, another stub is replaced with real module

Regression testing (conduct all or some of previous tests) may be done to ensure no new errors have been introduced


Bottom-Up IntegrationBegins construction and testing with

atomic modules (lowest level in hierarchy)

No need for stubs, all subordinate modules tested and available

Test drivers required and can be shared as integration move towards the higher levels


Bottom-Up Integration Diagram

Mc

D1 D2 D3

Ma Mb

Cluster 1

Cluster 2

Cluster 3


Bottom-Up Integration Process

Low-level modules are combined into clusters

A driver is written to coordinate test case input and output

The cluster is testedDrivers are removed and clusters are

combined moving upward in the program structure


Regression TestingNew module or major changes in software.Helps to ensure changes does not introduce

Unintended behavior Additional errors

Re-execute the subset of all test cases Most advised : Use automated capture-play back

toolsThis tool enable the software engineer to capture test

cases and results for subsequent playback and comparison.


Validation TestingA series of black box tests that

demonstrate conformity with requirements

Test that software functions as expected by customer

Test plan and details can be based on software requirements specification


Alpha TestConducted at developer’s site by

customerSoftware is used in a natural setting with

the developer “looking over the shoulder”Errors and usage problems recorded,

correction can possibly be done quicklyTest conducted in a controlled

environment


Beta TestConducted at one or more customer sites

by end user(s)Developer generally not present“Live” application of software in

uncontrolled environmentCustomer records all problems and reports

them to developer at regular intervalsSoftware modified based on problems

reported before final release to customers


System TestingA series of tests to fully exercise a

computer-based systemVerify that all system elements have been

properly integrated and perform allocated functions

Software is one element to be integratedSoftware engineer to anticipate potential

interfacing problems


Recovery TestingForce the software to fail in a variety of

waysVerify that recovery is properly performedIf recovery is automatic, re-initialisation,

checkpointing mechanisms, data recovery, and restart are evaluated for correctness

If recovery requires human intervention, mean time to repair is evaluated


Security TestingVerify that protection mechanisms

built into the system will protect it from improper penetration

Tester plays the role(s) of individual who desires to penetrate the system

System designer to make penetration cost greater than the value of information obtained


Stress & Performance Testing

Stress Testing Execute system in a manner that

demands resources in abnormal quantity, frequency or volume

Performance Testing Test run-time performance of software

within context of an integrated system


DebuggingTo find and correct the cause of a

software errorAttempt to match symptom to causeBug characteristics:

Symptom and cause may be geographically remote

Symptom disappear (temporarily) when another error is corrected


Debugging (Continued)

Symptom caused by nonerrors Symptom caused by human errors Symptom a result of timing problems Difficult to accurately reproduce input

conditions Symptom may be intermittent Symptom due to causes that are

distributed across a number of tasks on different processors


Debugging Process

TestCases

ExecutionOf cases

Results

Debugging

Additional test

SuspectedcausesRegression

test

CorrectionsIdentifiedcauses


Debugging ApproachesBrute force

“let the computer find the error” memory dumps, run-time traces, lots of

WRITE statements clues to errorBacktracking

trace source code backwards from errorCause elimination

induction/deduction, “binary partitioning” isolate potential causes, test each one


Debugging AidsDebugging Tools

Debugging compiler Dynamic debugging aids (“tracers”) Automatic test case generators Memory dumps Cross-reference maps

Good design documentationClear source codeOther people!


References“Software Engineering: A

Practitioner’s Approach” 5th Ed. by Roger S. Pressman, Mc-Graw-Hill, 2001

“Software Engineering” by Ian Sommerville, Addison-Wesley, 2001

Software Testing Strategies

Documents

Transcript of Software Testing Strategies