Integration and System TestingIntegration and System Testing

CSSE 376, Software Quality Assurance

Rose-Hulman Institute of Technology

March 29, 2007

2

Lifecycle Testing RelationshipsLifecycle Testing Relationships

Requirements

High-level Design

Low-level Design

Code Unit Testing

Integration Testing

System Testing

Acceptance Testing

3

Testing in the LargeTesting in the Large

1. Integration testing: putting the pieces together

2. System testing: function and performance

4

1. Integration Testing1. Integration Testing

5

Integration Testing ObjectivesIntegration Testing Objectives

■ Gain confidence in the integrity of overall system design

■ Ensure proper interaction of components

■ Run simple system-level tests

6

Integration StrategiesIntegration Strategies

■ Top-down■ Bottom-up■ Critical-first■ Function-at-a-time■ As-delivered■ Big bang

7

Top-Down IntegrationTop-Down Integration

■ Start with top-level modules

■ Use stubs for lower-level modules

■ As each level is completed, replace stubs with next level of modules

8

Top-Down IssuesTop-Down Issues

■ Pros:– Always have a top-level system– Stubs can be written from interface

specifications

■ Cons:– May delay performance problems until too

late

– Stubs can be expensive

9

Bottom-Up IntegrationBottom-Up Integration

■ Start with bottom-level modules

■ Use drivers for upper-level modules

■ As each level is completed, replace drivers with next level of modules

10

Bottom-Up IssuesBottom-Up Issues

■ Pros:– Primitive functions get most testing– Drivers are usually cheap

■ Cons:– Only have a complete system at the end

11

Critical-First IntegrationCritical-First Integration

■ Integrate the most critical components first

■ Add remaining pieces later■ Issues:

– guarantees that most important components work first

– may be difficult to integrate

12

Function-at-a-Time IntegrationFunction-at-a-Time Integration

■ Integrate all modules needed to perform one function at the same time

■ For each function, add another group of modules

■ Issues:– makes for easier test generation– may postpone function interaction too long

13

As-delivered IntegrationAs-delivered Integration

■ Integrate all of the modules that can be integrated as they become available

■ Issues– efficient: just-in-time integration – lazy: may lead to missed schedules

14

Big Bang IntegrationBig Bang Integration

■ Wait until all the components are ready, then put everything together at once

■ Issues:– avoids cost of scaffolding (stubs or drivers)– does not provide any locality for finding

faults

15

Summary of Integration Summary of Integration StrategiesStrategies

■ Top-down■ Bottom-up■ Critical-first■ Function-at-a-time■ As-delivered■ Big bang

16

Question:Question:What should be used as What should be used as integration test cases?integration test cases?

17

2. System Testing2. System Testing

18

System Testing ObjectivesSystem Testing Objectives

■ Gain confidence in the integrity of the system as a whole

■ Ensure compliance with functional requirements

■ Ensure compliance with performance requirements

19

Testing Functional Testing Functional RequirementsRequirements

1. Prepare a test plan from the functional specification of the system

2. Prepare tests for all areas of functionality

3. Review test plan and tests

4. Execute tests

5. Monitor fault rate

20

Fault RateFault Rate

0

20

40

60

80

100

120

1/1/

00

1/2/

00

1/3/

00

1/4/

00

1/5/

00

1/6/

00

1/7/

00

1/8/

00

1/9/

00

1/10

/00

1/11

/00

1/12

/00

1/13

/00

1/14

/00

1/15

/00

1/16

/00

1/17

/00

1/18

/00

1/19

/00

1/20

/00

To

tal

Fau

lts

21

Testing Performance Testing Performance RequirementsRequirements

1. Identify stress points of system

2. Create or obtain load generators– might use existing system

– might buy/make special purpose tools

3. Run stress tests

4. Monitor system performance– usually need instrumentation