Object Orientated Integration Testing - Swanseacsmarkus/CS339/dissertations/ThomasG.pdf ·...

26
Object Orientated Integration Testing Gareth Thomas December 14, 2006 Abstract A report on Object Orientated Integration Testing, looking at dif- ferent methods to produce test cases and approaches to this form of testing. 1

Transcript of Object Orientated Integration Testing - Swanseacsmarkus/CS339/dissertations/ThomasG.pdf ·...

Page 1: Object Orientated Integration Testing - Swanseacsmarkus/CS339/dissertations/ThomasG.pdf · 2007-01-19 · Object Orientated Integration Testing Gareth Thomas December 14, 2006 Abstract

Object Orientated Integration Testing

Gareth Thomas

December 14, 2006

Abstract

A report on Object Orientated Integration Testing, looking at dif-ferent methods to produce test cases and approaches to this form oftesting.

1

Page 2: Object Orientated Integration Testing - Swanseacsmarkus/CS339/dissertations/ThomasG.pdf · 2007-01-19 · Object Orientated Integration Testing Gareth Thomas December 14, 2006 Abstract

CONTENTS LIST OF FIGURES

Contents

1 Introduction 41.1 Definition of Terms . . . . . . . . . . . . . . . . . . . . . . . . 41.2 Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.2.1 Functional Testing . . . . . . . . . . . . . . . . . . . . 51.2.2 Structural Testing . . . . . . . . . . . . . . . . . . . . 61.2.3 Grey Box Testing . . . . . . . . . . . . . . . . . . . . . 7

1.3 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2 Examples 82.1 Next Date . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3 Integration Testing 143.1 Introduction to Integration Testing . . . . . . . . . . . . . . . 143.2 Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . 15

3.2.1 Big Bang . . . . . . . . . . . . . . . . . . . . . . . . . 153.2.2 Top Down . . . . . . . . . . . . . . . . . . . . . . . . . 163.2.3 Bottom Up . . . . . . . . . . . . . . . . . . . . . . . . 183.2.4 Sandwich . . . . . . . . . . . . . . . . . . . . . . . . . 19

3.3 Graphical . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193.3.1 Pair Wise . . . . . . . . . . . . . . . . . . . . . . . . . 193.3.2 Neighborhood . . . . . . . . . . . . . . . . . . . . . . . 213.3.3 Other . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

3.4 MM Paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

4 Conclusion 25

List of Tables

1 Example Tests with a Big Bang Approach . . . . . . . . . . . 162 Example Tests with a Top Down Approach . . . . . . . . . . 173 Example Tests with a Bottom Up Approach . . . . . . . . . . 19

List of Figures

1 Venn Diagram of Test Cases from a Functional Approach . . 62 Venn Diagram of Test Cases from a Structural Approach . . 73 The StateChart representing the Next Date program . . . . . 13

2

Page 3: Object Orientated Integration Testing - Swanseacsmarkus/CS339/dissertations/ThomasG.pdf · 2007-01-19 · Object Orientated Integration Testing Gareth Thomas December 14, 2006 Abstract

LIST OF FIGURES LIST OF FIGURES

4 V-Model of the software development cycle . . . . . . . . . . 145 Illustration of concept of layers introduced in 3.2.2 . . . . . . 176 Illustration of concept introduced in 3.2.3 . . . . . . . . . . . 187 One test scenario as described in 3.3.1 . . . . . . . . . . . . . 208 A Second test scenario as described in 3.3.1 . . . . . . . . . . 209 Another test scenario as described in 3.3.1 . . . . . . . . . . . 2110 An illustration of the approach outlined in 3.3.2 . . . . . . . . 2211 An illustration of the approach outlined in 3.3.2 . . . . . . . . 23

3

Page 4: Object Orientated Integration Testing - Swanseacsmarkus/CS339/dissertations/ThomasG.pdf · 2007-01-19 · Object Orientated Integration Testing Gareth Thomas December 14, 2006 Abstract

1 INTRODUCTION

1 Introduction

1.1 Definition of Terms

In testing there are many terms, which need specific definitions as they canbe used to mean many different things, all slightly different but providingmany big problems when trying to be understood, because of this I willstart with a definition of all the terms that fall into this category so thatwhen reading this document you shouldnt confuse my meanings with yourown.

[1, 2] were the main source’s for my work, and as such most of the documentis written with knowledge gained from them.

Error - An error is a misconception about a program; anybody can makea mistake, either while coding or using a program and as such is really themost basic problem a tester can observe.

Fault - A fault is when an Error leads to an unexpected behavior withinthe program. There can be two kinds of Fault one of commission and one ofomission, Faults of commission are when a user enters something into a pro-gram they know to be wrong/incorrect/incomplete while Faults of omissionare when a user does not enter something that is correct [1]. Of the two itis obvious that Faults of omission are more elusive to resolve as they occurwhen there are misconceptions either about the specification and hence theFault is with the program, or misconceptions with the implementation andhence the Fault is with the user.

Failure - A failure is when the Fault leads to a visible problem with theprogram such as a crash or an error message appears.

Bug - Not really a formal word however I will be using it a lot, and whatI mean by it is any Error, Fault or Failure that may occur during testingand needs to be fixed.

Coverage - Testing coverage is a measure of how extensively a program istested, and can be easily understood to be the greater the coverage the lessBugs should be in the final product.

Test - A test is part of a Test Case, it is used to either show correct executionof a program or to find a Failure in the program.

Test Case - A test case is a test with a series of inputs and expected outputs,

4

Page 5: Object Orientated Integration Testing - Swanseacsmarkus/CS339/dissertations/ThomasG.pdf · 2007-01-19 · Object Orientated Integration Testing Gareth Thomas December 14, 2006 Abstract

1.2 Testing 1 INTRODUCTION

it has the same purpose of a Test but provides a more robust system forfinding Failures.

Test Suite - A test suite is a series of related Test Cases and/or Tests.Testing a program can take long periods of time, and have hundreds, possiblythousands of Tests/Test Cases to run, breaking them down intro smallersub-sections can be beneficial especially if the Tests/Test Cases are relatedas then people can be assigned Test Suites to run and analyze thus providinga better division of work and therefore more efficient testing.

1.2 Testing

As with many things there are several different approaches to a problem, inthe field of testing there are two main approaches they are Functional andStructural.

1.2.1 Functional Testing

Functional testing which can also be referred to as Black Box testing is amethod of testing where the tester only know how the program should work,he/she doesnt know how actually works. This is only a subtle difference butactually in the world of software engineering means a great deal when itcomes to generating Test Cases.

In a Functional approach the tester as previously stated only knows howthe program should work, and this is based on the specification originallydefined by the user, the tester does not know how the specification has beenimplemented, this means that he/she can only test what they believe to bein the program, and how they believe it to work.

Hidden methods and code which although included in the program but wasntspecified can slip through, this hidden code could be malicious in nature orsimply added by mistake, each presents its own problems when given tothe user, but ultimately leads to that user being very unhappy with yourproduct.

Luckily Functional Testing does has its advantages, firstly Test Cases onceidentified of course are independent of the implementation and so can berun unmodified after changes to the code have been made, this means thattesting can be taking place during the development of the program and thus

5

Page 6: Object Orientated Integration Testing - Swanseacsmarkus/CS339/dissertations/ThomasG.pdf · 2007-01-19 · Object Orientated Integration Testing Gareth Thomas December 14, 2006 Abstract

1.2 Testing 1 INTRODUCTION

Figure 1: Venn Diagram of Test Cases from a Functional Approach

reducing the total time it takes to make the finished product. We can see therange of Test Cases generated by a Functional approach in 1 from [1].

1.2.2 Structural Testing

Structural testing, which can also be referred to as White Box testing, isa method which is pretty much the opposite of Functional. In StructuralTesting the tester will simply test the code itself, this provides a wholerange of advantages over Functional, which will now be discussed.

Firstly the most obvious advantage is test coverage, although no programcan be said to be thoroughly tested without using both methods to test theprogram for reasons which will be looked at shortly, the Structural approachallows for much better test coverage as the tester can really try to break thecode as opposed to really just checking that the code that Is there worksproperly when given correct inputs in the Functional approach.

As with most things the Structural approach does of course have its downsides, although it will be able to root out any redundant or malicious code,something that Functional cannot do the Structural approach cannot checkagainst the specification to see if all the functionality is actually there, andit is for this reason then that both approaches must be used when testing aprogram, Functional to be sure that the program does not only work as it isexpected to, but also has all functionality included within it, and Structuralto check that all the code in the program works regardless of input. Wecan see the range of Test Cases that can be generated with a Structural

6

Page 7: Object Orientated Integration Testing - Swanseacsmarkus/CS339/dissertations/ThomasG.pdf · 2007-01-19 · Object Orientated Integration Testing Gareth Thomas December 14, 2006 Abstract

1.3 Outline 1 INTRODUCTION

Figure 2: Venn Diagram of Test Cases from a Structural Approach

approach in 2 from [1].

1.2.3 Grey Box Testing

This isnt one of the fundamental approaches used while testing a program,however it is used quite a lot, as the name suggests its really a combinationof White and Black box testing methodologies. The situation with it is thatyou may know some of the implementation details but not all of them, andhence both Functional and Structural approaches can be used in conjunctionwith one another.

1.3 Outline

This document starts off illustrating some examples that I will be usingthroughout in quite a bit of detail, and I will introduce Integration Testing,then finally will look at ways to approach it, from MM-Paths to Graphicalmethods to Decomposition and conclude with some observations of the phaseitself and other things.

7

Page 8: Object Orientated Integration Testing - Swanseacsmarkus/CS339/dissertations/ThomasG.pdf · 2007-01-19 · Object Orientated Integration Testing Gareth Thomas December 14, 2006 Abstract

2 EXAMPLES

2 Examples

During the course of the this document I will be using the following examplesto illustrate various points, included here are descriptions of each program,UML State diagrams, method descriptions and code listings.

2.1 Next Date

The Next Date program is a very well used testing example, as it is simplein its nature however can be tricky when trying to make Test Cases. Asthe name suggests it traditionally takes a date from the user, and returnsthe next date for the users input. For example, if given August 2nd 2006 itwould return August 3rd 2006.

My implementation is slightly different, the Class Date takes as input a Day,a Month and a Year, it then tests whether this is correct, and then can eitherprint that date, or the user can call the increment() method one or moretimes to increment the date, at any point the date can be printed.

Here we have the code for the Next Date implementation in my chosenlanguage of Java.

The Day Class - Checks the Day given to it is viable, then stores it.

public class Day {

private int day ;

public Day( int day ) throws NumberFormatException {i f ( day > 0 && day < 32) {

this . day = day ;} else {

throw new NumberFormatException (”Day not between 1 and 31 i n c l u s i v e ” ) ;

}}

public int getDay ( ) {return day ;

}

public void increment ( int dim , Month m, Year y ){

8

Page 9: Object Orientated Integration Testing - Swanseacsmarkus/CS339/dissertations/ThomasG.pdf · 2007-01-19 · Object Orientated Integration Testing Gareth Thomas December 14, 2006 Abstract

2.1 Next Date 2 EXAMPLES

i f ( day + 1 > dim){day = 1 ;m. increment (y ) ;

} else {day++;

}}

}

The Month Class - Checks the Month given to it is viable, then storesit.

public class Month {

private int month ;

public Month( int month) throws NumberFormatException {i f (month > 0 && month < 13) {

this . month = month ;} else {

throw new NumberFormatException (”Month not between 1 and 12 i n c l u s i v e ” ) ;

}}

public int getMonth ( ) {return month ;

}

public void increment ( Year y ){i f (month + 1 > 12){

month = 1 ;y . increment ( ) ;

} else {month++;

}}

}

The Year Class - Checks the Year given to it is viable, then stores it.

public class Year {

private int year ;

9

Page 10: Object Orientated Integration Testing - Swanseacsmarkus/CS339/dissertations/ThomasG.pdf · 2007-01-19 · Object Orientated Integration Testing Gareth Thomas December 14, 2006 Abstract

2.1 Next Date 2 EXAMPLES

public Year ( int year ) throws NumberFormatException {i f ( year > 1799) {

this . year = year ;} else {

throw new NumberFormatException (”Year not g r e a t e r than 1800” ) ;

}}

public int getYear ( ) {return year ;

}

public void increment ( ){year++;

}}

The Date Class - Takes all inputs, checks they are viable on their own,then checks they are viable combined, if they are then stores the date as aString.

public class Date {

private Day d ;private Month m;private Year y ;private St r ing date ;

private int [ ] dim ={ 31 , 28 , 31 , 30 , 30 , 31 , 31 , 31 , 30 , 30 , 31 , 31 } ;

public Date ( int d , int m, int y ) {try {

this . d = new Day(d ) ;} catch ( NumberFormatException e ) {

System . out . p r i n t l n ( e . t oS t r i ng ( ) ) ;System . e x i t ( 0 ) ;

}

try {this .m = new Month(m) ;

} catch ( NumberFormatException e ) {

10

Page 11: Object Orientated Integration Testing - Swanseacsmarkus/CS339/dissertations/ThomasG.pdf · 2007-01-19 · Object Orientated Integration Testing Gareth Thomas December 14, 2006 Abstract

2.1 Next Date 2 EXAMPLES

System . out . p r i n t l n ( e . t oS t r i ng ( ) ) ;System . e x i t ( 0 ) ;

}

try {this . y = new Year (y ) ;

} catch ( NumberFormatException e ) {System . out . p r i n t l n ( e . t oS t r i ng ( ) ) ;System . e x i t ( 0 ) ;

}

try {checkDate ( ) ;date = this . d . getDay ( ) + ”/”

+ this .m. getMonth ( ) + ”/”+ this . y . getYear ( ) ;

} catch ( NumberFormatException e ) {System . out . p r i n t l n ( e . t oS t r i ng ( ) ) ;System . e x i t ( 0 ) ;

}}

private void checkDate ( ) throws NumberFormatException {i f (m. getMonth ( ) == 2) {

i f ( ! i sLeap ( ) && d . getDay ( ) > 28) {throw new NumberFormatException (

”Not a l eap year , but day in feb > 28” ) ;} else i f ( i sLeap ( ) && d . getDay ( ) > 29) {

throw new NumberFormatException (”Leap year , but day in feb > 29” ) ;

}}

i f (d . getDay ( ) == 31) {switch (m. getMonth ( ) ) {case 4 :

throw newNumberFormatException( ”30 days in t h i s month” ) ;

case 6 :throw newNumberFormatException( ”30 days in t h i s month” ) ;

case 9 :throw new

11

Page 12: Object Orientated Integration Testing - Swanseacsmarkus/CS339/dissertations/ThomasG.pdf · 2007-01-19 · Object Orientated Integration Testing Gareth Thomas December 14, 2006 Abstract

2.1 Next Date 2 EXAMPLES

NumberFormatException( ”30 days in t h i s month” ) ;

case 11 :throw newNumberFormatException( ”30 days in t h i s month” ) ;

}}

}

private boolean i sLeap ( ) {boolean l eap = fa l se ;i f ( y . getYear ( ) % 100 == 0 && y . getYear ( ) % 400 == 0) {

// Leap yearl eap = true ;

}

i f ( y . getYear ( ) % 4 == 0) {// Leap yearl eap = true ;

}

return l eap ;}

public void pr intDate ( ) {System . out . p r i n t l n ( date ) ;

}

public St r ing getDate ( ) {return date ;

}

public void increment ( ) {i f ( i sLeap ( ) ) {

d . increment (dim [m. getMonth ( ) ] + 1 , m, y ) ;} else {

d . increment (dim [m. getMonth ( ) ] , m, y ) ;}

}}

3 is the StateChart Diagram showing the Next Date implementation’s dataflow.

12

Page 13: Object Orientated Integration Testing - Swanseacsmarkus/CS339/dissertations/ThomasG.pdf · 2007-01-19 · Object Orientated Integration Testing Gareth Thomas December 14, 2006 Abstract

2.1 Next Date 2 EXAMPLES

Figure 3: The StateChart representing the Next Date program

13

Page 14: Object Orientated Integration Testing - Swanseacsmarkus/CS339/dissertations/ThomasG.pdf · 2007-01-19 · Object Orientated Integration Testing Gareth Thomas December 14, 2006 Abstract

3 INTEGRATION TESTING

3 Integration Testing

3.1 Introduction to Integration Testing

Integration Testing is 1 part of three main testing phases during the devel-opment cycle, it sits between Unit and System Testing as illustrated in 4and I feel is the hardest part of the testing that is done. There are manyreasons why this is the case and I will attempt to briefly cover them herebut my main focus will be on ways to approach Integration Testing for thebetter.

Figure 4: V-Model of the software development cycle

From 4 we can see the three main testing phases; Unit, Integration andSystem. Firstly I will talk a little about each so we can see where Integrationcomes into play.

Unit testing is the phase where each individual Unit is tested on its own, ifit requires the use of other Units stubs are inserted to mimic their behavior.For this document I will consider a Unit to be a class or more generally anindividual object, a stub in this case then is also a class or an object, and isused to mimic the behavior of any object needed, for example if a methodin one unit needed the result of a calculation done by another unit the stubfile might just return a constant value pre-calculated by the tester to be thecorrect response.

14

Page 15: Object Orientated Integration Testing - Swanseacsmarkus/CS339/dissertations/ThomasG.pdf · 2007-01-19 · Object Orientated Integration Testing Gareth Thomas December 14, 2006 Abstract

3.2 Decomposition 3 INTEGRATION TESTING

Integration testing is the phase in which these individual Units are combinedto form larger and larger aspects of the program, they are tested to determineIf the units interact together correctly, for example to check that a Unitis returning the result of a calculation in the correct format. Integrationtesting requires that the Unit testing phase has been completed successfully.Integration testing will take up much of the whole testing phase and one ofits biggest problems is determining exactly how long to spend since thisphase could potentially, if you wanted to exhaustively test the program,take a very long time.

System testing is the phase where the complete program is tested, this isslightly different to Integration testing as there if needed only one or twomodules can be tested on their own, here the whole program is compiled.Often it will be given to the user as an Alpha or Beta build for them touse to see if its what they want and also because the more people using theprogram the more Bugs they will find before its deployed for real.

To tackle the problems with Integration Testing there are several approachesto consider, these approaches fall into three main categories they are De-composition, Graphical and MM Paths. The next section of the documentwill discuss these categories in detail and outline the different methods foreach, although it will not discuss the actual Test Case generation, I willleave that for later.

3.2 Decomposition

3.2.1 Big Bang

Big Bang testing is quite useful when developing small programs like theNext Date program I outlined in chapter 2, basically a Big Bang test is aSystem Test where the whole program is tested at once. Big Bang testinglends itself to Black Box testing, as no knowledge of the implementationis actually required to run the program. However if a developer was doingthe testing it would be beneficial as he/she could be able to use his/herknowledge of the implementation details to attempt to run the program insuch a way that it would cause problems, so Big Bang is really a Grey Boxtesting method as it works best with both White and Black Box strategiescombined.

This as I mentioned before is useful when developing small programs, as

15

Page 16: Object Orientated Integration Testing - Swanseacsmarkus/CS339/dissertations/ThomasG.pdf · 2007-01-19 · Object Orientated Integration Testing Gareth Thomas December 14, 2006 Abstract

3.2 Decomposition 3 INTEGRATION TESTING

other methods require a lot more overhead work to be able to test theprogram in small subsections, work such as generation of stub files andmore. Although with a smaller system these other ways may provide moreinsight as to where the Bugs if any do occur the overhead of that methodmost likely out ways its benefits on such a small project.

Since Big Bang testing requires only that a person use the program no suchoverhead is required, however Bugs can be harder to find since the entireprogram has had no previous Integration Testing. This method shouldntbe used on larger programs for that same reason, Bugs will be very hardto find and now the benefits of not having to split the program into smallsubsections to test has been wasted going through the program, most likelyline by line searching for any Bugs found.

An Example for testing with the Big Bang approach on the Next Date pro-gram outlined in 2.1 would require no stub files, but would however requirea driver as no User Interface has been made for the example, some tests thatcould be run are listed in 1.

Expected Result Test To PerformPASS Create new date for 28/02/2006FAIL Create new date for 29/02/2006

Table 1: Example Tests with a Big Bang Approach

3.2.2 Top Down

Top Down Testing is an incremental approach, and by this I mean that youstart at a point and gradually move down through the layers until you reachthe finish at a constant rate. With a Top Down approach a tester will startat the User Interface whether it be a GUI or a simple command line andtests its functionality alone, to do this it must replace the Units on the nextlayer down when looking at the program as a UML State Chart with stubs.This is illustrated in 5, the boxes in grey are the stub files and the box inblack are the Unit’s being tested. A Top Down approach is obviously aWhite Box method as in depth knowledge of lower layers of the programsfunctionality is required for the generation of the stub files.

The advantages of this is that once a level has completed its testing thetester knows that any problems that appear in the future are more thanlikely caused by newly added Units, this greatly decreases the scope of places

16

Page 17: Object Orientated Integration Testing - Swanseacsmarkus/CS339/dissertations/ThomasG.pdf · 2007-01-19 · Object Orientated Integration Testing Gareth Thomas December 14, 2006 Abstract

3.2 Decomposition 3 INTEGRATION TESTING

Figure 5: Illustration of concept of layers introduced in 3.2.2

to search once Bugs arise. Its disadvantages are that the generation of stubfiles may not be able to be automated and even if it is still introduces acostly overhead to this approach, also because with integration testing youlook at how two or more Units interact with one another its possible thateach Unit would have to be individually tested against every Unit in theLayer below it which it is connected to in some way.

This approach works well with both small and larger programs although thelarger the project the more stub files will likely need generating, I cant for-seeany major issues with this approach being used to test very big programs.In 2 I have outlined some tests which could be made using this approach onthe Next Date program outlined in 2.1. As you can see even with this smallprogram each layer to be tested will require at least one stub file.

Unit Under Test List Of StubsDate Day, Month, YearDay Month, YearMonth Year

Table 2: Example Tests with a Top Down Approach

17

Page 18: Object Orientated Integration Testing - Swanseacsmarkus/CS339/dissertations/ThomasG.pdf · 2007-01-19 · Object Orientated Integration Testing Gareth Thomas December 14, 2006 Abstract

3.2 Decomposition 3 INTEGRATION TESTING

Figure 6: Illustration of concept introduced in 3.2.3

3.2.3 Bottom Up

Bottom Up Testing is very similar to that of Top Down, however its reallyin reverse, in this approach a tester will start with the most basic Units ofa program and work his way up, to test each Unit he/she will use a driverand possibly several stub files depending on how strictly they want to stickwith the approach. 6 illustrates this approach diagrammatically again theblack box indicates the Unit under test, and the grey stub files, but here theblue indicates a Unit in the program but not under test.

In practice the overhead of the stub file generation may be too great and theactual Units are used whenever possible to lessen the computational or morelikely manual load of stub file generation. Again this stub file generationis a prominent disadvantage with this approach, which is also a White Boxone as knowledge of the implementation must be used for driver and stubfile generation. Again also the advantages with this are that once a layerhas been completely tested, it’s less likely that any Bug found has occurredin that layer.

The bottom up approach to testing applied to the Next Date program re-quires no stub files only the use of drivers which is a definite benefit, outlinedin 3 is order in which I would test the program using this approach.

18

Page 19: Object Orientated Integration Testing - Swanseacsmarkus/CS339/dissertations/ThomasG.pdf · 2007-01-19 · Object Orientated Integration Testing Gareth Thomas December 14, 2006 Abstract

3.3 Graphical 3 INTEGRATION TESTING

Unit Under Test Number Of Stubs NeededYear 0Year + Month 0Year + Month + Day 0Year + Month + Day + Date 0

Table 3: Example Tests with a Bottom Up Approach

3.2.4 Sandwich

Last but by no means least of the Decomposition methods is the Sandwichmethod, a kind of hybrid of Top Down and Bottom up, it does them bothsimultaneously in a way, it works by testing the top layers of the program,again in relation to a UML State Chart of the program and also the bottomlayers of the program and working its way into the middle. Sandwich testingis most defiantly a White Box testing strategy as it needs stub and driversto supplement the tester to be able to run the tests, and without havingfirst hand knowledge of the implementation details it would be difficult togenerate the stub and driver files.

Its advantages are that Bugs can be found quite easily as you know in whatsub system to look at, and because it is done in layers the number of Unitsadded at one time although more than 1 are limited, and thus the area wherethe problem can exist is narrowed substantially. Its disadvantages are thatboth stub and driver files must be generated to be able to do this approachand therefore this method has quite a bit of overhead.

3.3 Graphical

3.3.1 Pair Wise

Pair Wise Testing is the first of our graphical approaches, it entails takingtwo Units of the program that in some way communicate with each othereither by the sending of receive of messages and testing there communication.It falls into the category of graphical, as its easy to visualize which two Unitsare tested and to see how they are connected. The diagrams in 7,8,9 showssome of how the Pair Wise strategy would test the program, again blackrepresents a Unit under test, grey a stub and blue a Unit in the programthats not under test.

19

Page 20: Object Orientated Integration Testing - Swanseacsmarkus/CS339/dissertations/ThomasG.pdf · 2007-01-19 · Object Orientated Integration Testing Gareth Thomas December 14, 2006 Abstract

3.3 Graphical 3 INTEGRATION TESTING

Figure 7: One test scenario as described in 3.3.1

Figure 8: A Second test scenario as described in 3.3.1

20

Page 21: Object Orientated Integration Testing - Swanseacsmarkus/CS339/dissertations/ThomasG.pdf · 2007-01-19 · Object Orientated Integration Testing Gareth Thomas December 14, 2006 Abstract

3.3 Graphical 3 INTEGRATION TESTING

Figure 9: Another test scenario as described in 3.3.1

The Pair Wise approach is not the most efficient as you can see even withthis small program at least 1 stub file needs to be generated in nearly everycase and so this adds overhead to the approach, with a larger project as youmight imagine this will only get worse. With this approach also by limitingthe Test Cases to only include two Units at a time many combinations haveto be tested to provide good coverage. However since with this approachyou only look at two Units at a time, it is vastly easier to find Bugs than itwould be with the other approaches discussed so far as there are only twoUnits to search for a problem in.

The Pair Wise approach works in larger programs however the more Unitsthere are to test the more overhead work will need to be done to generate TestCases between each pair of Units. With the Pair Wise approach knowledgeof a programs implementation is necessary as the tester will need to knowwhich and more importantly how all the Units interact and so this approachis a White Box one.

3.3.2 Neighborhood

The second of the graphical approaches I’m going to talk about is the Neigh-borhood one, and as the name suggests tests neighboring Units. The ap-

21

Page 22: Object Orientated Integration Testing - Swanseacsmarkus/CS339/dissertations/ThomasG.pdf · 2007-01-19 · Object Orientated Integration Testing Gareth Thomas December 14, 2006 Abstract

3.3 Graphical 3 INTEGRATION TESTING

Figure 10: An illustration of the approach outlined in 3.3.2

proach will start at a central point, most likely what is known as the socalled ultra-center which is will be a Unit on the graph which minimizes themaximum distances to the other Units in the program [1]. From this pointthe approach will move outwards, [1] gives I think to be an excellent analogywhich is ripples in a pond after dropping a stone in. In 10, 11 we can see anillustration of this approach for a small program.

In 10, the neighborhood of that top left Unit is the one to its right, thisright Unit needs the Units attached to it for it to work though, so that firstbatch of testing will require the generation of 3 stub files. In 11 we can seethat the top left Unit isnt included in the test but those 3 stub file Unitsare now being tested which increases the places to look for a Bug shouldone occur.

This approach although elegant in its nature has quite a few down sides,depending on where it starts there is a strong possibility that there will bea considerable amount of stub files, also because a State Chart of a programis likely to have many interconnected Units nearly every time the approachmoves outwards more than one Unit will be added, increasing the scope ofplaces to look for Bugs when they occur.

This approach is however not all bad and although some stub files will needto be generated there wont be as many as with the Pair Wise approach,

22

Page 23: Object Orientated Integration Testing - Swanseacsmarkus/CS339/dissertations/ThomasG.pdf · 2007-01-19 · Object Orientated Integration Testing Gareth Thomas December 14, 2006 Abstract

3.3 Graphical 3 INTEGRATION TESTING

Figure 11: An illustration of the approach outlined in 3.3.2

selection of what Units to test next is also simplified and this is an incre-mental approach so once a Unit is tested it is then used in the testing ofother Units thus increasing this approaches coverage.

Again this graphical approach works with larger programs and because ofthe need to know implementation details is like the Pair Wise approach aWhite Box approach to testing.

3.3.3 Other

This little section will discuss other graphical approaches to testing, as youmight imagine there are many ways to traverse a graph in a methodicalmanor that can be applied to the world of testing.

Such methods such as a Minimal Spanning Tree could prove to be veryuseful, as if you took the weight of a branch to be the number of methodcalls between two objects then the testing would work from a point in thegraph and test all of what I would describe as base Units (the simple Unitswith each others are built upon) and work its way up to more complex ones.This strategy would be good as it would limit the number of places to lookfor Bugs if one did cause a Failure to the newest Unit added, and at thesame time wouldnt need to limit itself to testing just two Units at a time

23

Page 24: Object Orientated Integration Testing - Swanseacsmarkus/CS339/dissertations/ThomasG.pdf · 2007-01-19 · Object Orientated Integration Testing Gareth Thomas December 14, 2006 Abstract

3.4 MM Paths 3 INTEGRATION TESTING

which as we saw with the Pair Wise approach can be costly.

3.4 MM Paths

There are several different takes on MM Paths, for this document on ObjectOrientated Integration Testing, the definition of an MM Path in use hereis from [1] and is as follows, An MM Path starts with a method and endswhen it reaches a method that does not issue any messages of its own. Itsobvious that this along with many of the other approaches is very much aWhite Box approach to testing.

MM Paths can be very useful for testing as they provide a detail outline ofhow a program will execute when a given method is called, this preciselygreatly improves the efficiency of a tester or developer in finding a Bug if oneshould occur during the test. However care must be taken when selectingthe paths themselves to ensure that all messages are covered i.e. all lines ofcode are executed at least once.

MM Paths do not require stub files, however if a method was called thatcontained hundreds of other method calls the advantage of precision wouldbe lost and more time would need to be taken for a Bug to be found. Anotherdisadvantage of this approach is that the paths themselves although easyfor a tester to see for a single test, manual or even automated selection ofMM Paths for the whole program could be quite computationally expensiveespecially for larger programs.

24

Page 25: Object Orientated Integration Testing - Swanseacsmarkus/CS339/dissertations/ThomasG.pdf · 2007-01-19 · Object Orientated Integration Testing Gareth Thomas December 14, 2006 Abstract

4 CONCLUSION

4 Conclusion

This document has covered a wide variety of Integration testing approachesuseful for testing Object Orientated programs, all of which have their ownpros and cons. It has also introduced several types of testing such as Func-tional and Structural and has hopefully shown that no one way can provide acomplete testing solution, but rather a only through a combination of manyapproaches can an individual test a program successfully to any degree ofCoverage.

Each category of approaches whether it be decomposition or graphical orMM Paths can be useful in its own right, and correct selection of a test-ing methodology can provide a significant reduction in the time it takes tocomplete the testing of a program.

Integration is only the middle step in the testing of a program and as statedoriginally can be very difficult to judge exactly how long should be spentdoing it, the next testing phase is System Testing and then the program isreleased to a consumer so it is also a very important phase as System Testingwill unlikely pick up discrete Faults which may only appear once in use andthen be difficult to fix without incurring major costs to a company.

25

Page 26: Object Orientated Integration Testing - Swanseacsmarkus/CS339/dissertations/ThomasG.pdf · 2007-01-19 · Object Orientated Integration Testing Gareth Thomas December 14, 2006 Abstract

REFERENCES REFERENCES

References

[1] Paul C. Jorgensen: Software Testing: A Craftman’s Approach, 2ndEdition, CRC Press 2002.

[2] Robert V. Binder: Testing Object-Oriented Systems CRC June 2005

26