Bug Patterns in Java

Bug Patterns in Java

of 168

CHAPTER 1: AGILE METHODS IN A CHAOTIC ENVIRONMENT ................................................... 5 EXAMINING TRENDS IN SOFTWARE DESIGN, IMPLEMENTATION, AND MAINTENANCE ............................. 5

Increase in demand for safe, secure systems ............................................................................... 5 Recognition of the limitations of traditional software engineering ............................................... 5 Availability of open-source software projects ................................................................................. 6 Demand for languages with platform-independent semantics .................................................... 6

LEARNING IN A FAST-PACED WORLD........................................................................................................ 7 DISSECTING BUG PATTERNS: WHY IT'S USEFUL...................................................................................... 7 A QUICK RECAP ........................................................................................................................................ 8

CHAPTER 2: BUGS, SPECIFICATIONS, AND IMPLEMENTATIONS.............................................. 9 WHAT IS A BUG? ....................................................................................................................................... 9 SPECIFICATION AS MONOLITHIC TREATISE............................................................................................... 9

C++..................................................................................................................................................... 10 Python ................................................................................................................................................ 10 ML....................................................................................................................................................... 11 Pascal ................................................................................................................................................ 11

BENEFITS OF SPECIFICATIONS ................................................................................................................ 11 IMPLEMENTATIONS ARE NOT SPECIFICATIONS....................................................................................... 12 BUILDING COST-EFFECTIVE SPECIFICATIONS WITH STORIES................................................................ 13

Include tests to eliminate specification errors .............................................................................. 14 Unit tests can't do everything ......................................................................................................... 20

A QUICK RECAP ...................................................................................................................................... 21 CHAPTER 3: DEBUGGING AND THE DEVELOPMENT PROCESS .............................................. 22

DEBUGGING AS SCIENTIFIC EXPERIMENT ............................................................................................... 22 Software is specified, integrated, and released incrementally .................................................. 22 Design is kept as simple as possible............................................................................................. 22 Programming is done in pairs......................................................................................................... 23 An on-site customer is always available ....................................................................................... 23 Code is owned by all developers ................................................................................................... 24 Tests exist for "anything that could possibly break" .................................................................... 24

INCORPORATE DEBUG TESTS INTO UNIT TEST SUITES.......................................................................... 24 THE FUTURE: TEST-ORIENTED LANGUAGES .......................................................................................... 25 A QUICK RECAP ...................................................................................................................................... 26

CHAPTER 4: DEBUGGING AND THE TESTING PROCESS ........................................................... 27 DESIGNING FOR TESTABILITY.................................................................................................................. 27

Keeping code in the model, not the view ...................................................................................... 27 Use static type checking to find errors .......................................................................................... 28 Use mediators to encapsulate functionality across fault lines ................................................... 28 Write methods with small signatures and default arguments .................................................... 29 Use accessors that do not modify memory state ........................................................................ 29 Specify out-of-program components with interfaces................................................................... 29 Write the tests first ........................................................................................................................... 30

THE GLOBALMODEL INTERFACE ............................................................................................................. 30 A QUICK RECAP ...................................................................................................................................... 37

CHAPTER 5: THE SCIENTIFIC METHOD OF DEBUGGING ........................................................... 38 SOFTWARE AS IMMORTAL MACHINE ....................................................................................................... 38 BUG PATTERNS HELP DIAGNOSE BUGS MORE QUICKLY ...................................................................... 40 A QUICK RECAP ...................................................................................................................................... 42

CHAPTER 6: ABOUT THE BUG PATTERNS ..................................................................................... 43


of 168

WHY IS IT IMPORTANT TO KNOW PATTERNS? ........................................................................................ 43 WHY THESE BUG PATTERNS? ................................................................................................................ 43 HOW THE PATTERNS ARE ORGANIZED ................................................................................................... 43 A QUICK REFERENCE FOR TROUBLESHOOTING ..................................................................................... 44

CHAPTER 7: THE ROGUE TILE............................................................................................................ 49 OVERVIEW ............................................................................................................................................... 49 ABOUT THIS BUG PATTERN .................................................................................................................... 49

The Symptoms.................................................................................................................................. 49 Cause, Cures, and Prevention ....................................................................................................... 50

OTHER OBSTACLES TO FACTORING OUT CODE..................................................................................... 56 Generic Types .................................................................................................................................. 56 Aspect-Oriented Programming....................................................................................................... 57

WHAT WE'VE LEARNED ........................................................................................................................... 58 CHAPTER 8: NULL POINTERS EVERYWHERE! .............................................................................. 59

THEY'RE UNINFORMATIVE ....................................................................................................................... 59 THEY'RE ELUSIVE .................................................................................................................................... 59

CHAPTER 9: THE DANGLING COMPOSITE...................................................................................... 60 OVERVIEW ............................................................................................................................................... 60 ABOUT THIS BUG PATTERN .................................................................................................................... 60

The Symptoms.................................................................................................................................. 61 The Cause ......................................................................................................................................... 61 Cures and Prevention ...................................................................................................................... 65

WHAT WE'VE LEARNED ........................................................................................................................... 68 CHAPTER 10: THE NULL FLAG ........................................................................................................... 69

OVERVIEW ............................................................................................................................................... 69 ABOUT THIS BUG PATTERN .................................................................................................................... 69


ROBUSTNESS VS. LACK OF DIAGNOSTIC EVIDENCE............................................................................... 72 Handling Exceptions in the Best Place ......................................................................................... 72 And What About Legacy Code?..................................................................................................... 73

WHAT WE'VE LEARNED ........................................................................................................................... 73 CHAPTER 11: THE DOUBLE DESCENT............................................................................................. 74



WHAT WE'VE LEARNED ........................................................................................................................... 79 CHAPTER 12: THE LIAR VIEW ............................................................................................................. 80


The Symptoms.................................................................................................................................. 80 The Cause ......................................................................................................................................... 81 Cures and Preventions .................................................................................................................... 84

GUIS AREN'T THE ONLY LIARS! .............................................................................................................. 87 WHAT WE'VE LEARNED ........................................................................................................................... 88

CHAPTER 13: SABOTEUR DATA ........................................................................................................ 89


of 168


The Symptoms.................................................................................................................................. 89 A Syntactic Cause ............................................................................................................................ 90 A Semantic Cause ........................................................................................................................... 91 Cures and Preventions .................................................................................................................... 91

WHAT WE'VE LEARNED ........................................................................................................................... 93 CHAPTER 14: THE BROKEN DISPATCH ........................................................................................... 94


The Symptoms.................................................................................................................................. 95 The Cause ......................................................................................................................................... 98 Cures and Preventions .................................................................................................................... 99

WHAT WE'VE LEARNED ......................................................................................................................... 100 CHAPTER 15: THE IMPOSTOR TYPE ............................................................................................... 101

OVERVIEW ............................................................................................................................................. 101 ABOUT THIS BUG PATTERN .................................................................................................................. 101

The Symptoms................................................................................................................................ 101 The Cause ....................................................................................................................................... 103 Cures and Preventions .................................................................................................................. 103

HYBRID PATTERNS ................................................................................................................................ 103 WHAT WE'VE LEARNED ......................................................................................................................... 105

CHAPTER 16: THE SPLIT CLEANER ................................................................................................ 107 OVERVIEW ............................................................................................................................................. 107 ABOUT THIS BUG PATTERN .................................................................................................................. 107


WHAT WE'VE LEARNED ......................................................................................................................... 112 CHAPTER 17: THE FICTITIOUS IMPLEMENTATION..................................................................... 113


The Symptoms................................................................................................................................ 114 The Cause ....................................................................................................................................... 114 Detecting Fictitious Implementations........................................................................................... 114 Cures and Preventions .................................................................................................................. 114

WHAT WE'VE LEARNED ......................................................................................................................... 118 CHAPTER 18: THE ORPHANED THREAD ....................................................................................... 120



ORPHANED THREADS AND GUIS .......................................................................................................... 124 Unit Tests and Multithreading....................................................................................................... 125

WHAT WE'VE LEARNED ......................................................................................................................... 126 CHAPTER 19: THE RUN-ON INITIALIZATION ................................................................................. 127



of 168

The Symptoms and the Cause..................................................................................................... 127 Cures and Preventions .................................................................................................................. 129

YOU'RE BETTER OFF FIXING THEM ...................................................................................................... 136 WHAT WE'VE LEARNED ......................................................................................................................... 136

CHAPTER 20: PLATFORM-DEPENDENT PATTERNS .................................................................. 138 OVERVIEW ............................................................................................................................................. 138 ABOUT PLATFORM DEPENDENCE ......................................................................................................... 138

Vendor-Dependent Bugs............................................................................................................... 139 Version-Dependent Bugs .............................................................................................................. 140 OS-Dependent Bugs...................................................................................................................... 141

WHAT WE'VE LEARNED ......................................................................................................................... 142 CHAPTER 21: A DIAGNOSTIC CHECKLIST .................................................................................... 143

GENERAL CONCEPTS ............................................................................................................................ 143 CHECKLIST FOR THE PATTERNS ........................................................................................................... 144

CHAPTER 22: DESIGN PATTERNS FOR DEBUGGING ................................................................ 150 OVERVIEW ............................................................................................................................................. 150 MAXIMIZING STATIC TYPE CHECKING ................................................................................................... 150

Make Fields final Whenever Possible ......................................................................................... 151 Make Methods final When They Shouldn't Be Overridden ...................................................... 151 Include Classes for Default Values.............................................................................................. 151 Use Checked Exceptions to Ensure That All Clients Handle Exceptional Conditions ......... 152 Define New Exception Types to Nail Down the Various Exceptional Conditions Precisely 152 Break Classes Whose Instances Take On One of a Fixed Number of States into Distinct Subclasses in a Composite Hierarchy ........................................................................................ 152 Minimize Casts and instanceof Tests.......................................................................................... 153 Use the Singleton Pattern to Help Minimize Use of instanceof ............................................... 153

MINIMIZING ERROR INTRODUCTION ...................................................................................................... 154 Factor Out Common Code............................................................................................................ 154 Factor Out Common Code............................................................................................................ 154 Make Methods That Are Purely Functional Whenever Possible ............................................. 154 Initialize All Fields in Constructors ............................................................................................... 154 Throw Exceptions When Exceptional Conditions Occur .......................................................... 155 Signal Errors as Soon as They're Discovered ........................................................................... 155 Discover Errors as Soon as Possible .......................................................................................... 155 Place Assertions into Code........................................................................................................... 155 Test Code as Closely to the User-Observable State as Possible........................................... 155 Factor Out Common Code............................................................................................................ 155

NOT THE LAST WORD............................................................................................................................ 156 CHAPTER 23: REFERENCES.............................................................................................................. 157

WEB SITES ............................................................................................................................................ 157 BOOKS ................................................................................................................................................... 160

APPENDIX A: STRING-PARSING LIST CONSTRUCTOR ............................................................. 162


of 168

Chapter 1: Agile Methods in a Chaotic Environment We discuss the context in which modern software is developed, and identify some of the shortcomings of the old approaches to development and debugging.

Examining Trends in Software Design, Implementation, and Maintenance

Over the past few years, new trends have emerged that are drastically affecting the way software is designed, implemented, and maintained: � Increase in demand for safe, secure systems. � Recognition of the limitations of traditional software engineering. � Availability of open-source software projects. � Demand for languages with platform-independent semantics.

Let's look at each of these issues in detail.

Increase in demand for safe, secure systems

The demand for safe, secure systems has grown tremendously. The terms "safe" and "secure" mean different things depending on who you ask, By "safe," I mean systems that don't break even when a user makes a mistake. By "secure" I mean systems that don't break even when they are deliberately attacked. For example, consider a web service for airline reservations. Suppose a user accidentally tries to get reservations for an impossible date (say, February 30). If the system is safe, it won't crash; instead, it will display an error message and allow the user to try again (or, better yet, the UI itself will prevent the user from ever entering this erroneous date). Now suppose that an attacker tries to snoop and play back someone else's credit card information when booking his ticket. If the system is secure, it will have measures in place to prevent him from doing so. To be sure, these two notions aren't completely distinct. A sufficiently inept attacker will be thwarted simply by a safe system. A sufficiently dopey user might trip over a sequence of actions that are prevented only by security measures. Once, in my early days on Unix, I accidentally put into my startup script for shell windows a command that started up yet another shell window. As soon as I opened one shell window, more and more windows were created, sucking more and more resources from the network, and bringing down several servers. This continued until a sys admin, who believed his network was under attack, shut down the account responsible (i.e., mine). It took awhile before he let me log in again.

With the growth in Internet commerce, the increasingly ubiquitous use of networked computing services, and the progress in embedded systems (many of which are intended for users with little or no computer experience), it is easy to see why demand for safety and security is on the rise.

Recognition of the limitations of traditional software engineering

There is growing recognition that the traditional methods of software engineering—focused on heavy up-front design and a strict separation of programmer roles into separate individuals (such as architect, coder, tester, etc.)—are often of limited use. Inevitably, designs change as a development team learns more about the advantages and disadvantages of various approaches. Such knowledge can be gained only through the process of actually building a system; a priori modeling is no substitute. The result is that many aspects of up-front designs are thrown away, and the effort spent in constructing them is wasted.


of 168

Additionally, the separation of programmers into various roles tends to impede knowledge transfer. Even if the coders discover that a design is flawed, it may still be quite difficult to convince the architects, and the testers may not be familiar enough with the code to test it thoroughly. The result of this division is poorly designed, poorly tested software that nobody's happy with. And, by the way, it doesn't work.

Even under ideal conditions in which deadlines are both reasonable and flexible, ample programmers of great proficiency are available, and software requirements are perfectly specified (and never change), traditional software engineering doesn't work very well. But real-world software is never developed under ideal conditions. It's developed in the presence of all sorts of complications, such as:

� Business realities. Business realities often result in systems that are rushed to market and are neither well documented nor well tested. When a product is already two weeks late and everything has been done except test it, you can bet that testing will be given short shrift.

� Lack of design experience. The developers of many new systems often have no experience in designing similar systems. This may not be a result of poor management, but an inevitable consequence of the massive shortage in experienced software developers.

� Difficult communication with users. The customers of software typically are not able to define their requirements well, often because they're not yet sure what their requirements are. In most cases, this isn't because they are "bad" customers; usually, it is because many software projects are the first systems of their kind to be put in place, and the customers have no real-world experience to help them determine which features would be of most use.

The real world of software development is one of chaos. As a result, the traditional rigid processes of software engineering are beginning to give way to more agile, adaptive methods such as extreme programming (XP). Agile methods hold promise in helping to meet the demand for increased reliability. In particular, an increased emphasis on unit testing can dramatically improve the reliability of a software system.

Availability of open-source software projects

Another trend that is affecting reliability is the growing body of freely available open-source software projects. The open-source concept and the code produced by the open-source community is challenging the business models of traditional software companies, providing competing products at no cost and (due to the openness of the source code) of often superior quality. The most striking example is the Linux operating system, which provides an alternative to existing commercial operating systems that is more stable and robust than many of its competitors.

Demand for languages with platform-independent semantics

There has been a surge in demand for languages with platform-independent semantics, along with virtual machines to execute them. The most famous language in this category is Java. Java programmers' compiled binaries enjoy an unprecedented level of portability; this portability can drastically decrease the cost of software development, since developers don't need to maintain separate source code or compile separate binaries for each target platform.


of 168

Learning in a Fast-Paced World

The trends in agile computing and open-source projects can help to meet some of the increased demand for reliable software. But the quality and safety of the software produced inevitably depends on the skill and experience of the programmers involved. And the available supply of experienced programmers is smaller than the demand for them.

Tip Teaching developers to recognize bug patterns is a way to leverage the

experience of many programmers to improve the effectiveness of each.

To address this shortage, we need ways to quickly convey to new programmers more than just the theoretical knowledge traditionally taught in computer science classes. We need to convey the kind of practical skill in developing robust systems that is normally gained through many years of experience.

Some of this experimental knowledge consists of various design patterns that have proven themselves to work in a variety of contexts and are now taught regularly in computer science curricula, alongside basic algorithms and data structures. But quite a bit of the working knowledge gained by experienced developers isn't demonstrated by such design patterns.

Part of this working knowledge consists of the ability to efficiently diagnose and fix bugs exhibited by a software system—in other words, effective debugging.

Effective debugging is far from a trivial skill. In fact, tracking and eliminating bugs constitutes a significant portion of the development time in a software project. If this task can be made more efficient, the resulting software will be more reliable and it will be developed more quickly.

While educating new developers, both in industry and academia, I've noticed some general tendencies in the ways they learn to debug software. Beginning programmers often work against themselves when considering the potential causes of a bug. They tend to:

1. Blame the underlying system (as opposed to their own code) far too quickly; 2. Convince themselves that the bug they are witnessing can't possibly occur (an idea that

could never be anything but a delusion); and 3. Thrash about, randomly changing code until the symptoms of the bug go away.

All of these tendencies diminish with experience. If novice programmers are made aware of them, they can learn to avoid them before they start real-world programming. But there is more to becoming a good debugger (which is a component of becoming an agile, effective developer) than just overcoming bad tendencies. Dissecting Bug Patterns: Why It's Useful

Just as good programming skills involve the knowledge of many design patterns (which can be combined and applied in various contexts), good debugging skills include knowledge of the common causes of bugs and how to fix them. In my column for IBM developerWorks, I first referred to these common causes as bug patterns.

Bug patterns are recurring relationships between signaled errors and underlying bugs in a program. Knowledge of these patterns and their symptoms helps the programmer to quickly identify new occurrences of the bug, as well as to prevent them from occurring in the first place.

Bug patterns are related to anti-patterns, which are patterns of common software designs that have been proven to fail time and again. Such negative examples of design are an essential complement to traditional, positive design patterns. But while anti-patterns are patterns of design, bug patterns are patterns of erroneous program behavior correlated with programming mistakes. The concern is not with design at all, but with the coding and debugging process.


of 168

The problem is that it can take many years for programmers to learn to recognize these patterns through experience alone. If we identify such patterns and teach them explicitly, we can leverage the experiences of many programmers to improve the effectiveness of each.

This concept is not novel to programming. Medical doctors rely on similar types of recurring relationships when diagnosing disease. They learn to do so by working closely with senior doctors during their internships. Their very education focuses on learning to make such diagnoses.

Once a beginning programmer can recognize these patterns, he is able to diagnose the cause of a bug and correct it more quickly. Furthermore, by explicitly identifying and communicating these correlations, developers can mutually benefit from each other's experience in debugging, thereby acquiring proficiency far more quickly than they would have otherwise.

A Quick Recap

In this chapter, we've: � Discussed the reasons why software is produced the way it is today. � Examined the changing trends in software design. � Learned about both the traditional and innovative methods by which programming is

taught. � Discovered why bug patterns are important to modern-language programmers and

designers.

In Chapter 2 we'll look at how to define the concept of a bug, explain why a specification is crucial for controlling software bugs, understand the differences between a specification and an implementation, use stories and unit tests when developing specifications, and introduce cost-effective means for developing specifications.


of 168

Chapter 2: Bugs, Specifications, and Implementations We rigorously define the concept of a bug, explain why a specification is crucial for controlling software bugs, highlight the differences between a specification and an implementation, and discuss a cost-effective means for developing specifications.

What Is a Bug?

This book is about debugging software. In order to discuss the act of debugging, it is important to define precisely what does and does not constitute a bug.

For the purposes of this text, I will define a bug as "program behavior that deviates from its specification." This definition does not include: � Poor performance, unless a threshold level of performance is included as part of the

specification. � An awkward or inefficient user interface. Although user interface design is an important

topic, it's not the subject of this book. � Lack of features, lack of a particular useful feature, or lack of any feature not included in

the program specification (even if it was intended to be in the specification).

The lack-of-features category illustrates an important aspect of our definition of bugs: they are inextricably linked to a program specification. If there is no program specification, then there literally are no bugs. To be sure, there are some generally accepted behavioral qualities expected from any software, e.g., it won't crash, it won't run forever without producing output, etc. Properties like these are implicitly part of the specification of any software. But these properties are the exception; most behavior must be explicitly specified. Because specifications define behavior which defines bugs, we had better discuss what constitutes a specification.

Tip The simplest definition of a bug is "program behavior that deviates from its

specification."

Intuitively, a program specification is a description of the behavior of a program. Therefore, having some kind of specification is essential to determining when the system is misbehaving. What form would we like this specification to take? First, let's consider how traditional software engineering answers this question.

Tip Bugs and program specifications are inextricably linked. Since specifications

define behavior, without a specification, bugs are not possible Specification as Monolithic Treatise

The traditional method of software engineering is to develop a thorough specification of the system's functionality before entering a single line of code. This specification is made as formal as possible, so as to minimize ambiguities. The programmers then slog through the various details of this specification (often a large book) as they implement the system.

This method of specification was adapted from other engineering disciplines, where it can be extremely costly to make any changes to a specification after deployment begins. Microprocessor design is one of these disciplines. Currently, the specifications of microprocessors are interpreted and analyzed automatically. In fact, many aspects of a microprocessor design can be proven sound by unaided machines. But such techniques would be impossible if the specification weren't formalized.

In the software arena, where changes to a specification after deployment aren't nearly as costly, it's natural to question whether this style of up-front, formal specification is so useful. To consider


of 168

this question, let's first examine how well that specification style works for a particular type of software artifact: a programming language.

Among software systems, programming languages are most similar to microprocessors in terms of the cost of modifying a specification. The cost of making even minor modifications to a language design after people have begun using it can be especially high. All the existing programs written in that language will have to be modified and recompiled.

As we might expect, the specifications of programming languages, compared with other software systems, are often quite formal, especially in the case of syntax. Virtually all modern programming languages have a formally specified syntax. Most parsers are constructed through the use of automatic parser-generators that read in such grammars and produce full-fledged parsers as output.

Tip The specifications of virtually all modern programming languages contain a

formally specified syntax.

What about language semantics? Let's take a look at the following four languages, all either currently or formerly popular, and examine the relative degree of precision in the semantic specification for each: � C++ � Python � ML � Pascal

For each language, let's look at how the degree of precision in the specification has helped determine its effectiveness.

C++

The C++ language specification leaves many parts of the specification implementation dependent, and even declines to define the behavior of many valid C++ programs. Although the designers of C++ would claim that the programs for which C++ semantics is undefined are not valid C++ programs, it is impossible in principle for a machine to determine automatically whether a program is valid under this criteria, implying that many (most?) real world software applications written in C++ are not valid C++ programs.

The result is that many C++ programs don't behave as intended when ported from one platform to another.

Python

The Python Language Reference is an informal language specification that leaves many details implementation dependent. In this case, the decision not to use a formal specification was made deliberately, with full awareness of the formalisms available for language semantics. In the words of Guido van Rossum, Python's inventor:

While I am trying to be as precise as possible, I chose to use English rather than formal specifications for everything except syntax and lexical analysis. This should make the document more understandable to the average reader, but will leave room for ambiguities. Consequently, if you were coming from Mars and tried to re-implement Python from this document alone, you might have to guess things and in fact you would probably end up implementing quite a different language. On the other hand, if you are using Python and wonder what the precise rules about a particular area of the language are, you should definitely be able to find them here.


of 168

But the ambiguities of the English language aren't just a problem for Martians. Various implementations of Python, such as JPython and CPython, have faced a formidable challenge in providing compatible behavior across platforms. This problem would be much worse if it weren't for the relative simplicity and elegance of the Python language.

ML

The ML specification formally defines the full operational semantics of the language. Consequently, ML programmers enjoy an unprecedented level of precision and cross-platform standardization. The formal specification of ML has even allowed computer scientists to discover subtle inconsistencies in the ML type system, and correct them. Such inconsistencies no doubt exist in many other languages, but they are difficult to find without a formal specification.

Pascal

Pascal is one language that suffered for quite a while with an inconsistency in its specification: the rules for determining type equivalence were left unspecified. For example, consider the following two Pascal types:

type complex = record

left : integer;

right : integer;

end;

type coordinate = record

left : integer;

right : integer;

end;

Are these two types identical? Clearly, they contain the same types of subcomponents. Depending on how we define the language, a value of type coordinate may be passed to a procedure that takes an argument of type complex, and vice versa. If so, our language would be said to use structural equivalence when identifying types. Alternatively, we might define two types as equivalent if and only if they have the same name. Then our language would use name equivalence.

What choice is made in the Pascal language specification? Originally, no choice was made at all; nobody had yet realized that there was more than one way to define type equivalence! As a result, each implementation team for Pascal had to make this choice on its own (and, often, those teams didn't realize they were making a choice either). But the result of this ambiguity is that Pascal code written for one implementation can behave in a drastically different fashion on others. Benefits of Specifications

Although no formal specification (akin to that of ML) exists for Java, a good deal of care was put into development of a precise informal specification. Many smaller, toy versions of Java have been formalized from this specification, and correctness properties have been proven about them. Furthermore, Java is typically compiled to bytecode for the Java Virtual Machine, which itself is well specified (although, at the time of this writing, the process of bytecode verification is not). The result is an unprecedented level of portability for programs written in this language.

Tip Java programs have a higher degree of portability because of the language's

precise, albeit informal, specification.


of 168

The conclusion we can draw from this is that there really are advantages to having as precise a specification as possible. The costs of an ambiguity or inconsistency can be quite high, leading to decreased portability, reliability, or even to a security hole.

But even in the world of programming languages, where problems in a specification are most costly, formal specifications are rare. Some of the reasons for this are:

� Few language properties are checked automatically. The process of proving properties about a programming language specification hasn't been automated, as of yet, to the same degree that proving properties about hardware design has. As a result, there's not quite as much advantage to formalizing them.

� Many language users prefer the informal. Informal specifications are preferred by most of the people who will actually read them, like compiler writers. (In fact, compiler writers often revel in less formal specifications because it gives them more room to optimize a program.) The other, occasional, users of a language specification are the programmers, and most of them greatly appreciate an informal specification that they can easily understand.

� It costs money to produce a formal specification. Producing a formal specification up front is expensive. Companies have found it more cost effective to ship early and flesh out the details of a specification later (or, more often, never). Admittedly, if a development team commits to producing a specification, it may not finish formally specifying its system before its competitors have already shipped! If Sun had waited to produce formal language semantics for Java before releasing it, the language may not have come out in time to ride to fame as the preferred language for Internet programming.

But if up-front and formal specification is too costly, what approach should a development team take in specifying software? Many development teams have been so turned off by the cost of up-front specification that they've renounced specification entirely. But that's never a good idea. Implementations Are Not Specifications

Like it or not, a great deal of industrial software is implemented without a discernible specification. If and when the software is completed, the implementation is then presented as the specification. Whatever behavior the software exhibits is said to be the specified behavior. Some poor souls might argue that this is a good approach, since it doesn't bog the developers down with working out some sort of formal plan that is bound to change anyway. But, while it is true that project specifications often change, an implementation makes for a lousy specification in several respects:

� Many of the choices made in an implementation are arbitrary. Thus, a team that wishes to implement the system on another platform has nothing to go on but the existing implementation. The developers will have to wade through numerous implementation details to determine the behavior that the implementation entails. It is much easier to determine such behavior when it is specified at a higher level of abstraction.

� You cannot define a bug. If an implementation is literally taken as its own specification, then, as in the case where there is no specification at all, it is impossible to identify any behavior as a bug!

� Initial developers have no model of behavior. Obviously, an implementation cannot serve as the specification for the initial developers, since no such implementation yet exists. These developers must rely on some model of behavior for the system they're creating. But the source of this model should then serve as the software's specification.


of 168

This last point sheds some light on what sort of a specification a developer might use with reasonable cost. While it's true that developers must have some mental model of the feature they are implementing, they needn't have a mental model of the entire application.

In other words, specifications can be developed a piece at a time. Not only does this make them more tractable, but it also allows them to be modified more efficiently as the customers' needs change. Building Cost-Effective Specifications with Stories

One good way to develop software requirements incrementally is in the manner advocated by extreme programming (XP), an agile software-development method that has become quite popular over the past few years. The name comes from the concept that many commonly accepted and uncontroversial development practices that are usually executed alone (such as testing, incremental development, pair programming, etc.) can create a synergistic effect when all practiced together in a radical, or extreme, form. Let's focus on the way in which XP teams specify the functionality of their software. (More on XP in Chapter 3: Debugging and the Development Process.)

Tip With extreme programming, functionality is specified incrementally via stories—

brief descriptions of an aspect of system behavior.

In an XP project, the required functionality of a system is specified incrementally through the use of stories. Each story briefly describes one aspect of the system's behavior. Let's consider a simple story from a real-world XP project: a free, open-source Java IDE called DrJava.

DrJava was developed at the JavaPLT research laboratory of Rice University and was designed to provide an extremely simple but powerful user interface that enables programmers at all levels to manipulate, test, and debug their code. It not only integrates testing and debugging support, but also provides an integrated "read-eval-print loop" that allows users to evaluate arbitrary Java expressions interactively. DrJava will be the basis of many examples in this book for the following reasons: � It provides a great example of code developed in an XP project. � I'm familiar with its code base. � All of the code is open source, so you can download the DrJava jar file, along with all of

the source code, at http://drjava.sourceforge.net.

Let's consider the following story from DrJava's early stages of development:

As the user types words into the editor, occurrences of Java keywords are automatically colored blue. String and character literals are colored green, and comments are colored red.

Believe it or not, that's actually a hard story to get right in Java, partly because of some peculiar properties of block comments. Depending on the velocity of the development team, it may be advisable to break that story up into two or more smaller ones.

Tip Small, new stories can be easily added to modify functionality. This method

works well in situations where requirements change frequently and bits of new functionality need to be rolled out quickly.

Still, notice that the functionality specified by this story is of a tiny scope when compared to a traditional, full, up-front specification for an IDE. Also, this story is written in simple, clear language. That makes it easy to split up into smaller stories when necessary, and it prevents coupling (unwanted entanglements) between the parts of a specification.

Let's look at another story, this time for an "interactions window" in which the user can enter Java statements and expressions dynamically and then see the results. We wanted to add to this window the ability to scroll through earlier commands with the up and down arrow keys:


of 168

The user enters a new command at the prompt in the interactions window. Once the user hits Enter, the command is executed, the result is displayed, and the cursor is moved down to a new prompt. Previous commands can be recalled to the current prompt using the up and down arrows, allowing the user to scroll through a history of commands.

In this case, the story is slightly longer, but the specified functionality is still very limited. Some time after we implemented this story, a user complained that he couldn't easily get back to a blank prompt after he started scrolling through the old commands by going to the bottom of the list. (Actually, the user could have pressed the Escape key to clear the line, but his suggestion made for a more natural interface.) No problem. We extended the existing specification by writing a new story:

When scrolling through the history in the interactions window using the arrow keys, the user moves down to the most recently entered command. The user hits the down arrow once more, and a blank line appears. He then types in a new command. This feature is convenient for entering new commands after scrolling through previously entered ones.

Very little new functionality is specified in this story. If support for scrolling through previous commands was already implemented, it's not hard to imagine that a pair of programmers could implement this new story in less than an hour. Because the stories are small, new ones can be added to modify program functionality without having to completely overhaul the specification. For this reason, stories work particularly well when the requirements for the software product change frequently.

Additionally, by specifying and implementing stories incrementally, the programmers are able to release new functionality quite rapidly, allowing the customers to get more value from the software more quickly.

Tip Stories may be prone to the same ambiguities and inconsistencies as any

informal specification. Accompanying tests can eliminate these errors.

Although stories allow us to specify software incrementally, they have the disadvantage of not being as formal as those traditional up-front software specs. Therefore, they are prone to the same ambiguities and inconsistencies as any other informal specification. But if the traditional formal specifications are too costly, is there any way that these errors can be eliminated?

Include tests to eliminate specification errors

One way to eliminate ambiguities and inconsistencies in a story is to include tests with it. If there's a section or clause in a story that has multiple interpretations, just write a test that helps to define that aspect of the interpretation. Provided the programming language you write the test in isn't ambiguous, that test will nail down the behavior of the program. In addition, if a set of unit tests specifies inconsistent functionality, it will be impossible for a program to pass them all.

Extreme programming uses two forms of tests: acceptance tests and unit tests. Acceptance tests check user-observable functionality. Unit tests are small tests that check specific "units" of program functionality.

A key feature of both kinds of tests is that they automatically check the desired functionality; it's not necessary for the programmer to examine the output of each test to ensure it's correct. If a test fails when run, the programmer is notified; otherwise, he knows that it passed.

In extreme programming, testing is a way of life. The programmers start writing tests before they write any of the implementation at all, and they continue writing more unit tests for each new aspect of program functionality. A rigorous suite of tests laid over a software project provides several advantages: � The tests are an important form of documentation.


of 168

� The tests expedite the process of refactoring. � The tests complement stories as part of the specification.

The tests are an important form of documentation

Since the tests (ideally) cover every aspect of the implementation, and since they invoke the functionality in simple ways to make sure it is working, it is easy for a programmer who is joining a project (or taking over maintenance of code) to read through the tests and determine what the various functional components do.

When first hearing of the concept that a test can be considered documentation, some people are skeptical: "How can you write documentation for a program in the same language that the program is written in?" This question misses the point of code documentation. Code should never be documented to explain what the code is doing; the code itself already does that.

Instead, documentation should explain why a block of code is doing what it does. Anyone reading the code should be already familiar with the language used; if not, then documentation in any language is unlikely to help. Granted, it is not always clear how a block of code interacts with the rest of a program, and documentation is good for that purpose. But since the reader of the code is (or should be) familiar with the language, it is perfectly valid to explain the intention behind the code in the same language as the code.

Tip Documentation should explain why a block of code is doing what it is doing.

Tests expedite the process of refactoring

When a suite of tests can be run over the code at any time to determine if any of the functionality has been broken, programmers can refactor the code with much more confidence that they aren't stomping over the invariants of each other's code. The vast majority of bugs introduced can be detected as soon as they're introduced.

Tests complement stories as part of the specification

But what is mentioned less often is that tests complement stories as part of the specification. And just as stories allow for the incremental and informal specification of a system, unit tests allow for the incremental and formal specification of the same system. Although no set of unit tests can nail down all aspects of a system, a test suite can define the most ambiguous aspects. Furthermore, tests have huge advantages over most forms of formal specification: � Each test can be written independently of the rest. � The tests can be automatically verified. Few other forms of formal specification have this

property (the most notable exception being static types). Compare the process of running tests to a manual proof of correctness for a program. If there is a flaw in the proof, all bets are off. But if only one test fails, at least we know that the rest of them passed. Plus, we don't need to rewrite the unit tests whenever we modify the implementation as we would for a manual proof.

� The tests can be written in the same language as the program. Thus, programmers needn't learn another formalism to formally specify functionality.

Tip Tests are important as documentation, to expedite refactoring, and to

complement stories as part of the specification.

For an example of how unit tests can help to better define a specification, let's return to our story concerning the history of commands in the DrJava interactions window. As we mentioned, the user can scroll back through this history with the up and down arrows, and extract text for forming new commands. One of the classes used to implement this story in DrJava is a History class, which stores the list of commands that have occurred so far.


of 168

What happens when the user issues the same command twice in a row? This question isn't answered by the story shown previously. But we'd like the History to store only one of the two commands; it's tedious to have to scroll through a series of identical commands. We could write the following unit test to enforce this property:

public void testMultipleInsert() {

_history.add("new Object()");

_history.add("new Object()");

assertEquals("Duplicate elements inserted", 1, _history.size());

}

Notice that the method takes no arguments and returns void. That's because it is run automatically. We don't need to feed it input or check its output; if it doesn't pass, it'll throw an exception. (This test is written in the form used by JUnit, a free, open-source testing harness for Java. JUnit is part of the xUnit suite of test harnesses, providing open-source testing tools for most popular programming languages.)

The test starts with a fresh History object (set in the _history field) and adds two identical commands. It then checks that the length of the History is exactly one. The assertEquals method takes three arguments: a message to signal if the test fails and two values. If the values are equal, the test succeeds; otherwise it fails.

What are some other tests we could put in our History class? Why don't we formalize the property stated in the final story example: that we can move back to a blank line at the end of the History. Following is a test for that:

public void testCanMoveToEmptyAtEnd() {

_history.add("some text");

_history.movePrevious();

assertEquals("Prev did not move to correct item",

"some text",

_history.getCurrent());

_history.moveNext();

assertEquals("Can't move to blank line at end",

"",


}

Notice that these tests are gradually winnowing down the definitions for the set of methods that the History class will have to implement.

Writing tests is a great way to determine the interface that a class should implement. Because you have to program to that interface yourself, you'll see just how difficult, or easy, you're making it to work with your interface. Your own preference for using simple interfaces will help you keep your own interfaces simple. It'll also help you maintain your tests in an easy-to-read form.

Of course, there are many other tests we can include over class History. But we shouldn't write them all before implementing some of the functionality. The better procedure is to:

1. Write just a few tests;


of 168

2. Write code to pass the tests from Step 1; 3. Repeat Steps 1 and 2 as many times as needed.

This way, we can integrate the code at each step (and make sure we didn't break anything).

The History class was implemented in DrJava in the following way:

/**

* Keeps track of what was typed in the interactions pane.

* @version $Id: History.java,v 1.9 2002/03/06 18:59:02 eallen Exp $

*/

public class History {

private Vector<String> _vector = new Vector<String>();

private int _cursor = -1;

/**

* Adds an item to the history and moves the cursor to point

* to the place after it.

*

* To access the newly inserted item, you must movePrevious first.

*/

public void add(String item) {

if (item.trim().length() > 0) {

if (_vector.isEmpty() || ! _vector.lastElement().equals(item)) {

_vector.addElement(item);

}

moveEnd();

}

}

/**

* Move the cursor to just past the end. To access the last element,

* you must movePrevious.

*/

public void moveEnd() {

_cursor = _vector.size();

}

/** Moves cursor back 1, or throws exception if there is none. */

public void movePrevious() {

if (!hasPrevious()) {


of 168

throw new ArrayIndexOutOfBoundsException();

}

_cursor—;

}

/** Moves cursor forward 1, or throws exception if there is none. */

public void moveNext() {

if (!hasNext()) {

throw new ArrayIndexOutOfBoundsException();

}

_cursor++;

}

/** Returns whether moveNext() would succeed right now. */

public boolean hasNext() {

return _cursor < (_vector.size());

}

/** Returns whether movePrevious() would succeed right now. */

public boolean hasPrevious() {

return _cursor > 0;

}

/**

* Returns item in history at current position, or throws exception if none.

*/

public String getCurrent() {

if (hasNext()) {

return _vector.elementAt(_cursor);

}

else {

return "";

}

}

/**

* Returns the number of items in this History.

*/

public int size() {

return _vector.size();


of 168

}

}

Now, here's a great example of just how easy it is to incrementally add to the formal specification of a program with unit tests. Let's say that, after writing the code above, we decided we wanted to limit the length of the History to 500 items in order to prevent runaway memory consumption in long-standing processes. So, we add the following unit test to our suite:

/**

* Ensures that Histories are bound to 500 entries.

*/

public void testHistoryIsBounded() {

int maxLength = 500;

for (int i = 0; i < maxLength + 100; i++) {

_history.add("testing "+ i);

}

while(_history.hasPrevious()) {

_history.movePrevious();

}

assertEquals("history length is not bound to "+ maxLength,

"testing 100",


}

This new test adds 600 elements to the History and checks that a few assertions hold. Notice that it doesn't just check that only 500 entries are included in the History; it checks that items are removed in a FIFO (first-in-first-out) order. It accomplishes that check by ensuring that the oldest element in the History is the 100th element added, which is exactly what it should be if the oldest elements are removed with every command after the 500th entry.

Modifying class History to pass this test was easy: first, we added the following constant to class History:

private static final int MAX_SIZE = 500;

Then we modified the add() method as follows:

/**

* Adds an item to the history and moves the cursor to point

* to the place after it.

* Note: Items are not inserted if they would duplicate the last item,

* or if they are empty. (This is in accordance with bug #522123 and

* feature #522213.)

*


of 168

* Thus, to access the newly inserted item, you must movePrevious first.

*/

public void add(String item) {

if (item.trim().length() > 0) {

if (_vector.isEmpty() || ! _vector.lastElement().equals(item)) {

_vector.addElement(item);

// If adding the new element has filled _vector to beyond max

// capacity, spill the oldest element out of the History.

if (_vector.size() > MAX_SIZE) {

_vector.removeElementAt(0);

}

}

moveEnd();

}

}

With this fix, the code behaves as specified.

Unit tests can't do everything

As the preceding example demonstrates, unit tests are an essential complement to stories for the incremental specification of a software system. In fact, some might be tempted to use a suite of unit tests as the sole specification of a system. But using unit tests to form the only specification has one big disadvantage: the set of tests over a system are inevitably incomplete.

No matter how many tests we specify over a system, there will always be more inputs and states of the system than we could ever hope to represent. We could interpret the tests as specifying the "most reasonable" extension, but such an extension will often be ambiguous. That's where the strength of stories comes in. Just as unit tests can clarify the intended specific aspects of a story, a story can clarify the intended general aspects of a unit test. Both are needed for an effective and agile software specification.

Tip Unit tests can clarify the intended specific aspects of a story. A story can clarify

the intended general aspects of a unit test.

The use of stories and unit tests can aid software development in many ways, but here we've described their use solely for efficiently specifying software systems. And we've also emphasized the need to use specifications by pointing out that they're necessary for precisely identifying bugs in a program. Thus, a serious concern for debugging can influence the way we program, even at the level of specifying software.


of 168

A Quick Recap

In this chapter, we've learned to: � Define the concept of a bug. � Explain why a specification is crucial for controlling software bugs. � Understand the differences between a specification and an implementation. � Use stories and unit tests when developing specifications. � Introduce cost-effective means for developing specifications.

Now, let's discuss the impact that the debugging process can have on designing and coding software systems.

In Chapter 3, we'll look at how six tenets of extreme programming make the debugging process easier and more effective. We'll also discuss the crucial interdependence between effective debugging and effective development, highlight extreme programming development methods, and peek at a future of test-oriented languages.


of 168

Chapter 3: Debugging and the Development Process We discuss the crucial interdependence between effective debugging and effective development, highlight extreme programming development methods, and peek at the potential future of test-oriented languages.

Debugging as Scientific Experiment

To debug software is to perform a type of scientific experiment. (This topic is explored in detail in Chapter 5.) And just as scientific experiments are best done in a laboratory environment, with controlled procedures and equipment, debugging software is best done as part of an integrated method of software development. Indeed, an appropriate method of development can easily help prevent many bugs from ever occurring in the first place.

Even when bugs are found, the development method can make a big difference in how quickly a bug can be diagnosed. As you might have guessed, I strongly advocate extreme programming as a particularly effective development method when you want to diagnose bugs quickly. And who doesn't want to find and dispose of bugs quickly?

Let's talk specifically about how extreme programming facilitates the debugging process. Although a thorough introduction to extreme programming is beyond the scope of this text (see the Resources chapter for some good references), I have found the following aspects of XP to be quite useful in debugging: � Software is specified, integrated, and released incrementally. � Design is kept as simple as possible. � Programming is done in pairs. � An on-site customer is always available. � Code is owned by all developers. � Tests exist for "anything that could possibly break."

How do each of these practices help in debugging software? Let's see.

Software is specified, integrated, and released incrementally

In extreme programming, each aspect of the functionality is specified just before it is implemented. A pair of programmers will then work on implementing that aspect for a couple of weeks or less (remember, it's just a small bit of functionality), and then release the new version. Along the way, they will integrate the work they've done thus far into the system, ensuring that all the tests they wrote and the tests everyone else wrote still pass.

In this way, any bugs that are introduced have a very good chance of being caught right away by the unit tests. Additionally, since implementation is incremental, the programmers will be wholly focused at each integration on making sure that one small piece of new functionality works.

Why do it? Catch bugs as they're introduced; focus is on a single, small bit of functionality.

Design is kept as simple as possible

Complexity is the enemy of robustness. If there is one aspect of software construction that laymen find most puzzling, it's that good programmers treat complexity like an unwanted weed, rooting it out to the extent possible. Why? Because programmers know that software systems are


of 168

often used much longer than anyone expects, in contexts never anticipated, growing larger and larger all the while. Unless complexity is actively resisted, it'll quickly overtake a project.

Extreme programming continually strives to keep the system as simple as possible. Collective code ownership and constant refactoring (facilitated by the heavy use of unit tests) help to keep the code simple. Of course, that helps to prevent bugs from being written in the first place. But it also helps debugging; the simpler a program is, the fewer possibilities have to be considered as potential causes of a bug.

Why do it? Simpler code means less potential causes for a bug.

Tip Complexity is the enemy of robustness.

Programming is done in pairs

In extreme programming, all programming is done in pairs. Partners take turns driving at the keyboard. The partner who isn't driving is navigating—that is, inspecting the code as it's written, considering the big picture, and determining if there's a better way to do things. In essence, the navigator performs a continuous code review.

The motivation behind pair programming is that the act of coding involves simultaneously considering high-level aspects of a program, such as encapsulation and abstraction, and thinking about low-level aspects, such as syntax, argument order, etc. By using two people, we can distribute this cognitive burden more effectively.

Believe it or not, two programmers working together can be just as efficient (or even more efficient) than the same two programmers would be if working separately. At first, that sounds absurd. Since they're both working on just one task, as opposed to the two they'd work on separately, we would expect that productivity would be cut in half. But, as amazing as it sounds, time and time again pair programming increases overall productivity.

No doubt there are many reasons why pair programming is so efficient, but there's one that anyone who has ever debugged a program can appreciate. Consider the most common form of bug introduced in a program: a typo or syntax error. Even when these bugs are caught by static type checking, it takes time to go through the tedious cycle of compile, fix, compile, etc. The navigator will prevent many such typos from making it to the compile stage. This alone can save a surprising amount of time. But the real time savings occurs when the navigator discovers a typo that actually would have made it past static checking. In those cases, tracking down the bug can involve a substantial amount of cognitive effort and time (often several hours). If the navigator finds just two such bugs in a four-hour pairing session, the pairing will have paid for itself. After that, any further time savings is in the black.

Why do it? Divides thinking about highand low-level issues; increases person-hour productivity; provides backup on error introduction.

An on-site customer is always available

In contexts in which requirements are ambiguous and changing frequently, having a live customer available to ask questions is extremely helpful. It can often prevent developers from wasting precious time guessing how the customers would like some functionality to be implemented, only to discover that they were completely wrong.

In the context of effective debugging, the important aspect here is that the programmers are getting continuous feedback from someone who is actively using the product, and the more


of 168

people using it, the better. Inevitably, these users will discover bugs that escaped even the most thorough test suite. As always, the sooner a bug is discovered, the easier it is to fix.

Why do it? Provides active user/client feedback.

Code is owned by all developers

When all programmers have the authority to refactor the code written by any programmer on the team, they tend to become more familiar with that code. Extreme programming involves not just collective ownership of code, but uniform coding standards and constant refactoring. When programmers are given the authority to review and improve all code in the project, they naturally become more familiar with the code they didn't write. This facilitates effective debugging in several respects.

Most obviously, they will be much better able to debug each other's code, since they'll understand it better. But they will also be less likely to blame a bug on code they didn't write. Humans tend to be suspicious of things they're unfamiliar with. That's a great survival trait when deciding which wild plants to eat, but it's deadly when debugging software. Programmers place blame all too often on anything but their own code. Collective code ownership ameliorates this phenomenon.

Why do it? Increases familiarity of all code for all programmers; lessens time spent blaming "foreign" code.

Tests exist for "anything that could possibly break"

In extreme programming, programmers write tests for functionality even before they start implementing that functionality, and they continue writing tests as they finish implementing it. Every time a programming pair integrates their code, all tests are run to ensure that the new code doesn't break anything. As mentioned previously in Chapter 2, unit tests form part of the specification of a program and they help prevent the members of a programming team from breaking each other's code. And since unit tests are run every time new code is integrated, they help to catch many bugs as soon as they're introduced. Even in cases where a bug isn't detected until long after it was introduced, the unit tests help programmers to eliminate many possible causes of the erroneous behavior and diagnose the cause of the problem quickly.

What does it mean to say "test anything that could possibly break"? Any method with a non-trivial body should be tested. Getters and setters on the other hand may not require testing since they are trivial and utterly fail to work if they are mistyped.

Why do it? It ensures that new functionality is consistent with existing functionality.

Taken together, these concepts make up a very powerful environment for debugging software. In fact, I've found that bugs resulting from mere typos can be virtually eliminated with extreme programming. An effective development method is the single most important quality of an effective debugging strategy. Incorporate Debug Tests into Unit Test Suites

Because one small test will often lead us to discover the true source of a bug, it follows that effective debugging should involve lots of testing. There's simply no way to build a reliable system without thoroughly testing it.

Although many programmers may be unfamiliar with the XP style of coding (in which writing the tests is interleaved with writing the code), there is one form of testing that everyone does while coding: debugging. Some programmers write these tests using print statements, others write them to work with a standalone debugger, others with the debugger in their IDE of choice.


of 168

Quite often, you will see programmers discard these tests once they've gotten the program to run correctly. This is a waste of very good tests. Why not incorporate them into the unit test suite over a program? After all, if one of them were to exhibit an unexpected result, we'd want to know about it.

Incorporating debugging tests into the unit tests can be done with relatively little effort, but there are some adaptations that have to be made. Frequently, we'll write tests for debugging to display information about the internal state of the program, whereas we'll write unit tests to signal only if they fail. That's because debugging and unit testing serve different purposes.

When debugging, we are trying to form a more accurate model of the program's behavior, so as to diagnose a bug. But when testing, all we want to know is whether the code passed the tests. If instead we wrote the unit tests to print out a result and then manually checked that it was what we expected, we'd waste a lot of time (or, more likely, we would seldom run the unit tests). So, unit tests tend to be written such that the result is solely one of "pass" or "fail".

Debugging tests can still be used as unit tests with only slight modification. Consider that when the program is working correctly, the result of running a debugging test will match some expected result. It is straightforward to modify such a test so that, instead of simply printing out this result, it compares the result to what's expected. It can then be incorporated into a unit test suite quite easily.

Just as debugging helps in developing a large suite of unit tests over a program, unit tests can help significantly when debugging. When diagnosing a bug, if you can first run a suite of unit tests and verify that they pass, you can rule out a huge number of potential explanations for a bug. In this way, unit tests allow you to leverage your cognitive energy when modeling program behavior. This is yet another way that debugging is like performing a scientific experiment.

When a physicist forms an explanation of an experimental result, he automatically rules out all sorts of explanations that would defy a set of accepted principles about the way the world works. For example, he assumes that the results of his experiment will not change depending on the current weather conditions on Jupiter (well, unless he is performing an experiment on Jupiter), or depending on what he plans to eat for dinner. Unit tests enforce accepted principles of program behavior. The Future: Test-Oriented Languages

Just as many design patterns add multiple classes to a program (such as visitors, decorators, and such) that serve no purpose other than to add extensibility to the program, it is acceptable to develop new patterns to facilitate testing.

Indeed, many of the features of object-oriented languages are included to facilitate extensibility; why shouldn't future versions of such languages (or entirely new languages) include features to facilitate testing?

In the case of the Java language, this is already beginning to happen. Future versions are scheduled to include much more powerful type systems, assertions, and the like. Just as object-oriented languages have increased the degree to which we can reuse and extend existing code, future, test-oriented designs and features will help to increase the robustness of our code, both old and new.


of 168

A Quick Recap

In this chapter, we've learned how the following tenets of extreme programming make the debugging process easier and more effective: � Software is specified, integrated, and released incrementally. � Design is kept as simple as possible. � Programming is done in pairs. � An on-site customer is always available. � Code is owned by all developers. � Tests exist for "anything that could possibly break."

In Chapter 4, we'll discuss how to design code with testability in mind


of 168

Chapter 4: Debugging and the Testing Process We highlight the essential interplay between continuous testing and effective debugging.

Designing for Testability

Before leaving the topic of development, let's discuss the issue of software that's hard to test. As I have stressed, effective debugging involves effective unit testing. But often, programmers not used to "infesting" their code with unit tests will run into situations where they believe there's just no way they can effectively test the behavior of their system.

When such cases occur, try to take a step back from the question, "How can I possibly test this kind of code?" Instead, ask, "How could I write this code in such a way that I can test it?" This shift in thinking will often result in the addition of many pieces of functionality that serve no purpose other than to facilitate the testing process. As long as the new functionality is necessary for testing, that's perfectly justified. I call this strategy test-oriented programming.

The time and effort spent writing tests and testing code pay off in dramatically reduced maintenance costs. However, unless you're careful, the effort involved in testing code can amount to several times the effort of writing the code in the first place! I've seen programmers make concerted efforts to fully cover their code with unit tests, and many of them end up disheartened at how much time it takes.

Fortunately, it doesn't have to be this way. With the application of a few basic principles that we'll discuss, it is possible to write code that is easy, and even fun, to test. Like any other set of coding principles, these principles are not meant to be unquestionable or unalterable dogma. There are times when it is necessary to break the rules. For this reason, it is important to understand the motivations behind each principle and to be able to determine when these motivations don't apply (or when they are usurped by more important concerns).

The following sections discuss these topics: � Keeping code in the model, not the view. � Using static type checking to find errors. � Using mediators to encapsulate functionality across fault lines. � Writing methods with small signatures and default arguments. � Using accessors that do not modify memory state. � Specifying out-of-program components with interfaces. � Writing tests first.

Keeping code in the model, not the view

When writing GUI code, move as much code as possible out of the view of a GUI. The various GUI actions can then be simple method invocations on the model. Why would you want to do this? Because it is much easier to test functionality through method invocations than indirectly with GUI testers. An additional benefit is that it makes it easier to modify the functionality of the program without affecting the view.

Of course, there can be bugs in the view, too. Ideally, the tests for a program will check both the model and the view. The Model-View-Controller architecture, where the view and model are completely decoupled and a controller sets up the connections between the two at runtime, is a particularly effective way to allow for testing of both model and view. The respective tests on these components will be quite different in each case, so by separating them, the tests for each can be greatly simplified.


of 168

The DrJava IDE example (introduced in the section of Chapter 2 entitled "Building Cost-Effective Specifications with Stories") provides a good example of how much functionality can be put into a model. The only contact that the view has with the model is through a specially designed interface we call the GlobalModel. This interface includes methods for every functional modification a user can make while using DrJava. In essence, it provides a handle that our tests can use to interact with DrJava in any way that a user can, except that the tests don't have to interact through the view. (If you're curious about what it looks like, the GlobalModel interface is included at the end of this chapter.)

Use static type checking to find errors

Types are your friends. Use the type system as much as possible to automatically check for errors. Doing so will save you from having to write a lot of extra tests just to check the invariants that type checking gives you for free.

Types can automatically catch a bug in your program before it is ever run. Without static type checking, a type error may linger in your program as a saboteur until just the right execution path happens to uncover it. See Chapter 13: The Saboteur Data bug pattern.

But the issue of how to use types to maximum advantage can be tricky. Often, a collection of data structures can be used together at one level of abstraction, or incorporated into a single, higher-level abstraction with a new associated data type. If we incorporate them into a higher-level abstraction, static type checking can ensure that they are used only in the manner allowed by that abstraction. Such checks can prevent errors, but they can also limit expressiveness. It can be difficult or impossible to use one of the data structures below the new level of the abstraction, even when we want to. In Java, we can do so only by inserting casts into the code.

In fact, the history of programming languages itself can be viewed as a gradual increase in the levels of abstraction at which one can program. Assembly language abstracted numerical opcodes into named mnemonics and numerical addresses into symbols. This was followed by abstractions such as records and functions, which were then followed by abstractions such as objects, classes, threads, and exceptions. At each higher level of abstraction, programming becomes simpler and more robust, but the expressiveness of the language is decreased.

In object-oriented languages (as well as other modern languages), the individual programmer is given a great deal of flexibility in devising abstractions. The level of abstraction at which to design a program then becomes a decision based on trade-offs, such as the added robustness provided by an abstraction level and the expressiveness (and sometimes performance) lost by not working at a lower level of abstraction.

In general, the added robustness and simplicity of higher levels of abstraction are seldom outweighed by other considerations. For more discussion on this issue, refer to Chapter 15 on the Impostor Type bug pattern.

Use mediators to encapsulate functionality across fault lines

By fault lines, I mean the interfaces between separate components that have little interaction compared with the internal interaction of their respective subcomponents. A classic example of such a fault line would be the interface between the view of a GUI and the corresponding model (as in the GlobalModel interface example described previously). Other examples include the interfaces between various phases of processing in a compiler or the interface between the kernel and user interface of an operating system.

The Mediator pattern is one of the design patterns discussed by Gamma, Helm, Johnson, and Vlissides (1994). A mediator is defined as "an object that encapsulates how a set of objects


of 168

interact." It allows the client of that set of objects to more easily interact with all of them; the client only interacts directly with a single object.

Find the fault lines of your program, then use mediators with forwarding functions to quickly access aggregate components. To be sure, it may be easier on some occasions to test each component along a fault line in isolation. But if there are many objects exposed by each component, or if some of the objects you would like to test in a component are accessible only by following several nested references, testing can become quite tedious. Instead of testing in isolation, it often helps to have a single mediator object on which you call the various methods you want to test. This object can then forward these method calls to the appropriate places.

Along the same lines (no pun intended), it is useful to design interfaces to program components in tandem with the tests over them. This will focus your efforts on keeping these interfaces as simple as possible.

Write methods with small signatures and default arguments

Using small method signatures and overloading methods with default method arguments will make it much more pleasant to invoke these methods in your tests. Otherwise, you'll have to construct the extra arguments when testing the methods. If the denotations of the arguments are large, this can quickly lead to code bloat. Even worse, it can tempt you into writing fewer tests than you otherwise would.

Use accessors that do not modify memory state

Use accessors that do not modify the state of memory to check the state of objects in your tests. By accessors, I mean methods that retrieve some view of the state of an object (most commonly, the value of a field).

Remember the analogy of tests as scientific experiments: they both attempt to verify that particular hypotheses hold. But this is much harder to do if the very act of inspection alters the state of the world. Unlike the state of a particle in quantum mechanics, the state of a computer process can be checked without modifying that state. Use this to your advantage.

One example of an accessor that modifies state is the next method in java.util.Iterator. It is natural to call this method to retrieve the next element while iterating over a Collection, but if it's called more than once in a single iteration, an element may be lost. It would have been better to separate next() into two methods: one to retrieve the current element, and one to move forward by one element.

Specify out-of-program components with interfaces

Using interfaces to specify outside program components allows for easy simulation of those components in the test cases.

This principle can save a tremendous amount of time, especially if the implementation of the outside component isn't complete. All too often, the most essential components aren't available on time. If you can't test your own code without these components in place, you are headed for disaster. Your customers won't care that you only had two hours to integrate a component that was two weeks late. All they know is that the integrated product is late and that it's broken.


of 168

Write the tests first

This is standard practice in extreme programming, but it is always tempting to ignore it. Nevertheless, every time I succumb to this temptation, I regret it. Given that you're trying to produce correct code, the time you appear to save by postponing the writing of tests is nothing but an illusion.

This doesn't mean that you should write the entirety of the tests in one shot, followed by the entirety of the implementation. It is better to write a few tests, implement them, write a few more, implement those, and so on, integrating at each iteration. In this way, the design evolves; oversights are caught during the implementation phase and corrected in the next set of tests.

It is also less daunting to write the tests in this way. And by continually integrating, you'll reduce the chance of conflicts with other modifications to the code base. The GlobalModel Interface

As promised earlier in this chapter in the section entitled "Keeping Code in the Model, Not the View," Listing 2-1 contains a snapshot of the GlobalModel interface as it exists today. Listing 2-1: DrJava's GlobalModel Provides an Interface that Maintains Limited Connections between the View and the Model �

package edu.rice.cs.drjava.model;

import javax.swing.text.*;

import javax.swing.ListModel;

import java.io.*;

import java.util.*;

import edu.rice.cs.util.swing.FindReplaceMachine;

import edu.rice.cs.drjava.DrJava;

import edu.rice.cs.util.UnexpectedException;

import edu.rice.cs.drjava.model.definitions.*;

import edu.rice.cs.drjava.model.repl.*;

import edu.rice.cs.drjava.model.compiler.*;

/**

* Handles the bulk of DrJava's program logic.

* The UI components interface with the GlobalModel through its public

* methods, and GlobalModel responds via the GlobalModelListener interface.

* This removes the dependency on the UI for the logical flow of the program's

* features. With the current implementation, we can finally test the compile


of 168

* functionality of DrJava, along with many other things.

*

* @version $Id: GlobalModel.java,v 1.30 2002/02/25 22:52:14 thepropsman Exp $

*/

public interface GlobalModel {

/**

* Add a listener to this global model.

* @param listener a listener that reacts on events generated by the GlobalModel

*/

public void addListener(GlobalModelListener listener);

/**

* Remove a listener from this global model.

* @param listener a listener that reacts on events generated by the GlobalModel

*/

public void removeListener(GlobalModelListener listener);

/**

* Fetches the {@link javax.swing.EditorKit} implementation for use

* in the definitions pane.

*/

public DefinitionsEditorKit getEditorKit();

/**

* Gets a ListModel of the open definitions documents.

*/

public ListModel getDefinitionsDocuments();

/**

* Gets the interactions document.

*/

public StyledDocument getInteractionsDocument();

/**

* Gets the console document.

*/

public StyledDocument getConsoleDocument();


of 168

/**

* Gets the array of all compile errors without Files.

*/

public CompilerError[] getCompilerErrorsWithoutFiles();

/**

* Gets the total number of current errors.

*/

public int getNumErrors();

/**

* Creates a new document in the definitions pane and

* adds it to the list of open documents.

* @return The new open document

*/

public OpenDefinitionsDocument newFile();

/**

* Open a file and read it into the definitions.

* The provided file selector chooses a file, and on a successful

* open, the fileOpened() event is fired.

* @param com a command pattern command that selects what file

* to open

* @return The open document, or null if unsuccessful

* @exception IOException

* @exception OperationCanceledException if the open was canceled

* @exception AlreadyOpenException if the file is already open

*/

public OpenDefinitionsDocument openFile(FileOpenSelector com)

throws IOException, OperationCanceledException, AlreadyOpenException;

/**

* Closes an open definitions document, prompting to save if

* the document has been changed. Returns whether the file

* was successfully closed.

* @return true if the document was closed

*/

public boolean closeFile(OpenDefinitionsDocument doc);

/**


of 168

* Attempts to close all open documents.

* @return true if all documents were closed

*/

public boolean closeAllFiles();

/**

* Saves all open documents, prompting when necessary.

*/

public void saveAllFiles(FileSaveSelector com) throws IOException;

/**

* Saves all open documents, used for testing

*/

public void saveAllFiles(FileSaveSelector com[]) throws IOException;

/**

* Exits the program.

* Only quits if all documents are successfully closed.

*/

public void quit();

/**

* Returns the OpenDefinitionsDocument for the specified

* File, opening a new copy if one is not already open.

* @param file File contained by the document to be returned

* @return OpenDefinitionsDocument containing file

*/

public OpenDefinitionsDocument getDocumentForFile(File file)

throws IOException, OperationCanceledException;

/**

* Clears and resets the interactions pane.

* First it makes sure it's in the right package given the

* package specified by the definitions. If it can't,

* the package for the interactions becomes the default

* top level. In either case, this method calls a helper

* which fires the interactionsReset() event.

*/

public void resetInteractions();


of 168

/**

* Resets the console.

* Fires consoleReset() event.

*/

public void resetConsole();

/**

* Clears the current interaction text and then moves

* to the end of the command history.

*/

public void clearCurrentInteraction();

/**

* Forwarding method to remove logical dependency of InteractionsPane on

* the InteractionsDocument. Gets the previous interaction in the

* InteractionsDocument's history and replaces whatever is on the current

* interactions input line with this interaction.

*/

public void recallPreviousInteractionInHistory(Runnable failed);

/**

* Forwarding method to remove logical dependency of InteractionsPane on

* the InteractionsDocument. Gets the next interaction in the

* InteractionsDocument's history and replaces whatever is on the current

* interactions input line with this interaction.

*/

public void recallNextInteractionInHistory(Runnable failed);

/**

* Interprets the current given text at the prompt in the interactions

* pane.

*/

public void interpretCurrentInteraction();

/** Returns the first location in the document where editing is allowed. */

public int getInteractionsFrozenPos();


of 168

/**

* Aborts the currently running interaction.

*/

public void abortCurrentInteraction();

/** Called when the repl prints to System.out. */

public void replSystemOutPrint(String s);

/** Called when the repl prints to System.err. */

public void replSystemErrPrint(String s);

/**

* Signifies that the most recent interpretation completed successfully,

* returning no value.

*/

public void replReturnedVoid();

/**

* Signifies that the most recent interpretation completed successfully,

* returning a value.

*

* @param result The .toString-ed version of the value that was returned

* by the interpretation. We must return the String form

* because returning the Object directly would require the

* data type to be serializable.

*/

public void replReturnedResult(String result);

/**

* Signifies that the most recent interpretation was ended

* due to an exception being thrown.

*

* @param exceptionClass The name of the class of the thrown exception

* @param message The exception's message

* @param stackTrace The stack trace of the exception

*/


of 168

public void replThrewException(String exceptionClass,

String message,

String stackTrace);

/**

* Signifies that the most recent interpretation contained a call to

* System.exit.

*

* @param status The exit status that will be returned.

*/

public void replCalledSystemExit(int status);

/**

* Returns all registered compilers that are actually available.

* That is, for all elements in the returned array, .isAvailable()

* is true.

* This method will never return null or a zero-length array.

*

* @see CompilerRegistry#getAvailableCompilers

*/

public CompilerInterface[] getAvailableCompilers();

/**

* Sets which compiler is the "active" compiler.

*

* @param compiler Compiler to set active.

*

* @see #getActiveCompiler

* @see CompilerRegistry#setActiveCompiler

*/

public void setActiveCompiler(CompilerInterface compiler);

/**

* Gets the compiler is the "active" compiler.

*

* @see #setActiveCompiler

* @see CompilerRegistry#getActiveCompiler

*/

public CompilerInterface getActiveCompiler();


of 168

/**

* Gets an array of all sourceRoots for the open definitions

* documents, without duplicates.

* @throws InvalidPackageException if the package statement in one

* of the open documents is invalid.

*/

public File[] getSourceRootSet();

}

The DrJava IDE example (in the Chapter 2 section entitled "Building Cost-Effective Specifications with Stories") provides a good example of how much functionality can be put into a model. The only contact that the view has with the model is through a specially designed interface we call the GlobalModel, shown in the following listing.

This interface includes methods for every functional modification a user can make while using DrJava. In essence, it provides a handle that our tests can use to interact with DrJava in any way that a user can, except that the tests don't have to interact through the view. A Quick Recap

In this chapter, we've described the following keys to effectively designing software with testability in mind: � Keeping code in the model, not the view. � Using static type checking to find errors. � Using mediators to encapsulate functionality across fault lines. � Writing methods with small signatures and default arguments. � Using accessors that do not modify memory state. � Specifying out-of-program components with interfaces. � Writing tests first.

In Chapter 5, we'll look at how the application of the scientific method can define, refine, and facilitate your code-debugging efforts. In subsequent chapters, we will assume the context of an XP development method and discuss the most common types of bugs that remain in such an environment.


of 168

Chapter 5: The Scientific Method of Debugging We explain that effective debugging is best carried out in the manner dictated by the scientific method.

Software as Immortal Machine

A software system is a type of machine that can be run, stopped, and modified. But unlike other machines, software possesses a very special form of elegance: every copy is exactly the same and its parts never wear down. Granted, the hardware on which it runs may eventually degrade with time. But the software is not really a physical system—it's a system of designed relationships.

Because a software system doesn't wear with time, it continues to act precisely in the manner we have told it to. We, the programmers, are solely responsible for any misbehavior. Since there is an absence of extraneous noise in the system (as well as no physical wear to deal with), narrowly focused reasoning about a software system, considering only the properties of that system in isolation from the rest of the world, can pay off as an appropriate way to deal with programming issues. The payoff can be much higher than in other contexts in which we are less certain about the facts we have to work with.

Tip Because a software system doesn't physically degrade, any erroneous behavior

is the sole responsibility of the programmers.

For just one example of how narrow reasoning doesn't work so well in other contexts, consider how the following statistical information could be interpreted:

Suppose you plan to open a martini bar, and so as to optimize your inventory on the opening night, you're trying to determine what percentage of people prefer gin martinis and what percentage prefer vodka martinis. You decide to experiment, so you take a survey and discover that, sure enough, approximately 61.27% of your respondents prefer vodka martinis. That must mean that more people prefer vodka martinis than gin martinis—maybe.

It might also mean that your random sample is not indicative of the percentage among the whole population. Or it might be that 15% of your respondents simply filled in the top answer ("vodka") on your survey without really thinking about it. Or that 5% of your respondents accidentally filled in the wrong circle, or ... you get the idea. At any rate, your sample population was wrong; on your first night in business, more people ordered gin martinis than vodka martinis.

In the wonderful world of software debugging, there is no noise in the data. If our program exhibits a bug, there is a problem. When we examine the value of some variable at runtime and it turns out to be 8 when it should be 12, it's not just the result of a bad sampling—we know that the value is not being set correctly. The ability to deduce properties of programs with unswerving certainty is our greatest asset when fixing programs. Sherlock Holmes should be the patron saint of software debugging.

Tip In the wonderful world of software debugging, there is no noise in the data. That

puts the responsibility for system behavior squarely in the lap of the programmer.

�

How Much Does Your Software Weigh?

I am reminded of a story told to me by Professor Claire Cardie of the AI laboratory at Cornell University. She was once involved in a team to develop guidance software for the space shuttle. NASA required every project developing technology for the shuttle to determine how much its contribution weighed. For weeks, Claire's team insisted that its software didn't weigh anything! But NASA didn't buy it. One day, someone proposed that they could store the software on old-


of 168

fashioned punch cards, and weigh the resultant stack of cards (that weight would still be insignificant compared to the weight of the space shuttle). But then someone pointed out the problem with that approach: the software doesn't consist of the cards, it consists of the holes.

Ultimately, it was decided to simply enter the smallest weight NASA would allow.

�

�

As I mentioned in Chapters 3 and 4, the act of diagnosing a software bug has a lot in common with performing a scientific experiment. In both cases, we want to take the following steps:

1. Observe the system under investigation. 2. Form a hypothesis to explain the observations. 3. Test this hypothesis with new experiments. 4. Repeat until you arrive at a hypothesis that explains all of the observed behavior.

Now, this is a reasonably good approximation of what a scientist does when experimenting, and, historically, it has worked pretty well. But for scientists, complications often arise due to the inevitability of noise in the data, faulty or inaccurate measuring devices, or other reasons. The good news for a software debugger is that in the programming world, these complications simply don't exist; the "scientific method" we described applies even better to software than it does to a physical system! By practicing this method, we can be quite effective in diagnosing bugs.

The most important part of this process (and the part that is most often overlooked) is the development of a hypothesis as to why the system is behaving as it is. To do this effectively, you need to form a model of how each component of the system interacts with the others. Some of these components will be modeled only at a high level. Others (the ones believed to be the direct cause of a bug) will be modeled quite precisely.

As new tests are performed on the system, it is essential that the programmer updates this model to account for the results of those tests. Again, since we are dealing with a digital system, it's never okay to ignore a piece of data when forming a model. If your model doesn't explain even one small test result, your model is incorrect. There is no room for shades of gray in the black-and-white world of software development.

This last point is perhaps the single most important lesson of debugging. The one small result that was ignored is inevitably the key to the true problem with the system. Don't be fooled into inferring, from the fact that some anomalous behavior is hardly noticed, that the implications on the structure of the system are also miniscule. In the world of software, small bugs can result from big problems with the source code. Similarly, small features of the source code can have large and nonlocal impact on the observed behavior of the whole system.

�

Small Anomalies, Big Problems

Here's one example of how a small anomaly can be the cause of much bigger problems.

In an early version of DrJava, my Ph.D. advisor, Robert Cartwright (we call him "Corky"), issued a bug report indicating that one of his Java files was displaying improperly when opened. Mangled text was appearing in the file. Because Corky's code was written in an experimental extension of the Java language, we immediately assumed that the problem was related to displaying this extended language.

I tried reproducing the bug while running Windows. I opened up the bug report in Internet Explorer, copied the text of the file, and pasted it into DrJava. The text displayed just fine. I tried eliminating all uses of the experimental language features; that action had no effect.


of 168

Next, because the bug originally occurred on Linux, I tried reproducing it on that platform. I opened the bug with Netscape Navigator. (Navigator wouldn't let me open the Java file directly in the browser, so instead I saved the file directly to disk.) Eureka! When I opened the saved file on Linux, I was able to reproduce the bug. Did that mean the bug was specific to the Linux JVM I was running?

Not so fast. When Corky rebooted his machine to Windows and opened the file, he was able to reproduce the bug, so we knew it wasn't Linux-specific. That left us in a quandary. Why couldn't I reproduce the bug on Windows?

Suddenly, a small discrepancy blazed up in my head! I remembered that I had transferred the file into DrJava differently on the two different platforms. On Windows, I had copied and pasted it out of Internet Explorer. On Linux, I had explicitly saved the file from Navigator. This seemed like a minor detail, but as soon as I directly saved the file on Windows, I was able to reproduce the bug on that platform too.

This provided a key clue about the true cause of the problem: the offending file contained tabs! But when I copied and pasted the file out of Internet Explorer, the tabs were converted to spaces.

Normally, DrJava inserts only spaces into a file. When opening a file, DrJava converts all tabs into spaces. In this broken version, it converted each tab to two spaces. Because this was done in a subclass of javax.swing.text.PlainDocument, it broke an undocumented invariant in that Swing class. This invariant was discovered by Brian Stoler, the lead developer of DrJava at the time. In his words:

The standard Swing code that handles reading from a file into a Document (in DefaultEditorKit.read()) assumes that each chunk it inserts will have the effect of changing the length [by] exactly the same amount as its argument. I proved this by changing the tab remover to replace it with only one space; that got rid of the problem!

So, now, DrJava inserts only one space per tab (but you can easily reformat the code in your file once it's opened).

As this example demonstrates, it's the details that are relevant when diagnosing bugs. The minor detail as to how the text was transferred into DrJava when trying to reproduce the bug provided the key clue as to what the underlying problem was.

Another point that this example illustrates: don't overlook the simple explanations when trying to explain a bug. We were sure that the problem had something to do with the complexities of rendering text from an extended version of Java, but really the problem was just with tabs. More often than not, bugs are caused by simple problems. Bug Patterns Help Diagnose Bugs More Quickly

Sherlock Holmes, the world's greatest detective and master debugger, was so adept at his trade partly because he possessed an unparalleled degree of rationality. But debugging software, like thwarting crime, involves more than just an unswervingly rational mind. It also involves knowledge of a great many specific cases of bugs, their causes, and their remedies.

Expert ability in any field is not just the result of raw intelligence. It's also the result of a great deal of experience. That experience provides many specific examples that can be generalized into patterns.

Experts apply these patterns to new situations and reason about them more effectively. In particular, they can disregard irrelevant details more quickly and focus on what's important. Consider the following behavior of a variety of experts:


of 168

� Chess masters study numerous famous games and openings. The knowledge they acquire isn't simply rote memorization of the moves of those games. Instead, they extract patterns and principles and apply them to new games.

� Doctors study more than just anatomy and disease, but also numerous case studies, from which they extract patterns to apply to new cases.

� Astronaut crews spend countless hours running through drill after drill of systems failures on simulated missions. It's unlikely that the particular sequence of failures occurring on a simulated mission would occur during the real thing, but the astronauts extract patterns from this training that allow them to respond to a variety of problems more quickly.

� Historians study the political events of the past, extract patterns, and apply the knowledge when recommending actions to take in the future. Economists perform a similar function.

� Architects were the first to actually record various patterns of their discipline directly. New architects have been able to study these patterns and quickly learn from the years of experience of numerous past architects. As long as the rationale behind the patterns is taught with the patterns, this style of teaching can be an extremely efficient way to transfer knowledge to new students. (For more on teaching new programmers, see the section entitled "Learning in a Fast-Paced World" in Chapter 1.)

Software engineers followed this behavior, too. They started copying the approach of the architects with the book Design Patterns: Elements of Reusable Object-Oriented Software (Gamma et. al., 1994). In that book, much of the acquired wisdom of the software community concerning how to design new systems was condensed into a set of patterns, with discussions as to when each pattern was applicable. This was followed by several books on anti patterns, providing negative examples to complement the positive. But all of these patterns are at the level of design. There are patterns involved with programming that occur below this level.

Sometimes, it's possible to identify a recurring bug pattern, complete with common symptoms, causes, and remedies. If you're aware of the bug pattern, you can then identify occurrences of that pattern more quickly and fix them.

In the case of simple typos, which occur in extraordinary variety, there aren't many common patterns of behavior that result. Fortunately, if you're programming with XP techniques, the occurrences of undetected typos should be extremely rare. That's good, because bugs resulting from typos can be just as difficult to diagnose as much more subtle bugs.

The more subtle bugs to which I am referring generally scuttle at the edge of coding and design. They involve more than just mistyping a name, but they aren't problems purely at the level of class diagrams; there's a lower-level problem with the code.

By examining these bug patterns, we can do more than just identify them; we can also consider what coding practices could help prevent recurrences of each pattern in the future. We will then have a much better appreciation of that coding practice than we would have if the methods were simply recommended for no reason at all.


of 168

A Quick Recap

As the first few chapters have shown, consideration for the debugging process can have a tremendous impact on the high-level method we use to develop software, as well as the priorities we set when designing it.

In this chapter, we've described the scientific method: 1. Observe the system. 2. Form a hypothesis to explain the observations. 3. Test the hypothesis with new experiments. 4. Repeat until you arrive at a hypothesis that explains the observed behavior in its entirety.

And we've demonstrated how you can use the scientific method to define, refine, and facilitate your code-design and -debugging efforts.

In Chapters 6-21, we'll discuss 13 common types of bug patterns: how to identify them, how to fix them, and how to prevent them. Once we're familiar with these patterns, we'll examine the impact of what we've learned on the program design techniques we choose.


of 168

Chapter 6: About the Bug Patterns We discuss why knowledge of patterns is important to effective creation, implementation, debugging, and maintenance of software; we explain why these patterns have been chosen (and why they are important to your coding efforts); we cover how the patterns are dissected in this book; and we offer a table of "symptoms" for those programmers and designers who want to jump right in and play detective.

Why Is It Important to Know Patterns?

As mentioned earlier (in Chapter 1), bug patterns are recurring relationships between underlying bugs and signaled errors in a program. Knowledge of these patterns and the symptoms they demonstrate is a tool that will let the programmer not only quickly identify occurrences of a bug, but will help him prevent bugs occurring in the first place.

Just as a good programmer needs to know design patterns—patterns which can be combined and applied in various contexts—a good debugger needs to know the common causes of bugs, and how to fix them.

Bug patterns are related to anti-patterns, which are patterns of common software designs that have been proven to fail time and again. Such "negative examples" of design are an essential complement to traditional, positive design patterns. But while anti-patterns are patterns of design, bug patterns are patterns of erroneous program behavior correlated with programming mistakes. The concern is not with design at all, but with the coding and debugging process.

The concept of using patterns, both positive and negative ones, is not novel to programming. Physicians rely on similar types of recurring relationships when diagnosing disease. The problem is that it can take many years for programmers to learn to recognize these patterns through experience alone.

If we identify such patterns and teach them explicitly, we can leverage the experiences of many programmers to improve the effectiveness of each. Why These Bug Patterns?

So why did I choose the particular patterns outlined in this book? Why should these bug patterns be of more use to you than others?

Simple. These patterns are among the most frequently encountered. Many of them are introduced into software by common programming errors, whether they are misunderstandings in the conceptual methods used to direct code behavior, or just entry typos. How the Patterns Are Organized

In this book, to aid in identifying and classifying the following bug patterns, we'll summarize each description using the following form (borrowing some terminology from the medical establishment):

� Pattern name. An easy-to-identify name for the bug.

� Symptoms. Behaviors or results that clue you in to the presence of a bug.

� Causes. The actions, inactions, misdirections, or errors that may be causing the problem.


of 168

� Cures and preventions. Methods to fix an existing problem and techniques to keep the problem from raising its annoying head.

Along with each pattern, I'll also discuss broader programming principles and language design issues—such as language expressiveness, static checking, class design, and so on—that may either contribute to or help minimize the occurrence of the pattern. Code samples will be included with each pattern to help the illuminate the problem and ways to correct it.

A Quick Reference for Troubleshooting

The list of symptoms in Table 6-1 can be used to lead you to the potential cause (and cure) of a problem that may currently be troubling you. You can also use it as a reference over the life of this book to help you quickly diagnose and fix problems that occurrences of these patterns may create. Table 6-1: A Concept-Oriented Troubleshooting Guide

FOR THIS CONCEPT ... SEE THIS PATTERN ...

adding new methods Broken Dispatch, Chapter 14

assertion (definition and types) Fictitious Implementation, Chapter 17

assertion (determining place in interface implementation)

Fictitious Implementation, Chapter 17

assertion (overhead) Fictitious Implementation, Chapter 17

avoiding type mismatches with if-then-else block

Impostor Type, Chapter 15

best time to check the view properties Liar View, Chapter 12

changes to the stop() method Orphaned Thread, Chapter 18

checking for initialization without casting at runtime

Run-On Initializer, Chapter 19

checking the data on input | existing data Saboteur Data, Chapter 13

class fields | null values | initialization Run-On Initializer, Chapter 19

class state Run-On Initializer, Chapter 19

ClassCastException | casting | null values

Null Pointers, Chapter 8, Dangling Composite, Chapter 9, Null Flag, Chapter 10, Double Descent, Chapter 11

ClassCastException | recursive data descent

Double Descent, Chapter 11

ClassCastException | Visitor() method

Rogue Tile, Chapter 7

ClassCastException avoidance by wrapping cast in an instanceof check


clean-up code Split Cleaner, Chapter 16

code crash | same task, different data Saboteur Data, Chapter 13


of 168

Table 6-1: A Concept-Oriented Troubleshooting Guide


code dispatch after a method call Double Descent, Chapter 11

code extension Split Cleaner, Chapter 16

commenting invariants in code Double Descent, Chapter 11, Fictitious Implementation, Chapter 17

constructor signatures | legacy code Run-On Initializer, Chapter 19

constructor signatures Run-On Initializer, Chapter 19

constructor Broken Dispatch, Chapter 14, Run-On Initializer, Chapter 19

copying code Rogue Tile, Chapter 7

copy-and-paste errors Rogue Tile, Chapter 7

data corruption (automatic file generation) Saboteur Data, Chapter 13

data corruption (manual editing) Saboteur Data, Chapter 13

data corruption (semantics) Saboteur Data, Chapter 13

data corruption (syntax) Saboteur Data, Chapter 13

data-input code Saboteur Data, Chapter 13

determining version of method to invoke Fictitious Implementation, Chapter 17

determining whether an instance is initialized


discovering corrupt data before a crash Saboteur Data, Chapter 13

discrepancies in testing and runtime results

Liar View, Chapter 12

documenting invariants Fictitious Implementation, Chapter 17

easily automating GUI tests Liar View, Chapter 12

equals() Double Descent, Chapter 11

execution paths Split Cleaner, Chapter 16

expressiveness | type signatures | unit tests


extension Split Cleaner, Chapter 16

freeing resources Split Cleaner, Chapter 16

general method constructor overloaded with a more specific method

Broken Dispatch, Chapter 14

GUI design pitfalls Liar View, Chapter 12

GUI passes tests, fails in real world Liar View, Chapter 12

how GUI tests work Liar View, Chapter 12


of 168



ID semantically corrupt data | type checking

Saboteur Data, Chapter 13

including exception-throwing methods | initialization


inconsistent code on various JVMs Platform-Dependent Bugs, Chapter 20

inconsistent code on various operating systems

Platform-Dependent Bugs, Chapter 20

inconsistent code on various versions of a JVM

Platform-Dependent Bugs, Chapter 20

infesting code with tests Broken Dispatch, Chapter1 4

initialization | class fields | null values Run-On Initializer, Chapter 19

initialization and new contexts Run-On Initializer, Chapter 19

initialized/uninitialized fields Run-On Initializer, Chapter 19

input with large data structure Saboteur Data, Chapter 13

instanceof Double Descent, Chapter 11

interface semantic requirements Fictitious Implementation, Chapter 17

interface Fictitious Implementation, Chapter 17

invariants Double Descent, Chapter 11, Fictitious Implementation, Chapter 17

IsInitialized() method Run-On Initializer, Chapter 19

iteration of data Saboteur Data, Chapter 13

Java Robot class Liar View, Chapter 12

Leaf class | Leaf nodes | null values | Branch | ClassCastException | casting


legacy code | constructor signatures Run-On Initializer, Chapter 19

limited specification | extra invariants | interface


memory leaks Split Cleaner, Chapter 16

method arguments don't match Broken Dispatch, Chapter 14

Model-View-Controller architecture (MVC) Liar View, Chapter 12

modified method Rogue Tile, Chapter 7

multiple identical instances of the same class.


multithreaded code freezes up | (does not) Orphaned Thread, Chapter 18


of 168


FOR THIS CONCEPT ... SEE THIS PATTERN ... print to standard error

null pointers Dangling Composite, Chapter 9

null values | class fields | initialization Run-On Initializer, Chapter 19

null values | Leaf class | Leaf nodes | Branch | ClassCastException | casting


NullPointerException. Null Pointers, Chapter 8, Dangling Composite, Chapter 9, Null Flag, Chapter 10, Run-On Initializer, Chapter 19

overloaded methods Broken Dispatch, Chapter 14

overloading method #1, breaking method #2

Broken Dispatch, Chapter 14

parsing | compilers Saboteur Data, Chapter 13

passing unit tests, failing program Liar View, Chapter 12

persistent bug Rogue Tile, Chapter 7

previously corrected bug Rogue Tile, Chapter 7

recursive data descent | ClassCastException


recursively defined data types Dangling Composite, Chapter 9

refactoring (other components) Liar View, Chapter 12

refactoring (perpetual) Liar View, Chapter 12

representing default values Run-On Initializer, Chapter19

resource clean-up code Split Cleaner, Chapter 16

restricting specification of interface invariants to type signatures


semantic requirements Fictitious Implementation, Chapter 17

single recursive call Double Descent, Chapter11

single-threaded vs. multithreaded design Orphaned Thread, Chapter18

special classes representing default values | performance hit

Run-On Initializer, Chapter19

specification of an interface Fictitious Implementation, Chapter 17

specification- vs.

implementation-related bug

Bugs/Specs/Implement, Chapter 2, Platform-Dependent Bugs, Chapter 20

specification/implementation bug round-up Platform-Dependent Bugs, Chapter 20

splitting a text line into two Strings Saboteur Data, Chapter 13


of 168



statement execution order Run-On Initializer, Chapter 19

static type system Impostor Type, Chapter 15

stop() method Orphaned Thread, Chapter 18

system resources Split Cleaner, Chapter 16

tracing through execution paths Split Cleaner, Chapter 16

treating all types of data as the same type Impostor Type, Chapter 15

TreeVisitor | ClassCastException | Visitor() method


TreeVisitor interface | accept() methods


type | value field | class Rogue Tile, Chapter 7

type checking | identifying semantically corrupt data

Saboteur Data, Chapter 13

type mismatches Impostor Type, Chapter 15

uninitialized/initialized fields Run-On Initializer, Chapter 19

unit tests | interface implementation | all possible inputs


unit tests and runtime behavior don't match

Liar View, Chapter 12

unit tests Broken Dispatch, Chapter 14, Fictitious Implementation, Chapter 17

using automated tests to check GUI code Liar View, Chapter 12

using instanceof and equals() to check for class and object identity


using tags in special fields to distinguish data types


using the static type system to distinguish data types


value field | class | type Rogue Tile, Chapter 7

variable assigned to null Null Pointers, Chapter 8

Visitor() method | ClassCastException | TreeVisitor


why so many unit tests Liar View, Chapter 12

won't recognize some data types Impostor Type, Chapter 15


of 168

Chapter 7: The Rogue Tile

Overview

When a program keeps exhibiting erroneous behavior after you've repaired it, you may have a Rogue Tile. We discuss programmer errors and original design errors as potential causes.

The Rogue Tile pattern is a bug pattern that all developers strive to minimize, but, as we'll demonstrate, it's not always an easy matter to eliminate the potential for such rogue tiles in a statically typed language like Java.

Here's our first bug pattern in a nutshell:

� Pattern: Rogue Tile.

� Symptoms: The code seems to act as if a previously corrected bug is still there.

� Cause: At least one copy of a copy-and-paste code fragment still contains a bug fixed in the other copies.

� Cures and Preventions: Factor out the common code, if possible; otherwise, update it. Avoid copying and pasting code

About This Bug Pattern

Perhaps the most common bug pattern—seen by all and often caused by beginning programmers—results from copying and pasting a block of code from one section of a program to another. Sometimes, small parts of the copy are changed because of slightly different functional requirements. For example, you may have written a tokenizer for the configuration file of an application. Later on, you may find that it is almost exactly the tokenizer you want when interactively reading user input for an unrelated application. Ideally, you would create a new class hierarchy of tokenizers, factoring out the common code into a new base class, and creating new subclasses for each use of the tokenizer. But that would require refactoring the old application; at the very least, you would have to change all references to the original tokenizer into references to the new subclass. Perhaps you don't have the time to do all of that refactoring, or perhaps you no longer have authority to modify the original code base. Or perhaps you're just being lazy that day. Whatever the reason, rather than doing all of this refactoring, you decide to just copy the tokenizer into a new package and tweak the parts that need tweaking.

Inevitably, bugs in the shared code will be found and fixed in one copy but not the other, leaving you scratching your head when the symptoms of errors that you have eliminated recur.

Although programmers quickly become familiar with this pattern of bug, few take appropriate measures to minimize its occurrence. It's always tempting to take a break from thinking and simply copy code that you believe to be working, but the productivity lost from fixing bugs due to indiscriminate copy-and-paste actions quickly dwarfs the productivity gained from copying the code.

I call this the Rogue Tile pattern because the various copies of a code block can be thought of as "tiles" covering the program. As the code in the various copies diverges, the copies become "rogue tiles."

The Symptoms


of 168

The most common symptom of this pattern of bug is a program that continues to exhibit erroneous behavior after you believe you've fixed the code causing that behavior.

Cause, Cures, and Prevention

To understand how this can happen, let's consider the following class hierarchy for binary trees: Listing 7-1: A Common Binary Tree Class Hierarchy �

abstract class Tree {}

class Leaf extends Tree {

private Object _value;

public Leaf(Object value) {

_value = value;

}

public Object getValue() {

return _value;

}

}

class Branch extends Tree {

private Object _value;

private Tree _left;

private Tree _right;

public Branch(Object value, Tree left, Tree right) {

_value = value;

_left = left;

_right = right;

}

public Object getValue() {

return _value;

}

public Tree getLeft() {

return _left;

}

public Tree getRight() {


of 168

return _right;

}

}

�

�

The first thing to notice about these classes is that both concrete classes contain a value field of type Object. If you decide later to make trees containing, say, Integers, you might forget to update one of these field declarations.

If some other part of the program were to expect these fields to be Integers, the program likely would not compile. You'll probably remember that you changed the type of the value field in one of the classes, but you might overlook the fact that you did not make the change in the other.

Of course, this simple example is one that a beginning programmer would quickly learn to avoid by factoring out the common code. In this case, the field declaration should be moved to class Tree. Both subclasses will then inherit this field, and any changes to the field declaration need only occur in one place.

In order to make that work, we'll add an accessor method to class Tree and modify all field references to _value in Leaf and Branch to invocations of the accessor instead. Otherwise, the subclasses won't have access to it.

This is a typical example of the tension between encapsulating code and keeping a single point of control for each functional aspect of a program.

Note There is a tension between encapsulating code and keeping a single point of

control for each functional aspect of a program.

Sometimes factoring out code is not so simple a task, especially when it involves factoring out not just common data, but common functionality as well. For example, suppose we factor out the _value field in our example above and change its static type to Integer. We could then write methods for adding and multiplying all the nodes in a Tree as follows: Listing 7-2: Factoring Out Common Code from Listing 7-1. �

abstract class Tree {

private Integer _value;

public Tree(Integer value) {

_value = value;

}

public Integer getValue() {

return _value;

}

public abstract int add();

public abstract int multiply();


of 168

}


public Leaf(Integer value) {

super(value);

}

public int add() {

return getValue().intValue();

}

public int multiply() {


}

}


private Tree _left;


public Branch(Integer value, Tree left, Tree right) {

super(value);

_left = left;

_right = right;

}


return _left;

}


return _right;

}

public int add() {

return getValue().intValue() + _left.add() + _right.add();

}



of 168

return getValue().intValue() + _left.add() * _right.add();

}

}

�

�

You may have noticed that there's a bug in this code. It's inside the multiply() method for class Branch. It adds the first term instead of multiplying by it.

The error occurred because I created the multiply() method by copying the code from the add() method and making slight (but incomplete) alterations.

This bug is particularly insidious because calling the multiply() method will compile just fine and will never signal an error. In fact, in many cases it will return what appears to be a perfectly reasonable result.

Just as before, we can minimize bugs of this sort by factoring out the common code. In this case, we could write a single method that accumulates an operator (passed as an argument) over a Tree. We can use a design pattern (not a bug pattern!) known as the Command pattern to encapsulate this operator in an object: Listing 7-3: Using a Design Pattern to Factor Out Common Code (Correctly) �

abstract class Operator {

public abstract int apply(int l, int r);

}

class Adder extends Operator {

public int apply(int l, int r) {

return l + r;

}

}

class Multiplier extends Operator {

public int apply(int l, int r) {

return l * r;

}

}

�

�

Then we can modify the code in our Tree classes as follows: Listing 7-4: Modifying Class Code After Common Code Is Factored Out �


private Integer _value;


of 168

public Tree(Integer value) {

_value = value;

}

public Integer getValue() {

return _value;

}

public abstract int accumulate(Operator o);

public int add() {

return this.accumulate(new Adder());

}


return this.accumulate(new Multiplier());

}

}


public Leaf(Integer value) {

super(value);

}

public int accumulate(Operator o) {


}

}


private Tree _left;


public Branch(Integer value, Tree left, Tree right) {

super(value);

_left = left;

_right = right;

}


of 168


return _left;

}


return _right;

}

public int accumulate(Operator o) {

return o.apply(getValue().intValue(),

o.apply(_left.accumulate(o),

_right.accumulate(o)));

}

}

By factoring out the common code, we eliminated the possibility of a copy-and-paste error occurring in the method bodies of add() and multiply(). Also, notice that we no longer need separate add() and multiply() methods for each subclass of Tree.

The Command Pattern Design

As mentioned in the paragraph introducing Listing 7-3, we used a design technique known as the Command pattern to encapsulate an operator (multiply, add, etc.) in an object.

The Command pattern allows us to encapsulate an operation on data as data itself. That way, other objects can send and receive it, and apply it as needed.

The key to this pattern is to define a special interface, Command, with a single method declaration that I will call apply():

public interface Command {

public Object apply(Object o1, Object o2);

}

An implementation of this interface defines apply() differently depending on the particular operation it represents. For example, here is an implementation that concatenates the String representations of its arguments:

class Concatenator implements Command {

public Object apply(Object o1, Object o2) {

return o1.toString() + o2.toString();

}

}

Of course, we would need a separate Command interface for each distinct operation signature.

For more information on design patterns, see the Resources chapter.


of 168

Other Obstacles to Factoring Out Code

Factoring out common code is a good practice, but it can't be done in all cases. And, because it's not always an easy matter to actually eliminate the potential for such rogue tiles in a statically typed language like Java, additional considerations can be useful in determining a way around the original design of a language. Two such considerations are generic types and aspect-oriented programming.

Generic Types

The simplicity of the Java type system often forces us to choose between taking advantage of static checking and keeping a single point of control for each distinct functional aspect of a program.

For example, suppose we were to write a Visitor over the Tree class hierarchy described in the previous listing. A Visitor is a design pattern that allows for methods to be added to a composite class hierarchy without actually modifying the classes in that hierarchy. A Visitor class will hold methods for each constituent class. These constituent classes will then contain accept() methods that take a Visitor and invoke the appropriate method on it.

We could add Visitors to our Tree hierarchy from the previous listing as follows: Listing 7-5: Adding Visitors to the Tree Hierarchy

�

interface TreeVisitor {

public Object forLeaf(Leaf that);

public Object forBranch(Branch that);

}

// in class Tree

public abstract Object accept(TreeVisitor that);

// in class Leaf

public Object accept(TreeVisitor that) {

return that.forLeaf(this);

}

// in class Branch

public Object accept(TreeVisitor that) {

return that.forBranch(this);

}

�

Notice that the return types of the TreeVisitor methods are all necessarily of the type Object. We can't narrow this return type without limiting the generality of interface TreeVisitor. So, we're left with two options:

1. Use the same TreeVisitor interface for all occasions and insert casts as appropriate with each Visitor method invocation. That's not just a lot of work, it effectively


of 168

circumvents the static type system and it makes it much more likely that we will experience a ClassCastException at runtime.

2. Construct a separate TreeVisitor interface with separate accept() methods for each return type we want to use. But that's exactly how rogue tiles are formed.

To be sure, any statically typed language will limit expressiveness and will, to some degree, impair our efforts to prevent rogue tiles. But Java doesn't have to be as bad as it is in this regard.

Note Any statically typed language will limit expressiveness and will, to some

degree, impair our efforts to prevent rogue tiles.

A more powerful type system, such as that implemented by the Sun JSR-14 prototype compiler, would alleviate this issue significantly. In the JSR-14 compiler, the Java type system is augmented with what are known as generic types, or parametric polymorphism. Parametric polymorphism allows us to add type parameters—declared in angle brackets in the class header—that can be instantiated when a particular instance of the class is made.

Note Parametric polymorphism allows us to add type parameters—declared in angle

brackets in the class header—that can be instantiated when a particular instance of the class is made.

For example, in our Tree hierarchy, it would make sense to parameterize the contained value of class Tree. Then we could instantiate that value type in different ways depending on the circumstances.

Also, we may want to parameterize our Visitor class by the return type of the for-() methods. That way, when we construct a concrete Visitor class for a particular circumstance, the Visitor can extend an instantiation of the abstract Visitor class with the appropriate return type. Listing 7-6: A Parametric Version of Interface TreeVisitor �

interface TreeVisitor<ReturnType> {

public ReturnType forLeaf(Leaf that);

public ReturnType forBranch(Branch that);

}

�

One limitation of JSR-14 is that it does not allow run-time type operations (such as casts, instanceof tests, etc.) on generic types. At the JavaPLT laboratory at Rice University, we have extended the language implemented by JSR-14 to include full support for these run-time type operations. The extended language is referred to as NextGen. Because NextGen is implemented in a manner that is compatible with existing compiled binaries, we are hopeful that, eventually, it will be incorporated into the Java language.

This is just a taste of what can be done with generic types. For more information on them, see the References chapter of this book.

Aspect-Oriented Programming

Static typing isn't the only way that a language like Java can prevent us from eliminating rogue tiles. Other constraints on the language, such as the requirement that all checked exceptions be caught or declared to be thrown, force us to include code in our methods that simply can't be factored out.


of 168

One mechanism for handling such duplicated code is aspect-oriented programming. In aspect-oriented programming, a development tool systematically inserts code you specify into methods to maintain global properties, allowing you to organize a program according to various aspects rather than simply by classes or functions. These aspects correspond to global properties of the program, such as the way that checked exceptions are handled in methods or the way that fields are assigned in constructors.

There are several existing tools for adding aspect-oriented programming to Java. One popular tool is AspectJ. For more information, refer to the References chapter.

What We've Learned

In this chapter on the Rogue Tile bug pattern we've, learned the following: � Rogue Tile is the most common bug pattern and often results from copying and pasting a

block of code from one section of a program to another. � Cut-and-pasting "bug-free" code may not be the work-saving option you think it is. � Environmental variables may cause cut-and-paste code to work correctly in some places

but not in others. � Factoring out common code can be a preventative measure, but it may not be an easy

task. � There is a tradeoff between encapsulating code and keeping a single point of control for

each functional aspect. � Statically typed languages tend to limit expressiveness, thereby reducing our

effectiveness in preventing rogue tiles. Generic types have the potential to counter this drawback of Java to a certain extent.

� The requirement that all checked exceptions must be caught or declared as thrown means that some of the code in methods cannot be factored out.

� Aspect-oriented programming—adding a program's "aspects" to its classes and functions as a way of organizing it—can help handle duplicated code. (Aspects correspond to the global properties of a program, such as the way checked exceptions are handled in methods.)


of 168

Chapter 8: Null Pointers Everywhere! This chapter is a quick interlude to expose you to null pointers and the NullPointerException, a frequently occurring cause of several bug patterns.

One of the most common (and most commonly complained about) symptoms of bugs in Java programming is a thrown NullPointerException. Tracking down the cause of such exceptions can truly make you question your career decision. They provide no useful information as to why they were thrown and they are difficult—perhaps impossible in many cases—to predict.

They're Uninformative

Of all the exceptions a Java programmer might encounter, the NullPointerException is among the most dreaded, and for good reason: it is one of the least informative exceptions that a program can signal.

Unlike, for example, a ClassCastException, a NullPointerException says nothing about what was expected instead of the null pointer. Furthermore, it says nothing about where in the code the null pointer was actually assigned.

In many NullPointerExceptions, the true bug occurs where the variable is actually assigned the null value. To find the bug, we have to trace back through the flow of control to discover where the variable was assigned and determine whether doing so was incorrect. This process can be particularly frustrating when the assignment occurs in a package other than the one in which the error was signaled. They're Elusive

Many Java developers note that the vast majority of program crashes they encounter are NullPointerExceptions; they long for a tool that would statically identify these bugs before the program is run. Unfortunately, automata theory— the study of machines, robots, or systems which follow a preset sequence of instructions automatically—tells us that no tool could ever statically determine exactly which programs will throw NullPointerExceptions.

But it is possible for a tool to rule out NullPointerExceptions for many expressions in a program, leaving us with just a few small potential trouble spots that we must then examine manually. In fact, several research efforts are now underway to provide just such a tool for Java programs (see the Resources chapter). But a good tool can only take us so far.

Note Automata theory says that no tool can ever statically determine exactly which

programs will throw NullPointerExceptions.

NullPointerExceptions will never be completely eradicated. When they do occur, it helps us to know the bug patterns associated with them so that we can diagnose them quickly. In addition, we can use certain programming and design techniques to significantly minimize the occurrence of these patterns.

Now, on to two bug patterns that list null pointers as their cause: the Dangling Composite and the Null Flag.


of 168

Chapter 9: The Dangling Composite

Overview

When code that uses recursively defined data types signals an exception and the base-case definitions are handled inconsistently by client code, there may be a Dangling Composite bug; we discuss the core of the problem, the NullPointerException.

Dangling composites result from defining a recursive data type in such a way that certain base cases of the definition are not given their own classes. Instead, null pointers are inserted into the various composite data types. Instances of the data type are then used as if these null pointers were filled in properly. I call this the Dangling Composite bug pattern because the offending code is an application of the Composite design pattern that is imperfect because the composite data types contain dangling references (that is, null pointers).

Here's this bug pattern in a nutshell:

� Pattern: Dangling Composite.

� Symptoms: Code that uses a recursively defined data type is signaling a NullPointerException.

� Cause: The recursive data type is defined in such a way that certain base cases of the definition are not given their own classes. Instead, null pointers are inserted into the various composite data types. The base cases are then handled inconsistently by client code.

� Cures and Preventions: Ensure that the base cases are represented and checked for consistency. Give each base case its own class.

�

The Composite and Singleton Design Patterns

As mentioned in this chapter, the Dangling Composite bug pattern is an incomplete application of the Composite design pattern.

The Composite pattern lets you build complex objects by recursively composing similar objects in a tree-like manner. This pattern also allows the objects in the tree to be manipulated in a consistent manner by requiring all of the objects in the tree to have a common superclass or interface.

The Singleton pattern is also referenced in this chapter as a potential optimization pattern. This pattern ensures that only one instance of a class is created. All objects that use an instance of that class will use the same instance.

For more information on design patterns, see the Resources chapter.

�


This is the first bug pattern relating to NullPointerExceptions that we'll explore. (For a quick intro to the symptom of this pattern—the exception thrown by dereferencing a null pointer—step back to Chapter 8.)

This pattern is a reflection of the Composite design pattern—it "dangles" because some of the references are null pointers instead of classes.


of 168

The Symptoms

Code that uses a recursively defined data type—that is, a data type that is defined so that in certain base cases of the definition null pointers, instead of classes, are inserted into the composite—signals a NullPointerException.

The Cause

Consider the following (ugly) implementation of singly linked lists, which represents empty lists as lists containing the null pointer in all of their fields. Just to show how insidious bugs of this pattern can be, we've already introduced a bug in the following code. See if you can spot it. Listing 9-1: A Dangling Composite in a Singly Linked Implementation of Lists

�

import java.util.NoSuchElementException;

public class LinkedList {

private Object first;

private LinkedList rest;

/**

* Constructs an empty LinkedList.

*/

public LinkedList() {

this.first = null;

this.rest = null;

}

/**

* Constructs a LinkedList containing only the given element.

*/

public LinkedList(Object _first) {

this.first = _first;

this.rest = null;

}

/**

* Constructs a LinkedList consisting of the given Object followed by

* all the elements in the given LinkedList.

*/

public LinkedList(Object _first, LinkedList _rest) {


this.rest = _rest;


of 168

}

}

�

This code is truly awful. Instead of defining a separate class for empty lists, it represents empty lists by putting a null pointer in both fields.

It may at first appear that representing empty lists in this way makes the code simpler. After all, we've saved the effort of having to define an extra class just for empty lists. But, as we will demonstrate, this simplicity is an illusion.

Let's define some getter and setter methods for this class.

Notice that the action taken by both of the getters depends on whether the list is empty. This is exactly the sort of if-then-else chain that a properly constructed class hierarchy prevents. Because of these chains, we are prevented from considering the behavior of these getters on a single type of list in isolation. Furthermore, if at some future date we needed a third type of list (an immutable list, say), we would have to go into every one of these methods and change the code.

Note Simplicity may be an illusion. The true measure of simplicity is how easily we

can avoid introducing bugs into the program.

By this measure, this LinkedList implementation (in Listings 9-1 and 9-2) fails miserably. In fact, as mentioned earlier, our LinkedList class already contains a subtle but devastating bug—did you spot it? Listing 9-2: Defining getter and setter Methods for LinkedList

�

public Object getFirst() {

if (! (this.isEmpty())) {

return this.first;

}

else {

throw new NoSuchElementException();

}

}

public LinkedList getRest() {

if (! (this.isEmpty())) {

return this.rest;

}

else {


}

}

public void addFirst(Object o) {


of 168

LinkedList oldThis = (LinkedList)this.clone();

this.first = o;

this.rest = oldThis;

}

public boolean isEmpty() {

return this.first == null && this.rest == null;

}

public Object clone() {

return new LinkedList(this.first, this.rest);

}

�

Exactly what is our representation of empty lists? We said earlier that an empty list is a LinkedList containing a null pointer in both fields. The zero-argument constructor sets up just such an empty list. But notice that the single-argument constructor does not place an empty list into the rest field, as would be necessary to construct a list of one argument. Instead, it places the null pointer into that field.

Because a Dangling Composite confuses null pointers with placeholders for base cases, mistakes like this are very easy to make. To see how this bug can manifest itself as a NullPointerException, let's write an equals() method for our lists: Listing 9-3: Writing an equals() Method for the Linked Lists

�

public boolean equals(Object that) {

// If the objects are not of the same class, then they are not equal.

// Reflection is used in case this method is called from an instance of a

// subclass.

if (this.getClass() != that.getClass()) {

return false;

}

else { // that instanceof LinkedList

LinkedList _that = (LinkedList)that;

return (this.isEmpty() && _that.isEmpty()) ||

((this.getFirst().equals(_that.getFirst())) &&

(this.getRest().equals(_that.getRest())));

}

}

�


of 168

First, a brief aside. The first thing that can go wrong with this equals() method is that we will get a NullPointerException if it is ever called on null. Now, that might be what you'd expect to happen; after all, null.equals(x) will signal an error for any x, so shouldn't x.equals(null) signal the same error in order to preserve symmetry? In fact, the API for equals() specifies that equals() should form an equivalence relation on all "reference values" (implying symmetry). And the Java Language Specification is pretty clear on the fact that reference values include null.

But the plot thickens. As developerWorks reader Paul Lewis has pointed out in the "Diagnosing Java Code" discussion forum, Sun has declared that x.equals(null) should return false for any non-null x. But this is in direct conflict with that part of the spec indicating that equals() should form an equivalence relation! What are we to do? It's impossible to implement an inconsistent specification; so, in a very real sense, we can't eliminate all bugs from our equals() method. This is a great example of the strong interdependence between bugs and specifications.

After I discussed this issue with Corky, he decided to inform Sun of the inconsistency in the equals() specification. In conversation, Guy Steele of Sun Microsystems has acknowledged the problem, and has proposed fixing it in Java 1.4.2. The fix he proposes is to stipulate that equals() should form an equivalence relation only on non-null reference values, and will continue requiring that equals() return false for non-null x.

If it were possible to start over, it would be preferable to stipulate that equals() implements a true equivalence relation for all reference values. Clients should be responsible for checking that they're not passing in null pointers to equals() methods, just as they are responsible for not calling equals() on null pointers. However, given the tremendous amount of legacy Java codes currently in production use, Sun's planned conservative fix to the spec is well justified.

So let's try to rewrite our equals() method to match the planned specification of equals() in 1.4.2. The first thing we have to do to fix our method is put a test at the beginning: Listing 9-4: Fixing the equals() Method with a Preemptive Test

�

// The semantics for equals specified in the Sun JDK 1.3 API require that

// we return false for x.equals(null), x non-null.

if (that == null) { return false; }

�

If control flows past this test, then, because neither this nor that is empty, the equals() method correctly expects that it can call getFirst() and getRest() on both of them without an error signal. But if either of these lists contains any part that was constructed using the bug-laden single-argument constructor, then the recursive call will eventually uncover a null pointer where an empty list should have been waiting. If the null pointer is encountered in this.getRest(), it will cause a NullPointerException.

But perhaps the worse case occurs when a null pointer is discovered in that.getRest(). Here, the method will simply return false as if nothing were wrong at all! Then we'd have to wait even longer for some subsequent error to be signaled before we had a clue that there was a problem.

One potential solution might be to simply represent empty lists directly as null pointers. However, this would be wholly inadequate, because it would then be impossible to remove the last element of a list or to insert an element into an empty list. Furthermore, it would make a .equals call on an empty list signal a NullPointerException.


of 168

On the other hand, the bug could be fixed by rewriting the single-argument constructor as follows: Listing 9-5: A Fix That Rewrites the Single-Argument Constructor �

public LinkedList(Object _first) {


this.rest = new LinkedList();

}

�

But, as is the case with most bug patterns, it is much better to prevent bugs of this kind rather than to fix them. The fact that the code broke so easily and that even simple getters, setters, and the equals() method were all so bloated, suggests that there is a better design.

Cures and Prevention

In fact, there is a simple way to avoid Dangling Composite bugs: give each base case of the data type its own class. Instead of implementing singly linked lists as we did earlier, implement such lists with a class LinkedList that contains a field holding either an Empty or a Cons class. These classes implement a common interface, as shown in Figure 9-1.

Figure 9-1: Classes implementing a common interface

To implement mutator() methods, the new LinkedList class serves as a container for an internal, immutable list, as shown in Listing 9-6. This approach is necessary because truly empty lists have no fields to mutate and are therefore immutable. Listing 9-6: The LinkedList Class Acts as a Container for an Immutable List

�

import java.util.NoSuchElementException;


private List value;

/**


*/

public LinkedList() { this.value = new Empty(); }

/**



of 168

*/

public LinkedList(Object _first) { this.value = new Cons(_first); }

/**



*/

public LinkedList(Object _first, LinkedList _rest) {

this.value = new Cons(_first, _rest.value);

}

private LinkedList(List _value) { this.value = _value; }

public Object getFirst() { return this.value.getFirst(); }

public LinkedList getRest() { return new LinkedList(this.value.getRest()); }

public void addFirst(Object o) { this.value = new Cons(o, this.value); }

public boolean isEmpty() { return this.value instanceof Empty; }





if (this.getClass() != that.getClass()) { return false; }

else { // that instanceof LinkedList

return this.value.equals(((LinkedList)that).value);

}

}

}

�

Implementation of the immutable lists is then straightforward, as shown in Listing 9-7. Listing 9-7: Implementing the Immutable Lists

�

interface List {}

class Empty implements List {



of 168

return (that != null) && (this.getClass() == that.getClass());

}

}

class Cons implements List {

Object first;

List rest;

Cons(Object _first) {


this.rest = new Empty();

}

Cons(Object _first, List _rest) {


this.rest = _rest;

}

public Object getFirst() { return this.first; }

public List getRest() { return this.rest; }






else { // that instanceof Cons

Cons _that = (Cons)that;

return (this.getFirst().equals(_that.getFirst())) &&

(this.getRest().equals(_that.getRest()));

}

}

}

�

The logic for each method is now considerably simpler. Also notice that while it is still possible to introduce the same bug as before in the singleargument Cons constructor, the fact that we have made an explicit Empty class makes this much less likely. Additionally, any code that traverses


of 168

our lists and forgets to check for the empty case will get a NoSuchElementException, as opposed to the much less helpful NullPointerException.

A simple optimization to this code would be to apply the Singleton design pattern to class Empty because every instance of Empty is identical. I have left out this optimization because it is not relevant to eliminating NullPointerExceptions and it makes the code slightly more complex. What We've Learned

In this chapter on the Dangling Composite bug pattern, we've learned the following: � A Dangling Composite results from defining a recursive data type so that certain base

cases of the definition are not assigned their own classes. � Null pointers in place of classes create the "dangling" references. � A Dangling Composite confuses null pointers with placeholders for base cases. � Representing lists directly as null pointers is not a solution since it would be impossible to

remove the last list element; it is also impossible to insert an element into an empty list. In addition, under these circumstances a .equals call on an empty list would signal a NullPointerException.

� The easiest cure is also a prevention: give each base case of the data type its own class. � Again, creating unit tests for the code can help eliminate the introduction of bugs into the

program.

We aren't finished with null pointer dereferences just yet. In Chapter 10, we'll take a look at yet another common bug pattern that manifests itself as a NullPointerException. You'll learn how to recognize this pattern and how to avoid it


of 168

Chapter 10: The Null Flag

Overview

When exceptional conditions throw exceptions that cannot be easily diagnosed, you may be encountering the Null Flag bug pattern; we discuss how to get your code to throw informative exceptions.

Exceptional conditions in Java programs are ordinarily dealt with by throwing exceptions and catching them at an appropriate point of control. But you will often see methods that indicate such a condition by returning a null value (and perhaps printing a message to System.err). If the calling method does not explicitly check for a null pointer, it may attempt to dereference the return value and trigger a NullPointerException. (For an introduction to null pointers and the NullPointerException, see Chapter 8.) We call this the Null Flag bug pattern. It produces unexpected results that are difficult to debug.

Here's the breakdown of this bug pattern:

� Pattern: Null Flag.

� Symptoms: A code block that uses null pointers as flags for exceptional conditions signals a NullPointerException.

� Cause: The calling methods are not checking for null pointers as return values.

� Cures and Preventions: Throw exceptions to signal exceptional conditions.

In Chapter 9, we demonstrated how the substitution of null pointers for various base types of data is one of the most common causes of NullPointerExceptions. In this chapter, we will show how substituting null pointers for exceptional conditions also tends to cause problems.

Note If the calling method does not explicitly check for a null pointer, it may trigger a

NullPointerException


Signaling exceptional conditions by returning null values is not a good idea. Doing so limits control flow to the ordinary paths of method invocation and return, and buries evidence of where an exceptional condition occurred.

As you might have guessed, I call this pattern the Null Flag bug pattern because it is caused by the inconsistent use of null pointers as flags for exceptional conditions.

The Symptoms

A code block that uses null pointers as flags for exceptional conditions signals a NullPointerException.


of 168

The Cause

Let's consider the following simple bridge class from BufferedReaders to Iterators: Listing 10-1: A Simple Bridge Class

�

import java.io.BufferedReader;

import java.io.InputStreamReader;

import java.io.IOException;

import java.util.Iterator;

public class BufferedReaderIterator implements Iterator {

private BufferedReader internal;

public BufferedReaderIterator(BufferedReader _internal) {

this.internal = _internal;

}


try {

boolean result = true;

// Let's suppose that lines in the underlying input stream are known

// to be no greater than 80 characters long.

internal.mark(80);

if (this.next() == null) {

result = false;

}

internal.reset();

return result;

}

catch (IOException e) {

System.err.println(e.toString());

return false;

}

}

public Object next() {


of 168

try {

return internal.readLine();

}


System.err.println(e.toString());

return null;

}

}

public void remove() {

// This iterator does not support the remove operation.

throw new UnsupportedOperationException();

}

}

�

Because this class serves as a bridge implementation of the Iterator interface, the code must catch IOExceptions from BufferedReader. Each of the methods handles IOExceptions by returning some default value. In the case of hasNext(), the value false is returned. This is reasonable: if an IOException is thrown, the client should not expect that another element can be retrieved from the Iterator.

On the other hand, next() simply returns null, both in the case of an IOException and—because it relies on the return value of internal.readLine()—in the case in which internal is empty. But this is not what the client of an Iterator object would expect.

Normally, when next() is called on an Iterator with no more elements, it throws a NoSuchElementException. If a client of our Iterator is relying on this behavior, it may likely attempt to dereference a null value returned from a call to next(), resulting in a NullPointerException.

Whenever a NullPointerException occurs, check for scenarios like the one described previously. Occurrences of this bug pattern are common.


Despite the fact that it happens so frequently, the use of null flags is often unwarranted. Let's rewrite next() so that it throws NoSuchElementException, as expected: Listing 10-2: Rewriting to Throw the Proper, Expected Exception

�

public Object next() {

try {

String result = internal.readLine();

if (result == null) {



of 168

}

else {

return result;

}

}


// The original exception is included in the message to notify the

// client that an IOException has occurred.

throw new NoSuchElementException(e.toString());

}

}

�

Notice that to make the rest of the code work with this altered method, we would also have to 1. Import java.util.NoSuchElementException. 2. Fix hasNext() so that it no longer calls next() to test. The simplest way to do this

would be to simply call internal.readLine() directly.

An alternative to our handling of IOExceptions would be to catch them and throw RuntimeExceptions in their stead. Base your decision to do this on an assessment of how frequently you expect IOExceptions are expected on your target platform. If they will be common, then you might want to try recovering from them.

Any code that calls this new next() method may have to deal with thrown NoSuchElementExceptions. Of course, the code may simply choose to ignore them and allow the program to abort. If so, the resulting error message and the place where the exception was thrown would be much more informative than a NullPointerException thrown by the original code.

If the exception thrown were a checked exception (such as IOException), it would be even more helpful, because no client code for the class would compile unless it handled the exception. In this way, we could eliminate even the possibility of certain errors occurring before the program is ever run.

But, in this example, we couldn't throw such a checked exception without violating the Iterator interface. So we've sacrificed some static checking for the sake of reusing code that operates on Iterators. Trade-offs such as these, between the goals of static checking and reuse, are quite common Robustness vs. Lack of Diagnostic Evidence

Before we leave this topic, let's address the concerns of the programmers who use null flags regularly. Programmers often argue that the use of null flags makes their programs more "robust." After all, a robust system handles varied situations gracefully rather than throwing exceptions at every little problem that comes up.

Handling Exceptions in the Best Place

What this argument overlooks is the fact that exceptions are a great tool to increase the robustness of code by allowing control to pass quickly during an exceptional condition to the


of 168

place most appropriate for controlling it. The use of null flags, on the other hand, limits control flow to the ordinary paths of method invocation and return—until the whole program crashes, of course.

Additionally, by using null flags in this way, the programmer effectively buries the evidence of where the exceptional condition occurred. Who knows how far that null pointer was passed from method to method before it was dereferenced? This simply makes it harder to diagnose errors and determine how to fix them.

Note Exceptions are a great tool to increase the robustness of code. During an

exceptional condition, they allow control to quickly pass to the best place to handle the exception.

Experience shows that code breaks often. Our primary concern should be to avoid such obfuscation and make the diagnosis as easy as possible. Therefore, as a matter of principle, I try to write code that signals exceptional conditions as soon as possible and attempts to recover only from exceptional conditions that do not indicate bugs in the program.

And What About Legacy Code?

Even if you try to avoid using null flags in your own code, you will inevitably have to deal with legacy code that uses them. In fact, many of the Java library classes themselves, such as the Hashtable and BufferedReader classes that we used earlier in this chapter, use null flags.

When maintaining such classes, make sure that it is made clear to all clients that the null value is reserved as a flag; it should never be returned under ordinary circumstances (e.g., as the stored value of a key in a Hashtable). When using such classes, you can avoid bugs by explicitly checking whether an operation will return null before performing it. For example, with Hashtables, I always test with containsKey() before calling get(). Such preventative measures can go a long way in eliminating occurrences of this pattern What We've Learned

In this chapter on the Null Flag bug pattern, we've learned the following: � Calling methods can cause this problem if they don't check for null pointers as return

values. � Exceptional conditions are usually handled by throwing and catching exceptions. If a

NullPointerException is thrown, it is not much of a diagnostic help. � Throwing checked exceptions provides more safety than throwing unchecked exceptions,

since client code cannot compile without handling the exception. However, this may be impossible if you've elected to employ a high level of code reuse.

� There are often trade-offs between the goals of static checking and code reuse. � Exceptions are a great tool to increase the robustness of code. During an exceptional

condition, they allow control to quickly pass to the best place to handle the exception.

In Chapter 11, we will discuss a bug pattern related to ClassCastExceptions.


of 168

Chapter 11: The Double Descent

Overview

When your program recursively descends a composite data structure so that occasionally more than one step down is taken via a single recursive call, you've come upon the Double Descent pattern. We discuss ClassCastExceptions and the conceptual errors behind this bug.

Although usually easier to debug than other, more insidious erroneous behaviors, ClassCastExceptions are often symptoms of conceptual errors in recursively descending a composite data structure. We'll discuss where programmers should look to find this pattern of bug, how to recognize the pattern, and what to do to minimize its occurrence.

Unlike the dreaded NullPointerException—which says nothing about what was expected to occur instead of the null pointer (see more in Chapters 8, 9, and 10)—a ClassCastException is relatively easy to debug. A ClassCastException often occurs in a program that is performing a recursive descent over a data structure. I call this the Double Descent bug pattern.

Here's the breakdown of this bug pattern:

� Pattern: Double Descent.

� Symptoms: A program that throws a ClassCastException while performing a recursive descent over a data structure.

� Cause: Some part of the code is descending two levels per method call without dispatching appropriately on the second descent.

� Cures and Preventions: Factor the casting code out into separate methods for each class. Alternatively, examine the invariants to ensure that the casts will succeed


The Double Descent bug pattern manifests itself as a ClassCastException. It is caused by recursively descending a composite data structure in such a way that sometimes more than one step in the descent is taken in a single recursive call.

Descending in this way often necessitates the addition of casts to get the code to compile. But, in such a descent, it is easy to forget to check that appropriate invariants are satisfied to guarantee that these casts will succeed.

Consider the following class hierarchy for binary trees of ints . We want to allow for empty trees, so we will not put a value field into the Leaf class. Because this decision makes all Leafs identical, we can make use of the Singleton design pattern for class Leaf (see Chapter 9 for more on this design pattern). This design pattern stores a single public instance of a class as a static field (to be used whenever an instance is needed) and restricts outside invocations of the class constructor.

The advantages of doing it this way? � It saves storage space. Why create multiple identical instances of the same class? � It allows us to use == in place of instanceof or equals() to check for class and

object identity.

Here's the resulting code for the Tree class hierarchy: Listing 11-1: A Class Hierarchy for Binary Trees of ints That Allows for Empty Trees


of 168

�


}


public static final Leaf ONLY = new Leaf();

private Leaf() {

}

}


public int value;

public Tree left;

public Tree right;

public Branch(int _value, Tree _left, Tree _right) {

this.value = _value;

this.left = _left;

this.right = _right;

}

}

�

Now, suppose we want to add a method on Trees that determines whether any two adjacent nodes (such as a branch and one of its children) both contain 0 as their value. We might add the following methods (notice that the last method will not compile in its current form): Listing 11-2: Adding Methods to Ferret Out 0 Values in Adjacent Nodes

�

// in class Tree:

public abstract boolean hasAdjacentZeros();

// in class Leaf:

public boolean hasAdjacentZeros() {

return false;

}

// in class Branch:


return this.value == 0 && (this.left.value == 0 || this.right.value == 0)


of 168

|| this.left.hasAdjacentZeros()

|| this.right.hasAdjacentZeros();

}

�

The method in class Branch will not compile because this.left and this.right are not guaranteed to have value fields.

The fact that we cannot compile strongly suggests that there is a logical problem with our manipulation of these data structures. But suppose we ignore this warning sign and simply cast this.left and this.right to Branches in the appropriate if statement, as follows: Listing 11-3: Casting to Branches To Fix Inability to Compile

�

// in class Branch:


return this.value == 0 && (((Branch)this.left).value == 0 || ((Branch)this.right).value == 0)



}

�

Now the code will compile. In fact, it will succeed on many test cases.

The Symptoms

Suppose we were to run this code on the tree shown in Figure 11-1, where the branches are represented as circles with their values in the center and the leaves are represented as squares. Calling hasAdjacentZeros() on this tree causes a ClassCastException.

Figure 11-1: A hasAdjacentZeros() call will throw a ClassCastException.

The problem occurs with the left branch. Because the value of that branch is 0, hasAdjacentZeros() casts its children to type Branch—which fails, of course.

The Cause


of 168

Why does it fail? Some part of the code is descending two levels per method call without dispatching appropriately on the second descent.


The way to fix the problem in Figure 11-1 is the same as the way to prevent it. But before we discuss the best fix, we should discuss one way not to fix it.

A Quick but Incomplete Fix

The quick but incorrect way to remedy the problem would be to eliminate the Leaf class and represent Leaf nodes simply by putting null values in the left and right fields of a Branch. This approach would eliminate the need to cast in the code listings above, but it would not fix the bug.

Instead, the error signaled at runtime would be a NullPointerException instead of a ClassCastException. Because NullPointerExceptions are more difficult to diagnose, this "fix" would actually decrease the quality of the code. (In fact, the resultant class hierarchy would be a Dangling Composite. For more discussion on this issue, see Chapter 9.)

So, what's the best way (or ways) to fix this bug?

Note Think of a cast as a kind of an assertion; think of the invariants as arguments

for why the assertion is true.

The Real Fix

One way would be to wrap each cast in an instanceof check.

�

Comments in Code

In Listing 11-4, notice the comments asserting what invariant we expect to hold in the body of each if statement. It's good practice to add comments like this to your code. Listing 11-4: Wrapping Each Cast in an instanceof Check

�


boolean foundOnleft = false;

boolean foundOnRight = false;

if (! (this.left instanceof Leaf)) {

// this.left instanceof Branch

foundOnLeft = ((Branch)this.left).value == 0;

}

if (! (this.right instanceof Leaf)) {

// this.right instanceof Branch

foundOnRight = ((Branch)this.right).value == 0;

}


of 168

return this.valueIs(0) && (foundOnLeft || foundOnRight)



}

�

This is an especially helpful practice for else clauses. Because there is rarely an explicit check on the invariants that are expected to hold in an else clause, it's a good idea to make those invariants clear in your comments.

You can think of a cast as a kind of assertion and the invariants as arguments for why the assertion is true.

�

The disadvantage of using instanceof checks in this fashion is that, if we were to add another subclass of Tree (such as a LeafWithValue class), we would have to revise the instanceof checks. For this reason, I try to avoid instanceof checks whenever possible.

Instead, I add extra methods to the subclasses that perform the appropriate action for each subclass. After all, the ability to add such polymorphic methods is one of the key advantages of an object-oriented language.

In the current example, we could do this by adding a valueIs() method to the Tree class as follows: Listing 11-5: Using Extra Methods to Avoid Rewriting instanceof Checks

�

// in class Tree:

public abstract boolean valueIs(int n);

// in class Leaf:

public boolean valueIs(int n) { return false; }

// in class Branch:

public boolean valueIs(int n) {

return value == n;

}

// in class Branch, method hasAdjacentZeros

public boolean hasConsecutiveZeros() {

return this.valueIs(0) && (this.left.valueIs(0) || this.right.valueIs(0))

|| this.left.hasConsecutiveZeros()

|| this.right.hasConsecutiveZeros();

}


of 168

�

Notice that we've added a valueIs() method instead of a getValue() method. If we had added a getValue() method to class Leaf, we would either have to return some sort of flag value indicating that the method application is nonsensical or actually throw an exception.

Returning a flag value would bring up many of the problems associated with the Null Flag bug pattern we discussed in Chapter 10. Throwing an exception would not help in our case because we would then have to add instanceof checks in hasAdjacentZeros() to ensure that we didn't trigger the exception. And that is exactly what we are trying to avoid with the new method.

valueIs() avoids all of these problems by encapsulating what we really want each class to handle separately—checking whether an instance of the class contains the given value.

Note Holding each cast to this level of scrutiny—knowing that the invariants will

ensure that any casts will succeed— will lead you to eliminate many such casts What We've Learned

In this chapter on the Double Descent bug pattern, we've learned the following: � If a ClassCastException is thrown as your program recursively descends a composite

data structure, it usually means that a method call isn't dispatching appropriately on the second call.

� One cure is to factor out casting code into separate methods for each class. � Another cure is to examine the invariants to ensure that the casts will succeed. � Another is to wrap each cast in an instanceof check. However, I suggest you avoid

instanceof checks. Otherwise, you'll have to revise them every time you add another subclass of Tree.

� Instead of instanceof checks, try adding extra methods to the subclasses to perform the appropriate actions for each subclass. This makes use of the polymorphic aspect of object-oriented languages, one of such languages' advantages.

� A quick fix seems to be to get rid of the Leaf class and represent the Leaf nodes as null values in Branch's left and right fields. But this "fix" would introduce a Dangling Composite. The runtime error would throw a NullPointerException instead of a ClassCastException, a harder exception to diagnose.

� Commenting in code is helpful to logical debugging, especially for else clauses. (See the sidebar "Comments in Code" in this chapter.)

� Holding each cast to this level of scrutiny—knowing that the invariants will ensure that any casts will succeed—will lead you to eliminate many such casts.

In short, the moral to this story is to always convince yourself that the invariants inside a code block ensure that any casts in the block will always succeed. When each cast is held to this level of scrutiny, you may find yourself eliminating many of these casts by adding methods to the relevant subclasses.

In Chapter 12, we'll discuss the Liar View bug, a pattern that often occurs in GUI applications.


of 168

Chapter 12: The Liar View

Overview

GUIs are usually designed with a Model-View-Controller architecture in which the view is decoupled from the model. This separation presents a challenge to automated testing because it's difficult to verify that a state change in the model is reflected appropriately in the view—it spawns the Liar View bug pattern. We discuss how you can make sure both model and view get tested.

Picture this: You've crafted a golden GUI program designed for a distributed system, all that your customer asked for and then some. You've run it through an extensive suite of automated unit tests. It came back with a clean bill of health.

The pressure is on to deliver the GUI, but being the exacting programmer that you are, you fire it up for one last manual test only to discover erroneous behavior—behavior that should have been caught by the automated tests. If only you could have prevented this situation! Well, you can.

Sometimes, despite passing its suite of tests, a program will, upon manual inspection, exhibit erroneous behavior that should have been discovered by one of the tests. Such behavior is common in distributed and multithreaded systems. In these cases, the nondeterministic nature of the program is often the cause.

But in the case of GUIs, there is another common cause: the Liar View bug pattern.

Here's the bug pattern in a nutshell:

� Pattern: Liar View.

� Symptoms: A GUI program passes a suite of tests, but then exhibits behavior that should've been ruled out by those tests.

� Cause: The tests only check aspects of the model directly.

� Cures and Preventions: Write independent tests for the model and the view plus tests on the combined entity.


Good debugging starts with good testing. And with the vast number of invariants that must be checked in GUI code, automated testing is essential. But beware of test suites that only test the state of an underlying model without testing the view as well. When a view incorrectly displays a valid state of the model, you've got yourself a case of the Liar View.

The Symptoms

Most GUI program tests, like program tests in general, follow this structure: 1. Start the program. 2. Check some aspect of its state. 3. Attempt to modify the state. 4. Check that the state was modified as intended.

Sometimes a manual inspection of the runtime program behavior will contradict the successful results of the tests. Queues may appear on the screen containing elements that testing


of 168

(supposedly) confirmed were deleted. Objects may contain stale data that was reportedly updated.

Bugs like these may cause us to question our sanity or, worse yet, fall into an unhealthy skepticism of the validity of reason itself. Don't let this happen to you. When used as directed, reason really does work. And, despite reports to the contrary, it is the rare programmer who permanently loses his sanity while coding (permanently being the operative word).

Caution At least a portion of this bug may be in the test suite itself.

The Cause

A key to finding these bugs is to realize that at least part of the bug may be in the test suite.

The most common place for a bug to occur in a test on a GUI program is in the last step—checking that the state of the program was modified as intended. This is because GUIs are generally designed with a Model-View-Controller (MVC) architecture. The Swing class library builds this architecture into the structure of the GUI classes.

In an MVC architecture, the internal state of the program is kept in the model. The view responds to events signifying a new state of the model and updates the screen image accordingly. The controller connects these two components together.

The advantage of this architecture is that it decouples the view from the model, so that the implementation of either can change independently. But it poses a challenge to automatic testing methods: it can be difficult to verify that a state change in the model is reflected appropriately in the view. When there is a discrepancy between the two, we have an instance of the Liar View bug pattern.

Note It can be difficult to verify that a state change in the model is reflected

appropriately in the view.

For example, consider the following simple GUI. It displays the contents of a list of elements as it is updated. Note: Because this GUI is so simple, the architecture is much more straightforward than an industrial-strength MVC architecture. For instance, communication between the model and the view occurs only in one direction (from the model to the view), so the controller does not have to install callbacks (from the view to the model). Nevertheless, the example is adequate for our present purposes.

The main method of the Controller class is used as a simple test. In a real application, I'd move this method into a separate test class and hook it into JUnit.

I've added a pause() method and a PAUSE field to allow us to put the test into slow motion and manually inspect each event as it occurs. Listing 12-1: The Pause Lets Us Place the Test in Slow-Motion Mode

�

import java.awt.*;

import java.awt.event.*;

import java.util.Vector;

import javax.swing.*;

public class Controller {


of 168

private static final int PAUSE = 1;

private static void assert(boolean assertion) {

if (! assertion) {

throw new RuntimeException("Assertion Failed");

}

}

private void pause() {

try {

synchronized (this) {

wait(PAUSE);

}

}

catch (InterruptedException e) {

}

}

public static void main(String[] args) {

Controller controller = new Controller();

JFrame frame = new JFrame("Test");

Model model = new Model();

JList view = new JList(model);

view.setPreferredSize(new Dimension(200,100));

frame.getContentPane().add(view);

frame.pack();

frame.setVisible(true);

assert(model.getSize() == 0);

controller.pause();

model.add("test0");

controller.pause();

model.add("test1");

controller.pause();


controller.pause();


of 168

model.remove(0);

controller.pause();


controller.pause();

System.exit(0);

}

}

class Model extends AbstractListModel {

private Vector elements = new Vector();

public synchronized Object getElementAt(int index) {

return elements.get(index);

}

public synchronized int getSize() {

return elements.size();

}

public synchronized void add(Object o) {

int index = this.getSize();

this.elements.add(o);

this.fireIntervalAdded(this, index, index);

}

public synchronized void remove(int index) {

this.elements.remove(index);

}

}

�

You may have noticed that there is a serious bug in this code. If we run the test case, all assertions succeed, indicating that items are added and removed from the list appropriately. But if we slow things down by, say, setting PAUSE to 1000, we can inspect the test run manually. And guess what? We notice that no items are ever removed in the view.

The reason that the view is not updated is that the remove() method in class Model never calls fireIntervalRemoved() to notify any listeners that the state of the model has changed.

But all the assertions in our test method succeed. Why? Because these assertions check for changes in the model, not the view. Because the model is updated appropriately, the missing event firing is not detected by the assertions.


of 168

Note Perpetual refactoring is only practical when there is a strong network of unit

tests in place, to prevent perpetual code breakage along with it.

Cures and Preventions

There are two ways to fix or prevent the Liar View: � Check the model through the view. � Automate the physical manipulation of the GUI with the mouse and keyboard.

Checking the Model Through the View

One way to prevent this bug pattern is to check properties of the model as they are reflected in the view. Although this technique limits the properties we can check to those provided in the view, the assertions will at least reflect what's really happening on screen.

For example, we could rewrite Controller.main as follows: Listing 12-2: Rewriting to Check View Properties After Model State Changes �

import java.awt.*;

import java.awt.event*;


import java.swing.*;

public class Controller {

...


Controller controller = new Controller();

JFrame frame = new JFrame("Test");

Model model = new Model();

JList view = new JList(model);

view.setPreferredSize(new Dimension(200,100));

frame.getContentPane().add(view);

frame.pack();

frame.setVisible(true);

assert (model.getSize() == 0);

controller.pause();

boolean toggle = model.toggle;

model.add("test0");

assert (toggle == ! model.toggle);


of 168

controller.pause();

toggle = model.toggle;

model.add("test1");

assert (toggle == ! model.toggle);

controller.pause();


view.setSelectedIndex(0);

assert(view.getSelectedValue().equals("test0"));

controller.pause();

toggle = model.toggle;

model.remove(0);

assert(toggle == ! model.toggle);

controller.pause();


view.setSelectedIndex(0);

assert(view.getSelectedValue().equals("test1"));

controller.pause();

System.exit(0);

}

}

class Model extends AbstractListModel {

boolean switch = false;

private Vector elements = new Vector();

...

public void fireIntervalAdded(AbstractListModel m, int start, int end) {

super.fireIntervalAdded(m,start,end);

this.switch = ! this.switch;

}

public void fireIntervalRemoved(AbstractListModel m, int start, int end) {

super.fireIntervalAdded(m,start,end);


of 168

this.switch = ! this.switch;

}

}

�

By using setSelectedIndex() and getSelectedIndex(), we perform a slightly different but much improved test on the program. Not only does the modified test check the model through the view, it also checks the content of selected rows, rather than just the number of rows.

�

Use Recorders to Test Separately

Testing the model through the view is the best way to prevent Liar Views, as it tests model and view as a combined entity. But it is also advisable to test each component in isolation. By also testing the model (as well as the view) in isolation, we will be able to diagnose bugs introduced into each component much more quickly.

A good strategy for testing a model in isolation is to wire it up to a listener that simulates a view. This simulated view can record the calls made on it, and this record can be checked by the unit tests. Similarly, views can be tested in isolation by wiring them up to simulated models. I call these simulation classes "Recorders," as their sole purpose is to record the sequence of calls performed on them. In cases where we want to check only that a single call was made, a Recorder may simply throw an exception on each call. These exceptions should be caught by the unit tests.

�

Automate a GUI's Physical Manipulation

Another way to check the view directly is to use the Java Robot class (introduced in Java 1.3) to automate the physical manipulation of a GUI with the mouse and keyboard.

The Robot class lets you take snapshots of subsections of the screen, allowing you to build tests based on the actual physical layout of a GUI view. Of course, this ability can be a disadvantage if the physical layout is not as stable as the logical structure of the view. It can be painful to have to rewrite several tests every time the physical layout changes.

Note The Robot class lets you take snapshots of subsections of the screen, allowing

you to build tests based on the actual physical layout of a GUI view.

Therefore, I recommend using the Robot class only as a testing tool for a mature GUI whose view won't change very often. To test the logical aspects, call methods on the view as we did in the earlier examples.

By the way, some network installations of Java disable the Robot class functionality out of security concerns.

Avoid These Methods

Beware of methods in view objects that simply trampoline calls back to the model. Doing so can quickly introduce Liar Views. JTables, in particular, contain many such methods.

�

A Peek at a Real-World Example

Here is one example of a real-world Liar View that occurred in the DrJava project at Rice University.


of 168

Shortly after we set DrJava to report the cursor's current line and column numbers in the status bar, we realized that the line and column numbers reported were often wrong. Although a solid suite of unit tests over the GlobalModel ensured that the internal representation of the position was correct, the view wasn't always displaying it properly.

In particular, when moving the cursor with the arrow keys, the line reporting was consistently reporting the position before the last line change. Also, the current column was reported as 1 at every key movement after the first, even though test cases on the model side showed that the current column was computed properly.

Although we were trying to maintain the invariant that all listeners were notified of events through the GlobalModel, new programmers on the project who weren't yet completely familiar with the code base registered the listener for updating the position display directly with a subcomponent of GlobalModel called DefinitionsPane.

Once we realized this, we hypothesized that the problem was a race condition: DefinitionsPane was notifying this listener of a cursor movement before other subcomponents of the GlobalModel were updated. The listener would then poll other components in the GlobalModel to determine the new cursor location, but that new location could be in an inconsistent state. If this hypothesis were correct (it turned out that it was), it explained the lag in the update of line positions, as well as the strange column positions reported by the view.

Jim Van Fleet, one of our developers on the DrJava project, investigated this bug and discovered that it was indeed a race condition: When line changes occurred, a listener we had installed for highlighting text in the view was called by the GlobalModel after the listener for updating the line and column numbers was called by the DefinitionsPane. But calculating what text to highlight involved temporary modification of the GlobalModel position. This temporary modification was not reported to listeners registered directly with the GlobalModel, but our errant listener was able to notice it by polling directly.

In order to release a corrected version of the application as quickly as possible, we implemented a simple fix: we simply synchronized the polling by the errant listener with the calculation of other listeners on the GlobalModel.

In the long term, we've planned a refactoring task to make the errant listener into a GlobalModel listener. That way, there will be no need to synchronize listeners on the various subcomponents (after all, that's the whole point of the GlobalModel).

This bug case study is a good example of how even a moderately large project, with new developers coming on board continually, will need perpetual refactoring in order to maintain the intended invariants. And perpetual refactoring is only practical when there is a strong network of unit tests in place, to prevent perpetual code breakage along with it.

For more on the DrJava project, see the Resources chapter, the section entitled "The GlobalModel Interface" in Chapter 4, and the sidebar in Chapter 5 entitled "Small Anomalies, Big Problems." GUIs Aren't the Only Liars!

Although Liar Views occur quite often in GUI code, the underlying pattern is actually more general than that. Any time we are displaying data, be it in textual or graphical form, there is the potential for a Liar View.

For example, suppose you are writing a toString() method over a composite data structure and you forget to include the value of one of the fields (or worse, you include another field in its


of 168

place). If you print these Strings out during, say, debugging, it can send you on a serious wild goose chase. Textual Liar Views can be at least as dangerous as GUI-based Liar Views. Fortunately, they are a lot more straightforward to eliminate through testing.

In fact, toString() test methods are some of the easiest and most fun test methods to write (yes, testing really can be fun). Because String values can be represented literally in Java, test methods on toString() can consist entirely of a series of assertions that check that various calls to toString() match designated String literals What We've Learned

In this chapter on the Liar View bug pattern, we've learned the following: � Most GUIs are built on the Model-View-Controller architecture (MVC), which decouples

the model from the view. � In an MVC architecture, the internal state of the program is kept in the model. The view

responds to events signifying a new state of the model and updates the screen image accordingly. The controller connects these two components together.

� The decoupled aspect of the GUI architecture means that program test results may differ from runtime behavior.

� A key to diagnosing this bug is to realize that part of the bad behavior can be in the test itself.

� The most common place for a bug to occur in a test on a GUI program is in the last step: checking that the state of the program was modified as intended. It can be difficult to verify that a state change in the model is reflected appropriately in the view. Assertions in the test method may succeed even though the view is not updated because the assertions only check for changes in the model.

� One way to prevent this bug pattern is to check the model through the view. � Another way to check the view directly is to use the Java Robot class to automate the

physical manipulation of a GUI with the mouse and keyboard. This class also lets you take snapshots of subsections of the screen, allowing you to build tests based on the actual physical layout of a GUI view. However, I recommend using the Robot class only as a testing tool for mature GUIs whose views won't change very often.

� Use Recorders to test the model (and the view) in isolation. � Beware of methods in view objects that trampoline calls back to the model. Doing so can

introduce Liar Views. JTables, in particular, contain many such methods. � Any time we are displaying data, be it in textual or graphical form, there is the potential

for a Liar View. � Perpetual refactoring is only practical when there is a strong network of unit tests in

place, to prevent perpetual code breakage along with it.

Bit by bit we're working our way through solutions to the most common (and most frustrating) bugs, but we still have a lot of ground to cover. In Chapter 13, we'll tackle a more subversive bug pattern: Saboteur Data, data that is perfectly benign—until it's accessed!


of 168

Chapter 13: Saboteur Data

Overview

When a program crashes due to corrupted data, the reason can be elusive. Often a program can crash dead in its tracks while manipulating its own internal data, even after working flawlessly for long periods. You've probably already encountered the Saboteur Data bug pattern. We discuss the syntactic and semantic reasons for it and offer potential methods for eliminating it.

Being an industrious developer, you've deployed one of your well-written and well-tested applications for several of your clients who needed better access to their massive stores of complex data.

For each client, the on-site testing period went off without a hitch. You're on your way to the bank, only barely thinking about the six-month software checkup, when your pager goes off. One of your clients ran a report using your software and bombed the system.

You rush to the site and run a random test. It works fine. You run another. No problems. You run hundreds more. Still no problem. You check on other clients who have been running this application full tilt for six months. You get no complaints.

Then, you repeat the report that caused the problem. Crash! What's going on?

You've just met the Saboteur Data, a bug pattern that can be the culprit behind this sort of crash. We explain why it exists and offer several methods to eliminate it—before and after it occurs. Here is the summary of the Saboteur Data bug pattern:

� Pattern: Saboteur Data.

� Symptoms: A program that stores and manipulates complex input data crashes unexpectedly while performing a task similar to other tasks that cause no problem.

� Cause: Either the syntax or the semantics of some of the internal data is corrupted.

� Cures and Preventions: Perform as many integrity checks on input data as possible, as early as possible. For persistent data that is already corrupt, walk over it and check for integrity.


Many programs need to intensively access and manipulate internally stored data to perform various complex tasks. This data might be retrieved from a large data structure in memory, from a database, or over a network.

This type of program is highly susceptible to a crash caused by corrupt internal data. I call this bug pattern the Saboteur Data pattern because such data can stay in the system indefinitely, much like a Cold War sleeper spy, causing no trouble until the particularly troublesome bit of data is accessed. The corrupt data then explodes like a bomb.

The Symptoms

A program that stores and manipulates complex input data crashes unexpectedly while performing a task similar to other tasks that cause no problem.


of 168

A Syntactic Cause

Suppose we have a JDBC application that stores a database table called Mapping that maps String names to sets of elements. Each element of each set refers to a key stored in another table, Properties, containing various known properties of these elements. (JDBC serves up a common API for connecting to and eliciting services from databases on a variety of platforms. See the Resources chapter for more information on this powerful API.)

Let's say that both Mapping and Properties are initially read from a text file developed by an outside source (outside meaning any data source not generated by our JDBC application itself) in which each line starts with a name and is followed by a representation of the corresponding set, as follows: Listing 13-1: Data from an Outside Source Text File �

In the Mapping file:

apples {macintosh, gala, golden-delicious}

trees {elm, beech, maple, pine, birch}

rocks {quartz, limestone, marble, diamond}

...

In the Properties file:

macintosh {color: red, taste: sour}

gala {color: red, taste: sweet}

diamond {color: clear, rigidity: hard, value: high}

...

�

The Mapping and Properties table entries could be parsed and passed to a method that inserts them into a database. But there are potential pitfalls in this approach. For example, let's suppose that we have written a class that handles a JDBC-compliant database. Following the JDBC API, we could define a PreparedStatement object and use it to pass information into the database, like so: Listing 13-2: Defining PreparedStatement Object for Passing Data

�

PreparedStatement insertionStmt =

con.prepareStatement("INSERT INTO MAPPING VALUES(?,?)");

...

public void insertEntry(String domain, String range)

throws SQLException

{

insertionStatement.setString(1, domain);

insertionStatement.setString(2, range);


of 168

insertionStatement.executeUpdate();

}

�

Inserting two Strings this way may or may not be all right, depending on how the Strings are obtained from the text file. Suppose, for example, that a simple regular expression-matching tool is used to split each line into two Strings: � One String contains all the characters before the first space. � One String contains all the characters after the first space.

Such a rudimentary parse of the text file would not catch minor corruption in the data. For example, consider what would happen if one of the lines were in the following form:

trees {elm, beech, maple, pine birch}

The comma between pine and birch is missing. An error such as this can easily result from a bug in the tool that generates the file or from manual editing of the file.

At any rate, the data would enter the database in its corrupt form, waiting silently to be accessed. If the method used to access data expects entries to be separated by commas and spaces, it will crash when reading this entry.

If the program simply distinguishes the elements of the set by commas alone, an even more serious error can occur. The system could interpret pine birch as a single type of tree (a single entry of data) and propagate the bug further into the computation.

A Semantic Cause

Our example is one in which a simple, syntactic constraint of the data was violated. Of course, that's not the only way in which the data might be corrupted. Semantic-level constraints can be violated as well. In our example, one expectation of the data in the Mapping table is that every element in each set is a domain entry in the Properties table. If this invariant were violated, we might end up trying to read an element in the Properties table that wasn't there, causing an exception to be thrown.

In this chapter we use database entries as examples, but a Saboteur Data bug can come at you in a variety of ways—as many ways as there are data-input avenues. When data is read by a program, whether it is from a file, a keyboard, a microphone, a network port, or a digital camera, the potential for a saboteur exists.


The best defense against the Saboteur Data bug is one that is universally employed by compiler and interpreter developers. Because the input data to these programs is so complex, developers have no choice but to perform as thorough an integrity check as possible when first reading the input, rather than upon later access.

Let's look at several elimination methods.

Parsing as an Elimination Method

The very practice of parsing input is a way to eliminate most saboteurs. Unfortunately, programmers who would never think of writing a compiler without a parser fail to write adequate


of 168

parsing methods for simpler data. The parsing of simpler data is easy, but that's no excuse for not parsing it at all.

Any program that reads data—no matter how simple—should parse it. After all, such a program can be viewed as a compiler or an interpreter over the "language" defined by its set of valid inputs. Take it from someone who has been there. I plead guilty to having manipulated data without proper parsing in my young and reckless days, and I suffered the consequences—rampant saboteurs. I don't recommend the experience.

Type Checking as an Elimination Method

Another common form of checking performed by compilers for many languages (including the Java language) is type checking. Type checking is an example of a semantic-level check on the integrity of a program.

Provided that the type system is sound (as the Java type system is), this integrity check literally guarantees that a huge class of errors can never occur at runtime. Like parsing, this example from compiler writers can be applied to other programs, which often stipulate semantic-level invariants over their input data (as in our example). These invariants are often not explicit, but they can be made explicit by putting in the corresponding checks.

Iteration as an Elimination Method

Of course, if you suspect an occurrence of this bug pattern with data that has already been read in and stored, it would be prudent to iterate over the data, accessing each bit of data as it would be accessed in the deployed application and ensuring that everything works as expected. In the process, you might be able to correct simple errors as well.

In cases where your data is stored in an immutable database or other immutable finite store, such an offline integrity check can also serve as a performance optimization. If you check over all of the data offline and it all passes, there is no need to check it again when it's used online. You might as well conserve the processor cycles.

But this optimization should be done only when the data is truly immutable, and only when there is no chance that the data will be corrupted while reading it from storage. If there is even a remote chance that new data will be entered or if the connection to this data is any less reliable than a connection to the local filesystem, check it again while reading it. After all, these integrity checks rarely cause significant performance degradation; the data retrieval process itself will almost always be the bottleneck.

But even a small risk of saboteur data is too much risk. Just one case will easily outweigh any advantages of not doing the checks—both from the perspective of your customer when the software makes a catastrophic nosedive and from your perspective when you try to diagnose what happened. Because the symptoms are far removed from the cause, saboteur data can be a bear to diagnose.

A Caveat on Elimination Methods

I don't mean to imply that it is always possible to perform enough checks to eliminate each piece of saboteur data from a program. If that were the case, this would be a much less problematic bug pattern.

There are many reasons why a saboteur might be undetectable before it starts wreaking havoc: � The data necessary to perform all the checks is not available until after the saboteurs are

stored away, and they are not all accessible offline.


of 168

� The function checking the complete set of constraints is not even computable (as is the case for many programming languages).

� The set of constraints is computable, but the resources required to check them are beyond what's available to the program.

In such cases, the best we can do is eliminate as many forms of saboteurs as possible What We've Learned

In this chapter on the Saboteur Data bug pattern we've learned the following: � Saboteur Data is often responsible when either the syntactic or the semantic constraints

of complex input data or legacy data is violated. � The unpredictable nature of this bug rests in the fact that some actions call up the bits of

corrupt data while other actions don't. � This saboteur data can stay in the system indefinitely, much like a sleeper spy, causing

no trouble until the particularly troublesome bit of data is accessed. � The bug can come at you in a variety of ways, as many ways as there are data-input

avenues. � A syntactic data error can occur by manual editing or automated generation of a file. � With a syntactic error, a simple parse—like splitting a text line into two Strings, one

each before and after the first space—wouldn't catch a minor data corruption such as a missing comma separator between entries.

� The results of the syntactic error above: if the program expects entries to be separated with a space and a comma, the program may crash; if not, the program may accept the two entries as one and propagate the error.

� In a semantic error, expectations of elements can be violated. If an expectation of the data in one table is that every element in each set is a domain entry in another table, and we violate this expectation, we may throw an exception when we try to read an element in the second table that isn't there.

� The best defense against this pattern is one that is universally employed by compiler and interpreter developers. Because the input data to these programs is so complex, developers perform as thorough an integrity check as possible when first reading the input rather than later.

� The practice of parsing input is a way of eliminating bugs. In fact, any program that reads data—no matter how simple—should parse it.

� Type checking, another elimination method, is an example of a semanticlevel check on the integrity of a program.

� If you suspect an occurrence of this bug with data that has already been stored, you can iterate over the data, accessing each bit as it would be accessed in the deployed application and ensuring that everything works as expected. When the data is stored in an immutable database or finite store, such an offline integrity check can also serve as a performance optimization.

� One caveat: offline checking of data integrity should replace online checks only when the data is immutable and only when there is no chance that the data will be corrupted while reading it from storage.

� A saboteur might be undetectable because the data necessary to perform all the checks won't be available until after the saboteurs are stored away and inaccessible offline.

� A saboteur might be undetectable because the complete set of constraints is not even computable (as is the case for many programming languages).

� A saboteur might be undetectable because the constraints are computable, but the resources required to check them are beyond the access of the program.

The Golden Rule to eliminating data saboteurs: Any program that reads data should parse the data. Good luck in stamping them out!

In Chapter 14 we'll examine the Broken Dispatch—when an overloaded method suddenly breaks code you haven't modified.


of 168

Chapter 14: The Broken Dispatch

Overview

When you employ inheritance polymorphism—a feature of object-oriented languages that allows you to override methods—in combination with static overloading—a feature that allows you to overload methods based on the static types of the arguments—and an untouched method breaks, you may have come upon a Broken Dispatch. We discuss argument mismatches and practical and conceptual ways to eliminate the problem.

Remember this adage: "The whole is greater than the sum of its parts." When single events are combined as an interacting unit, that unit can demonstrate many more results than the sum of results from each event on its own. Programs follow this rule.

With each new method added to a program, the set of possible flows of control through the program grows dramatically. In large programs, it can quickly grow out of control. And like a perverse magic trick, sometimes the direction you expect is not where you go—it is similar to what can happen when you overload or override methods.

Before we continue, notice that I am distinguishing method overriding (where a subclass redefines an inherited method) from method overloading (where a class defines two or more methods using the same name but different argument types).

Method dispatch in an object-oriented language such as Java, in which methods can be overloaded and overridden, can cause difficulties of code management even in moderately complex programs.

Note When single events are combined as an interacting unit, that unit can

demonstrate many more results than the sum of results from each event on its own.

We will discuss the Broken Dispatch bug pattern, one of the more prominent patterns caused by these difficulties, outlining the symptoms that appear when argument mismatches cause the wrong method to be invoked, and offering a few solutions to combat the problem. Here's a summary of this bug pattern:

� Pattern: Broken Dispatch.

� Symptoms: A test case for code you haven't touched suddenly breaks, just after you've overloaded another method.

� Cause: The overloading has caused the untouched method to invoke a method other than the one intended.

� Cures and Preventions: Either put in explicit upcasts or rethink the set of methods you're providing on your various classes.

One of the most powerful features of an object-oriented language is inheritance polymorphism. This feature allows us to override methods depending on the run-time type of the receiver. Another powerful feature is static overloading, which allows us to overload methods depending on the static types of the arguments. However, these two language features can introduce problems when used together.

It can be easy in a large program to overload a method in one class and in the process break code that previously worked in another class.


of 168

Note Method dispatch in object-oriented languages—in which methods can be

overloaded and overridden—can cause code-management difficulties even in moderately complex programs.


Although Java programmers quickly learn the rules governing which method will be called on an invocation, it's not hard to accidentally break one method by overloading another method that it relies on.

The Broken Dispatch pattern can be described as follows: � The arguments to an overloaded method (such as foo()) are passed to another method

(such as bar()) that takes more general types. � bar() then invokes foo() with these arguments. � Because the static types of these arguments inside the scope of bar() are more

general, the wrong version of method foo() might be invoked.

Errors like these can be difficult to diagnose because they can be introduced simply by adding new methods (as opposed to modifying existing ones). Also, the program execution may continue for some time before problems are discovered.

The Symptoms

To illustrate the nature of this pattern, let's consider the following example code (originally Listing 9-7) for an implementation of immutable lists from Chapter 9: Listing 14-1: Implementing the Immutable Lists �

interface List {}

class Empty implements List {


return (that != null) &&

(this.getClass() == that.getClass());

}

}

class Cons implements List {

Object first;

List rest;

Cons(Object _first) {


this.rest = new Empty();

}



of 168


this.rest = _rest;

}

public Object getFirst() { return this.first; }

public List getRest() { return this.rest; }






else { // that instanceof Cons

Cons _that = (Cons)that;

return (this.getFirst().equals(_that.getFirst())) &&

(this.getRest().equals(_that.getRest()));

}

}

}

�

Recall that we implemented linked lists as containers for these immutable lists. But now let's suppose that we're implementing linked lists in a separate package in which we know that all instances of class LinkedList will be lists of Strings. We could write the constructors to enforce this invariant as follows: Listing 14-2: Defining Enforcement Parameters for Linked Lists

�


private List value;

/**


*/

public LinkedList() { this.value = new Empty(); }

/**


*/

public LinkedList(String _first) { this.value = new Cons(_first); }


of 168

/**



*/

public LinkedList(String _first, LinkedList _rest) {

this.value = new Cons(_first, _rest.value);

}

public Object getFirst() { return this.value.getFirst(); }

public LinkedList getRest() {

return new LinkedList(this.value.getRest());

}

public void push(String s) { this.value = new Cons(s, this.value); }

public String pop() {

String result = (String)this.value.getFirst();

this.value = this.value.getRest();

return result;

}

public boolean isEmpty() { return this.value instanceof Empty; }

public String toString() { ... }

...

}

�

Suppose we write this code and all the test cases work. Or, more realistically, let's suppose that the code doesn't work at first, but that we manage to get it working after a few debugging cycles.

Perhaps several months later you develop a new constructor on class Cons that takes a String representation of the list as its only argument (Appendix A contains some sample code that illustrates this). Such a constructor is quite useful—it allows us to construct new lists with expressions such as the following: Listing 14-3: New Constructor Takes Only String Representation of List as Argument

�

// equivalent to new Cons("this", new Cons("is", new Cons("a", new Cons("list", new Empty()))))

new Cons("(this is a list)")


of 168

// equivalent to new Cons("so", new Cons("is", new Cons("this", new Empty())))

new Cons("(so is this)")

// equivalent to new Cons("this",

// new Cons("list",

// new Cons("contains",

// new Cons("a",

// new Cons(new Cons("nested",

// new Empty()),

// new Cons("list",

// new Empty()))))))

new Cons("(this list contains a (nested) list)")

�

So, we write this constructor and all the test cases for it work. Great! But then we notice that some of the tests for methods on class LinkedList suddenly break. What's going on?

The Cause

The problem is with the constructor on class LinkedList that takes a single String as its argument. This constructor previously called the underlying constructor on class Cons. But now that we've overloaded that constructor with a more specific method, one that takes a String as an argument, this more specific method is invoked.

Unless the String passed to the LinkedList constructor is a valid representation of a Cons, the program will crash when trying to parse it. Worse yet, if that String does happen to be a valid representation of a Cons, the program will continue to execute with corrupt data. In that case, we would have introduced a saboteur into the data (see Chapter 12).

The Broken Dispatch bug, like other bug patterns, is easiest to diagnose in code that is test infested (to borrow from the terminology of extreme programming), in which all but the most trivial methods have corresponding unit tests. (For more on XP, see "Building Cost-Effective Specifications with Stories" in Chapter 2, as well as most of Chapter 3.)

In such an environment, the most common symptom will be a test case for code you haven't touched that suddenly breaks. When this happens, it's possibly a case of the Broken Dispatch pattern. If the test case breaks immediately after you overload another method, it almost certainly is.

If the code isn't loaded with tests, things become more difficult. The bug might produce symptoms such as method invocations that return much more quickly than expected (and with incorrect results). Alternatively, you might find that certain events that were supposed to occur never did (because the appropriate method was never executed).


of 168

Remember that symptoms like these can also be attributed to other bug patterns. If you encounter such symptoms, your best bet is to begin writing more unit tests, starting with the methods in which the error was discovered and working back through the program execution.


The nice thing about this bug pattern is that there are some simple solutions. The most straightforward cure is to upcast the arguments in the method invocation. In our example, this would mean rewriting the relevant LinkedList constructor as follows: Listing 14-4: Upcasting the Arguments in the Method Invocation

�

public LinkedList(String _first) {

this.value = new Cons((Object)_first);

}

�

Of course, this approach solves the problem only for this one call to the Cons constructor. There may be other calls where we have to upcast, and this technique might become cumbersome for clients of the Cons class. In cases like these, you have to weigh the advantage you gain from having this convenient constructor against the potential for errors it introduces.

This dilemma is yet another example of the tension that exists between the expressiveness and the robustness of an interface. One way to get the best of both worlds would be to replace the String constructor on Cons with a static method that takes a String and returns a new Cons object. In fact, this is a strictly better solution in examples like this one, where we want to perform sophisticated object initialization based on the constructor arguments. Including static methods that return new instances of a class is a design pattern known as the Factory Method pattern. See Resources for more information on design patterns.

Note Argument structure is a key component to making sure that when you overload

a method, the call invokes the intended method.

A good preventative measure for this bug pattern is to avoid overloading methods entirely. Unlike method overriding, overloading is seldom needed or justified.

�

A Brisk Walk Through the Problem

This is a speedy summary of the steps we took surrounding the code in Listings 14-1 and 14-2 in this chapter. Starting with our example, we've implemented LinkedLists as containers for immutable lists.

We write a constructor to enforce the invariant that "all instances of class LinkedList will be lists of Strings."

Later, we write a new constructor that takes a String representation of the list as its only argument.

Test cases for the new constructor work, but some method tests on class LinkedList break.

Why? We've overloaded the constructor with a more specific method.

Results: If the String passed to the LinkedList constructor isn't a valid representation, it won't parse. If the String is coincidentally valid, the program will continue, but with corrupt data.


of 168

What We've Learned

In this chapter on the Broken Dispatch bug pattern we've learned the following: � Broken Dispatch can be described as follows: (1) The arguments to an overloaded

method (method1) are passed to another method (method2) that takes more general types; (2) method2 invokes method1 with these arguments; (3) Because the static types of these arguments inside the scope of method2 are more general, the wrong version of method1 might be invoked.

� Broken Dispatch errors can be difficult to diagnose because they can be introduced simply by adding new methods. Also, program execution may continue for some time before problems are discovered.

� In a test-laden environment, the most common symptom will be a test case for code you haven't touched that suddenly breaks. If the test case breaks immediately after you overload another method, this bug pattern is almost certainly the culprit.

� The most straightforward cure for this bug is to upcast the arguments in the method invocation. A good preventative measure is to avoid overloading methods.

Remember, argument structure is a key component to assuring that when you overload or override a method, the call invokes the intended method.

In Chapter 15, we'll discuss the Impostor Type and the problems that occur when tags in fields—intended to distinguish object types—mislabel the associated data


of 168

Chapter 15: The Impostor Type

Overview

When special tags in fields are used to distinguish between types of objects, errors are possible in which a tag mislabels the associated data—a bug pattern known as the Impostor Type. We'll examine the symptoms and causes, define ways to prevent this error, and discuss some tempting hybrid implementations that don't use impostor types but hold danger nonetheless.

All but the most trivial of programs manipulate some types of data. Static type systems provide a way to ensure that a program doesn't manipulate data of a given type inappropriately.

One of the advantages of the Java language is that it is strongly typed, so that the possibility of a type error is eliminated before the program is ever run. As developers, we can use this type system to produce more robust and bug-free code. Often, though, the type system is not used to its full potential.

Many programs make less use of the static type system than they could, instead relying on special fields to contain tags that distinguish the types of data. By relying on these special fields, such programs give up the very protection that the type system was designed to provide. When one of these tags mislabels its data, it generates what I call an Impostor Type bug pattern.

In a nutshell, here is our latest bug pattern:

� Pattern: Impostor Type.

� Symptoms: A program that treats data of conceptually distinct types in the same way, or that doesn't recognize certain types of data.

� Cause: The program uses fields with tags in lieu of separate classes for the various types of data.

� Cures and Preventions: Divide conceptually distinct types of data into separate classes whenever possible.

Note The static type system is there to protect you from this type of bug. Use it.Weed

bugs out during static checking About This Bug Pattern

This bug pattern results from using special fields inside classes to distinguish conceptually distinct subtypes. Because these subtypes are grouped together in the same class, static type checking is unable to catch a misuse of data that could have been caught if the subtypes were split into separate classes.

The Symptoms

One common symptom of an Impostor Type bug is that many conceptually distinct types of data are all treated in the same (and incorrect) manner when the program is run. Another common symptom is that data doesn't match any of the designated types.

As a rule of thumb, suspect this bug pattern whenever there is a mismatch between the conceptual type of data and the way it is handled by your program.

Note Common symptoms include conceptually distinct types of data being treated in

the same manner and data not matching any of the designated types.


of 168

To illustrate how easily bugs of this pattern can be introduced, let's consider a simple example. Suppose we want to manipulate various Euclidean forms, such as circles, squares, and so on. These forms will have no position, but they will have a scale, so that it will be possible to compute their area. Listing 15-1: Implementation of Shapes with Impostor Types �

public class Form {

String shape;

double scale;

public Form(String _shape, double _scale) {

this.shape = _shape;

this.scale = _scale;

}

public double getArea() {

if (shape.equals("square")) {

return scale * scale;

}

else if (shape.equals("circle")) {

return Math.PI * scale * scale;

130

}

else { // shape.equals("triangle"), an equilateral triangle

return scale * (scale * Math.sqrt(3) / 4);

}

}

}

�

There are serious disadvantages to implementing a data type in this way, even though you see it done often. One of the most glaring drawbacks is that this method is not very extensible. If we wanted to introduce a new shape for our forms (such as "pentagon"), we'd have to go in and modify the source code for the getArea() method. But extensibility is a separate concern; how does this programming style increase our susceptibility to errors?

Consider what would happen if, in some other part of the program, we constructed a new Form object as follows: Listing 15-2: Constructing a New Form

�

Form f = new Form("sqaure", 2);

�


of 168

The form "square" has been misspelled. But as far as the compiler is concerned, this is perfectly valid code.

Now consider what will happen when we try to call getArea() on our new Form object. Because the shape of the Form won't match any of the tests in the if- then-else block, its area will be computed in the else clause, as if it were a triangle!

There will be no error signaled. Indeed, in many circumstances, the return value will appear to be a perfectly reasonable number. Even if we put in some redundancy and check that the implied condition in the else clause holds (with an assertion, for instance), the error won't be found until the code is run.

Many other similar bugs might occur with the previous code. A clause might be accidentally left out of the if-then-else block, causing all Forms of the type corresponding to that clause to be handled improperly. Additionally, because the impostor type is just a String in a field, it might be modified, either accidentally or maliciously. Either way, such modifications could wreak all sorts of havoc.

The Cause

Again, the program uses fields with tags in lieu of separate classes for the various types of data.


As you might have guessed, I suggest avoiding bugs of this type by using the type system to weed them out during static checking. Consider this alternative implementation:

Now consider what would happen if we were to mistype Square as Sqaure when creating a new Form. The compiler would signal an error, telling us that class Sqaure could not be found. The errant code would never even have a chance to run.

Similarly, the compiler would not allow us to forget to define getArea() for any of our subclasses. And, of course, it would be impossible for any object to change the type of a Form. Hybrid Patterns

Before leaving this topic, I'd like to discuss two more possible implementations, each a kind of cross between the two implementations I've discussed. In these hybrid implementations, no impostor types are used, but the code has many of the same susceptibilities to error that would be present if they were. Here's the first:

Although the compiler would still catch type misspellings and the types of objects could not be changed, we are again using an if-then-else block to dispatch on the appropriate type. Therefore, we are again susceptible to mismatches between the instanceof checks in the if-then-else block and the set of types we are operating on.

The second hybrid implementation is similar to the first. It also uses separate classes to represent the variants, and defines the area function in each variant, but adds a numeric tag to each variant. The tags are included so that switch statements can be used to write procedural code to process the variants (instead of using a visitor). This approach is susceptible to mismatches in procedural code to process the variants. Some programmers implement their code this way because they believe it's faster than using dynamic dispatch. In practice, however, this technique rarely results in a noticeable performance improvement. But it does result in an increased likelihood of error.


of 168

I should also mention that neither hybrid implementation is as extensible as the solution in Listing 15-3. Listing 15-3: Implementation of Forms with Real Types

�

public abstract class Form {

double scale;

public Form(double _scale) {


}

public abstract double getArea();

}

class Square extends Form {

public Square(double _scale) {

super(_scale);

}



}

}

class Circle extends Form {

public Circle(double _scale) {

super(_scale);

}



}

}

class Triangle extends Form {

public Triangle(double _scale) {

super(_scale);

}




of 168

}

}

What We've Learned

In this chapter on the Impostor Type bug pattern, we've learned the following: � Impostor Types occur when special tags in fields are used to distinguish between types

of objects; errors are possible when a tag mislabels the associated data. � One common symptom of an Impostor Type is that many conceptually distinct types of

data are all treated in the same, incorrect, manner when the program is run. � Another common symptom is that data doesn't match any of the designated types. � Suspect the Impostor Type whenever there is a mismatch between the conceptual type

of data and the way it is handled by a program. � Using real types (as opposed to special tags) means that our type checker would signal

an error on any new instance that is entered incorrectly; it would also remind us to define methods for our subclasses.

� There are hybrid implementations that may appear to be solutions but aren't. They don't employ Impostor Types, but they do use if-then- else blocks (or switch blocks) to dispatch on the appropriate type. They make us susceptible to mismatches between the cases in the block and the set of types we're using. (See Listing 15-4.) Listing 15-4: A Cross-Breed Implementation

�

public abstract class Form {

double scale;

public Form(double _scale) {


}


if (this instanceof Square) {


}

else if (this instanceof Circle) {


}

else { // this instanceof Triangle


}

}

}

class Square extends Form {


of 168

public Square(double _scale) {

super(_scale);

}

}

class Circle extends Form {

public Circle(double _scale) {

super(_scale);

}

}

class Triangle extends Form {

public Triangle(double _scale) {

super(_scale);

}

}

�� One of the advantages of the Java language is that it is strongly typed, so that the

possibility of a type error is eliminated before a program is run.

The most important point in this chapter is that the language offers you the best resources for avoiding this type of error—the static type system. Just remember to use it.

In Chapter 16, we'll examine the Split Cleaner, an instance in which code doesn't manage resources properly by freeing them up when finished as it should.


of 168

Chapter 16: The Split Cleaner

Overview

When a program leaks or frees resources too early—resources such as memory, files, or database connections—a Split Cleaner may be the problem. We look at how the resource is managed as a potential cause.

One key feature of Java is that storage is automatically managed, saving the programmer from the bug-prone task of freeing memory after it has been used. Nevertheless, many programs still have to manipulate resources that must be explicitly freed after use. As is true in manual storage management, there are pitfalls that a programmer can encounter when managing resources in this way.

When managing a resource such as a file or a database connection, it is necessary to free the resource once you're done with it. Of course, on any given execution of the code, you want to obtain and free the resource exactly once. There are a couple of ways you could go about doing this:

� Obtain and free the resource in the same method. This way, you can guarantee that each time the resource is obtained, it is also freed.

� Trace through each possible execution path of the code. This way, you have to check that the resource is eventually freed in each case.

The second option is problematic. Because your code base will inevitably grow, another programmer who is not familiar with your code may add another execution path in which the resource is not freed, resulting in resource leakage.

I call bugs that fit this pattern Split Cleaners because the clean up code for the resource is split along the various possible execution paths. Because the clean up code along each path is likely to be identical, most Split Cleaners are also examples of Rogue Tiles (see Chapter 7).

We can summarize this bug pattern as follows:

� Pattern: Split Cleaner.

� Symptoms: A program that improperly manages resources, either by leaking them or by freeing them too early.

� Cause: Some execution paths of the program do not free the resource exactly one time as they should.

� Cures and Preventions: Move the code that handles clean up into the same method that obtains the resource.


As we said earlier, since the clean up code along each path is likely to be identical, most Split Cleaners are also examples of Rogue Tiles.

For example, suppose you are using JDBC to manipulate a table of employee names. Many of the operations you want to perform involve walking over this table and computing over the contained data. One thing you may end up doing is printing out the first names of all your employees, like so: Listing 16-1: Code That Walks over a Table of Employees


of 168

�

import java.sql.*;

public class Example {


String url = "your database url";

try {

Connection con = DriverManager.getConnection(url);

new TablePrinter(con, "Names").walk();

}

catch (SQLException e) {

throw new RuntimeException(e.toString());

}

}

}

abstract class TableWalker {

Connection con;

String tableName;

public TableWalker(Connection _con, String _tableName) {

this.con = _con;

this.tableName = _tableName;

}

public void walk() throws SQLException {

String queryString =("SELECT * FROM "+ tableName);

Statement stmt = con.createStatement();

ResultSet rs = stmt.executeQuery(queryString);

while (rs.next()) {

execute(rs);

}

con.close();

}

public abstract void execute(ResultSet rs) throws SQLException;


of 168

}

class TablePrinter extends TableWalker {

public TablePrinter(Connection _con, String _tableName) {

super(_con, _tableName);

}

public void execute(ResultSet rs) throws SQLException {

String s = rs.getString("FIRST_NAME");

System.out.println(s);

}

}

�

First, a short digression. Notice that we've pulled out the code to handle walking over the table into an abstract Walker class, allowing for new subclasses to easily walk over the rows of a table. Although it is often a waste of time to try to anticipate and code for all the ways in which a program will be extended, let's suppose that in this case there is absolutely no doubt whatsoever that exactly this kind of extension will be made to the code. (In fact, I can guarantee that just such an extension will be made before the end of this chapter.)

Tip It is often a waste of time to try to anticipate and code for all the ways in which a

program will be extended. Instead, try to find the simplest design that will solve the problem at hand, and cover your code with unit tests.When you need to extend or modify the program's functionality, your unit tests will allow you to do so with confidence.

The Symptoms

Now, notice that the database connection is passed into the constructor of the TableWalker. Once it is done walking over the table, it closes the connection. So, in this case, we've used the second strategy for cleaning up the connection: we've attempted to close the connection separately along each execution path.

Let's suppose that in the context of our system it makes sense to close the connection after a single walk over the data (for example, perhaps this code is intended to be called from the command prompt). Even in that case, we haven't caught every possible execution path: if an SQLException is thrown, the program may abort before ever closing the connection.

Because SQLExceptions are not so common in mature code, this bug may not demonstrate any symptoms for a long time, perhaps not until the original developer is no longer available. Naturally, this will make things harder to diagnose when the symptoms do appear.

But if the code is extended, there are ways in which symptoms might appear much more rapidly. For instance, let's suppose that after the original code was written, it became clear that the bulk of the phone numbers on file were out of date. So management decides that all employee phone numbers should be replaced with "411." To perform this update, a new TableWalker could be written, as follows: Listing 16-2: Code Walker to Update Old Data

�


of 168

class TableFixer extends TableWalker {

public TableFixer(Connection _con, String _tableName) {

super(_con, _tableName);

}

public void execute(ResultSet rs) throws SQLException {

String s = rs.getString("FIRST_NAME");

String updateString =

"UPDATE "+ tableName +

"SET PHONE_NUMBER = "+ "411" +

"WHERE FIRST_NAME LIKE '"+ s + "'";

Statement stmt = con.createStatement();

stmt.executeUpdate(updateString);

}

}

�

Because TableFixer also extends TableWalker, calling walk() on an instance of this class will close the connection to the database, just like TablePrinter. If a program were to try to make instances of both walkers with the same connection, it would find that the connection was closed as soon as the first walker finished its walk.

It would be easy for an incoming programmer to make this mistake, especially if the invariant—that only one walker could be constructed—wasn't documented or tested.

The Cause

Some execution paths of the program do not free the resource exactly one time as they should.


When you find an execution path that doesn't include the appropriate clean up code, you might be tempted to simply add it to that path. For example, you could wrap the body of the walk() method in a try block, and put in a finally clause to guarantee that the connection will be closed. But such a solution would be a bad fix.

There is really no reason why our TableWalkers should need to worry about closing the connection at all. But even if each TableWalker did try to close the connection, we would run into the second way in which this bug pattern can manifest itself: when we want to run multiple walkers, too many attempts are made to close the connection.

Worse still, if we were to put in two calls to con.close() (one in the try block and one in the catch block, as opposed to a single call in a finally clause), we would have introduced a Rogue Tile into the code. If many Rogue Tiles were added, it would become quite difficult to ever refactor the code successfully. Some of the Rogue Tiles might handle rare execution paths that may not occur during testing.


of 168

A much better solution would be to refactor the code to use the first approach to managing such resources: put the code for obtaining and releasing a resource in the same method. In their excellent book, The Pragmatic Programmer, Andrew Hunt and Dave Thomas advocate this idea with the phrase "Finish What You Started."

Each method should be responsible for cleaning up the resources that it acquires. In our example, this would involve moving the call to con.close() into the main method of class Example, as follows: Listing 16-3: Code Refactored So That Resources Are Obtained and Released in the Same Method �

class Example2 {


String url = "your database url";

// con must be declared and initialized here, so as to be in

// the scope of both the try and finally clauses.

Connection con = null;

try {

con = DriverManager.getConnection(url, "Fernanda", "J8");

new TablePrinter(con, "Names").walk();

}

catch (SQLException e) {


}

finally {

try {

con.close();

}

catch (Exception e) {


}

}

}

}

�

Here, the call to con.close() is in a finally clause of the same try block that created the connection; thus, an execution path in which it's not called is impossible


of 168

What We've Learned

In this chapter on the Split Cleaner bug pattern we've learned the following: � With Split Cleaners, the clean up code for a resource is split along the various possible

execution paths. � Most Split Cleaners are also examples of Rogue Tiles because the clean up code along

each path is likely to be identical. � One way to guarantee that you obtain and free a resource exactly once is to obtain and

free it in the same method. � Another (bad) way is to trace through each possible execution path of the code to ensure

that the resource is eventually freed in each instance. This will cause maintenance nightmares when a programmer unfamiliar with your code adds another execution path in which the resource is not freed.

� When you find an execution path that doesn't include the appropriate clean up code, you might be tempted to simply add it to that path. That would be a bad fix.

� Each method should be responsible for cleaning up resources that it acquires. � Extending code can make Split Cleaners appear more rapidly. � It is often a waste of time to try to anticipate and code for all the ways in which a program

will be extended.

In Chapter 17 we'll look at instances of the Fictitious Implementation, a bug pattern that involves interface implementations that violate unchecked invariants


of 168

Chapter 17: The Fictitious Implementation

Overview

When an interface implementation doesn't satisfy all necessary invariants, you may have encountered the difficult-to-diagnose Fictitious Implementation bug pattern. We look at a double threat prevention method—assertions and unit tests.

Java interfaces are a great tool—they provide many of the advantages of multiple inheritance without all of the problems. By specifying an interface for all the services that a client expects to use, you make it possible to plug in various implementations of that interface as necessary.

Unfortunately, the only parts of an interface specification that can be expressed in code are method signatures. There may very well be other invariants that are expected to hold for any implementation, but the Java language provides no facility to check them. When an implementation of an interface fails to satisfy some of these unchecked invariants, we have an instance of the Fictitious Implementation bug pattern.

We can summarize this bug pattern as follows:

� Pattern: Fictitious Implementation.

� Symptoms: A client class that works with a specified interface breaks when a certain implementation of that interface is used.

� Cause: The interface includes many intended invariants that aren't satisfied by the implementation.

� Cures and Preventions: Fix the implementation to include those invariants. Make the invariants explicit in the documentation of the interface, if they aren't already


Because Java checks only type signatures of interface implementations, it is possible to "implement" an interface without actually meeting its intended semantics. For example, consider the following interface for stacks: Listing 17-1: An Interface for Stacks �

public interface Stack {

public Object pop();

public void push(Object top);

public boolean isEmpty();

}

�

Any class containing methods that match the above signatures would, from the perspective of the Java type checker, serve as a legal implementation of a Stack. But in practice there are several additional requirements we would expect a stack to fulfill. For instance: � If an object o is pushed on a stack s, and the next operation performed on the stack is

pop, then the return value of that operation should be o. � If, for a given stack s, the return value of s.isEmpty() is true, and the next operation

performed on the stack is pop, then that call to pop should throw a RuntimeException.


of 168

There are lots of other invariants we could specify, such as: � How do we expect a stack to handle multiple push operations? � What behavior do we expect with multiple threads?

It is difficult to enforce invariants such as these programmatically. We could (and should) mention them in the documentation, but a developer writing an implementation could easily ignore them. If that happens, then a client that relies on such invariants will not work with the implementation, and we'll have a bug.

I call bugs of this pattern Fictitious Implementations because I place the blame for them squarely on the implementation rather than on the client. Like any bug that deserves its own pattern, a Fictitious Implementation may not be immediately apparent, but can lurk hidden until some uncommon execution path uncovers it.

The Symptoms

A client class that works with a specified interface breaks when a certain implementation of that interface is used.

The Cause

The interface includes many intended invariants that aren't satisfied by the implementation.

Detecting Fictitious Implementations

The main problem with Fictitious Implementations is, of course, that they will pass compilation without incident. At runtime, the symptoms will often seem puzzling because the extra invariants that a programmer expects an interface to satisfy are often left unspoken—in fact, the programmer may not even be consciously aware that he is expecting them to be satisfied.

The process of correcting a bug often starts with a period of confusion. The programmer, tripped up by the Fictitious Implementation pattern, may at first try to convince himself that the problem he has observed cannot possibly have occurred.

If you find yourself in this situation, it's a good time to check your premises: � What hidden assumptions have you not stated? � How can you test these assumptions to thoroughly eliminate the possibility that they are

faulty?

If you rely on an interface to another part of the system, and the implementation of that interface has been modified since the last release, then you might be running up against a Fictitious Implementation.


Now let's examine various methods and techniques to fix and prevent this problem.

Some Ideas for Cures

In cases such as these, it is important that the maintainer of the interface document any invariants that may be assumed by a client programmer. If a client programmer discovers that an invariant he was relying on was not, in fact, documented, then the client programmer and the maintainer of the interface should sit down and discuss whether that invariant should be made


of 168

explicit. Often, it will be easy to add an assumed invariant to the specification, saving the client programmer the trouble of modifying all the code that relied on it.

If the maintainer of the interface is not available, then the client programmer cannot rely on anything but the interface's documented invariants. If these are scant, then the interface is much less valuable than it could be. If the client programmer chooses to rely on undocumented invariants, the client code he writes can quickly lose value, since it may be incompatible with future releases of the interface implementation.

Tip Client code that relies on undocumented interface invariants can quickly lose

value since it may be incompatible with future releases of the interface implementation.

Two Prevention Concepts

If you're designing an interface, you have two very powerful tools to prevent bugs of this pattern: assertions and unit tests. Let's discuss how these technologies can be used as a sort of executable documentation to aid in enforcing interface invariants.

Assertions

The addition of assertions to a program is an old but good technique that is underused. The idea is to put in boolean checks for certain conditions at various stages in the execution of the program. According to the idea of design by contract (see the Resources chapter for more information), assertions should be included in the agreement that the implementation of an interface makes with outside clients.

Usually, assertions come in one of three varieties: � A precondition checks that some condition holds just before entering a code block. � A postcondition checks that some condition holds when exiting a code block. � An invariant checks that some condition holds during the execution of a code block.

Because of their expense, assertions of the last category are rarely supported in their most general form. Instead, the programmer is allowed to check that various conditions hold before and after the code block's execution.

In the case of interface specifications in which no implementation code is given, the first two categories are most useful.

With the introduction of Java-based preprocessors such as iContract, it is possible to place assertions into your source code and have them automatically converted into Java code that checks to ensure that the assertions are never violated. Unfortunately, iContract (and similar tools) do not catch all of the errors that can occur when using assertions in the context of an object-oriented language. In particular, iContract does not ensure that assertions in a class maintain the appropriate relationship with assertions in a superclass. This limitation often results in uninformative error messages during assertion violations. Although the theory behind the complete set of checks to perform is well understood, no production tools yet perform all of these checks correctly for Java. So, right now, we have no choice but to make do with iContract. See the Resources chapter for more information on this issue.

�

Leaving Assertions in Production Code

Because the assertions processed by iContract are specified as Javadoc comments in the original file, it is easy to compile this file without running the iContract preprocessor in order to make a "production" copy of the code in which none of the assertions are checked. But assertions are removed in this way too often.


of 168

In all but the most performance-critical sections of a program, the overhead of assertion checking will not be significant. By leaving in assertions, you make it easier to diagnose bug reports from end users (and there will be bug reports).

In our stack example, we could add an assertion to pop that ensures that it is never called on an empty stack: Listing 17-3: An Assertion to Test a Stack's Interface

�

public interface Stack {

/**

*@pre ! this.isEmpty()

*/


public void push(Object top);


}

Adding assertions such as these to interface code can help ensure that such additional invariants hold when the methods of an implementation are called. Because the assertions can be compiled into the code, they are a powerful way to quickly diagnose the occurrence of fictitious implementations. What's more, they serve as added documentation for the interface.

But, because they are strictly functional boolean expressions, assertions are limited in their expressiveness. How would we encode our first rule for stacks into an assertion, for instance?

Like types, assertions are not expressive enough by themselves to capture all of the rules we may want to specify on an interface. For this reason, they are best used in tandem with unit tests.

Unit Tests

As discussed in Chapters 2 through 5, unit tests form a very precise form of documentation. Documentation in the form of unit tests also has the advantage of being executable. Using a unit testing framework such as JUnit (see Resources), you can easily check that such unit tests hold for any implementation of an interface.

Note Unit tests are a precise form of documentation, documentation that has the

added advantage of being executable.

The extent to which unit testing can aid in eliminating occurrences of fictitious implementations cannot be overemphasized. In fact, unit tests are an excellent way to provide limited specification of these extra invariants. An interface that comes with an accompanying set of unit tests gives the implementer a means to check that the extra invariants of the interface are satisfied.

I highly recommend providing such tests with any interface that will be used by outside clients. They'll thank you for it. Even in-house interfaces will be much easier to implement with an accompanying test suite.

Of course, unlike type declarations, a finite set of tests cannot check an implementation over all possible inputs. But unit tests can be thorough enough that we can reasonably expect them to catch most violations of the invariants. And they are, of course, much more expressive than type signatures.


of 168

When documenting an interface with unit tests, you can be much more precise in describing invariants than you can be in prose. For example, consider the following tests to check invariants on stacks: Listing 17-4: Unit Tests for Stacks

public void testPushAndPop() {

Stack s = new MyStack();

Object o = new Object();

s.push(o);

assertTrue(o == s.pop());

}

public void testPopOnEmpty() {

Stack s = new MyStack();

assertTrue(s.isEmpty());

try {

s.pop();

}

catch (RuntimeException e) {

return;

}

throw new RuntimeException("pop on empty stack does not fail");

}

Compare these tests to the invariants for stacks as we originally specified them in English. Unlike the unit tests, those English descriptions leave many things open for interpretation. For example, when the first rule states that "the return value of that operation will be o," does this mean that the return value will satisfy an equals test with the pushed object or that it will actually satisfy ==? The unit test makes this very clear.

A few more things to notice about these tests: � They are small and straightforward. Because the unit tests for an interface should also

serve as documentation, it is essential that they be as easy to read as possible. � Because they can be arbitrary Java code, they allow us to test complex behaviors of an

implementation. For example, notice that the second method actually tests that an exception is thrown when it should be; if the exception isn't thrown, the test fails!

The fact that unit tests are so expressive certainly has advantages. It allows us to capture the essence of any rule for an interface that we would want to specify. But this expressiveness also has a disadvantage—we can specify examples of a rule, but, as noted, we can't use unit tests to check that a rule holds for all possible inputs to a program.

Combining the Fixes

We can now consider the three languages for the specification of an interface—the unit-testing language, the assertion language, and the type system—to form a hierarchy of expressiveness. Each step up in the hierarchy is achieved at the expense of a decrease in the testability of the language.


of 168

This hierarchy captures the common and fundamental tension between expressiveness and testability. By incorporating several such specification languages for our interfaces, it is possible to get the best of both worlds.

As these examples have shown, assertions and unit tests are powerful ways to avoid Fictitious Implementations, providing checkable specifications for an interface. What's more, the kinds of invariants that they check are complementary. Ideally, an interface would include both.

Notice that the inclusion of such specifications doesn't just catch errors in completed implementations; it actually helps the would-be implementer to ensure that he is correctly implementing the interface while he is programming. Not only can this improve productivity, but it can also make for happier programmers. It's always nice to send your code through an automated checking tool—and watch it pass. What We've Learned

In this chapter on the Fictitious Implementation bug pattern we've learned the following: � Specifying an interface for all the services that a client expects to use makes it possible

to plug in various implementations of that interface as necessary. � The only parts of a specification that can be checked statically are method signatures. � When an implementation of an interface fails to satisfy some necessary invariants, we

have an instance of the Fictitious Implementation bug pattern. � Fictitious Implementations may not be immediately apparent, but can lurk hidden until

some uncommon execution path uncovers them. They may pass compilation without incident.

� Because Java checks only the type signatures of interface implementations, it is possible to "implement" an interface without actually meeting its intended semantics.

� A disadvantage: many invariants we would like to specify cannot be checked statically. � The addition of assertions—boolean checks for certain conditions at various stages in the

execution of a program—is a potential method for preventing Fictitious Implementations. Because they can be compiled into the code, assertions are a fast and powerful diagnostic tool.

� Assertions should be included in the agreement the implementation of an interface makes with outside clients.

� The three kinds of assertions are: (1) a precondition that checks that some condition holds just before entering a code block; (2) a postcondition that checks that some condition holds when exiting a code block; and (3) an invariant that checks that some condition holds during the execution of a code block. (Invariants are rarely supported in the general form due to expense.)

� In all but the most performance-critical sections of a program, the overhead of assertion checking will not be significant. Leaving assertions in makes it easier to diagnose bug reports from users.

� Like types, assertions are not expressive enough by themselves to capture all of the rules we may want to specify on an interface. For this reason, they are best used in tandem with unit tests.

� Unit tests provide limited specification of extra invariants as a means for an implementer to check that the extra invariants of an interface are satisfied.

� Unlike type declarations, unit tests cannot check an implementation over all possible inputs.

� Unit tests should be small, straightforward, and easy to read. Because they can be arbitrary Java code, they allow us to test complex behaviors of an implementation.

� It is important that the maintainer of the interface document any invariants that may be assumed by a client programmer.

� Client code that relies on undocumented invariants can quickly lose value since it may be incompatible with future releases of the interface implementation.


of 168

In Chapter 18, we'll look at instances of the Orphaned Thread, a bug pattern that can make using the marvelous ability of multithreading a bit of a headache


of 168

Chapter 18: The Orphaned Thread

Overview

When your multithread program freezes and stack traces don't print to standard error, there's a chance you've discovered the hard-to-track Orphaned Thread bug pattern. We discuss the obvious fix and detail ways to detect and prevent this bug.

Writing code with multiple threads can make the programming (and the program) go much faster—and the code can use resources much more effectively. However, as with most things in life, there are drawbacks. Because multithreaded code is inherently nondeterministic, the potential for errors is much greater. What's more, the errors that do occur will be much harder to reproduce, and therefore tougher to pin down.

The Java programming language provides abundant support for multithreaded code, including one feature that can be especially useful: the ability to throw an exception in one thread without affecting the others. But this feature can cause many hard-to-track bugs.

Tip Multithreaded code is inherently nondeterministic, so it can potentially introduce

errors that will be much harder to reproduce and identify.

In cases where it makes sense to recover from a crash in one of the threads, this ability can add a level of robustness to a program. It can, however, make it difficult to determine which thread has thrown an exception. Because the remaining threads will continue to run, the program may exhibit the signs of an unresponsive or frozen program. This is particularly true in a program where the threads communicate with each other often. I call this the Orphaned Thread bug pattern.

Here is this bug pattern in a nutshell:

� Pattern: Orphaned Thread.

� Symptoms: A multithreaded program locks up with or without printing stack traces to standard error.

� Cause: Various program threads are stuck waiting for input from a thread that exited after an uncaught exception was thrown.

� Cures and Preventions: Put exception-handling code in the main thread to notify the dependent threads of the exceptional condition. Alternatively, put a handler into the thread that exited so that it passes information to its clients instead


The fact that one thread can throw an exception without affecting other threads can make it difficult to determine which thread has actually thrown the exception. Consider the example shown in Listing 18-1, in which a pair of threads communicate via a producer-consumer model. Listing 18-1: A Simple, Multithreaded Consumer-Producer Program. �

public class Server extends Thread {

Client client;

int counter;


of 168

public Server(Client _client) {

this.client = _client;

this.counter = 0;

}

public void run() {

while (counter < 10) {

this.client.queue.addElement(new Integer(counter));

counter++;

}

throw new RuntimeException("counter >= 10");

}


Client c = new Client();

Server s = new Server(c);

c.start();

s.start();

}

}

class Client extends Thread {

Vector queue;

public Client() {

this.queue = new Vector();

}

public void run() {

while (true) {

if (! (queue.size() == 0)) {

processNextElement();

}

}

}

private void processNextElement() {

Object next = queue.elementAt(0);

queue.removeElementAt(0);

System.out.println(next);

}


of 168

}

�

In a case such as this, the second thread is entirely dependent upon the first for the data it needs to compute. If the first thread crashes (and in Listing 18-1 it is guaranteed to do so), the second thread will inevitably wait for further input that will never come, essentially freezing the program. The second thread has been abandoned, which is why I call this pattern of bug the Orphaned Thread pattern.

The Symptoms

The most common symptom of this bug pattern is the one I mentioned earlier—namely, that the program will appear to freeze. If you see stack traces printed to standard error and to standard out, that's another symptom.

Such stack traces are especially common with GUI applications on Unix-based systems in which the applications are launched from a terminal window. When a GUI application spills a stack trace to the terminal while freezing, strongly suspect an Orphaned Thread. (For more discussion of Orphaned Threads in the context of GUIs, see the section entitled "Orphaned Threads and GUIs" later in this chapter.)

The Cause

Various program threads are stuck waiting for input from a thread that exited after an uncaught exception was thrown.


Once a bug of this pattern has been diagnosed, the obvious cure is to find and fix the underlying error in the crashing thread. Prevention, however, is more difficult. If you can get away with using a single-threaded design, you'll eliminate many headaches. Chances are, however, that if you settled on a multithreaded design in the first place, there was a good reason for that design choice.

One way to aid in the diagnosis of such crashes is to catch the exceptions thrown in the various threads and notify the dependent threads of the problem before exiting. This is what we've done in Listing 18-2. Listing 18-2: Example in Which the Client Thread Is Notified of an Error �


public class Server2 extends Thread {

Client2 client;

int counter;

public Server2(Client2 _client) {

this.client = _client;

this.counter = 0;

}


of 168

public void run() {

try {

while (counter < 10) {

this.client.queue.addElement(new Integer(counter));

counter++;

}

throw new RuntimeException("counter >= 10");

}

catch (Exception e) {

this.client.interruptFlag = true;


}

}


Client2 c = new Client2();

Server2 s = new Server2(c);

c.start();

s.start();

}

}

class Client2 extends Thread {

Vector queue;

boolean interruptFlag;

public Client2() {

this.queue = new Vector();

this.interruptFlag = false;

}

public void run() {

while (! interruptFlag) {

if (! (queue.size() == 0)) {


}

}

// Processes whatever elements remain on the queue before exiting.

while (! (queue.size() == 0)) {


of 168


}

System.out.flush();

}

private void processNextElement() {

Object next = queue.elementAt(0);

queue.removeElementAt(0);

System.out.println(next);

}

}

�

If the exceptional circumstance indicates a bug in the program, another option for handling the thrown exception would be to put up a short message notifying the user of the problem and then call System.exit. This option is preferable to allowing the program to freeze, since a user can never really be sure that a program is frozen. He may decide that if he waits just a little longer, it'll come back. But an explicit message indicating a catastrophic error is unmistakable.

This option particularly makes sense when the crash occurs in the program's main thread and the other threads don't manage any critical resources. In other cases, though, it can be dangerous. For example, what if one of the other threads were accessing a shared database? In that case, simply exiting the program could prevent that thread from freeing a lock on the database. Even in the simple example above, calling System.exit in the server thread would cause the client to exit without processing any remaining elements on its queue.

In fact, problems such as these were what spurred Sun to deprecate the stop() method on threads. That method allowed the termination of a thread without its consent. Because it may leave resources in an inconsistent state, the stop() method violated the security model of the language.

Tip Sun deprecated the stop() method on threads because terminating a thread

without its consent leaves resources in an inconsistent state, thereby violating Java's security model.

In addition to freeing shared resources, we'd also like to offer the user the opportunity to save unsaved data and perhaps file a bug report (as is common in many newer applications, especially on Windows XP). Ideally, tasks like saving files, filing bug reports, and unlocking shared resources could all be done in the exception handling code before calling System.exit.

But remember, we are assuming that there is a bug in the program—the clean up code might trip over the same bug! That would result in a recursive cascade of error messages to the user, which is clearly not what we want. On the other hand, perhaps most users would prefer to risk this cascade of messages in exchange for the chance to save their data Orphaned Threads and GUIs

In the context of GUI programming, it is quite easy to introduce orphaned threads. Often, GUI programs will spawn threads in response to various events, each time locking the application until the spawned thread returns. But if the spawned thread throws an unchecked exception, you've got an orphaned thread.


of 168

In our first attempt to integrate support for JUnit into DrJava, we added a Test button to the toolbar. When pressed on a valid subclass of junit.framework.TestCase, this button would spawn a separate thread to invoke JUnit on the corresponding class file.

As with the interactions pane, DrJava would not require the user to explicitly put a class on the system classpath before trying to test it. Instead, DrJava would check the package name of the test class and automatically add the implied root directory of the source packages to the class path passed to JUnit. Provided that the classfiles were kept in the same directories as the corresponding source files (as is done by default in DrJava), this would work fine.

Also, if the user modified the file, DrJava would prompt him to save and compile before calling JUnit. But if the user had compiled the classfile for a test class to a package directory hierarchy distinct from that of the source file, the directory hierarchy for the classfile would not be on his system classpath. If the user opened the corresponding source file and immediately called Test on it, JUnit would be unable to locate the corresponding class file. It would throw a ClassNotFoundException.

But this scenario was overlooked, which meant that we didn't code in logic to handle these ClassNotFoundExceptions. So they propagated to the top level of the spawned thread, resulting in its termination.

Meanwhile, the application would lock until the spawned thread returned a report of the result of running JUnit on the test class—which, of course, would never happen. The result? Every user's worst nightmare: a frozen application.

Once this problem was identified, the solution was easy: catch the ClassNotFoundException thrown from JUnit and generate an appropriate report of the result. Of course, a new unit test was put into place to make sure that this particular bug never occurred again.

Unit Tests and Multithreading

Because DrJava makes heavy use of multithreading, we have encountered Orphaned Threads, along with other multithreading bugs, on many occasions. But, luckily, few of these bugs make it past our unit tests. One main reason that the unit tests save us so often from multithreading problems is that the tests interact with the application much more quickly than any human user could. This ultrafast interaction tends to cause all of the program threads to interact in unexpected ways, revealing bugs that a user wouldn't normally encounter. (Of course, on the rare occasion that a user did happen to trip up on such a bug, it would be extraordinarily difficult to diagnose, or even repeat.) For this reason, a strong suite of unit tests over multithreaded code is essential, even more so than in other contexts.

(For more on the DrJava project, see Resources, "The GlobalModel Interface" in Chapter 4, "Small Anomalies, Big Problems" in Chapter 5, and "A Peek at a Real-World Example" in Chapter 12.)

As the earlier examples demonstrate, Orphaned Threads can have devastating symptoms. And in the context of a multithreaded application, they aren't hard to generate.

Tip Use the sharp knife of multithreading with caution. Although it can deliver a more

efficient and understandable program, it is an easy way to break an invariant expected to hold in one thread


of 168

What We've Learned

In this chapter on the Orphaned Thread bug pattern we've learned the following: � The most common symptom of this pattern is that the program will appear to freeze.

Stack traces printed to standard error and standard out are another clue. � The obvious cure is to find and fix the underlying error in the crashing thread. Prevention

is a more difficult issue. � One way to diagnose this problem is to catch the exceptions thrown in the various

threads and notify the dependent threads of the problem before exiting. � An option for handling the thrown exception would be a message notifying the user of the

problem and a call to System.exit. This option doesn't allow the program to freeze and it serves as an unmistakable marker for the user. This option works when the crash occurs in the program's main thread and other threads don't manage critical resources—but it can be dangerous in other cases.

� The ideal situation is one in which tasks like saving files, filing bug reports, and unlocking shared resources could all be done in the exception handling code before calling System.exit.

� Multithreaded code is inherently nondeterministic, so its potential for errors is greater and those errors will be much harder to reproduce and identify.

� Unit tests are a great way to discover bugs in multithreaded code, as the interactions between the threads tend to occur differently than they do when a human user interacts with the program.

� Java provides abundant support for multithreaded code, including the useful ability to throw an exception in one thread without affecting the others—but this feature can cause hard-to-track bugs.

In Chapter 19, we'll look at instances of the Run-On Initialization bug pattern, in which class constructors in the definitions don't take enough arguments to properly initialize all the fields of a class


of 168

Chapter 19: The Run-On Initialization

Overview

When you get a NullPointerException and you suspect that class-definition constructors don't take enough arguments to initialize all the class's fields, you may be looking at the Run-On Initializer pattern. We discuss initializing all constructor fields with special classes and including multiple constructors.

For various reasons, mostly bad, you will often see class definitions in which the class constructors don't take enough arguments to properly initialize all the fields of the class. Such constructors require client classes to initialize instances in several steps (setting the values of the uninitialized fields) rather than with a single constructor call.

Initializing an instance in this way is an error-prone process that I refer to as a Run-On Initialization. The types of bugs that result from this process have similar symptoms and remedies, so we can group them together into the Run-On Initializer bug pattern.

Here is this pattern in a nutshell:

� Pattern: Run-on Initializer.

� Symptoms: A NullPointerException at the point that one of the uninitialized fields is accessed.

� Cause: A class whose constructors don't initialize all fields directly.

� Cures and Preventions: Initialize all fields in a constructor. Use special classes for default values when better values can't be used. Include multiple constructors to cover cases where better values can be used. When your hands are tied, at least include an isInitialized() method


In this bug pattern, several steps are necessary in order to initialize an instance of a class. As a result, initialization is more prone to error. If one of the fields is not initialized, a NullPointerException can result.

The Symptoms and the Cause

This pattern is indicated by a NullPointerException at the point that one of the uninitialized fields is accessed; there is a class whose constructors don't initialize all fields directly. For example, consider the following code:

Unfortunately, the initialization sequence for an instance of this class is prone to bugs. You may have noticed that an exception is thrown in the second initialization step. As a result, the field that should have been set after that step is not set.

But a handler for the thrown exception may not know that the field was not set. If, in the process of recovering from the exception, it accesses the value field of the RestrictedInt in question, it may trip over a NullPointerException itself.

If that happens, we are in worse shape than we would be if the handler weren't there at all. At least the checked exception contained some clue about its cause. But NullPointerExceptions are notoriously difficult to diagnose because they (necessarily)


of 168

contain very little information as to why a value was set to null in the first place. Furthermore, they occur only when the uninitialized field is accessed. That access will probably occur far away from the cause of the bug—that is, from the failure to initialize the field in the first place.

There are, of course, other errors that can occur from run-on initialization bugs. For instance: � The programmer writing the initialization code may forget to put in one of the initialization

steps. � There may be an order-based dependence in the initialization steps that is unknown to

the programmer, who therefore executes the statements out of order. � The class being initialized might change. New fields might be added, or old ones

removed. As a result, all the initialization code in every client must be modified to set the fields appropriately. Much of the modified code will be similar, but if just one copy is missed, a bug is introduced. For this reason, run-on initializers can easily become rogue tiles (see Chapter 7).

Because of all the problems involved with Run-On Initialization, it's much better to define constructors that initialize all fields. In Listing 19-1, the constructor for RestrictedInt should take an int to initialize its value field. There's never a good reason to include a constructor for a class that leaves any of the fields uninitialized. When writing classes from scratch, that's not a difficult principle to follow. Listing 19-1: A Simple Run-On Initialization

�

class RestrictedInt {

public Integer value;

public boolean canTakeZero;

public RestrictedInt(boolean _canTakeZero) {

canTakeZero = _canTakeZero;

}

public void setValue(int _value) throws CantTakeZeroException {

if (_value == 0) {

if (canTakeZero) {

value = new Integer(_value);

}

else {

throw new CantTakeZeroException(this);

}

}

else {

value = new Integer(_value);

}

}

}

class CantTakeZeroException extends Exception {


of 168

public RestrictedInt ri;

public CantTakeZeroException(RestrictedInt _ri) {

super("RestrictedInt can't take zero");

ri = _ri;

}

}

class Client {

public static void initialize() throws CantTakeZeroException {

RestrictedInt ri = new RestrictedInt(false);

ri.setValue(0);

}

}

�

�

Tip There's

never a good reason to include a constructor for a class that leaves any of the fields uninitialized.


Let's look at several things you can do to help eliminate this type of initializer.

For Legacy Code

What if you must work with a large codebase in which a class doesn't initialize all of its fields in the constructors—a codebase littered with run-on initializers?

Unfortunately, many programmers find themselves working with legacy codebases in which a class doesn't initialize all of its fields in the constructors more often than they'd like. If the legacy codebase is large and the offending class has many clients, you may not want to modify the constructor signatures, especially if the unit tests over the code are scant. Inevitably, you'll break undocumented invariants.


of 168

Often the best thing to do is to throw out that legacy code and start fresh! That may sound like crazy talk, but the time you'll spend patching up bugs in code like that can easily dwarf the time it would take to rewrite it. Many times, I have struggled to work with large bases of legacy code with problems like this, and ultimately I come away wishing I had just started fresh.

But if throwing the code away is not an option, we can still attempt to control the potential for errors by incorporating the following simple practices: � Initialize the fields to non-null, default values. � Include extra constructors. � Include an isInitialized() method in the class. � Construct special classes to represent the default values.

Let's take a look at why we should follow these practices.

Initialize the Fields to Non-Null, Default Values

By filling in the fields with default values, you help to ensure that instances of your class will be in a well-defined state at all times. This practice is particularly important for fields of reference type that will take on the null value unless you specify otherwise.

Why? Because gratuitous uses of null values inevitably result in NullPointerExceptions. And NullPointerExceptions are bad. For one thing, they provide very little information about the true cause of a bug. For another, they tend to be thrown very far away from the actual cause of the bug. Avoid them at all costs.

And if you decide you want to use null values so that you can signal that the class is not yet completely initialized, see Chapter 10 for assistance.

Tip Remember, gratuitous uses of null values inevitably result in

NullPointerExceptions. And NullPointerExceptions are bad.

Include Extra Constructors

When you include additional constructors, you can use them in new contexts, where you don't have to include new run-on initializations. Just because some contexts are forced to use this bad code, other contexts shouldn't have to pay the price.

Place an isInitialized() Method in the Class

You can include an isInitialized() method in the class to allow for quick determination as to whether an instance has been initialized. Such a method is almost always a good idea when working with classes that require run-on initialization.

In cases in which you don't maintain these classes yourself, you can even put such isInitialized() methods into your own utility class. After all, if there is a consequence of an instance not being initialized that is observable from the outside, you can write a method to check for this consequence (even if it entails using the usually ill-advised practice of catching a RuntimeException).

Construct Special Classes to Represent the Default Values

Instead of allowing the fields to be filled in with null values, construct special classes (most likely with Singletons) to represent the default values. Then fill instances of these classes into your fields in the default constructor. Not only will you decrease the chances of a NullPointerException, but you will be able to control precisely which error does occur if


of 168

these fields are accessed inappropriately. (For more on Singletons, see Resources and "The Composite and Singleton Design Patterns" in Chapter 9.)

For example, we could modify the RestrictedInt class as follows: Listing 19-2: RestrictedInts with NonValues

�

class RestrictedInt implements SimpleInteger {

public SimpleInteger value;




value = NonValue.ONLY;

}


if (_value == 0) {

if (canTakeZero) {

value = new DefaultSimpleInteger(_value);

}

else {


}

}

else {


}

}

public int intValue() {

return ((DefaultSimpleInteger)value).intValue();

}

}

interface SimpleInteger {

}

class NonValue implements SimpleInteger {

public static NonValue ONLY = new NonValue();

private NonValue() {}


of 168

}

class DefaultSimpleInteger implements SimpleInteger {

private int value;

public DefaultSimpleInteger(int _value) {

value = _value;

}


return value;

}

}

�

Now, if any of your client classes that access this field were to perform an intValue() operation on the resulting element, they would first have to cast to a DefaultSimpleInteger, since NonValues don't support that operation.

The advantage of this approach is that you'll be constantly reminded (with compiler errors) at every point in the code where you forgot to cast that this method call doesn't work on the default value. Also, if at runtime you happen to access this field and it contains the default value, you'll get a ClassCastException, which will be much more informative than a NullPointerException—the ClassCastException will tell you not only what was actually there, but what the program expected to be there as well, and it will occur when the cast is attempted rather than at some later point in the execution when you dereference the value.

The disadvantage is that you'll pay in performance. Every time the field is accessed, the program will also have to perform a cast.

If you're willing to do without the compilation error messages, another solution is to include the intValue() method in interface SimpleInteger. You can then implement this method in the default class with a method that throws whatever error you'd like (and you can include any information that you'd like in the error). To illustrate this, look at the following example: Listing 19-3: NonValues That Throw Exceptions

�

class RestrictedInt implements SimpleInteger {

public SimpleInteger value;




value = NonValue.ONLY;

}


of 168


if (_value == 0) {

if (canTakeZero) {


}

else {


}

}

else {


}

}


return value.intValue();

}

}

interface SimpleInteger {

public int intValue();

}

class NonValue implements SimpleInteger {

public static NonValue ONLY = new NonValue();

private NonValue() {}


throw new

RuntimeException("Attempt to access an int from a NonValue");

}

}

class DefaultSimpleInteger implements SimpleInteger {

private int value;

public DefaultSimpleInteger(int _value) {


of 168

value = _value;

}


return value;

}

}

�

This solution can provide even better error diagnostics than the ClassCastException. It's also more efficient, because no cast is required at runtime. But this solution won't require you to think about the possible values of the field at every access point.

Which solution is best? That depends partly on your personal style and partly on the performance constraints of your project.

Including Methods That Only Throw Exceptions

That last solution in the previous section may, at first glance, seem completely wrong. We added a method that does nothing but throw an exception, whenever it's called.

At first, this practice may strike you as inherently wrong and counterintuitive—after all, a class should only contain methods that actually make sense to perform on the data, right? Including classes such as these can be particularly confusing when you are teaching programmers about object-oriented programming.

For example, consider the two possible ways to define a class hierarchy for Lists, shown in Listings 19-4 and 19-5: Listing 19-4: Lists with No Universal getters

�

abstract class List {}

class Empty extends List {}

class Cons extends List {

Object first;

List rest;


first = _first;

rest = _rest;

}


return first;

}


of 168

public List getRest() {

return rest;

}

}

�Listing 19-5: Lists with getters in the Interface

�

abstract class List {

public abstract Object getFirst();

public abstract Object getRest();

}

class Empty extends List {


throw new RuntimeException("Attempt to take first of an empty list");

}


throw new RuntimeException("Attempt to take rest of an empty list");

}

}

class Cons extends List {

Object first;

List rest;


first = _first;

rest = _rest;

}


return first;

}


return rest;

}

}

�


of 168

For a programmer new to object-oriented languages, the motivations behind the first version of List (the one with no universal getters) will be less confusing. Your intuition tells you that a class shouldn't contain a method unless that method does real work. But the above considerations for dealing with default classes apply equally well to this example too.

It can be quite cumbersome to continually insert casts in your code; the code can become quite wordy. Additionally, the class casts can have significant repercussions in terms of performance, especially for an often-called utility class like List.

As with all design practices, this practice is best applied with a consideration for the underlying motivation of the practice. The motivation won't always be applicable; when it isn't, the practice shouldn't be used You're Better Off Fixing Them

You might have noticed that the Run-On Initializer bug is a bit different from some of the other patterns we've worked with. This time we've provided quite a few ideas on how to work around the root causes, rather than just fixing them outright. That's because, on many occasions, you will have to work around them.

As the considerations we've mentioned indicate, it is far better to avoid run-on initializations altogether. But when you have to deal with them, at least you can protect yourself What We've Learned

In this chapter on the Run-On Initializer bug pattern we've learned the following: � This bug is caused by class definitions in which the class constructors don't take enough

arguments to properly initialize all the fields of the class. � A potential error: The programmer writing the initialization code may forget to put in one

of the initialization steps. � Another potential error: There may be an order-based dependence in the initialization

steps, unknown to the programmer, who therefore executes the statements out of order. � Yet another error: The class being initialized might change. New fields might be added, or

old ones removed. � A sure prevention: It's much better to define constructors that initialize all fields. � If you encounter legacy code infected with this bug, often the best thing to do is to throw

out that legacy code and start fresh! � However, when you must work with a legacy codebase in which a class doesn't initialize

all of its fields in the constructors, you may not want to modify the constructor signatures, especially if the unit tests over the code are scant.

� When you must work with legacy code, attempt to control the errors by (1) initializing the fields to non-null, default values; (2) including extra constructors; (3) including an isInitialized() method in the class; and (4) constructing special classes to represent the default values.

� A good practice is to fill in the fields with default values; you help to ensure that instances of your class will be in a well-defined state at all times.

� Remember, gratuitous uses of null values inevitably result in NullPointerExceptions—and those are bad.

� Include additional constructors. You can use them in new contexts where you don't have to include new run-on initializations.

� An isInitialized() method in the class will allow for quick determination as to whether an instance has been initialized.

� Construct special classes to represent the default values. The advantage is that you'll be constantly reminded with compiler errors (at every point in the code where you forgot to cast) that this method call doesn't work on the default value. The disadvantage: a performance hit each time the field is accessed (as a cast is needed).


of 168

� There's never a good reason to include a constructor for a class that leaves any of the fields uninitialized.

In Chapter 20 we'll take a look at group of bugs that cluster around a single issue, that of platform dependence


of 168

Chapter 20: Platform-Dependent Patterns

Overview

When you detect any problem that can't be replicated across the vendor, version, or operating system spectrum, chances are the bug is linked to the platform in some way. We discuss the key to diagnosis with this family of patterns—recognition.

One of the main advantages to programming in the Java language is the tremendous degree of platform independence it allows. Rather than having to form separate builds of a product for each target platform, you can simply compile to bytecode and distribute to any platform with a Java Virtual Machine. Or at least that's the way the story is supposed to go.

Unfortunately, it's not quite that simple. Although Java programming can save untold hours of developer support for multiple platforms, there are many compatibility snags across different JVM versions.

Some of these snags, such as incorrect platform-specific separator characters in path names, are easy to spot and correct. But others can be difficult or impossible to intercept. For that reason, it's important to keep in mind the possibility that some anomalous program behavior that defies explanation may be a bug in a particular JVM.

The cost of writing cross-platform code is never zero! About Platform Dependence

We'll break down our discussion of platform-dependent bugs into three separate patterns: vendor-, version-, and operating system–dependent bug patterns.

� Pattern: Vendor-dependent bugs.

� Symptoms: Errors may occur on some JVMs, but not on others.

� Cause: There are some unspecified areas in the JVM specification (such as no required optimization of tail-calls, for example). This type of bug is less common than version-dependent bugs.

� Cures and Preventions: Varies for the problems encountered.

� Pattern: Version-dependent bugs.

� Symptoms: Errors may occur on some versions of a JVM, but not on others.

� Cause: Bugs in certain versions of a particular vendor's JVM. This is a more common cause than the vendor-dependent bugs.


� Pattern: OS-dependent bugs.

� Symptoms: Errors may occur on some operating systems, but not on others.

� Cause: Rules of system behavior are different on different operating systems (for instance, on Unix, open files can be deleted; on Windows, they cannot).



of 168

Vendor-Dependent Bugs

Of course, if you want to see some of the many subtle platform-dependent bugs that exist in JVMs, you need only make a casual inspection of Sun's Java Bug Parade (see Resources). Many of the bugs listed there are implementation bugs that apply only to JVMs on one specific platform. If you don't happen to be developing on that platform, you may not even know that your program trips over it.

But not all Java platform dependence results from JVM implementation bugs. Significant platform dependence is introduced by the JVM specification itself. When a detail of the JVM is left open at the specification level, it can produce vendor-dependent behavior across JVMs.

For example, the JVM spec does not require optimization of tail calls. You may be familiar with tail-recursive calls, which are recursive-method invocations that occur as the very last operation in a method. More generally, any method invocation, recursive or not, that occurs at the end of a method is a tail call. For example, consider the following simple code: Listing 20-1: A Tail-Recursive Factorial �

public class Math {

public int factorial(int n) {

return _factorial(n, 1);

}

private int _factorial(int n, int result) {

if (n <= 0) {

return result;

}

else {

return _factorial(n - 1, n * result);

}

}

}

�

In this example, both the public factorial() method and its private helper method, _factorial(), include tail calls; factorial()includes a tail call to _factorial() and _factorial() includes a recursive tail call to itself.

If that strikes you as a particularly convoluted way to write factorial(), you're not alone. Why not write it in the following, much more natural form? Listing 20-2: A Purely Recursive Factorial �

public class Math {

int factorial(int n) {

if (n <= 0) {

return 1;

}


of 168

else {

return n * factorial(n-1);

}

}

}

�

The answer is that tail calls allow for very powerful optimization—they let us replace the stack frame built for the calling method with that for the called method. This can drastically decrease the depth of the stack at runtime, preventing stack overflows (especially if the tail calls are recursive, like the one for _factorial() in Listing 20-1).

Some JVMs implement this optimization; some don't. As a result, some programs will cause stack overflows on some platforms and not others. So, when we are concerned about the possibility of stack overflow, we will often have to write methods that are naturally expressed recursively using iterative control constructs such as for and while.

Version-Dependent Bugs

The platform dependence resulting from tail calls is a result of the nature of the JVM spec itself. But the much more common causes of platform dependence are bugs in JVM implementations. In the case of Swing, such bugs are widespread.

For example, the JOptionPane component in JDK 1.4.0 (before 1.4.0_b3) has an associated bug. If a user adds text in a JOptionPane to a line that comes immediately after a blank line, and then presses the down arrow key, nothing happens. Try it for yourself:

1. Open a new JOptionPane. 2. In the OptionPane, press the Enter key twice. 3. Type "test." 4. Press the up arrow key. 5. Press the down arrow key.

Apparently, this sequence of operations, along with some similar sequences, put JOptionPanes into a strange state. If a user of your program discovers this bug, he might very well try to recover from it by frantically banging at his keyboard.

In fact, it's not hard to recover from such a state; pressing the right arrow key will do the trick. If, in the course of his banging, your user manages to hit the right arrow key, he will be able to move on as if nothing had happened. He may no longer care much that things froze up and may never even report the bug. Users' standards of acceptability have been lowered substantially by decades of buggy software.

Here's the kicker, though. This bug exists on all versions of Sun JDK 1.4.0 for every platform we've tested—Windows, Solaris, and Linux. So it's likely an operating system–independent bug in Sun's JDK. After our team reported this problem, Sun eliminated the bug in the 1.4.0_b3 release.

This example illustrates that platform dependence is not just about OS dependence and it's not just about vendor dependence—it's about JVM version dependence, both backward and forward.

Teams are usually concerned about providing backward compatibility, but they often expect their code to maintain its behavior under later versions of Java. Ideally, this expectation would be correct, but in reality it's not. In fact, it's not so surprising that Sun introduced a bug in Swing on


of 168

version 1.4 given the tremendous effort the company made in improving performance on that version.

Incidentally, Sun was not the only one dissatisfied with Swing's performance. Eclipse, an open-source project designed to deliver a robust, full-featured, commercial-quality platform for the development of highly integrated tools, implements an entirely new widget toolkit, called the Standard Widget Toolkit (SWT).

SWT is extremely lightweight because, unlike Swing, it leverages the underlying platform-specific windowing system. Many aspects of the API are identical across the platforms on which it is implemented, but the look and feel is entirely platform dependent. So we can expect a whole new set of platform-dependent issues to accompany it.

OS-Dependent Bugs

As the final example of some of the insidious forms of platform dependence you can experience on the Java platform, consider the following code, which at one point was used by the DrJava project editor for opening files and reading them into the editor window. As a first cut, we wrote the code as follows:

FileReader reader = new FileReader(file);

_editorKit.read(reader, tempDoc, 0);

The call to _editorKit.read() reads the contents of the file into a temporary document that is later added to the collection of open documents. But after these two lines, we never refer to reader again. (For more on the DrJava project, see Chapters 4, 5, 12, and 18, as well as Resources.)

Now, you may have noticed that this code contains a great example of a Split Cleaner (see Chapter 16). A FileReader is constructed to read the contents of the file, but that FileReader is never closed. Of course, like other instances of the Split Cleaner, this bug will not produce any symptoms until some other attempt is made to access the file. But, depending on the platform, it may not produce any symptoms even then!

Suppose the user later tries to delete this file. On Unix, open files may be deleted, so the vestigial, unclosed FileReader won't cause any problems there. But open files can't be deleted on Windows, so a Windows user would encounter an exception at this point. The bug in the previous code listing was discovered when one of our unit tests managed to pass on Unix but not on Windows. Once the problem was diagnosed, it wasn't hard to fix:

FileReader reader = new FileReader(file);

_editorKit.read(reader, tempDoc, 0);

reader.close(); // win32 needs readers closed explicitly!

The Java language is not immune to insidious platform-dependent bugs. The symptoms of these bugs are quite varied, but you can expect some of them to bite you at one time or another.

Although the cost of writing cross-platform code is much lower in Java than in many other languages, it's not zero. The best advice I can offer is to run your unit tests on as many platforms as possible, using as many JVM versions as possible.

And, as always, avoid writing bug-prone code. Bug-prone code and platform dependence are a deadly combination


of 168

What We've Learned

In this chapter on bugs that spring from platform dependencies, probably the most important thing we've learned is that the cost of cross-platform capabilities is never zero.

We've also learned the following: � There are many compatibility snags across different JVM versions, such as incorrect

platform-specific separator characters in path names. � There are three patterns of platform-dependent bugs—those tied to the vendor, those

tied to the version, and those tied to the operating system. � Vendor-dependent bugs, in which errors occur on only some JVMs, are caused by

unspecified areas in the JVM's specification. This is a fairly uncommon problem. � Version-dependent bugs, in which errors occur on only some versions of a JVM, are

caused by bugs in certain JVM implementations. This is seen more commonly. � OS-dependent bugs, in which errors occur only on some operating systems, are caused

by the rules of system behavior being different for different OSes. � For implementation bugs, there is a great tool: Sun's Java Bug Parade, a list of JVM-

specific implementation bugs. � An example: Tail calls allow us to replace the stack frame built for the calling method with

that for the called method, which can decrease the depth of the stack at runtime, preventing stack overflows. However, only some JVMs implement this optimization; others don't.

� Platform dependence is not just about OS dependence and vendor dependence—it's about JVM version dependence, both backward and forward. Teams are usually concerned about providing backward compatibility, and they often expect their code to maintain its behavior under later versions of Java. But in real life, this expectation doesn't always hold true.

This is the last chapter on specific bug patterns. Chapter 21 offers a diagnostic checklist to help you quickly find the chapter in this book that is most relevant to your particular needs


of 168

Chapter 21: A Diagnostic Checklist We offer a checklist of queries that question and probe problem behavior and offer potential corresponding references.

General Concepts

For the following list of questions regarding the general concepts of modern software design as they pertain to Java—debugging, implementation, maintenance, and the like—a potential answer can be found in the indicated chapters, as shown in Table 21-1. Table 21-1: Problem Checklist

PROBLEM SEE

How has the way we design software changed recently? Agile/Chaotic Chapter 1

Why has the way we design software changed? Agile/Chaotic Chapter 1

What are the traditional and new methods of teaching programmers?

Agile/Chaotic Chapter 1

What are patterns—bug patterns, design patterns, anti-patterns, etc.—and why are they important to software programmers and designers?

Agile/Chaotic Chapter 1

What is the definition of a bug? Bugs/Specs/Implement Chapter 2

Why are specifications so important in controlling bugs? Bugs/Specs/Implement Chapter 2

What's the difference between a specification and an implementation?

Bugs/Specs/Implement Chapter 2

What is a story and how can it help me develop a specification?


What are unit tests and how can they help me develop a specification?


What are some cost-effective methods for developing specifications?


What is extreme programming (XP) and how can it make debugging easier and more effective?

Debug/Development Chapter 3

Why should I design software with the incorporated ability to test it?

Debug/Testing Chapter 4

How can I design software with the incorporated ability to test it?

Debug/Testing Chapter 4

What is the scientific method of software debugging? Scientific Method Chapter 5

How can I use the scientific method of software debugging? Scientific Method Chapter 5

Why is it important to know about bug patterns? About Patterns Chapter 6


of 168

Table 21-1: Problem Checklist

PROBLEM SEE

Why is it important to know about the bug patterns in this book?

About Patterns Chapter 6

Where do I go to determine how this book is organized? About Patterns Chapter 6

Checklist for the Patterns

For the following list of problems—covered in our patterns section—that can occur whether you are designing, debugging, implementing, or maintaining software, a potential answer can be found in the indicated chapters, shown in Table 21-2. Table 21-2: Pattern Checklist

PROBLEM SEE

The code acts as if a previously corrected bug is still there.

Rogue Tile Chapter 7

Errors are popping up in a job with lots of cut-and-paste code.


A change in the type of a value field in a class is causing an error.


A modified method compiles but returns a reasonable, but wrong, value.


I've built a separate TreeVisitor interface with separate accept() methods for each return type I want to use.


At runtime, I get a ClassCastException even though I use the same TreeVisitor interface for all occasions and insert casts as appropriate with each Visitor method invocation.


I got a damned NullPointerException and I can't figure out what it's telling me!

Null Pointers Chapter 8

Dangling Composite Chapter

9 Null Flag Chapter 10

Run-On Initializer Chapter 9

I got a NullPointerException; how do I find where the variable is assigned to null?


Is there a rule or tool to help statically determine which programs will throw NullPointerExceptions?


I'm getting a NullPointerException in code that uses recursively defined datatypes.

Dangling Composite Chapter 9

Is there a problem in defining a recursive datatype so that some base cases of the definition are not given their own classes?

Dangling Composite Chapter 9

Is there a problem with inserting null pointers into the Dangling Composite Chapter 9


of 168

Table 21-2: Pattern Checklist

PROBLEM SEE various composite datatypes?

My code throws a ClassCastException when I do a recursive descent over the data.

Double Descent Chapter 11

Help! When I recursively descend my data, more than one step down is taken in a single recursive call.


Is there a problem with creating multiple identical instances of the same class?


I'm using instanceof and equals to check for class and object identity; is there a better method?


What can I do to weed out "0" values in consecutive nodes?


I can't get my code to dispatch properly after a method call.


Would eliminating the Leaf class and representing Leaf nodes with null values in the left and right fields of a Branch fix the ClassCastException and keep me from having to cast?

Null Pointers Chapter 8 Dangling Composite Chapter 9 Null Flag Chapter 10 Double Descent Chapter 11

I don't really see why I need to comment all the invariants in my code.


I've got it! To avoid a ClassCastException, I'll wrap each cast in an instanceof check. Right?


What are the disadvantages of using an instanceof check to avoid a ClassCastException?


Are there any "gotchas" when it comes to the design of GUIs?

Liar View Chapter 12

My GUI passed all my tests, but now my customer is calling me with problems. What's up?


I know about writing unit tests as a way to debug, but do I really need so many?


My tests and the runtime behavior of my code don't match. Why?


My tests and the runtime behavior give inconsistent results. Could the tests be wrong?


How do GUI tests work? Liar View Chapter 12

Can the Model-View-Controller (MVQ architecture be at fault in my code?


Is there a best time to check the model through the view?


If I can't easily automate GUI tests, how can I use Liar View Chapter 12


of 168


PROBLEM SEE tests to check my code?

Should I refactor all the time? Liar View Chapter 12

Are there other components necessary to \effective, perpetual refactoring of code?


I've heard the Java Robot class is good for GUI testing. Is it useful in every instance?


Can non-GUI data-displaying software also show discrepancies in testing and runtime results?


My data-input code works fine for a while, then crashes, even though it's working on the same task. What's wrong?

Saboteur Data Chapter 13

The data that causes a crash comes from a large data structure over a network (instead of in memory or from a database). Is that the problem?


Which is the probable cause of the data corruption—syntax or semantics?


Which is the probable cause of the data corruption—manual editing or automatic file generation?


Checking for corrupt input data by parsing seems like a lot of work! Isn't that what compilers are for?


Would type checking help identify semantically corrupt data?


What if I can't check the data on input, as I can with existing data?


Would iteration of the data be a good tool for weeding out corruption? Under what circumstances?


Should I iterate over data, accessing each bit of data as it would be accessed in the deployed application?


Is it always possible to discover corrupt data before it causes a problem? If not, why?


Hey, can splitting a text line into two Strings cause hard-to-detect, corrupt data?


OK. I overloaded one method, then another one (that I didn't touch) broke. What gives?

Broken Dispatch Chapter 14

Is it possible that my method arguments don't match? Broken Dispatch Chapter 14

Can adding new methods cause others to break? Broken Dispatch Chapter 14

If a constructor with a general method is overloaded with a more specific method, what happens?

Broken Dispatch Chapter 14

Should I "infest" my code with tests? Broken Dispatch Chapter 14


of 168


PROBLEM SEE

My code is treating different types of data as the same type. What's up?

Impostor Type Chapter 15

My code won't recognize some of my data types. Impostor Type Chapter 15

Are there any problems in using tags in special fields to distinguish data types?


Why should I use the static type system to distinguish data types?


I've got a solution to avoid type mismatches: I'll use an if -then-else block to dispatch on the appropriate type. Will that work?


My code leaks memory. What could be the cause? Split Cleaner Chapter 16

Why is my program freeing up resources, such as my database connection, too early?

Split Cleaner Chapter 16

To free resources, should I get and free the resource in the same method?


To ensure that my resource is freed, should I trace through each possible execution path of the code?


Some execution paths of my code aren't freeing the resource exactly one time. Why?


Should I try to anticipate and code for all the ways in which a program will be extended?


I discovered an execution path that doesn't include the appropriate clean up code, so I'm adding it to that path. That'll work, right?


How can I go about formally specifying an interface? Fictitious Implementation Chapter 17

Why, when I make a certain implementation of an interface, does one client class that's supposed to work with the interface break?

Fictitious Implementation Chapter 17

Is it OK not to explicitly document invariants in an interface?


What are the drawbacks of loading my code with extra invariants?


As a safeguard, can I specify invariants that can all be checked statically?


Is it OK to restrict the specification of interface invariants to type signatures?


What is an assertion? Are there different types assertions?

Fictitious Implementation of Chapter 17


of 168


PROBLEM SEE

Where should assertions be included in an interface implementation?


Does the inclusion of assertions add heavily to execution overhead?


Would assertions alone be enough to capture all of the rules I want to specify on an interface?


Can I use unit tests to provide limited specification of extra invariants for an interface?


Can my set of unit tests check an interface implementation over all possible inputs?


Which are more expressive, type signatures or unit tests?


My multithreaded code locks up and prints stack traces to standard error. Why?

Orphaned Thread Chapter 18

What about if I just use single-threaded design from now on?


My program just freezes, which is confusing. Are there other ways I can notify the user that there is a problem?


Hey! Did the stop() method on threads change? Orphaned Thread Chapter 18

In what type of programming am I more likely to encounter an abandoned second thread?


I've got a NullPointerException where an uninitialized field is accessed. What's up?


I don't think all the fields of my class are initializing. Did I not build the constructor correctly?


Is there something wrong with constructors that require client classes to initialize instances in several steps?


This class has had fields added several times. Is it possible that it is not initializing properly because of the additions?


Does it matter what order I execute statements in? Run-On Initializer Chapter 19

Should I convince my customer to throw out old code and start fresh?


I'm working with legacy code. Is it OK to modify the constructor signatures?


What's the best way to control NullPointerException errors when working with



of 168


PROBLEM SEE legacy code?

What about using an isInitialized() method in the class?


How can I ensure that instances of my class will be in a well-defined state at all times?


If I filled in class fields with null values, wouldn't that help with initializing?


What's a quick way to determine whether an instance has been initialized?


What should I do to avoid initialization problems with new contexts?


What is the best way to represent default values? Run-On Initializer Chapter 19

Will using special classes to represent default values cause a performance hit? If so, why?


Is there a way to check for initialization without casting at runtime and taking the associated performance hit?


Someone told me that including methods that always throw exceptions is sometimes justified, but that seems stupid. What do you think?


Why does my code work on some Java Virtual Machines but not on others?

Platform-Dependent Bugs Chapter 20

Why does my code work on some versions of a JVM and but on others?


Why does my code work on some operating systems and not on others?


What's the difference between a specification-related bug and an implementation-related bug?

Bugs/Specs/Implement Chapter 2 Platform-Dependent Bugs Chapter 20

Is there a roundup of the bugs caused by specifications and implementations of Java?



of 168

Chapter 22: Design Patterns for Debugging

Overview

In this final chapter, we discuss the yin-yang motivations for employing bug patterns when designing a program: maximizing static checking use and minimizing error introduction.

Now that we've taken a trip through some of the most common bug patterns and have seen how to deal with them, we're in the enviable position of asking this question: "What are the implications of bug patterns on program design?"

As we discussed in Chapter 4, the extreme programming practice of writing tests in tandem with writing code has several major ramifications on design. We have also demonstrated how the causes of the most common bugs can be eliminated by taking several preventative programming measures, but these measures have been largely below the level of design.

Considering these preventative measures as a whole, we can formulate a guiding set of design principles to avoid these bugs and many like them. In fact, a prudent application of some of the most popular design patterns can help to eliminate occurrences of many bug patterns.

After reviewing the preventative programming measures discussed in previous chapters, we can see that there are two primary motivations behind them:

1. To maximize the use of static checking. 2. To minimize the ways in which errors can be introduced.

These two motivations are sometimes in conflict. Errors can be minimized by keeping a single point of control for each functional aspect of the code, but static type checking doesn't always allow for such single points of control.

Conflicts like these make programming a hard task. As is always the case, resolution of conflicts among sets of principles must be resolved by an assessment of the costs and benefits of each in a given context.

Let's consider some of the ways to apply each of these two guiding principles and see how design patterns can help. Maximizing Static Type Checking

Java is a statically typed language, but, as we've discovered, the language alone does not determine how much of an advantage that static checking provides. The programmer must actively seek to utilize the type system to get as much static checking as possible.

The following points outline some ways to fully utilize Java's type system. The bug patterns we've discussed provide the motivation behind these suggestions; in fact, some of these points have been discussed already in the context of specific bug patterns, while others are more generally applicable. � Make fields final whenever possible. � Make methods final when they should never be overridden. � Include classes for default values. � Use checked exceptions to ensure that all clients handle exceptional conditions. � Define new exception types to nail down the various exceptional conditions precisely. � Break classes whose instances take on one of a fixed number of states into distinct

subclasses in a Composite hierarchy. � Stay clear of platform-dependent behavior. � Test on as many platforms as possible.


of 168

� Minimize casts and instanceof tests. � Use the Singleton design pattern to help minimize use of instanceof. � Use extra methods and dynamic dispatch to help minimize use of instanceof.

Let's consider each of these practices in turn, along with the motivations behind them.

Tip The Java language alone does not determine how much of an advantage that

static checking provides—programmers must actively seek to utilize the type system.

Make Fields final Whenever Possible

A final field can never be modified. The type checker itself will prevent its mutation. And, as we've seen from several bug patterns, whenever things can be modified, they can break.

A great rule of thumb: if you can get away with never modifying a field, then make that field impossible to modify.

Make Methods final When They Shouldn't Be Overridden

Dynamic dispatch is a powerful tool and a key component of object-oriented programming. But if a method can be overridden, it can be overridden with a method that does not act as expected in a particular context (see the Broken Dispatch bug pattern in Chapter 14 for more details).

An easy preventative measure is to simply declare methods as final when there will never be a good reason to override them. Of course, this measure should be applied with care; every time a method is declared as final, the extensibility of the program is diminished. As we have seen, extensibility is often in conflict with our goal of keeping bugs out of the program.

One place to apply this maxim in particular is in the context of the Template Method design pattern (for references on design patterns, see Resources). In this pattern, you define an abstract class that contains only part of the logic needed to accomplish its purpose. This abstract class includes both concrete methods and abstract methods. The concrete methods invoke the abstract methods, which are defined differently in each concrete subclass. Of course, the abstract methods are supposed to be overridden, but the defined concrete methods in the base class never should be. So why not make them final and avoid any problems?

Include Classes for Default Values

Don't use a null value as a placeholder for default values; it'll be dereferenced and you'll get the horribly nondescriptive NullPointerException.

Instead, define one class per distinct default value. In general, this will involve constructing an application of the Composite pattern in which the original classes and the new default classes will both become subclasses of a new abstract class. If the default classes contain no fields, apply the Singleton pattern. (For more on these design patterns, see the sidebar in Chapter 9, as well as Resources.)

By using these classes for default values, you won't just get a better error message when one of those values is used inappropriately; you'll also allow the type checker to rule out many ways in which the value can be used inappropriately before you execute a single line of code.


of 168

Use Checked Exceptions to Ensure That All Clients Handle Exceptional Conditions

Checked exceptions are an extremely powerful mechanism. Methods that throw checked exceptions must declare that they throw the exceptions; clients must either handle the exception in a try-catch block or explicitly throw the exception themselves.

In both cases, the client's attention is brought to the fact that the exception can occur; that way, a conscious decision must be made as to how to handle it. And that's all done by the type checker. The code won't compile until it's done. For these reasons, when used properly, checked exceptions can help to improve code robustness.

However, like any sharp knife, they can also cut you. Checked exceptions should not be used to signal run-time errors in a program. Forcing clients of a library containing such exceptions to handle these exceptions is a tedious waste of programmer time and clutters the code with misleading annotations.

The best approach is to use checked exceptions only when throwing such an exception is part of expected (albeit unusual) program behavior. But, in such cases, they can be a big win.

Define New Exception Types to Nail Down the Various Exceptional Conditions Precisely

Because the static type system checks that clients handle checked exceptions, and because exceptions in Java are instances of classes, we can design a class hierarchy of exceptions to increase the precision with which distinct exceptional conditions are handled.

The design tradeoffs when putting together this hierarchy are just like those for other class hierarchies: break groups of exceptions into separate cases when they should be handled separately and group exceptions with common properties together via Composite hierarchies.

Break Classes Whose Instances Take On One of a Fixed Number of States into Distinct Subclasses in a Composite Hierarchy

One way to think about types is as collections of values. Another way to think about them is as static assertions about expressions. How can we break up the collections of values that our types represent so we can make the corresponding assertions they represent as useful as possible?

As we've seen with the Impostor Type bug pattern (Chapter 15), single classes often represent collections of values that are naturally separated into distinct collections. One telltale sign of an Impostor Type is a class that contains a boolean (such as isEmpty) or other field with a fixed set of possible values whose value for a given instance has strong implications as to how that instance can be used.

In these cases, if we restructure the class as a Composite class hierarchy with subclasses—one for each possible value of the field—we can use static typing to more precisely specify which of these subclasses are valid in given contexts. The type checker will do the rest.

One potentially useful alternative approach is to use what is known as the State design pattern. Here, a class of objects dynamically takes on various states during a computation. The current state of the object is represented as a field. The set of states can then be defined in a Composite class hierarchy.


of 168

When applying this pattern, try to keep state-specific functionality in the state classes themselves. Then, rather than explicitly checking the state of an object, you can simply dispatch method calls on the state. In this way, we can continue to use static checking to maximum advantage.

Minimize Casts and instanceof Tests

In the current context, it is really casts we're worried about, as they allow us to circumvent the static type system, providing less static checking.

I mention casts and instanceof tests together, however, because you will often see casts inside an if clause guarded by an instanceof test. When that happens, it strongly suggests that you should use separate methods in the various classes to dispatch on the code appropriate for a given type.

Use the Singleton Pattern to Help Minimize Use of instanceof

When there is only one instance of a class, then checking that a value is identical to that instance is equivalent to checking instanceof on the class (except that it's less expensive).

Also, checking that a value of unknown type is equal to some instance of the class is equivalent to checking that the two instances are identical (==). Thus, Singletons give us stronger invariants over the set of possible values in a program.

�

Some Mentioned Design Patterns

In this chapter, we mentioned the following design patterns: Command, Composite, Singleton, Template, State, and Visitor.

Command. This pattern encapsulates commands in objects so that you can control their selection by sequencing, queuing, undoing, and otherwise manipulating them.

Composite. This pattern lets you build complex objects by recursively composing similar objects in a tree-like manner. It also allows the objects in the tree to be manipulated in a consistent manner, since they all have a common super-class or interface.

Singleton. This pattern ensures that only one instance of a class is created. All objects that use an instance of that class will use the same instance. This pattern is a potential optimization pattern.

Template Method. This pattern allows you to write an abstract class that contains only part of the logic needed to accomplish its purpose. You then organize the class so that its concrete methods call an abstract method where the missing logic would have appeared, and provide the missing logic in subclass methods that override the abstract methods.

State. This pattern encapsulates the states of an object as discrete objects, each belonging to a separate subclass of an abstract state class.

Visitor. This pattern provides an alternative way to implement an operation that involves objects in a complex structure by providing logic in each of their classes to support the operation. It lets you avoid complicating the classes of the objects in the structure by putting all of the necessary logic in a separate visitor class. It also allows the logic to be varied by using different visitor classes.

More on design patterns can be found in Resources


of 168

Minimizing Error Introduction

The following points, also motivated by the bug patterns we've discussed, are some important keys to minimize the introduction of bugs into your code. Remember, these methods may be in conflict with the goals of maximizing static type checking, the other side of the good-programming coin. � Factor out common code. (Say it over and over again—it's that important.) � Make methods that are purely functional whenever possible. � Initialize all fields in constructors. � Throw exceptions when exceptional conditions occur. � Signal errors as soon as they are discovered. � Discover errors as soon as possible via parsing, type checking, etc. � Place assertions into code via casts, assertTrue() methods, documentation, and

arguments in the form of documentation. � Test code as closely to the user-observable state as possible.

Factor Out Common Code

Code that has been copied and pasted is the root of all programming evil. Bugs must be fixed in every copy. Improvements in one copy must similarly be made in all. Programmers trying to understand the code will have to read each copy separately, wasting time. That's why there's a whole bug pattern—the Rogue Tile—dedicated to problems with copy-and-paste code. (See Chapter 7 for our coverage of the Rogue Tile pattern.)

As mentioned in the discussion on the Rogue Tile pattern, the Command design pattern is a great way to factor out common code. Similarly, the Visitor design pattern is useful for providing type-specific behavior on a Composite class hierarchy. Both are useful in flushing out Rogue Tiles.

Additionally, various extensions to Java, such as JSR-14, NextGen, and AspectJ (see Resources for more on each), help to factor out common code where it would be otherwise impossible in Java.


This point just can't be stressed enough.

Make Methods That Are Purely Functional Whenever Possible

Whenever data is mutated, it can be broken. Also, if there are multiple references to the data, modification via one reference can be made without the knowledge or consent of other clients—but those clients might be relying on invariants that are broken when the data is mutated. And, when data is being modified, it can take on inconsistent states, which is dangerous in the context of multithreading.

By making methods that are purely functional (that, in essence, involve no mutation), we avoid all of these problems at once.

Another advantage of purely functional methods is that their behavior (which in this case is merely the values they return in relation to their input) can be understood in isolation. That makes it much easier for other programmers to understand the code; they can absorb a piece of it at a time.

Initialize All Fields in Constructors


of 168

Run-on initializers (Chapter 19) result in NullPointerExceptions, which we hate. Just say no!

Throw Exceptions When Exceptional Conditions Occur

Don't use null flags to signal error conditions. They'll be dereferenced and your program will crash.

Signal Errors as Soon as They're Discovered

The longer the time between when a bug occurs and when it's reported, the more the state can change. The more the state changes, the fewer clues we have to work with.

As a wise man once said, "Fear leads to anger, anger leads to hate, hate leads to suffering." Avoid suffering by nipping problems in the bud.

Discover Errors as Soon as Possible

In order to signal errors as soon as possible, you have to discover them by active programming strategy; you can't wait for them to come to you.

This tenet is especially important in the context of persistent data in which the stakes are higher. If we don't discover a bug immediately, it can linger as a Saboteur (Chapter 13) indefinitely long before we discover it.

Parse input and check that it satisfies all required invariants. That'll involve designing Composite class hierarchies specifically to represent parsed data, and parsers to transform input into instances of those classes.

When checking higher-level constraints over the parsed input, the functionality can be put into the constituent classes, but I prefer to keep the logic for such an analysis all in one place by encapsulating it in a Visitor.

Place Assertions into Code

When writing code, try to make sure that the operations you perform are guaranteed to succeed without incident. Then put the argument that convinces you of this fact into the code, in the form of documentation and executable assertions.

Think of the program text itself as a means for arguing for its own correctness. This programming style is sometimes referred to as persuasive programming.

Declaring preconditions, postconditions, and invariants for methods is a good idea. So is documenting the invariants expected to hold in else clauses. One tool to help you check such invariants is iContract (see Resources).

Test Code as Closely to the User-Observable State as Possible

The longer the distance between what we test and what the user observes, the more room there is for a bug to introduce a Liar View (Chapter 12).



of 168

Enough said? Not the Last Word

Naturally, adopting principles like these will help to reduce the occurrence of bug patterns in your code. You may find that some of the bug patterns discussed here are not the ones you most frequently encounter; and, to be sure, many of the bug patterns we've discussed are problematic enough that we can't expect to stop them from recurring. We can only learn to diagnose them more quickly and to eliminate them with the right preventative measures.

As you discover that other patterns become more frequent, document them. Talk to others about them. Write articles about them and let people know.

Many new books, such as Bruce Tate's Bitter Java (see Resources), have started documenting various common bugs at the level of design. Tate also discusses many coding-level issues, including the Split Cleaner bug pattern (although Tate labels it an "anti pattern," a term more commonly reserved for design-level bugs).

Bug patterns can occur not just in Java, but in any programming language, even in markup languages, scripting languages, and query languages. The more we document such patterns, the more we can benefit from each other's experience. And in the fast-paced world of software development, leveraging the experience of others is a must.

May your days be filled with many successful bug sleuthings


of 168

Chapter 23: References

Web Sites

The IBM developerWorks Javazone contains a wealth of information on Java and many tools to use with it.

http://www.ibm.com/developerworks/java

Javaworld is another great site for information on the Java language.

http://www.javaworld.com

The Java Tutorial includes information on many Java APIs, including Swing, JDBC, graphics, and many other topics.

http://java.sun.com/docs/books/tutorial/

The Patterns home page contains links to many varied resources on patterns and their applications:

http://hillside.net/patterns/

AntiPatterns are patterns of design that have time and again shown themselves to result in errors. Check out the AntiPatterns home page:

http://www.antipatterns.com/

The Extreme Programming home page provides many links to useful resources for that methodology (including links to the xUnit tools):

http://xprogramming.com/

JUnit is the xUnit tool for Java unit testing. It has developed a tremendous amount of popularity:

http://www.junit.org/

The following JavaWorld article I wrote a few years back explains the motivation behind adding generic types to Java, and describes several of the major implementation proposals:

http://www.javaworld.com/javaworld/jw-02-2000/jw-02-jsr.html

One algorithm for higher level program analysis that has helped to automatically identify null-pointer exceptions in other languages is set-based analysis. Here's a pointer to a short introduction on it:

http://www-2.cs.cmu.edu/afs/cs/user/nch/www/sba.html

AspectJ, a tool for aspect-oriented programming in Java, has become increasingly popular lately:

http://aspectj.org/servlets/AJSite

The IBM developerWorks article "Incremental Development with Ant and JUnit" describes how to set up a development platform for extreme programming:

http://www-106.ibm.com/developerworks/java/library/j-ant/


of 168

ANTLR is a tool for parsing languages in Java.

http://www.antlr.org/

OroMatcher is a Java library that adds support for regular-expression matching (often a very useful thing to have, especially for weeding out saboteurs from program input):

http://www.savarese.org/oro/downloads/

If you're looking for a cheap JDBC driver for moderately large databases, check out Enhydra; it's free! Just be sure to keep out saboteurs.

http://www.enhydra.org/

"XP Distilled" is a short introduction to Extreme Programming, at IBM developerWorks.

http://www-106.ibm.com/developerworks/java/library/j-xp/

When you're working with the more subtle aspects of Java, the Java Language Specification is an invaluable tool for nailing down the language semantics.

http://java.sun.com/docs/books/jls/second_edition/html/j.title.doc.html

Tail recursion was first studied in the context of the Lisp-like languages. Here's a short introduction to the notion of tail recursion (discussed in the CMU Common Lisp User's Manual).

http://www.cis.ksu.edu/VirtualHelp/Info/develop/cmu-user.info.Tail_Recursion.html

Here's an online description of Sun's Java Hotspot Technology:

http://java.sun.com/products/hotspot/

The IBM Java Technology Tools and Products provides some great links to new tools at various stages in the pipeline.

http://www-105.ibm.com/developerworks/tools.nsf/dw/java-devkits-byname?OpenDocument&Count=100

One of the first concerns of many veteran programmers when first learning about extreme programming is how XP fits in with their wealth of accumulated knowledge on good design and design patterns. Martin Fowler, a key figure in both OO-design and Extreme Programming, addresses this concern brilliantly in his online article, "Is Design Dead?" I won't spoil the article by letting you know that his answer is "no."

http://www.martinfowler.com/articles/designDead.html

Jinsight is an IBM alphaWorks tool that lets you visualize and analyze the execution of Java applications, so you can identify potential bug patterns.

http://www.alphaworks.ibm.com/tech/jinsight

iContract allows you to include assertions in your Java programs.

http://www.reliable-systems.com/tools/iContract/iContract.htm


of 168

"Contract Soundness for Object-Oriented Languages" by Findler and Felleisen provides an excellent discussion on why iContract (and every other existing contract checker for Java) fails to perform all of the assertion checks necessary in an object-oriented language.

http://www.ccs.neu.edu/scheme/pubs/oopsla01-ff.pdf

Bertrand Meyer explains his approach of "Design by Contract" in his online article, "The Eiffel approach to Separating Interface from Implementation."

http://www.elj.com/eiffel/bm/intimp/

If you're working with multithreaded code, be sure to check out Sun's explanation as to why Thread.stop was deprecated.

http://java.sun.com/j2se/1.3/docs/guide/misc/threadPrimitiveDeprecation.html

"Writing Multithreaded Java Applications" is a good article on that subject from IBM developerWorks.

http://www-106.ibm.com/developerworks/java/library/j-thread.html

"The Rules and Practices of Extreme Programming" is another good online introduction to the subject.

http://www.extremeprogramming.org/rules.html

The Java Tree Builder, is a syntax tree builder used with the Java Compiler Compiler (JavaCC) parser generator. It takes a plain JavaCC grammar file as input and automatically generates the following: a set of syntax tree classes based on the productions in the grammar (using the Visitor design pattern); two interfaces (Visitor and ObjectVisitor); two depth-first visitors (DepthFirstVisitor and ObjectDepthFirst); and a JavaCC grammar with the proper annotations to build the syntax tree during parsing.

http://www.cs.purdue.edu/jtb/

Download a copy of Jython, an implementation of the dynamically typed, object-oriented Python that is written in and 'seamlessly' integrates with the Java language and platform.

http://www.jython.org/

Retrieve your own version of the Extended ML specification here, in PDF format; Extended ML is a framework for specification and formal development of Standard ML programs. It provides a great example of how the semantics of a real-world language can be formally specified.

http://www.dcs.ed.ac.uk/home/dts/pub/eml-final.pdf

Don't forget to download DrJava and take advantage of the power of an IDE with a built-in repl.

http://drjava.sf.net

bitterjava.com is "dedicated to finding and ridiculing the most vile, bitter, disgusting Java programs, architectures, and designs imaginable—all the bad coding practices available."

http://www.bitterjava.com


of 168

In "A Taste of Bitter Java" (developerWorks, March 2002), Bruce Tate demonstrates how and why antipatterns (a design solution to a problem that actually results in decidedly negative consequences) are a necessary and complementary companion to design patterns.

http://www-106.ibm.com/developerworks/java/library/j-bitterjava/?dwzone=java

Check out the Eclipse Web site, for more information about the Standard Widget Toolkit.

http://www.eclipse.org/

Check out the Java Bug Parade for many examples of platform-dependent bugs. (You must be registered with the Java Developer Connection to access this list — it's free!)

http://developer.java.sun.com/servlet/SessionServlet?url=http://developer.java.sun.com/developer/bugParade/index.jshtml Books

The Definition of Standard ML—Revised (MIT Press, May 1997), provides a formal definition of Standard ML, a general-purpose programming language designed for large projects.

The Structure and Interpretation of Computer Programs, by Abelson and Sussman (MIT Press, 1985), is an old but extraordinary book. After all of these years, it's still the single best computer science text I know of. If one were to read only one book on computer science, it should be this book.

Security Engineering: A Guide to Building Dependable Distributed Systems, by Anderson (John Wiley & Sons, 2001), presents a comprehensive design tutorial to designing secure systems that illustrates basic concepts through real-world system design successes and failures.

Object-Oriented Analysis and Design, by Booch (Addison-Wesley, 1994), was a pivotal book in the OO-design movement, and is still quite relevant today.

The Mythical Man-Month, by Brooks (Addison-Wesley, 1975), was the seminal text on software engineering. It's interesting to read this text and consider how extreme programming addresses many of the problems arising from software construction discussed in that book.

Analysis Patterns, by Fowler (Addison-Wesley, 1997), is a great source of ideas on how to think about a problem domain before you even start coding for it.

Design Patterns: Elements of Reusable Object-Oriented Software, by Gamma (Addison-Wesley, 1995), is the canonical source for design patterns. The 1998 update includes a CD with sample code and 23 cut-and-paste patterns.

Extreme Programming Installed, by Jeffries, Anderson, and Hendrickson (Addison-Wesley, 2001), is a great introduction to extreme programming, with a heavy emphasis on specific practices and techniques. This book is required reading for all developers on the JavaPLT development team.

The Pragmatic Programmer, by Hunt and Thomas (Addison-Wesley, 2000), provides a wealth of specific programming advice and tips.

Types and Programming Languages, by Pierce (MIT Press, 2002), is an advanced text on the semantics of programming languages and type systems. It's not light reading, but it is extremely


of 168

well written, and it includes a chapter on Featherweight Java, a simple formalized sublanguage of Java.

Toward Zero-Defect Programming, by Stavely (Addison-Wesley, 1999), is a bold attempt to outline a programming approach that tries to minimize programming bugs. Although this book neglects unit tests and focuses on formalisms, assertions, and proofs, there's no reason why the techniques discussed can't be used as an augmentation to an extreme programming approach. When trying to minimize bugs, we should bring all arms to bear on the problem.

Component Software: Beyond Object-Oriented Programming, by Szyperski (Addison-Wesley, 1998), discusses the distinction between interface and implementation inheritance in this excellent book.

User-Interface Design for Programmers, by Spolsky (Apress, 2001), provides great advice on how to set up GUIs that users will enjoy using (and will be productive with).

Extreme Programming Explained, by Succi and Marchesi (Addison-Wesley, 2001), contains many great articles on various aspects of extreme programming, applied in various settings (including academic ones).

Bitter Java, by Tate (Manning Publications, 2002), is an excellent and wellwritten introduction to the world of antipatterns.

John Zukowski's Guide to Swing, by Zukowski (Apress, 2000), is an indispensable guide for working with the subtleties of the Java Swing libraries.


of 168

Appendix A: String-Parsing List Constructor The following code implements a constructor for nonempty lists that takes a String representation of a list and parses it into a list, as described in Chapter 14. The parser used is a general-purpose parser for S-expressions. S-expressions (short for symbolic expressions) are built from a set of atomic elements (called atoms or symbols). Given the set of atoms, any S-expression is either an atom or a list of S-expressions.

In order to avoid namespace clash with java.util, all S-expression classes (including List classes) are prepended with S. The constructor itself appears in class SCons. It also appears as a Factory method in class SExp.

This code is also available online at http://www.cs.rice.edu/~eallen. Listing A-1: String-Parsing List Constructor �

import java.util.*;

interface StackI {

/**

* @ pre ! isEmpty()

*/

public Object top();


public void push(Object o);


}

class SExpParseException extends Exception {

public SExpParseException(String msg) {

super(msg);

}

}

abstract class SExp {

public static final String LEFT_PAREN = "(";

public static final String RIGHT_PAREN = ")";

public static SExp parse(String tokens) throws SExpParseException {

return parseSExp(new StringStack(tokens));

}

public static SExp parseSExp(StackI tokens) throws SExpParseException {


of 168

SExp nextSExp = parseNextSExp(tokens);

if (tokens.isEmpty()) {

// The stack of tokens consisted of a single S-expression

// (with possible subexpressions), as expected.

return nextSExp;

}

else {

throw new SExpParseException("Extraneous material "+

"at end of stream.");

}

}

public static SExp parseNextSExp(StackI tokens) throws SExpParseException {

if (tokens.isEmpty()) {

throw new SExpParseException("Unexpected end of token stream.");

}

else { // tokens.pop() succeeds

Object next = tokens.pop();

if (next.equals(LEFT_PAREN)) {

// The S-expression is a list. Accumulate the subexpressions

// this list contains, and return the result.

SList result = SEmpty.ONLY;

while (! tokens.isEmpty()) { // tokens.pop() succeeds

next = tokens.top();

if (next.equals(RIGHT_PAREN)) {

// We've reached the end of the list. We need only

// pop off the ending right parenthesis before returning.

// Since subexpressions were accumulated in the front

// of the list, we must return the reverse of the list

// to reflect the proper structure of the S-expression.

tokens.pop();

return result.reverse();

}

else {

// Recursively parse the next subexpression and

// add it to result.


of 168

result = new SCons(parseNextSExp(tokens), result);

}

}

// If we haven't yet returned, then we've reached the end

// of the token stream without finding the matching right

// paren.

throw new SExpParseException("Unmatched left parenthesis.");

}

else if (next.equals(RIGHT_PAREN)) {

// A right parenthesis was encountered at the beginning of

// the S-expression!

throw new SExpParseException("Unmatched right parenthesis.");

}

else {

// The next S-expression is an atom.

return new Atom(next);

}

}

}

}

class StringStack implements StackI {

StringBuffer _values;

public StringStack(String values) {

_values = new StringBuffer(values);

}

public Object top() {

if (_values.length() == 0) {

throw new RuntimeException("Attempt to call top on an empty stack.");

}

else { // _values.length() >= 1

return new StringBuffer().append(_values.charAt(0)).toString();

}

}

public Object pop() {

if (_values.length() == 0) {


of 168

throw new RuntimeException("Attempt to pop an empty stack.");

}

else { // _values.length() >= 1

String result = new StringBuffer().append(_values.charAt(0)).toString();

_values.deleteCharAt(0);

return result;

}

}

public void push(Object o) {

_values.append(o);

}

public boolean isEmpty() {

return _values.length() == 0;

}

}

abstract class SList extends SExp {

abstract SList reverse();

/**

* Although accessing first() and rest() is valid only on non-empty

* lists, we can simplify many list-based operations by including

* these accessors in the SList interface and then implementing them

* in SEmpty with methods that throw exceptions. Since the alternative

* would be to insert casts in many cases (such as the inner class

* Iterator, below), this design does not decrease the effectiveness

* of static checking. Additionally, the information provided during

* an error at run-time is at least as informative.

*/

public abstract SExp first();

public abstract SList rest();

class Iterator {

SList values;

public Iterator() {


of 168

values = SList.this;

}

public SExp next() {

SExp result = values.first();

values = values.rest();

return result;

}


return !(values == SEmpty.ONLY);

}

}

}

class SEmpty extends SList {

public static final SEmpty ONLY = new SEmpty();

private SEmpty() {}

public String toString() {

return "()";

}

SList reverse() {

return this;

}

public SExp first() {

throw new RuntimeException("Attempt to access the first element "+

"of an empty list.");

}

public SList rest() {

throw new RuntimeException("Attempt to access rest of an empty list.");

}

}

class SCons extends SList {

private SExp first;

private SList rest;


of 168

public SCons(SExp _first, SList _rest) {


this.rest = _rest;

}

/**

* @pre SExp.parse(s) instanceof SCons

*/

public SCons(String s) throws SExpParseException {

// Notice that the precondition guarantees that the cast

// below will always succeed.

SCons that = (SCons)SExp.parse(s);

first = that.first();

rest = that.rest();

}

public SExp first() {return this.first;}

public SList rest() {return this.rest;}

SList reverse() {

SList result = SEmpty.ONLY;

SList.Iterator iter = this.new Iterator();

while (iter.hasNext()) {

result = new SCons(iter.next(), result);

}

return result;

}


StringBuffer result = new StringBuffer("(");

SList.Iterator iter = this.new Iterator();

while (iter.hasNext()) {

result.append(iter.next().toString());

}

result.append(")");

return result.toString();

}

}


of 168

class Atom extends SExp {

public Object value;

public Atom(Object _value) {

this.value = _value;

}


return value.toString();

}

}

Bug Patterns in Java

Documents

Transcript of Bug Patterns in Java