Applied and Numerical Harmonic Analysis

Series Editor John J. Benedetto University of Maryland

Editorial Advisory Board

Akram Aldroubi NIH, Biomedical Engineering/ Instrumentation

Ingrid Daubechies Princeton University

Christopher Heil Georgia Institute of Technology

James McClellan Georgia Institute of Technology

Michael Unser Swiss Federal Institute of Technology, Lausanne

M. Victor Wickerhauser Washington University

Douglas Cochran Arizona State University

Hans G. Feichtinger University of Vienna

MuratKunt Swiss Federal Institute of Technology, Lausanne

Wim Sweldens Lucent Technologies Bell Laboratories

Martin Vetterli Swiss Federal Institute of Technology, Lausanne

Applied and Numerical Harmonic Analysis

Published titles

J.M. Cooper: Introduction to Partial Differential Equations with MATLAB (ISBN 0-8176-3967-5)

C.E. D'Attellis and E.M. Femandez-Berdaguer: Wavelet Theory and Harmonic Analysis in Applied Sciences (ISBN 0-8176-3953-5)

H.G. Feichtinger and T. Strohmer: Gabor Analysis and Algorithms (ISBN 0-8176-3959-4)

T.M. Peters, J.H.T. Bates, G.B. Pike, P. Munger, and J.C. Williams: Fourier Transforms and Biomedical Engineering (ISBN 0-8176-3941-1)

AI. Saichev and W.A. Woycz:yriski: Distributions in the Physical and Engineering Sciences (ISBN 0-8176-3924-1)

R. Tolimierei and M. An: Time-Frequency Representations (ISBN 0-8176-3918-7)

G.T. Herman: Geometry of Digital Spaces (ISBN 0-8176-3897 -0)

A. Prochazka, J. Uhlir, PJ. W. Rayner, and N.G. Kingsbury: Signal Analysis and Prediction (ISBN 0-8176-4042-8)

J. Ramanathan: Methods of Applied Fourier Analysis (ISBN 0-8176-3963-2)

A. Teolis: Computational Signal Processing with Wavelets (ISBN 0-8176-3909-8)

W.O. Bray and C.V. Stanojevic: Analysis of Divergence OSBN 0-8176-4058-4)

G.T. Herman and A. Kuba: Discrete Tomography (ISBN 0-8176-41 01-7)

J.J. Benedetto and P.J.S.G. Ferreira: Modem Sampling Theory (ISBN 0-8176-4023-1)

A. Abbate, C.M. DeCusatis, and P.K. Das: Wavelets and Subbands (ISBN 0-8176-4136-X)

L. Debnath: Wavelet Transforms and Time-Frequency Signal Analysis (ISBN 0-8176-4104-1)

K. Gr6chenig: Foundations of Time-Frequency Analysis (ISBN 0-8176-4022-3)

D.F. Walnut: An Introduction to Wavelet Analysis (ISBN 0-8176-3962-4)

0. Bratelli and P. Jorgensen: Wavelets through a Looking Glass (ISBN 0-8176-4280-3)

H. Feichtinger and T. Strohmer: Advances in Gabor Analysis (ISBN 0-8176-4239-0)

0. Christensen: An Introduction to Frames and Riesz Bases (ISBN 0-8176-4295-1)

L. Debnath: Wavelets and Signal Processing (ISBN 0-8176-4235-8)

J. Davis: Methods of Applied Mathematics with a MATLAB Overview (ISBN 0-8176-4331-1)

G. Bi andY. Zeng: Transforms and Fast Algorithms for Signal Analysis and Representations (ISBN 0-8176-4279-X)

J.J. Benedetto and A. Zayed: Sampling, Wavelets, and Tomography (0-8176-4304-4)

E. Prestini: The Evolution of Applied Harmonic Analysis (0-8176-4125-4)

(Continued after the Index)

David F. Walnut

An Introduction to

Wavelet Analysis

With 8 8 Figures

Springer Science+Business Media, LLC

David F. Walnut Department of Mathematical Sciences George Mason University Fairfax, VA 22030 USA

Library of Congress Cataloging-in-Publication Data

Walnut, David F. An introduction to wavelet analysis I David F. Walnut

p. em.- (Applied and numerical harmonic analysis) Includes bibliographical references and index.

I. Wavelets (Mathematics) I. Title. II. Series. QA403.3 .W335 2001 515' .2433-dc21 2001025367

CIP

ISBN 978-1-4612-6567-2 ISBN 978-1-4612-0001-7 (eBook) DOI 10.1007/978-1-4612-0001-7

©2002 Birkhiiuser Boston ~®

Birkhiiuser ley) © 2004 Springer Science+Business Media New York Originally published by Birkhiiuser Boston in 2004 Softcover reprint of the hardcover 1st edition 2004

All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+ Business Media, LLC), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of in­formation storage and retrieval, electronic adaptation, computer software, or by similar or dis­similar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to property rights.

9 8 7 6 5 4 3 2 SPIN 10967157

www.birkhauser-science.com

To my parents

and to Megan

Unless the LORD builds the house, its builders labor in vain.

-Psalm 127:1a (NIV)

Contents

Preface

I Preliminaries

1 Functions and Convergence 1.1 Functions . . . . . . . . .

1.1.1 Bounded (L00 ) Functions 1.1.2 Integrable (L1) Functions 1.1.3 Square Integrable (L2 ) Functions 1.1.4 Differentiable (en) Functions . .

1.2 Convergence of Sequences of Functions . 1.2.1 Numerical Convergence .. 1.2.2 Pointwise Convergence ... 1.2.3 Uniform (L00 ) Convergence 1.2.4 Mean (L1 ) Convergence .. 1.2.5 Mean-square (L2 ) Convergence 1.2.6 Interchange of Limits and Integrals .

2 Fourier Series 2.1 Trigonometric Series ....... .

2.1.1 Periodic Functions .... . 2.1.2 The Trigonometric System 2.1.3 The Fourier Coefficients .. 2.1.4 Convergence of Fourier Series

2.2 Approximate Identities . . . . . . . . 2.2.1 Motivation from Fourier Series 2.2.2 Definition and Examples . 2.2.3 Convergence Theorems .

2.3 Generalized Fourier Series .... 2.3.1 Orthogonality . . . . . . . 2.3.2 Generalized Fourier Series 2.3.3 Completeness ...... .

3 The Fourier Transform 3.1 Motivation and Definition 3.2 Basic Properties of the Fourier Transform 3.3 Fourier Inversion . . . . . . . . . . . . . .

xiii

1

3 3 3 3 6 9

11 11 13 14 17 19 21

27 27 27 28 30 32 37 38 40 42 47 47 49 52

59 59 63 65

viii Contents

3.4 Convolution . . . . . . . . . . . . . . . . 68 3.5 Plancherel's Formula . . . . . . . . . . . 72 3.6 The Fourier Transform for L2 Functions 75 3. 7 Smoothness versus Decay . . . . . . . . 76 3.8 Dilation, Translation, and Modulation . 79 3.9 Bandlimited Functions and the Sampling Formula 81

4 Signals and Systems 87 4.1 Signals . . . . . . . . . . . . . . 88 4.2 Systems . . . . . . . . . . . . . 90

4.2.1 Causality and Stability 95 4.3 Periodic Signals and the Discrete Fourier Transform 101

4.3.1 The Discrete Fourier Transform . 102 4.4 The Fast Fourier Transform 107 4.5 L2 Fourier Series ............ . 109

II The Haar System 113

5 The Haar System 115 5.1 Dyadic Step Functions . . . . . . . . . . . 115

5.1.1 The Dyadic Intervals . . . . . . . . 115 5.1.2 The Scale j Dyadic Step Functions 116

5.2 The Haar System . . . . . . . . . . . . . . 117 5.2.1 The Haar Scaling Functions and the

Haar Functions . . . . . . . . . . . 117 5.2.2 Orthogonality of the Haar System 118 5.2.3 The Splitting Lemma . . . . . . . 120

5.3 Haar Bases on [0, 1] . . . . . . . . . . . . . 122 5.4 Comparison of Haar Series with Fourier Series . 127

5.4.1 Representation of Functions with Small Support 128 5.4.2 Behavior of Haar Coefficients Near

Jump Discontinuities ........... . 5.4.3 Haar Coefficients and Global Smoothness

5.5 Haar Bases on R ................. . 5.5.1 The Approximation and Detail Operators 5.5.2 The Scale J Haar System on R . 5.5.3 The Haar System on R .....

130 132 133 134 138 138

6 The Discrete Haar Transform 141 6.1 Motivation . . . . . . . . . . . . . . . . . . . 141

6.1.1 The Discrete Haar Transform (DHT) . 142 6.2 The DHT in Two Dimensions . . . . . . . . . 146

6.2.1 The Row-wise and Column-wise Approximations and Details . . . . . . . . . . . . . . . . . . . . . 146

6.2.2 The DHT for Matrices ... . 6.3 Image Analysis with the DHT ... .

6.3.1 Approximation and Blurring 6.3.2 Horizontal, Vertical, and Diagonal Edges 6.3.3 "Naive" Image Compression ....... .

III Orthonormal Wavelet Bases

7 Multiresolution Analysis 7.1 Orthonormal Systems of Translates . 7.2 Definition of Multiresolution Analysis

7.2.1 Some Basic Properties of MRAs 7.3 Examples of Multiresolution Analysis .

7.3.1 The Haar MRA ...... . 7.3.2 The Piecewise Linear MRA 7.3.3 The Bandlimited MRA .. 7.3.4 The Meyer MRA ..... .

7.4 Construction and Examples of Orthonormal Wavelet Bases . . . . . . . . . . . . . 7.4.1 Examples of Wavelet Bases .. 7.4.2 Wavelets in Two Dimensions 7.4.3 Localization of Wavelet Bases .

7.5 Proof of Theorem 7.35 ........ . 7.5.1 Sufficient Conditions for a Wavelet Basis. 7.5.2 Proof of Theorem 7.35 ........ .

7.6 Necessary Properties of the Scaling Function 7. 7 General Spline Wavelets . . . . . ..... .

7.7.1 Basic Properties of Spline Functions 7.7.2 Spline Multiresolution Analyses .

8 The Discrete Wavelet Transform 8.1 Motivation: From MRA to a Discrete Transform

Contents 1x

147 150 151 153 154

161

163 164 169 170 174 174 174 179 180

185 186 190 193 196 197 199 203 206 206 208

215 215

8.2 The Quadrature Mirror Filter Conditions . . . . 218 8.2.1 Motivation from MRA . . . . . . . . . . . 218 8.2.2 The Approximation and Detail Operators and

Their Adjoints . . . . . . . . . . . . . . . . . . 221 8.2.3 The Quadrature Mirror Filter ( QMF) Conditions . 223

8.3 The Discrete Wavelet Transform (DWT) . 231 8.3.1 The DWT for Signals . . . . . . . . . . . . . 231 8.3.2 The DWT for Finite Signals . . . . . . . . . . 231 8.3.3 The DWT as an Orthogonal Transformation 232

8.4 Scaling Functions from Scaling Sequences 236 8.4.1 The Infinite Product Formula . 237 8.4.2 The Cascade Algorithm . . . . . . 243

X Contents

8.4.3 The Support of the Scaling Function 245

9 Smooth, Compactly Supported Wavelets 249 9.1 Vanishing Moments. . . . . . . . . . . . . 249

9.1.1 Vanishing Moments and Smoothness . . 250 9.1.2 Vanishing Moments and Approximation 254 9.1.3 Vanishing Moments and the Reproduction

of Polynomials . . . . . . . . . . . . . . . . 257 9.1.4 Equivalent Conditions for Vanishing Moments. 260

9.2 The Daubechies Wavelets . . . . . . 264 9.2.1 The Daubechies Polynomials . 264 9.2.2 Spectral Factorization . . . . . 269

9.3 Image Analysis with Smooth Wavelets 277 9.3.1 Approximation and Blurring . 278 9.3.2 "Naive" Image Compression with

Smooth Wavelets . . . . . . . . . . . . . . . . . . . . 278

IV Other Wavelet Constructions

10 Biorthogonal Wavelets 10.1 Linear Independence and Biorthogonality 10.2 Riesz Bases and the Frame Condition .. 10.3 Riesz Bases of Translates . . . . . . . . . 10.4 Generalized Multiresolution Analysis (GMRA)

10.4.1 Basic Properties of GMRA ...... . 10.4.2 Dual GMRA and Riesz Bases of Wavelets

10.5 Riesz Bases Orthogonal Across Scales ..... . 10.5.1 Example: The Piecewise Linear GMRA .

10.6 A Discrete Transform for Biorthogonal Wavelets 10.6.1 Motivation from GMRA ....... . 10.6.2 The QMF Conditions ........ .

10.7 Compactly Supported Biorthogonal Wavelets 10.7.1 Compactly Supported Spline Wavelets 10. 7.2 Symmetric Biorthogonal Wavelets 10. 7.3 Using Symmetry in the DWT .

11 Wavelet Packets 11.1 Motivation: Completing the Wavelet Tree 11.2 Localization of Wavelet Packets ..

11.2.1 Time/Spatial Localization .... . 11.2.2 Frequency Localization ..... .

11.3 Orthogonality and Completeness Properties of Wavelet Packets . . . . . . . . . . . . . . . . . 11.3.1 Wavelet Packet Bases with a Fixed Scale

287

289 289 290 293 300 301 302 311 313 315 315 317 319 320 324 328

335 335 337 337 338

346 347

11.302 Wavelet Packets with Mixed Scales 0 0 0 0 0 11.4 The Discrete Wavelet Packet Transform (DWPT) 0

110401 The DWPT for Signals 0 0 0 0 11.402 The DWPT for Finite Signals 0 0 0 0 0

11.5 The Best-Basis Algorithm 0 0 0 0 0 0 0 0 0 0 0 11.501 The Discrete Wavelet Packet Library 0 11.502 The Idea of the Best Basis 0 0 11.503 Description of the Algorithm 0 0 0 0 0

V Applications

Contents xi

350 354 354 354 357 357 360 363

369

12 Image Compression 371 1201 The Transform Step 0 0 0 0 0 0 0 0 0 372

1201.1 Wavelets or Wavelet Packets? 372 1201.2 Choosing a Filter 0 373

1202 The Quantization Step 0 0 373 1203 The Coding Step 0 0 0 0 0 375

120301 Sources and Codes 376 120302 Entropy and Information 378 120303 Coding and Compression 380

1204 The Binary Huffman Code 0 0 0 0 385 1205 A Model Wavelet Transform Image Coder 387

120501 Examples 0 0 0 0 0 0 0 0 0 0 0 0 0 0 388

13 Integral Operators 397 1301 Examples of Integral Operators 0 0 0 0 0 0 0 0 0 0 0 397

1301.1 Sturm-Liouville Boundary Value Problems 0 397 1301.2 The Hilbert Transform 0 402 1301.3 The Radon Transform 0 0 0 0 0 0 406

1302 The BCR Algorithm 0 0 0 0 0 0 0 0 0 0 0 414 130201 The Scale j Approximation to T 415 130202 Description of the Algorithm 0 0 418

VI Appendixes 423

A Review of Advanced Calculus and Linear Algebra 425 Ao1 Glossary of Basic Terms from Advanced Calculus and

Linear Algebra 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 425 Ao2 Basic Theorems from Advanced Calculus 0 0 0 0 0 0 0 431

B Excursions in Wavelet Theory 433 Bo1 Other Wavelet Constructions 433

Bol.1 M-band Wavelets 0 0 433

xii Contents

B.1.2 Wavelets with Rational Noninteger Dilation Factors 0 •••• 0 •••• 434

B.1.3 Local Cosine Bases . . . . . . . . . 434 B.1.4 The Continuous Wavelet Transform 435 B.1.5 Non-MRA Wavelets 436 B.1.6 Multiwavelets ..... 436

B.2 Wavelets in Other Domains 437 B.2.1 Wavelets on Intervals 437 B.2.2 Wavelets in Higher Dimensions 438 B.2.3 The Lifting Scheme 438

B.3 Applications of Wavelets . . . . . . 439 B.3.1 Wavelet Denoising ..... 439 B.3.2 Multiscale Edge Detection . 439 B.3.3 The FBI Fingerprint Compression Standard . 439

c References Cited in the Text 441

Index 445

Preface These days there are dozens of wavelet books on the market, some of which are destined to be classics in the field. So a natural question to ask is: Why another one? In short, I wrote this book to supply the particular needs of students in a graduate course on wavelets that I have taught several times since 1991 at George Mason University. As is typical with such offerings, the course drew an audience with widely varying backgrounds and widely varying expectations. The difficult if not impossible task for me, the instructor, was to present the beauty, usefulness, and mathematical depth of the subject to such an audience.

It would be insane to claim that I have been entirely successful in this task. However, through much trial and error, I have arrived at some basic principles that are reflected in the structure of this book. I believe that this makes this book distinct from existing texts, and I hope that others may find the book useful.

(1) Consistent assumptions of mathematical preparation. In some ways, the subject of wavelets is deceptively easy. It is not difficult to under­stand and implement a discrete wavelet transform and from there to analyze and process signals and images with great success. However, the underly­ing ideas and connections that make wavelets such a fascinating subject require some considerable mathematical sophistication. There have been some excellent books written on wavelets emphasizing their elementary nature (e.g., Kaiser, A Friendly Guide to Wavelets; Strang and Nguyen, Wavelets and Filter Banks; Walker, Primer on Wavelets and their Scien­tific Applications; Frazier, Introduction to Wavelets through Linear Alge­bra; Nievergelt, Wavelets Made Easy; Meyer, Wavelets: Algorithms and Applications). For my own purposes, such texts required quite a bit of "filling in the gaps" in order to make some connections and to prepare the student for more advanced books and research articles in wavelet theory.

This book assumes an upper-level undergraduate semester of advanced calculus. Sufficient preparation would come from, for example, Chapters 1-5 of Buck, Advanced Calculus. I have tried very hard not to depart from this assumption at any point in the book. This has required at times sacrificing elegance and generality for accessibility. However, all proofs are completely rigorous and contain the gist of the more general argument. In this way, it is hoped that the reader will be prepared to tackle more sophisticated books and articles on wavelet theory.

(2) Proceeding from the continuous to the discrete. I have always found it more meaningful and ultimately easier to start with a presenta-

xiv Preface

tion of wavelets and wavelet bases in the continuous domain and use this to motivate the discrete theory, even though the discrete theory hangs to­gether in its own right and is easy to understand. This can be frustrating for the student whose primary interest is in applications, but I believe that a better understanding of applications can ultimately be achieved by doing things in this order.

(3) Prepare readers to explore wavelet theory on their own. Wave­lets is too broad a subject to cover in a single book and is most interesting to study when the students have a particular interest in what they are study­ing. In choosing what to include in the book, I have tried to ensure that students are equipped to pursue more advanced topics on their own. I have included an appendix called Excursions in Wavelet Theory (Appendix B) that gives some guidance toward what I consider to be the most readable articles on some selected topics. The suggested topics in this appendix can also be used as the basis of semester projects for the students.

Structure of the Book

The book is divided into five parts: Preliminaries, The Haar System, Mul­tiresolution Analysis and Orthonormal Wavelet Bases, Other Wavelet Con­structions, and Applications.

Preliminaries

Wavelet theory is really very hard to appreciate outside the context of the language and ideas of Fourier Analysis. Chapters 1-4 of the book provide a background in some of these ideas and include everything that is sub­sequently used in the text. These chapters are designed to be more than just a reference but less than a "book-within-a-book" on Fourier analysis. Depending on the background of the reader or of the class in which this book is being used, these chapters are intended to be dipped into either superficially or in detail as appropriate.

Naturally there are a great many books on Fourier analysis that cover the same material better and more thoroughly than do Chapters 1-4 and at the same level (more or less) of mathematical sophistication. I will list some of my favorites below. Walker, Fourier Analysis; Kammler, A First Course in Fourier Analysis; Churchill and Brown, Fourier Series and Boundary Value Problems; Dym and McKean, Fourier Series and Integrals; Korner, Fourier Analysis; and Benedetto, Harmonic Analysis and Applications.

The Haar System

Chapters 5 and 6 provide a self-contained exposition of the Haar system, the earliest example of an orthonormal wavelet basis. These chapters could

Preface xv

be presented as is in a course on advanced calculus, or an undergraduate Fourier analysis course. In the context of the rest of the book, these chap­ters are designed to motivate the search for more general wavelet bases with different properties, and also to illustrate some of the more advanced concepts such as multiresolution analysis that are used throughout the rest of the book. Chapter 5 contains a description of the Haar basis on [0, 1] and on R, and Chapter 6 shows how to implement a discrete version of the Haar basis in one and two dimensions. Some examples of images analyzed with the Haar wavelet are also included.

Multiresolution Analysis and Orthonormal Wavelet Bases

Chapters 7-9 represent the heart of the book. Chapter 7 contains an expo­sition of the general notion of a multiresolution analysis (MRA) together with several examples. Next, we describe the recipe that gives the construc­tion of a wavelet basis from an MRA, and then construct corresponding examples of wavelet orthonormal bases. Chapter 8 describes the passage from the continuous domain to the discrete domain. First, properties of MRA are then used to motivate and define the quadrature mirror filter (QMF) conditions that any orthonormal wavelet filter must satisfy. Then the discrete wavelet transform (DWT) is defined for infinite signals, pe­riodic signals, and for finite sets of data. Finally the techniques used to pass from discrete filters satisfying the QMF conditions to continuously defined wavelet functions are described. Chapter 9 presents the construc­tion of compactly supported orthornomal wavelet bases due to Daubechies. Daubechies's approach is motivated by a lengthy discussion of the impor­tance of vanishing moments in the design of wavelet filters.

Other Wavelet Constructions

Chapters 10 and 11 contain a discussion of two important variations on the theme of the construction of orthonormal wavelet bases. The first, in Chapter 10, shows what happens when you allow yourself to consider nonorthogonal wavelet systems. This chapter contains a discussion of Riesz bases, and describes the semi-orthogonal wavelets of Chui and Wang, as well as the notion of dual MRA and the fully biorthogonal wavelets of Daubechies, Cohen, and Feauveau. Chapter 11 discusses wavelet packets, another natural variation on orthonormal wavelet bases. The motivation here is to consider what happens to the DWT when the "full wavelet tree" is computed. Waveletpacket functions are described, their time and fre­quency localization properties are discussed, and necessary and sufficient conditions are given under which a collection of scaled and shifted wavelet­packets constitutes an orthonormal basis on R. Finally, the notion of a best basis is described, and the so-called best basis algorithm (due to Coifman and Wickerhauser) is given.

xvi Preface

Applications

Many wavelet books have been written emphasizing applications of the theory, most notably, Strang and Nguyen, Wavelets and Filter Banks, and Mallat's comprehensive, A Wavelet Tour of Signal Processing. The book by Wickerhauser, Applied Wavelet Analysis from Theory to Software, also contains descriptions of several applications. The reader is encouraged to consult these texts and the references therein to learn more about wavelet applications.

The description of applications in this book is limited to a brief de­scription of two fundamental examples of wavelet applications. The first, described in Chapter 12, is to image compression. The basic components of a transform image coder as well as how wavelets fit into this picture are described. Chapter 13 describes the Beylkin-Coifman-Rokhlin (BCR) algo­rithm, which is useful for numerically estimating certain integral operators known as singular integral operators. The algorithm is very effective and uses the same basic properties of wavelets that make them useful for image compression. Several examples of singular integral operators arising in ordi­nary differential equations, complex variable theory, and image processing are given before the BCR algorithm is described.

Acknowledgments

I want to express my thanks to the many folks who made this book possible. First and foremost, I want to thank my advisor and friend John Benedetto for encouraging me to take on this project and for graciously agreeing to publish it in his book series. Thanks also to Wayne Yuhasz, Lauren Schultz, Louise Farkas, and Shoshanna Grossman at Birkhauser for their advice and support. I want to thank Margaret Mitchell for LaTeX advice and Jim Houston and Clovis L. Tondo for modifying some of the figures to make them more readable. All of the figures in this book were created by me using MATLAB and the Wavelet ToolBox. Thanks to the Math Works for creating such superior products.

I would like also to thank the National Science Foundation for its support and to the George Mason University Mathematics Department (especially Bob Sachs) for their constant encouragement. I also want to thank the students in my wavelets course who were guinea pigs for an early version of this text and who provided valuable feedback on organization and found numerous typos in the text. Thanks to Ben Crain, James Holdener, Amin Jazaeri, Jim Kelliher, Sami Nefissi, Matt Parker, and Jim Timper.

I also want to thank Bill Heller, Joe Lakey, and Paul Salamonowicz for their careful reading of the text and their useful comments. Special thanks go to David Weiland for his willingness to use the manuscript in an undergraduate course at Swarthmore College. The book is all the better

Preface xvii

for his insights, and those of the unnamed students in the class. I want give special thanks to my Dad, with whom I had many conversa­

tions about book-writing. He passed away suddenly while this book was in production and never saw the finished product. He was pleased and proud to have another published author in the family. He is greatly missed.

Finally, I want to thank my wife Megan for her constant love and support, and my delightful children John and Genna who will someday read their names here and wonder how their old man actually did it.

Fairfax, Virginia David F. Walnut

Albrecht Durer (1471-1528), Melencholia I (engraving). Courtesy of the Fogg Art Museum, Harvard University Art Museums, Gift of William Gray from the collection of Francis Calley Gray. Photograph by Rick Stafford, ©President and Fellows of Harvard College. A detail of this engraving, a portion of the magic square, is used as the sample image in 22 figures in this book. The file processed is a portion of the image file detail.mat packaged with MATLAB version 5.0.