Applied and Numerical Harmonic Analysis

Series Editor John J. Benedetto University of Maryland

Editorial Advisory Board

Akram Aldroubi Vanderbilt University

Ingrid Daubechies Princeton University

Christopher Heil Georgia Institute of Technology

James McClellan Georgia Institute of Technology

Michael Unser Ecole Polytechnique Federale de Lausanne

M. Victor Wickerhauser Washington University, St. Louis

Douglas Cochran Arizona State University

Hans G. Feichtinger University of Vienna

Murat Kunt Ecole Polytechnique Federale de Lausanne

Wim Sweldens Lucent Technologies Bell Laboratories

Martin Vetterli Ecole Polytechnique Federale de Lausanne

Modern Sampling Theory Mathematics and Applications

John 1 Benedetto Paulo lS.G. Ferreira

Editors

With 37 Figures

Springer Science+Business Media, LLC

John 1. Benedetto Department of Mathematics University of Maryland CoIIege Park, MD 20742 USA

Paulo 1.S.G. Ferreira Departamento de Electronica

e Telecommunicacoes Universidade de Aveiro Aveiro 3810-193 Portugal

Library of Congress Cataloging-in-Publication Data Benedetto, John.

Modern sampling theory : mathematics and applications I John J. Benedetto, Paulo J.S.O. Ferreira.

p. cm.-(Applied and numerica! harmonic ana!ysis) Includes bibliographica! references and index. ISBN 978-1-4612-6632-7 ISBN 978-1-4612-0143-4 (eBook) DOI 10.1007/978-1-4612-0143-4 1. Sampling (Statistics) I. Ferreira, Pauio 1.S.0. II. Title. III. Series.

QA276.6 .B46 2000 5 1 9.5'2--dc2 1 00-060867

Printed on acid-free paper.

© 2001 Springer Science+Business Media New York Originally published by Birkhăuser Boston in 2001

Softcover reprint of the hardcover lst edition 2001

CIP

All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher Springer Science+Business Media, LLC,

except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieva!, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use of general descriptive names, trade names, trademarks, etc., in this publication, even if the former are not especiall y identified, is not to be taken as a sign that such names, as understood by the Trade Marks and Merchandise Marks Act, may accordingly be used freely by anyone.

ISBN 978-1-4612-6632-7 SPIN 10645585

Produc ti an managed by Louise Farkas; manufacturing supervised by Erica Bresler. Typeset by the authars in TeX.

9 8 7 6 5 432 1

Contents

Preface xi

Contributors xiii

1 Introduction 1 JOHN J. BENEDETTO AND PAULO J. S. G. FERREIRA 1.1 The Classical Sampling Theorem . . 1 1.2 Non-Uniform Sampling and Frames. . . . . . . . 10 1.3 Outline of the Book ................ 21

1.3.1 Sampling, Wavelets, and the Uncertainty Principle .................. 22

1.3.2 Sampling Topics from Mathematical Analysis 23 1.3.3 Sampling Tools and Applications . . . . . . . 24

2 On the Transmission Capacity of the "Ether" and Wire in Electrocommunications 27 V. A. KOTEL'NIKOV (TRANSLATED BY V. E. KATSNELSON)

I Sampling, Wavelets, and the Uncertainty Principle 47

3 Wavelets and Sampling 49 GILBERT G. WALTER 3.1 Introduction ...... 49

3.1.1 Background .. 49 3.1.2 The RKHS Setting 50 3.1.3 The Wavelet Setting 51

3.2 Sampling in Other Spaces 52 3.2.1 Impulse Train Convergence 53 3.2.2 RKHS and Sampling . . . . 57

3.3 Sampling in Wavelet Subspaces .. 58 3.3.1 Elements of Wavelet Theory. 58 3.3.2 Sampling Functions ..... 60 3.3.3 Sampled Values as Coefficients 63

3.4 Interpolating Multiwavelets ...... 66 3.4.1 Properties of Hermite Sampling Functions 67

Vl Contents

4 Embeddings and Uncertainty Principles for Generalized Modulation Spaces 73 J. A. HOGAN AND J. D. LAKEY

4.1 Introduction......................... 73 4.1.1 Weighted Fourier Inequalities Imply Uncertainty

Principle Inequalities. . . . . . . . . . . . 75 4.1.2 Sharp Constants and Endpoint Estimates . . . . 76 4.1.3 Embeddings for Modulation Spaces. . . . . . . . 77

4.2 The Short-Time Fourier Transform and Modulation Spaces 77 4.2.1 Weighted Fourier Inequalities Imply Modulation

Embeddings. . . . . . . . . . . . . . . . . . . . . .. 80 4.2.2 Representation Theory and the Link with Littlewood-

Paley Theory . . . . . . . . . . . . . . . . . . 81 4.2.3 More Remarks on the Uncertainty Principle. 82

4.3 Modulation Embeddings and Uncertainty Principles 82 4.3.1 Grochenig's Two-Sided Embedding Theorem 82 4.3.2 Lieb's Sharp Single-Sided Embedding Theorem 88

4.4 Symmetric Localization . . . . . . . . . . . . . . . . . 89 4.5 Generalized Modulation Spaces . . . . . . . . . . . . . 93

4.5.1 Generalized Square-Integrability, Frames, and the Metaplectic Group . . . . . . . . 94

4.5.2 The Metaplectic Group .... . . . . . . . . . 95 4.5.3 Generalized Square Integrability . . . . . . . . 96 4.5.4 Connection with Time-Frequency Localization 97 4.5.5 Metaplectic Frames. . . . . . . . . . . . . 97

4.6 Generalized Modulation Spaces and Embedding. 100

5 Sampling Theory for Certain Hilbert Spaces of Bandlimited Functions 107 JEAN-PIERRE GABARDO

5.1 Introduction................. 107 5.2 Notation................... 108 5.3 Positive-Definite Distributions on (-R, R) 109 5.4 The Case Where MR(/1) Has Finite Codimension in L~ 115 5.5 The General Case. . . . 129 5.6 Riesz Bases and Frames . . . . . . . . . . . . . . . . . . 134

6 Shannon-Type Wavelets and the Convergence of Their Associated Wavelet Series 135 AHMED I. ZAYED

6.1 Introduction.... 135 6.2 Preliminaries . . . 136 6.3 Shannon's Wavelet 138 6.4 Generalized Shannon Wavelets 140 6.5 Pointwise Convergence of Shannon-type Wavelet Series. 147

Contents Vll

II Sampling Topics from Mathematical Analysis 153

7 Non-Uniform Sampling in Higher Dimensions: From Trigonometric Polynomials to Bandlimited Functions 155 KARL HEINZ GROCHENIG

7.1 Introduction......................... 155 7.2 Non-Uniform Sampling With Trigonometric Polynomials. 157 7.3 Toward Bandlimited Functions 161 7.4 Proof of Theorem 7.1 . . . . . . . . . . . . . . . . . . . .. 167

8 The Analysis of Oscillatory Behavior in Signals Through Their Samples 173 RODOLFO H. TORRES

8.1 Introduction and Motivation. . . . . . . . . . . 173 8.2 Besov Spaces of Functions . . . . . . . . . . . . 178 8.3 Sampling Theorem for Besov Spaces and Their

Discrete Counterpart . . . . . . . . . . . . . . . 182 8.4 Other Characterizations of the Discrete Besov Spaces 185 8.5 Nonlinear Approximation of Band Limited Signals

Using Samples ... 188 8.6 Concluding Remarks . . . . . . . . . . . . . . . . . 192

9 Residue and Sampling Techniques in Deconvolution 193 STEPHEN CASEY AND DAVID WALNUT

9.1 Introduction........................ 193 9.1.1 Statement of the Problem . . . . . . . . . . . . 193 9.1.2 The Coprime Condition and Local Deconvolution. 194 9.1.3 Other Types of Deconvolvers 195 9.1.4 Example: Cubes in]Rd . 196 9.1.5 Example: Balls in ]Rd. . . . . 197 9.1.6 Practical Deconvolution . . . 198

9.2 Residue Methods for Deconvolution. 199 9.3 Sampling Methods for Deconvolution. 209

9.3.1 Deconvolution and Sets of Uniqueness in cm~2[-R,R] . . . . . . . . . . . . . . . . 210

9.3.2 Nonperiodic Frames and Bases for L2 [-R, R] 215 9.3.3 Deconvolution and the Completeness of:F . . 217

10 Sampling Theorems from the Iteration of Low Order Differential Operators 219 J. R. HIGGINS

10.1 Introduction. . . . . . . . . . . 219 10.2 Preliminary Facts and Methods 220 10.3 Examples of the Method . . . 223 10.4 Remarks and Open Problems . 227

viii Contents

11 Approximation of Continuous Functions by Rogosinski-Type Sampling Series 229 AND! KIVINUKK

11.1 Notation and Introduction to the Sampling Series. 229 11.2 Rogosinski-Type Sampling Series . . . . . . . 232 11.3 Applications to Generalized Sampling Series. 239 11.4 Conclusion ................... 244

III Sampling Tools and Applications

12 Fast Fourier Transforms for Nonequispaced Data: A Tutorial DANIEL POTTS, GABRIELE STEIDL, AND MANFRED TASCHE

12.1 Introduction .................... . 12.2 NDFT for Nonequispaced Data Either in Time or

Frequency Domain . . . . . . . . . . . . . . . 12.3 NDFT for Nonequispaced Data in Time and

Frequency Domain . . 12.4 Roundoff Errors 12.5 Fast Bessel Transform

245

247

247

250

258 261 265

13 Efficient Minimum Rate Sampling of Signals with Frequen-cy Support over Non-Commensurable Sets 271 CORMAC HERLEY AND PING WAH WONG

13.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . 271 13.2 Periodic Non-Uniform Sampling. . . . . . . . . . . . . . . . 273 13.3 Reconstruction of a Discrete-Time Signal from N Samples

Out of M . . . . . . . . . . . . . . . . . . . . . 275 13.4 Filter Design Using POCS ........... . 13.5 Minimum Rate Sampling of Multiband Signals 13.6 Slicing the Spectrum ......... .

13.6.1 Dividing the Bands into Slices 13.6.2 Freedom in Pairing. 13.6.3 Pairing the Edges ...... .

14 Finite- and Infinite-Dimensional Models for Oversampled

283 284 288 290 290 291

Filter Banks 293 THOMAS STROHMER

14.1 Introduction. . . . . . . . . . . . . . . . . . . 293 14.1.1 Oversampled Filter Banks and Frames 295

14.2 Finite-Dimensional Models for Filter Banks . 296 14.2.1 The Finite Section Method for Oversampled

Filter Banks .................. 297

Contents ix

14.2.2 Finite Sections, Laurent Operators, and Polyphase Matrices . . . . . . . . . . . . . . . . . . . . . . . . . 298

14.2.3 Polyphase Matrices and "Perfect Symbol Calculus" in Finite Dimensions . . . . . . . . . . . . . . 300

14.3 Convergence and Rates of Approximation for Finite Dimensional Approximations . . . . . . . . . . . . . 301 14.3.1 Rates of Approximation for Oversampled Filter

Banks. . . . . . . . . . . . . . . . . . . . . . . . 302 14.4 Convergence Using Polyphase Representation . . . . . . 305 14.5 Finite-Dimensional Approximation of Paraunitary Filter

Banks Via S-! . . . . . . . . . . . . . . . . . . . . . . . 309 14.6 Oversampled DFT Filter Banks - Beyond

Polyphase Representation . . . . . . . . . . . . . . . 311 14.6.1 Matrix Factorization of the Frame Operator. 312

15 Statistical Aspects of Sampling for Noisy and Grouped Da-ta 317 M. PAWLAK AND U. STADTMULLER

15.1 Introduction. . . . . . . . . . . . . . . . . . . . . 317 15.2 Sampling from Noisy Data and Signal Recovery. 319 15.3 Signal Recovery from Grouped Data 324 15.4 Statistical Accuracy 327 15.5 Concluding Remarks 334 15.6 Proofs . . . . . . . . 336

16 Reconstruction of MRI Images from Non-Uniform Sampling and Its Application to Intrascan Motion Correction in Functional MRI 343 MARC BOURGEOIS, FRANK T. A. W. WAJER,

DIRK VAN ORMONDT, AND DANIELLE GRAVERON-DEMILLY

16.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . 343 16.2 MRI Sampling Trajectories . . . . . . . . . . . . . . . 344 16.3 Motion Influence and Correction of Intrascan Motion

Artifacts . . . . . . . . . . . . . . . . . . . . . . . 346 16.3.1 Physiological and Subject Motions . . . . 346 16.3.2 Reduction of Motion Artifacts in Images. 347 16.3.3 Correction of Intrascan Motions in fMRI . 347

16.4 Image Reconstruction from Non-Uniform Sampling 350 16.4.1 Gridding from a Non-Uniform Grid to a Cartesian

Grid and Vice Versa . . . . . . . . . . . . . 350 16.4.2 Density Correction and Voronoi Algorithm 16.4.3 Limits of Gridding .

16.5 Bayesian Image Estimation ... . 16.5.1 Preamble ......... . 16.5.2 The Most Probable Image .

351 352 353 353 354

x Contents

16.5.3 The Likelihood . 355 16.5.4 The Prior . . . . 355 16.5.5 Calculation of Ti 358

16.6 Applications of Intrascan Motion Correction to fMRI . 358 16.6.1 Shepp-Logan Simulation. . . 358 16.6.2 Simulated fMRI Experiment 359

16.7 Conclusions. . . . . . . . . . . . . . 362

17 Efficient Sampling of the Rotation Invariant Radon Transform 365 LAURENT DESBAT AND CATHERINE MENNESSIER

17.1 Introduction. . . . . . . . . . . . . 365 17.1.1 Principle ............ 365 17.1.2 Inverse Problem Formalism . . 366

17.2 Efficient Sampling in 2D Tomography 369 17.2.1 Results in Standard Tomography 369 17.2.2 Generalization to the Rotation Invariant

Radon Transform. . . . . . . 371 17.3 Application to Doppler Imaging. . . . . . . . . . 374

17.3.1 Null Inclination Case: 0: = O. . . . . . . . 374 17.3.2 Perpendicular Inclination Case: 0: = 7r /2 . 374

17.4 Discussion. . . . . . . . . . . . . . . . . . . . . . 376

References 379

Index 414

Preface

This book was planned at the close of SampTA'97, an international work­shop on sampling theory and applications held in 1997 in Aveiro, Portugal. These workshops are held biennially. The first-named co-editor attributes the remarkable success of the conference in A veiro exclusively to the ex­traordinary effort and ability of his fellow co-editor.

A major feature of this book is Victor Katsnelson's English transla­tion from the Russian of Kotel'nikov's classical sampling paper, On the Transmission Capacity of the "Ether" and Wire in Electrocommunications (Chapter 2). It is probably safe to say that, until now, Kotel'nikov's paper has been referenced more than it has been read by the Western scientific community. Our Introduction is Chapter 1, and it contains a mathematical history and perspective on sampling theory, as well as an outline of the volume and an overview of each chapter.

The remainder of the book is divided into three parts reflecting, re­spectively, the role of wavelet theory in sampling, a broad range of other modern mathematical methods used in sampling theory, and some current and important tools and applications. We deeply appreciate the excellent contributions by the authors, and it has been a pleasure working with each of them.

It is natural to envisage a sequel to this volume devoted more extensively to tools and applications. In fact, sampling theory is a natural componen­t in the broad emerging area of mathematical engineering. For example, it is realistic to assert that mathematical engineering will be to today's mathematics departments what mathematical physics was to those a cen­tury ago; and it is no extravagance to note the fundamental impact of mathematics in engineering subjects such as speech and image processing, information theory, and biomedical engineering, to name but a few. Of course, Birkhiiuser's book series, Applied and Numerical Harmonic Anal­ysis, in which this volume appears, is devoted to such an interleaving of ideas and disciplines.

Xll Preface

We have been fortunate beneficiaries of patience and professionalism, par excellence, from Birkhiiuser's editorial offices in Boston. In particular, it is a great pleasure to acknowledge the help and guidance of Wayne Wheeler, Wayne Yuhasz, and, most extensively, Lauren Lavery. She tolerated our idiosyncracies, corrected our errant tendencies, and delivered the book. We hope you enjoy it.

John J. Benedetto College Park, Maryland

Paulo J. S. G. Ferreira A veiro, Portugal

Contributors John J. Benedetto

Marc Bourgeois

Stephen Casey

Laurent Desbat

Paulo J. S. G. Ferreira

Jean-Pierre Gabardo

Department of Mathematics, University of Maryland, College Park, MD 20742, USA.

E-mail: j jb@math. umd. edu

Laboratoire de RMN, CNRS UPRESA 5012, Universite Claude Bernard Lyon I - CPE, 43 Boulevard du 11 Novembre 1918, 69622 Villeurbanne Cedex, France.

E-mail: marc@jade.univ-lyon1.fr

Department of Mathematics and Statistic­s, American University, 4400 Massachusetts Ave., N.W., Washington, DC 20016-8050, US­A. E-mail: scasey@american.edu

TIMC-lMAG, UMR CNRS 5525, lAB, Fac­ulte de Medecine, UJF, 38706 La Tronche, France.

E-mail: Laurent.Desbat@imag.fr

Departamento de Electronica e Telecomuni­ca<;oes / IEETA, Universidade de Aveiro, 3810-193 Aveiro, Portugal.

E-mail: pjf@det.ua.pt

Department of Mathematics and Statistics, McMaster University, 1280 Main Street West, Hamilton, Ontario, L8S 4K1, Canada.

E-mail: gabardo@mcmaster.ca

Danielle Graveron-Demilly Laboratoire de RMN, CNRS UPRESA 5012, Universite Claude Bernard Lyon I - CPE, 43 Boulevard du 11 Novembre 1918, 69622 Villeurbanne Cedex, France. E-mail: graveron@azur.univ-lyonl.fr

X1V Contributors

Karlheinz Grochenig

Cormac Herley

J. R. Higgins

J. A. Hogan

V. E. Katsnelson

Andi Kivinukk

J. D. Lakey

Catherine Mennessier

Dirk van Ormondt

Department of Mathematics, U-9, The U­niversity of Connecticut, Storrs, CT 06269-3009, USA.

E-mail: groch@math. uconn. edu

Microsoft Research, One Microsoft Way, Red­mond, WA 98052, USA.

E-mail: cormac@microsoft.com

Department of Sciences, Anglia Polytechnic University, East Road, Cambridge CBl IP­T, UK.

E-mail: rhiggins@bridge.anglia.ac . uk

School of MPCE, Macquarie University, NSW 2109, Australia.

E-mail: jeffh@mpce.mq.edu.au

Department of Mathematics, The Weizmann Institute, Israel.

E-mail: katze@wisdom.weizmann.ac.il

Department of Mathematics and Computer Science, Tallinn Pedagogical University, Nar­va Str. 25, 10 120 Tallinn, Estonia.

E-mail: andik@tpu.ee

College of Arts and Sciences, Department of Mathematical Sciences, Box 30001, Dept. 3M­B, Las Cruces, New Mexico 88003-8001, USA.

E-mail: j lakey@nmsu. edu

TIMC-IMAG, UMR CNRS 5525, lAB, Fac­uM de Medecine, UJF, 38706 La Tronche, France.

E-mail: Catherine.Mennessier@imag.fr

Spin Imaging Group, Department of Applied Physics, Delft University of Technology, PO Box 5046, 2600 GA Delft, The Netherlands.

E-mail: ormo@si.tn.tudelft.nl

M. Pawlak

Daniel Potts

U. Stadt muller

Gabriele Steidl

Thomas Strohmer

Manfred Tasche

Rodolfo H. Torres

Frank T. A. W. Wajer

David Walnut

Gilbert G. Walter

Contributors xv

Department of Electrical and Computer En­gineering, University of Manitoba, Winnipeg, Man. R3T 5V6, Canada.

E-mail: pawlak©ee.umanitoba.ca

Medical University Lubeck, WallstraJ3e 40, 23560 Lubeck, Germany.

E-mail: potts©math.mu-luebeck.de

Abteilung fUr Mathematik III, University of Ulm, D 89069 Ulm, Germany.

University of Mannheim, D 7, 29, 68131 Mannheim, Germany.

E-mail: steidl©math.uni-mannheim.de

Department of Mathematics, 1 Shields Ave., University of California, Davis, CA 95616-8633, USA.

E-mail: strohmer©math. ucdavis. edu

Medical University Lubeck, WallstraJ3e 40, 23560 Lubeck, Germany.

E-mail: tasche@math.mu-luebeck.de

Department of Mathematics, University of Kansas, Lawrence, KS 66045, USA.

E-mail: torres©math. ukans. edu

Spin Imaging Group, Department of Applied Physics, Delft University of Technology, PO Box 5046, 2600 GA Delft, The Netherlands.

E-mail: waje©si.tn.tudelft.nl

Department of Mathematical Sciences, George Mason University, Fairfax, VA, 22030, USA.

E-mail: dwalnut©gmu. edu

Department of Mathematical Sciences, Uni­versity of Wiscousin, PO Box 413 Milwaukee WI 53201, USA.

E-mail: ggw©csd. uwm. edu

XVi Contributors

Ping Wah Wong

Ahmed 1. Zayed

Hewlett Packard, 11000 Wolfe Road, Cuper­tino, CA 95014, USA.

Department of Mathematics, University of Central Florida, Orlando, FL 32816, USA.

E-mail: zayed@pegasus. ee. uef . edu