Otto Bretscher

Linear Algebra with ApplicationsFourth Edition

Otto BretscherColby College

Prentice Hallis an imprint of

Upper Saddle River, New Jersey 07458

Library of Congress Cataloging-in-Publication DataBretscher, Otto.

Linear algebra with applications / Otto Bretscher.—4th ed.p. cm.

Includes index.ISBN 978-0-13-600926-9

1. Algebras, Linear—Textbooks. I. Title.QA184.2.B73 2009512'.5-dc22 2008028930

Editor-in-Chief, Collegiate Mathematics: Deirdre LynchSenior Acquisitions Editor: William HoffmanAssociate Editor/Print Supplements: Caroline CelanoProject Manager: Raegan Keida HeeremaProduction Coordination/Composition: ICC Macmillan Inc.Associate Managing Editor: Bayani Mendoza deLeonSenior Managing Editor: Linda Mihatov BehrensSenior Operations Supervisor: Alan FischerOperations Specialist: Lisa McDowellExecutive Marketing Manager: Jeff WeidenaarMarketing Assistant: Jon ConnellyArt Director: Maureen EideInterior Designer: Dina CurroCover Designer: Daniel ConteAV Project Manager: Thomas BenfattiManager, Cover Visual Research & Permissions: Karen SanatarArt Studio: Laserwords Private LimitedCover Image Credit: Andy BurmfielaVIstockphoto.comThe cover is a close-up image of a guitar, showing four vibrating strings and the sound hole.Using Fourier analysis, we can measure the amplitudes of a plucked string's harmonics or evensynthesize the guitar's sound. See pages 239-244.

Prentice Hallis an imprint of

PEARSON© 2009, 2005, 2001, 1997 Pearson Education, Inc.Pearson Prentice HallPearson Education, Inc.Upper Saddle River, NJ 07458

All rights reserved. No part of this book maybe reproduced, in any form or by any means,without permission in writing from the publisher.

Pearson Prentice Hall® is a trademark of Pearson Education, Inc.

Printed in the United States of America1 0 9 8 7 6 5 4 3 7 I

I S B N - 1 0 : 0 - 1 3 - 6 0 0 9 2 6 - 3ISBN-13 : 978-0 -13-600926-9

Pearson Education Ltd., LondonPearson Education Singapore, Pte. LtdPearson Education Canada, Inc.Pearson Education-JapanPearson Education Australia PTY, Limited

Pearson Education North Asia, Ltd., Hong KongPearson Education de Mexico, S.A. de C.V.Pearson Education Malaysia, Pte. Ltd.Pearson Education Upper Saddle River, New Jersey

To my parentsOtto and Margrit Bretscher-Zwicky

with love and gratitude

ContentsText Features ix

Preface xi

1 L i n e a r E q u a t i o n s 11 . 1 I n t r o d u c t i o n t o L i n e a r S y s t e m s 11.2 Matrices, Vectors, and Gauss-Jordan Elimination 81.3 On the Solutions of Linear Systems; Matrix Algebra 25

2 L i n e a r T r a n s f o r m a t i o n s 4 02.1 Introduction to Linear Transformations and

T h e i r I n v e r s e s 4 02 . 2 L i n e a r T r a n s f o r m a t i o n s i n G e o m e t r y 5 42 . 3 M a t r i x P r o d u c t s 6 92.4 The Inverse of a Linear Transformation 79

3 Subspaces of R" and Their Dimensions 1013.1 Image and Kernel of a Linear Transformation 1013.2 Subspaces of R"; Bases and Linear Independence 1133.3 The Dimens ion o f a Subspace o f R" 1233 . 4 C o o r d i n a t e s 1 3 7

4 L i n e a r S p a c e s 1 5 34 . 1 I n t r o d u c t i o n t o L i n e a r S p a c e s 1 5 34.2 Linear Transformations and Isomorphisms 1654.3 The Matr ix of a Linear Transformation 172

5 Orthogonality and Least Squares 1875.1 Orthogonal Projections and Orthonormal Bases 1875.2 Gram-Schmidt Process and QR Factorization 2035.3 Orthogonal Transformations and Orthogonal Matrices 2105 . 4 L e a s t S q u a r e s a n d D a t a F i t t i n g 2 2 05 . 5 I n n e r P r o d u c t S p a c e s 2 3 3

vii i Contents

6 D e t e r m i n a n t s 2 4 96 . 1 I n t r o d u c t i o n t o D e t e r m i n a n t s 2 4 96 . 2 P r o p e r t i e s o f t h e D e t e r m i n a n t 2 6 16.3 Geometrical Interpretations of the Determinant;

C r a m e r ' s R u l e 2 7 7

7 E igenva lues and E igenvec tors 2947.1 Dynamical Systems and Eigenvectors:

A n I n t r o d u c t o r y E x a m p l e 2 9 47.2 F ind ing the E igenva lues o f a Mat r i x 3087.3 F inding the Eigenvectors of a Matr ix 3197 . 4 D i a g o n a l i z a t i o n 3 3 27 . 5 C o m p l e x E i g e n v a l u e s 3 4 37 . 6 S t a b i l i t y 3 5 7

8 Symmetric Matrices and Quadratic Forms 3678 . 1 S y m m e t r i c M a t r i c e s 3 6 78 . 2 Q u a d r a t i c F o r m s 3 7 68 . 3 S i n g u l a r V a l u e s 3 8 5

9 L i n e a r D i f f e r e n t i a l E q u a t i o n s 3 9 79.1 An Introduction to Continuous Dynamical Systems 3979 .2 The Complex Case : Eu le r ' s Formu la 4109.3 Linear Differential Operators and

L i n e a r D i f f e r e n t i a l E q u a t i o n s 4 2 3

Appendix A Vectors 437Answers to Odd-Numbered Exercises 446Subject Index 471Name Index 478

SUBJECT INDEX

Affine system, 329Algebraic multiplicity of an eigenvalue, 312

and complex eigenvalues, 352and geometric multiplicity, 326

Algorithm, 18Alternating property of determinant,

251,265Angle, 196

and orthogonal transformations, 211Approximations,

Fourier 232, 239least-squares 17, 220

Argument (of a complex number), 346of a product, 348

Associative law for matrix multiplication, 75Asymptotically stable equilibrium, 358

isee stable equilibrium)Augmented matrix, 11

BBasis, 116, 158

and coordinates, 139and dimension, 125, 159and unique representation, 120finding basis of a linear space, 161of an image, 116, 128, 131of a kernel, 127, 131ofRM32standard, 126

Binary digits, 112Block matrices, 75

determinant of, 257inverse of, 87multiplication of, 76

Bounded trajectory, 363

Carbon dating, 407Cardano's formula, 316, 319, 356Cartesian coordinates, 10, 137Cauchy-Schwarz inequality, 195, 237

and angles, 196and triangle inequality, 200

Cayley-Hamilton Theorem, 332Center of mass, 21Change of basis, 177, 179Characteristic polynomial, 311

and algebraic multiplicity, 312

and its derivative, 318of linear differential operator, 426of similar matrices, 326

Chebyshev polynomials, 246Cholesky factorization, 383Circulant matrix, 356Classical adjoint, 287Closed-formula solution

for discrete dynamical system, 303for inverse, 287for least-squares approximation, 224for linear system, 286

Coefficient matrixof a linear system, 11of a linear transformation, 41

Column of a matrix, 9Column space of a matrix, 105Column vector, 10Commuting matrices, 73Complements, 135Complex eigenvalues, 350

and determinant, 353and rotation-scaling matrices, 352,

359,416and stable equilibrium, 359and trace, 353

Complex numbers, 155, 345and rotation-scaling, 348in polar form, 347

Complex-valued functions, 411derivative of, 411exponential, 412

Component of a vector, 10Composite functions, 69Computational complexity, 89, 96Concatenation, 323Conies, 23, 134Consistent linear system, 25Continuous least-squares condition, 238Continuous linear dynamical

system, 400stability of, 415with complex eigenbasis, 416with real eigenbasis, 405

Contraction, 56Coordinates, 139, 158Coordinate transformation, 159, 167Coordinate vector, 139, 159Correlation (coefficient), 198Cramer's Rule, 286

471

472 Subject Index

Cross productand polar (QS) factorization, 394in R3, 52, 443, 445in R", 275rules for, 444

Cubic equation (Cardano's formula),316,356

Cubic splines, 19Cubics, 134Curve fitting

with conies, 23, 134with cubics, 134

Data compression, 393Data fitting, 225

multivariate, 228De Moivre's formula, 348Determinant, 85, 255

alternating property of, 251, 265and characteristic polynomial,

310,311and eigenvalues, 315, 353and Cramer's rule, 286and elementary row operations, 265and invertibility, 266and Laplace expansion, 268, 270and QR factorization, 279as area, 85, 278as expansion factor, 282as volume, 280is linear in rows and columns,

252, 262of inverse, 268of linear transformation, 272of orthogonal matrix, 278of permutation matrix, 260of product, 267of rotation matrix, 278of similar matrices, 267of 3 x 3 matrix, 249of transpose, 262of triangular matrix, 257of 2 x 2 matrix, 85patterns and, 255Vandermonde, 273Weierstrass definition of, 276

Diagonalizable matrices, 333and eigenbases, 333and powers, 336orthogonally, 367, 372simultaneously, 342

Diagonal matrix, 9

Diagonal of a matrix, 9Dilation, 56Dimension, 125, 159

and isomorphism, 168of image, 129, 165of kernel, 129,165of orthogonal complement, 193

Direction field, 401Discrete linear dynamical system, 303

and complex eigenvalues, 360, 362and stable equilibrium, 358, 359

Distance, 236Distributive Laws, 75Domain of a function, 43Dominant eigenvector, 405Dot product, 20, 28, 215, 441

and matrix product, 73, 215and product Ax, 29rules for, 442

Dynamical systemisee continuous, discrete linear

dynamical system)

Eigenbasis, 322and continuous dynamical system, 405and diagonalization, 333and discrete dynamical system, 303and distinct eigenvalues, 324and geometric multiplicity, 324

Eigenfunction, 337, 426Eigenspaces, 320

and geometric multiplicity, 322and principal axes, 381

Eigenvalue(s), 300algebraic multiplicity of, 312and characteristic polynomial, 311and determinant, 315, 353and positive (semi)definite

matrices, 379and QR factorization, 316and singular values, 387and stable equilibrium, 359, 415and trace, 315, 353complex, 350, 362geometric multiplicity of, 322of linear transformation, 337of orthogonal matrix, 302of rotation-scaling matrix, 351of similar matrices, 326of symmetric matrix, 370of triangular matrix, 310power method for finding, 355

Subject Index 473

Eigenvectors, 300and linear independence, 323dominant, 405of symmetric matrix, 369

Elementary matrix, 93Elementary row operations, 16

and determinant, 265and elementary matrices, 93

Ellipse, 69as image of the unit circle, 69, 372, 387as level curve of quadratic form, 381as trajectory, 362, 416

Equilibrium (state), 358Equivalence Relation, 146Error (in least-squares solution), 222Error-correcting codes, 112Euler identities, 240Euler's formula, 413Euler's theorem, 328Exotic operations (in a linear space), 172Expansion factor, 282Exponential functions, 398

complex-valued, 412

Factorizations,Cholesky, 383LDLT, 220, 383LDU, 93, 220LLT, 383LU, 93ßfl, 207, 208, 279, 316QS, 394SOS-1, 336SDST, 367[/EVT,390

Field, 350Finite dimensional linear space, 162Flow line, 401Fourier analysis, 239Function, 43Fundamental theorem of algebra, 349Fundamental theorem of linear

algebra, 129

Gaussian integration, 246Gauss-Jordan elimination, 17

and determinant, 265and inverse, 82

Geometric multiplicity of an eigenvalue, 322and algebraic multiplicity, 326and eigenbases, 324

Golden section, 331Gram-Schmidt process, 206

and determinant, 278and orthogonal diagonalization, 372and QR factorization, 207

HHarmonics, 241Hubert space £2,201,235

and quantum mechanics, 243Homogeneous linear system, 35Hyperbola, 381Hyperplane, 134

IIdentity matrix /„, 45Identity transformation, 45Image of a function, 101Image of a linear transformation,

105, 165and rank, 129is a subspace, 105, 165orthogonal complement of, 221written as a kernel, 111

Imageof the unit circle, 69, 372, 387

Imaginary part of a complex number, 346Implicit function theorem, 276Inconsistent linear system, 5, 17, 25

and least-squares, 222Indefinite matrix, 378Infinite dimensional linear space, 162Inner product (space), 234Input-output analysis, 5, 20, 90, 95, 150Intermediate value theorem, 68Intersection of subspaces, 123

dimension of, 135Invariant Subspace, 307Inverse (of a matrix), 80

and Cramer's rule, 287determinant of, 268of an orthogonal matrix, 215of a product, 83of a transpose, 216of a 2 x 2 matrix, 85of block matrix, 87

Inversion (in a pattern), 255Invertible function, 79Invertible matrix, 80

and determinant, 266and kernel, 109

Isomorphism, 167and dimension, 168

474 Subject Index

KKernel, 106, 165

and invertibility, 109and linear independence, 120and rank, 129dimension of, 129is a subspace, 108, 165

Kyle numbers, 131,320

Laplace expansion, 268, 270LDL1 factorization, 220, 383LDU factorization, 93, 220Leading one, 16Leading variable, 13Least-squares solutions, 17, 201, 222

and normal equation, 223minimal, 230

Left inverse, 122Legendre polynomials, 246Length of a vector, 187, 442

and orthogonal transformations, 210Linear combination, 30, 156

and span, 105Linear dependence, 116Linear differential equation, 161, 423

homogeneous, 423order of, 423solution of, 425, 433solving a first order, 431solving a second order, 428, 429

Linear differential operator, 423characteristic polynomial of, 426eigenfunctions of, 426image of, 432kernel of, 427

Linear independenceand dimension, 126and kernel, 120and relations, 118inR«, 116in a linear space, 158of eigenvectors, 323of orthonormal vectors, 189

Linearity of the determinant, 252, 262Linear relations, 118Linear space(s), 154

basis of, 158dimension of, 159finding basis of, 161finite dimensional, 162isomorphic, 167

Linear systemclosed-formula solution for, 286consistent, 25homogeneous, 35inconsistent, 5, 25least-squares solutions of, 222matrix form of, 32minimal solution of, 230number of solutions of, 25of differential equations: see continuous linear

dynamical systemunique solution of, 27vector form of, 32with fewer equations than unknowns, 27

Linear transformation, 41, 44, 48, 165image of, 105, 165kernel of, 106, 165matrix of, 41, 44, 47, 143, 173

Lower triangular matrix, 9LU factorization, 93

and principal submatrices, 94

MMain diagonal of a matrix, 9Mass-spring system, 422Matrix, 9

(for a composite entry such as "zero matrix"see zero)

Minimal solution of a linearsystem, 230

Minors of a matrix, 269Modulus (of a complex number), 346

of a product, 348Momentum, 21Multiplication (of matrices), 72

and determinant, 267column by column, 72entry by entry, 73is associative, 75is noncommutative, 73of block matrices, 76

NNegative feedback loop, 420Neutral element, 154Nilpotent matrix, 136, 375Norm

of a vector, 187in an inner product space, 236

Normal equation, 223Nullity, 129, 165Null space, 106

Subject Index 475

Orthogonal complement, 193of an image, 221

Orthogonally diagonalizable matrix, 367, 372Orthogonal matrix, 211, 216

determinant of, 278eigenvalues of, 302inverse of, 215transpose of, 215

Orthogonal projection, 58, 189, 191,225, 237

and Gram-Schmidt process, 203and reflection, 59as closest vector, 222matrix of, 217, 225

Orthogonal transformation, 210and orthonormal bases, 217preserves dot product, 218preserves right angles, 211

Orthogonal vectors, 187, 236, 443and Pythagorean theorem, 194

Orthonormal bases, 189, 192and Gram-Schmidt process, 203and orthogonal transformations, 212and symmetric matrices, 368

Orthonormal vectors, 188are linearly independent, 189

Oscillator, 184

Positive (semi)definite matrix, 378and eigenvalues, 379and principal submatrices, 379

Positively oriented basis, 290Power method for finding eigenvalues, 355Principal axes, 381Principal submatrices, 94

and positive definite matrices, 379Product Ax, 29

and dot product, 29and matrix multiplication, 72

Projection, 56, 67isee also orthogonal projection)

Pseudo-inverse, 230Pythagorean theorem, 194, 237

QR factorization, 207and Cholesky factorization, 383and determinant, 279and eigenvalues, 316is unique, 219

QS factorization, 394Quadratic forms, 377

indefinite, 378negative (semi)definite, 378positive (semi)definite, 378principal axes for, 381

Quaternions, 220, 356, 364form a skew field, 356

Parallelepiped, 280Parallel vectors, 440Parametrization (of a curve), 102Partitioned matrices, 76

isee block matrix)Pattern (in a matrix), 255

inversion in, 255signature of, 255

Permutation matrix, 89determinant of, 260

Perpendicular vectors, 187, 236, 443Phase portrait,

of continuous system, 400, 419of discrete system, 304, 419summary, 419

Piecewise continuous function, 239Pivot, 16Polar factorization, 394Polar form (of a complex number), 347

and powers, 348and products, 348with Euler's formula, 413

Rank (of a matrix), 26, 129and image, 129and kernel, 129and row space, 136and singular values, 389of similar matrices, 326of transpose, 216

Rank-nullity theorem, 129, 165Real part of a complex number, 346Reduced row-echelon form (rref), 16

and determinant, 265and inverse, 80, 82and rank, 26

Redundant vector(s), 116, 158image and, 116, 131kernel and, 131

Reflection, 59Regular transition matrix, 317, 354Relations (among vectors), 118Resonance, 184Riemann integral, 232, 234

476 Subject Index

Rotation matrix, 62, 278Rotation-scaling matrix, 63

and complex eigenvalues, 351and complex numbers, 348

Row of a matrix, 9Row space, 136Row vector, 10Rule of 69, 408

Sarrus's rule, 250Scalar, 12Scalar multiples of matrices, 28Scaling, 54Second derivative test, 383Secular equation, 308Separation of variables, 398, 407Shears, 64Signature of a pattern, 255Similar matrices, 145

and characteristic polynomial, 326Simultaneously diagonalizable

matrices, 342Singular value decomposition

(SVD), 390Singular values, 387

and ellipses, 387and rank, 389

Skew field, 356Skew-symmetric matrices, 214, 382

determinant of, 274Smooth function, 157Space of functions, 155Span

inR", 105in a linear space, 159

Spectral theorem, 368Spirals, 344, 349, 362, 413, 416, 419Square matrix, 9Square-summable sequences, 201, 235Stable equilibrium, 358

of continuous system, 415of discrete system, 359

Standard basis, 126, 160State vector, 295Subspace

invariant, 307of R", 113of a linear space, 157

Sudoku,75Sum of subspaces, 123

dimension of, 135Sums of matrices, 28

Symmetric matrices, 214are orthogonally diagonalizable, 368have real eigenvalues, 370

Target space (of a function), 43Tetrahedron, 92, 150, 289Theorem of Pythagoras, 194, 237Trace, 235, 310

and characteristic polynomial, 311and eigenvalues, 315, 353and inner product, 235of similar matrices, 326

Trajectory, 304, 400bounded, 363

Transpose (of a matrix), 214and determinant, 262and inverse, 216and rank, 216of an orthogonal matrix, 215of a product, 216

Triangle inequality, 200Triangular matrices, 9

and determinant, 257and eigenvalues, 310invertible, 89

Triangulizable matrix, 375Trigonometric polynomials, 240

uUnit vector, 187,443Upper triangular matrix, 9

Vandermonde determinant, 273Vector(s), 10

angles between, 196addition, 437column, 10coordinate, 139, 159cross product of, 52, 249, 443dot product of, 441geometric representation of, 439length of, 187,442orthogonal, 187, 443parallel, 440position, 439redundant, 116row, 10rules of vector algebra, 438scalar multiplication, 437standard representation of, 10, 439unit, 187,443

Subject Index 477

velocity, 400vs. points, 10, 439zero, 438

Vector field, 401Vector form of a linear system, 32Vector space, 10, 154

isee linear space)Velocity vector, 400

wWeierstrass definition of determinant, 276Wigner semicircle distribution, 246

XZero matrix, 9Zero vector, 438

NAME INDEXAbel, Niels Henrik, 316Alcuin of York, 24al-Kashi, Jamshid, 24al-Khowarizmi, Mohammed, 1, 18nal-Ma'mun, Caliph, 1Archimedes, 53, 331Argand, Jean Robert, 346n

Banach, Stefan, 163Berlinski, David, 17Bezout, Etienne, 272Binet, Jacques, 272Burke, Kyle, 131

Cajori, Florian, 272Cardano, Gerolamo, 316, 319Cauchy, Augustin-Louis, 195, 255n, 272, 309Cayley, Arthur, 272, 332Chebyshev, Pafnuty, 246Cholesky, Andre-Louis, 383Cramer, Gabriel, 135, 272, 286

d'Alembert, Le Rond, 125, 309Dantzig, Tobias, 345De Moivre, Abraham, 348Descartes, Rene, 118n, 137, 155De Witt, Johan, 378

Einstein, Albert, 356-357Euler, Leonhard Paul, 24, 135, 240, 243, 328,

345, 413

Fermat, Pierre de, 137Fibonacci (Leonardo of Pisa), 330-331Fourier, Jean Baptiste Joseph, 239n

Galois, Evariste, 316Gauss, Carl Friedrich, 17-18, 51, 246, 346n, 350Gram, Jörgen, 206Grassmann, Hermann Günther, 125

Hamilton, William Rowan, 220, 332, 356Hamming, Richard W., 112Harriot, Thomas, 118nHeisenberg, Werner, 201, 244Helmholtz, Hermann von, 309Hubert, David, 162,201,309

Jacobi, Carl Gustav Jacob, 255nJordan, Wilhelm, 17

Kepler, Johannes, 233Kronecker, Leopold, 255n

Lagrange, Joseph Louis, 309, 364Lanchester, Frederick William, 409nLaplace, Pierre-Simon Marquis de, 268n, 272Legendre, Adrien-Marie, 246Leibniz, Gottfried Wilhelm von, 272Leonardo da Vinci, 200Leontief, Wassily, 5, 150

Mahavira, 24Mas-Colell, Andreu, 201Mazur, Barry, 345Medawar, Peter. B., 64Mumford, David, 423

Newton, Isaac, 24

Olbers, Wilhelm, 17

Peano, Giuseppe, 154nPeirce, Benjamin, 413nPiazzi, Giuseppe, 17Pythagoras, 194, 243, 272

Riemann, Bernhard, 232, 234, 246Roosevelt, Franklin D., 90n

Samuelson, Paul E., 364Sarrus, Pierre Frederic, 250nSchläft, Ludwig, 126Schmidt, Erhard, 162, 206nSchrödinger, Erwin, 244Schwarz, Hermann Amandus, 195Seki, Köwa (or Takakazu), 255, 272Simpson, Thomas, 246Strang, Gilbert, 393Strassen, Völker, 96Sylvester, James Joseph, 9n

Thompson, d'Arcy, 64

Vandermonde, Alexandre-Theophile, 255n, 274

Weierstrass, Karl, 273, 276Wessel, Caspar, 346nWeyl, Hermann, 125Wigner, Eugene Paul, 246Wilson, Edward O., 285n

478