Numerical Linear Algebra for Applications in Statistics

by Gentle, James E.; Chambers, J.; Eddy, W.; Haerdle, W.; Sheather, S.; Tierney, L.

• ISBN13: 9780387985428
• ISBN10: 0387985425
• Format: Hardcover
• Copyright: 1998-11-01
• Publisher: Springer Verlag

Summary

Numerical linear algebra is one of the most important subjects in the field of statistical computing. Statistical methods in many areas of application require computations with vectors and matrices. This book describes accurate and efficient computer algorithms for factoring matrices, solving linear systems of equations, and extracting eigenvalues and eigenvectors. Although the book is not tied to any particular software system, it describes and gives examples of the use of modern computer software for numerical linear algebra. An understanding of numerical linear algebra requires basic knowledge both of linear algebra and of how numerical data are stored and manipulated in the computer. The book begins with a discussion of the basics of numerical computations, and then describes the relevant properties of matrix inverses, matrix factorizations, matrix and vector norms, and other topics in linear algebra; hence, the book is essentially self- contained. The topics addressed in this book constitute the most important material for an introductory course in statistical computing, and should be covered in every such course. The book includes exercises and can be used as a text for a first course in statistical computing or as supplementary text for various courses that emphasize computations. James Gentle is University Professor of Computational Statistics at George Mason University. During a thirteen-year hiatus from academic work before joining George Mason, he was director of research and design at the world's largest independent producer of Fortran and C general-purpose scientific software libraries. These libraries implement many algorithms for numerical linear algebra. He is a Fellow of the American Statistical Association and member of the International Statistical Institute. He has held several national

Table of Contents

Preface	vii
1 Computer Storage and Manipulation of Data	1	(46)
1.1 Digital Representation of Numeric Data	3	(15)
1.2 Computer Operations on Numeric Data	18	(8)
1.3 Numerical Algorithms and Analysis	26	(15)
Exercises	41	(6)
2 Basic Vector/Matrix Computations	47	(40)
2.1 Notation, Definitions, and Basic Properties	48	(33)
2.1.1 Operations on Vectors; Vector Spaces	48	(4)
2.1.2 Vectors and Matrices	52	(3)
2.1.3 Operations on Vectors and Matrices	55	(3)
2.1.4 Partitioned Matrices	58	(1)
2.1.5 Matrix Rank	59	(1)
2.1.6 Identity Matrices	60	(1)
2.1.7 Inverses	61	(1)
2.1.8 Linear Systems	62	(1)
2.1.9 Generalized Inverses	63	(1)
2.1.10 Other Special Vectors and Matrices	64	(3)
2.1.11 Eigenanalysis	67	(2)
2.1.12 Similarity Transformations	69	(1)
2.1.13 Norms	70	(2)
2.1.14 Matrix Norms	72	(2)
2.1.15 Orthogonal Transformations	74	(1)
2.1.16 Orthogonalization Transformations	74	(1)
2.1.17 Condition of Matrices	75	(4)
2.1.18 Matrix Derivatives	79	(2)
2.2 Computer Representations and Basic Operations	81	(3)
2.2.1 Computer Representation of Vectors and Matrices	81	(1)
2.2.2 Multiplication of Vectors and Matrices	82	(2)
Exercises	84	(3)
3 Solution of Linear Systems	87	(36)
3.1 Gaussian Elimination	87	(5)
3.2 Matrix Factorizations	92	(11)
3.2.1 LU and LDU Factorizations	92	(1)
3.2.2 Cholesky Factorization	93	(2)
3.2.3 QR Factorization	95	(2)
3.2.4 Householder Transformations (Reflections)	97	(2)
3.2.5 Givens Transformations (Rotations)	99	(3)
3.2.6 Gram-Schmidt Transformations	102	(1)
3.2.7 Singular Value Factorization	102	(1)
3.2.8 Choice of Direct Methods	103	(1)
3.3 Iterative Methods	103	(4)
3.3.1 The Gauss-Seidel Method with Successive Overrelaxation	103	(1)
3.3.2 Solution of Linear Systems as an Optimization Problem; Conjugate Gradient Methods	104	(3)
3.4 Numerical Accuracy	107	(2)
3.5 Iterative Refinement	109	(1)
3.6 Updating a Solution	109	(2)
3.7 Overdetermined Systems; Least Squares	111	(4)
3.7.1 Full Rank Coefficient Matrix	112	(1)
3.7.2 Coefficient Matrix Not of Full Rank	113	(1)
3.7.3 Updating a Solution to an Overdetermined System	114	(1)
3.8 Other Computations for Linear Systems	115	(2)
3.8.1 Rank Determination	115	(1)
3.8.2 Computing the Determinant	115	(1)
3.8.3 Computing the Condition Number	115	(2)
Exercises	117	(6)
4 Computation of Eigenvectors and Eigenvalues and the Singular Value Decomposition	123	(14)
4.1 Power Method	124	(2)
4.2 Jacobi Method	126	(3)
4.3 QR Method for Eigenanalysis	129	(2)
4.4 Singular Value Decomposition	131	(3)
Exercises	134	(3)
5 Software for Numerical Linear Algebra	137	(24)
5.1 Fortran and C	138	(10)
5.1.1 BLAS	140	(2)
5.1.2 Fortran and C Libraries	142	(4)
5.1.3 Fortran 90 and 95	146	(2)
5.2 Interactive Systems for Array Manipulation	148	(5)
5.2.1 Matlab	148	(3)
5.2.2 S, S-Plus	151	(2)
5.3 High-Performance Software	153	(2)
5.4 Test Data	155	(2)
Exercises	157	(4)
6 Applications in Statistics	161	(22)
6.1 Fitting Linear Models with Data	162	(1)
6.2 Linear Models and Least Squares	163	(9)
6.2.1 The Normal Equations and the Sweep Operator	165	(1)
6.2.2 Linear Least Squares Subject to Linear Equality Constraints	166	(1)
6.2.3 Weighted Least Squares	166	(1)
6.2.4 Updating Linear Regression Statistics	167	(2)
6.2.5 Tests of Hypotheses	169	(1)
6.2.6 D-Optimal Designs	170	(2)
6.3 Ill-Conditioning in Statistical Applications	172	(1)
6.4 Testing the Rank of a Matrix	173	(2)
6.5 Stochastic Processes	175	(1)
Exercises	176	(7)
Appendices	183	(14)
A Notation and Definitions	183	(8)
B Solutions and Hints for Selected Exercises	191	(6)
Bibliography	197	(16)
Literature in Computational Statistics	198	(1)
World Wide Web, News Groups, List Servers, and Bulletin Boards	199	(3)
References	202	(11)
Author Index	213	(4)
Subject Index	217