Matrix Algebra Useful for Statistics

by Searle, Shayle R.

• ISBN13: 9780470009611
• ISBN10: 0470009616
• Edition: 1st
• Format: Paperback
• Copyright: 2006-03-20
• Publisher: Wiley-Interscience

Summary

WILEY-INTERSCIENCE PAPERBACK SERIES The Wiley-Interscience Paperback Series consists of selected books that have been made more accessible to consumers in an effort to increase global appeal and general circulation. With these new unabridged softcover volumes, Wiley hopes to extend the lives of these works by making them available to future generations of statisticians, mathematicians, and scientists. "This book is intended to teach useful matrix algebra to 'students, teachers, consultants, researchers, and practitioners' in 'statistics and other quantitative methods'.The author concentrates on practical matters, and writes in a friendly and informal style . . . this is a useful and enjoyable book to have at hand." -Biometrics This book is an easy-to-understand guide to matrix algebra and its uses in statistical analysis. The material is presented in an explanatory style rather than the formal theorem-proof format. This self-contained text includes numerous applied illustrations, numerical examples, and exercises.

Author Biography

SHAYLE R. SEARLE, PhD, is Professor Emeritus of Biometry at Cornell University. He is the author of Linear Models, Linear Models for Unbalanced Data, and Generalized, Linear, and Mixed Models (with Charles E. McCulloch), all from Wiley.

Table of Contents

1. Introduction	1	(21)
1. The scope of matrix algebra	1	(2)
2. General description of a matrix	3	(1)
3. Subscript notation	4	(2)
4. Summation notation	6	(5)
5. Dot notation	11	(1)
6. Definition of a matrix	12	(3)
7. Vectors and scalars	15	(1)
8. General notation	16	(1)
9. Illustrative examples	16	(1)
Exercises	17	(5)
2. Basic Operations	22	(38)
1. The transpose of a matrix	22	(2)
a. A reflexive operation,	23	(1)
b. Vectors,	24	(1)
2. Partitioned matrices	24	(3)
a. Example,	24	(2)
b. General specification,	26	(1)
c. Transposing a partitioned matrix,	26	(1)
d. Partitioning into vectors,	26	(1)
3. The trace of a matrix	27	(1)
4. Addition	28	(1)
5. Scalar multiplication	29	(1)
6. Subtraction	30	(1)
7. Equality and the null matrix	31	(1)
8. Multiplication	32	(18)
a. The inner product of two vectors,	32	(1)
b. A matrix–vector product,	33	(3)
c. A product of two matrices,	36	(2)
d. Existence of matrix products,	38	(1)
e. Products with vectors,	39	(3)
f. Products with scalars,	42	(1)
g. Products with null matrices,	43	(1)
h. Products with diagonal matrices,	43	(1)
i. Identity matrices,	44	(1)
j. The transpose of a product,	44	(1)
k. The trace of a product,	45	(1)
l. Powers of a matrix,	46	(2)
m. Partitioned matrices,	48	(1)
n. Hadamard products,	49	(1)
9. The Laws of algebra	50	(62)
a. Associative laws,	50	(1)
b. The distributive law,	51	(1)
c. Commutative laws,	51	(1)
10. Contrasts with scalar algebra	52	(1)
Exercises	53	(7)
3. Special Matrices	60	(24)
1. Symmetric matrices	60	(5)
a. Products of symmetric matrices,	61	(1)
b. Properties of AA' and A'A,	61	(2)
c. Products of vectors,	63	(1)
d. Sums of outer products,	64	(1)
e. Elementary vectors,	65	(1)
f. Skew-symmetric matrices,	65	(1)
2. Matrices having all elements equal	65	(3)
3. Idempotent matrices	68	(1)
4. Orthogonal matrices	69	(4)
a. Definitions,	69	(2)
b. Special cases,	71	(14)
(i) Helmert matrices,	71	(1)
(ii) Givens matrices,	72	(1)
(iii) Householder matrices,	72	(1)
5. Quadratic forms	73	(3)
6. Positive definite matrices	76	(2)
Exercises	78	(6)
4. Determinants	84	(35)
1. Expansion by minors	84	(6)
a. First- and second-order determinants,	85	(1)
b. Third-order determinants,	86	(3)
c. n-order determinants,	89	(1)
2. Formal definition	90	(2)
3. Basic properties	92	(7)
a. Determinant of a transpose,	92	(1)
b. Two rows the same,	93	(1)
c. Cofactors,	93	(2)
d. Adding multiples of a row (column) to a row (column),	95	(1)
e. Products,	96	(5)
(i) Reduction to triangular form,	96	(1)
(ii) Two useful lemmas,	97	(1)
(iii) Determinant of a product,	98	(1)
4. Elementary row operations	99	(4)
a. Factorization,	101	(1)
b. A row (column) of zeros,	102	(1)
c. Interchanging rows (columns),	102	(1)
d. Adding a row to a multiple of a row,	102	(1)
5. Examples	103	(3)
6. Diagonal expansion	106	(3)
7. The Laplace expansion	109	(2)
8. Sums and differences of determinants	111	(1)
Exercises	112	(7)
5. Inverse Matrices	119	(36)
1. Introduction: solving equations	119	(4)
2. Products equal to I	123	(2)
3. Cofactors of a determinant	125	(1)
4. Derivation of the inverse	125	(4)
5. Conditions for existence of the inverse	129	(1)
6. Properties of the inverse	130	(1)
7. Some simple special cases	131	(2)
a. Inverses of order 2,	131	(1)
b. Diagonal matrices,	132	(1)
c. I and J matrices,	132	(1)
d Orthogonal matrices,	132	(1)
e. Idempotent matrices,	133	(1)
8. Equations and algebra	133	(6)
a. Solving linear equations,	133	(3)
(i) Age distributions in wild populations,	133	(1)
(ii) Input–output analysis in economics,	134	(2)
(iii) Least squares equations,	136	(1)
b. Algebraic simplifications,	136	(3)
9. Computers and inverses	139	(9)
a. The arithmetic of linear equations,	140	(2)
b. Rounding error,	142	(17)
(i) Addition,	143	(1)
(ii) Inverting a matrix,	144	(1)
(iii) Solving linear equations,	145	(2)
10. Left and right inverses	147	(1)
Exercises	148	(7)
6. Rank	155	(29)
1. Linear combinations of vectors	155	(2)
2. Linear transformations	157	(2)
3. Linear dependence and independence	159	(3)
a. Definitions,	159	(2)
b. General characteristics,	161	(1)
(i) Sets of vectors,	161	(1)
(ii) Some a's zero,	161	(1)
(iii) Existence and non-uniqueness of nonzero a's,	161	(1)
(iv) Null vectors,	161	(1)
4. Linearly dependent vectors	162	(4)
a. At least two a's are nonzero,	162	(1)
b. Vectors are linear combinations of others,	162	(1)
c. Partitioning matrices,	163	(1)
d. Zero determinants,	164	(1)
e. Inverse matrices,	164	(1)
f Testing for dependence (simple cases),	164	(2)
5. Linearly independent (LIN) vectors	166	(3)
a. Nonzero determinants and inverse matrices,	166	(1)
b. Linear combinations of LIN vectors,	167	(1)
c. A maximum number of LIN vectors,	167	(2)
6. The number of LIN rows and columns in a matrix	169	(2)
7. The rank of a matrix	171	(1)
8. Rank and inverse matrices	172	(1)
9. Permutation matrices	173	(2)
10. Full-rank factorization	175	(2)
a. Basic development,	175	(2)
b. The general case,	177	(1)
c. Matrices of full row (column) rank,	177	(1)
11. Vector spaces	177	(4)
a. Euclidean space,	178	(1)
b. Vector spaces,	178	(1)
c. Spanning sets and bases,	179	(1)
d. Many spaces of order n,	179	(1)
e. Subspaces,	180	(1)
f. The range and null space of a matrix,	180	(1)
Exercises	181	(3)
7. Canonical Forms	184	(28)
1. Elementary operators	184	(2)
a. Row operations,	184	(1)
b. Transposes,	185	(1)
c. Column operations,	185	(1)
d Inverses,	185	(1)
2. Rank and the elementary operators	186	(1)
a. Rank,	186	(1)
b. Products of elementary operators,	186	(1)
c. Equivalence,	187	(1)
3. Finding the rank of a matrix	187	(3)
a. Some special LIN vectors,	187	(1)
b. Calculating rank,	188	(1)
c. A general procedure,	189	(1)
4. Reduction to equivalent canonical form	190	(6)
a. Row operations,	190	(1)
b. Column operations,	191	(1)
c. The equivalent canonical form,	192	(2)
d. Non-uniqueness of P and Q,	194	(1)
e. Existence is assured	194	(1)
f Full-rank factorization,	194	(2)
5. Rank of a product matrix	196	(3)
6. Symmetric matrices	199	(6)
a. Row and column operations,	199	(1)
b. The diagonal form,	200	(1)
c. The canonical form under congruence,	201	(1)
d Two special provisions,	202	(3)
(i) Null rows,	202	(1)
(ii) Negative elements in the diagonal form,	203	(2)
e. Full-rank factorization,	205	(1)
7. Non-negative definite matrices	205	(4)
a. Diagonal elements and principal minors,	205	(1)
b. Congruent canonical form,	206	(1)
c. Full-rank factorization,	206	(1)
d. Quadratic forms as sums of squares,	206	(2)
e. Full row (column) rank matrices,	208	(1)
Exercises	209	(3)
8. Generalized Inverses	212	(15)
1. The Moore—Penrose inverse	212	(1)
2. Generalized inverses	212	(4)
a. Derivation from row operations,	214	(1)
b. Derivation from the diagonal form,	215	(1)
3. Other names and symbols	216	(1)
4. An algorithm	217	(2)
a. An easy form,	217	(1)
b. A general form,	217	(2)
5. Arbitrariness in a generalized inverse	219	(1)
6. Symmetric matrices	220	(2)
a. Non-negative definite matrices,	220	(1)
b. A general algorithm,	221	(1)
c. The matrix X'X,	221	(1)
Exercises	222	(5)
9. Solving Linear Equations	227	(30)
1. Equations having many solutions	227	(1)
2. Consistent equations	228	(5)
a. Definition,	228	(1)
b. Existence of solutions,	229	(3)
c. Tests for consistency,	232	(1)
3. Equations having one solution	233	(2)
4. Deriving solutions using generalized inverses	235	(4)
a. Obtaining a solution,	235	(1)
b. Obtaining many solutions,	236	(1)
c. All possible solutions,	237	(2)
d. Combinations of solutions,	239	(1)
5. Linearly independent solutions	239	(3)
6. An invariance property	242	(3)
7. Equations Ax = 0	245	(4)
a. General properties,	245	(1)
b. Orthogonal solutions,	246	(2)
c. Orthogonal vector spaces,	248	(1)
8. A complete example	249	(2)
9. Least squares equations	251	(1)
Exercises	252	(5)
10. Partitioned Matrices	257	(15)
1. Orthogonal matrices	257	(1)
2. Determinants	258	(2)
3. Inverses	260	(1)
4. Schur complements	261	(1)
5. Generalized inverses	261	(2)
6. Direct sums	263	(2)
7. Direct products	265	(2)
Exercises	267	(5)
11. Eigenvalues and Eigenvectors	272	(48)
1. Introduction: age distribution vectors	272	(2)
2. Derivation of eigenvalues	274	(2)
3. Elementary properties of eigenvalues	276	(3)
a. Eigenvalues of powers of a matrix,	276	(1)
b. Eigenvalues of a scalar-by-matrix product,	277	(1)
c. Eigenvalues of polynomials,	277	(1)
d. The sum and product of eigenvalues,	278	(1)
4. Calculating eigenvectors	279	(3)
a. A general method,	279	(1)
b. Simple roots,	280	(1)
c. Multiple roots,	281	(1)
5. The similar canonical form	282	(8)
a. Derivation,	282	(3)
b. Uses,	285	(5)
6. Symmetric matrices	290	(3)
a. Eigenvalues all real,	290	(1)
b. Symmetric matrices are diagonable,	290	(1)
c. Eigenvectors are orthogonal,	290	(2)
(i) Different eigenvalues,	291	(1)
(ii) Multiple eigenvalues,	291	(1)
(iii) The canonical form under orthogonal similarity,	291	(1)
d. Rank equals number of nonzero eigenvalues,	292	(1)
7. Dominant eigenvalues	293	(5)
8. Factoring the characteristic equation	298	(1)
Exercises	299	(6)
11A. Appendix to Chapter 11	305	(13)
1. Proving the diagonability theorem	305	(3)
a. The number of nonzero eigenvalues never exceeds rank,	305	(1)
b. A lower bound on r(A — λkI),	306	(1)
c. Proof of the diagonability theorem,	307	(1)
d All symmetric matrices are diagonable,	307	(1)
2. Other results for symmetric matrices	308	(6)
a. Spectral decomposition,	308	(1)
b. Non-negative definite (n.n.d.) matrices,	309	(3)
c. Simultaneous diagonalization of two symmetric matrices,	312	(2)
3. The Cayley—Hamilton theorem	314	(2)
4. The singular-value decomposition	316	(2)
Exercises	318	(2)
12. Miscellanea	320	(26)
1. Orthogonal matrices—a summary	320	(1)
2. Idempotent matrices—a summary	320	(2)
3. The matrix a I + bJ—a summary	322	(1)
4. Non-negative definite matrices—a summary	322	(1)
5. Canonical forms and other decompositions—a summary	323	(2)
6. Matrix Functions	325	(1)
a. Functions of matrices,	325	(1)
b. Matrices of functions,	325	(1)
7. Iterative solution of nonlinear equations	326	(1)
8. Vectors of differential operators	327	(5)
a. Scalars,	327	(1)
b. Vectors,	328	(1)
c. Quadratic forms,	329	(3)
9. Vec and vech operators	332	(28)
a. Definitions,	332	(1)
b. Properties of vec,	333	(1)
c. Vec-permutation matrices,	334	(1)
d Relationships between vec and vech,	334	(1)
10. Other calculus results	334	(7)
a. Differentiating inverses,	335	(1)
b. Differentiating traces,	335	(1)
c. Differentiating determinants,	336	(2)
d. Jacobians,	338	(2)
e. Aitken's integral,	340	(1)
f. Hessians,	341	(1)
11. Matrices with elements that are complex numbers	341	(1)
Exercises	342	(4)
13. Applications in Statistics	346	(17)
1. Variance—covariance matrices	347	(1)
2. Correlation matrices	348	(1)
3. Matrices of sums of squares and cross-products	349	(3)
a. Data matrices,	349	(1)
b. Uncorrected sums of squares and products,	349	(1)
c. Means, and the centering matrix,	350	(1)
d. Corrected sums of squares and products,	351	(1)
4. The multivariate normal distribution	352	(3)
5. Quadratic forms and χ²-distributions	355	(2)
6. Least squares equations	357	(1)
7. Contrasts among means	358	(2)
Exercises	360	(3)
14. The Matrix Algebra of Regression Analysis	363	(29)
1. General description	363	(3)
a. Linear models,	364	(1)
b. Observations,	364	(1)
c. Nonlinear models,	365	(1)
2. Estimation	366	(2)
3. Several regressor variables	368	(1)
4. Deviations from means	369	(2)
5. The statistical model	371	(1)
6. Unbiasedness and variances	372	(1)
7. Predicted y-values	373	(1)
8. Estimating the error variance	374	(2)
9. Partitioning the total sum of squares	376	(2)
10. Multiple correlation	378	(1)
11. The no-intercept model	379	(1)
12. Analysis of variance	380	(2)
13. Testing linear hypotheses	382	(4)
a. Stating a hypothesis,	382	(1)
b. The F-statistic,	383	(2)
c. Equivalent statements of a hypothesis,	385	(1)
d. Special cases,	386	(9)
(i) H:b= 0,	386	(1)
(ii) H:b = b0,	386	(1)
(iii) H:λ'b = m,	386	(1)
14. Confidence intervals	386	(1)
15. Fitting subsets of the x-variables	387	(2)
16. Reductions in sums of squares: the R(·|·) notation	389	(3)
15. An Introduction to Linear Statistical Models	392	(37)
1. General description	392	(3)
2. The normal equations	395	(2)
a. A general form,	395	(1)
b. Many solutions,	396	(1)
3. Solving the normal equations	397	(2)
a. Generalized inverses of X'X,	397	(1)
b. Solutions,	397	(2)
4. Expected values and variances	399	(1)
5. Predicted y-values	400	(1)
6. Estimating the error variance	401	(2)
a. Error sum of squares,	401	(1)
b. Expected value,	402	(1)
c. Estimation,	402	(1)
7. Partitioning the total sum of squares	403	(1)
8. Coefficient of determination	404	(1)
9. Analysis of variance	405	(2)
10. The R(·|·) notation	407	(1)
11. Estimable functions	408	(3)
12. Testing linear hypotheses	411	(4)
13. Confidence intervals	415	(1)
14. Some particular models	416	(8)
a. The one-way classification,	416	(2)
b. Two-way classification, no interactions, balanced data,	418	(4)
c. Two-way classification, no interactions, unbalanced data,	422	(2)
15. The R(·|·) notation (Continued)	424	(5)
References	429	(4)
Index	433