Multivariate Statistical Inference and Applications

by Rencher, Alvin C.

• ISBN13: 9780471571513
• ISBN10: 0471571512
• Edition: 1st
• Format: Hardcover
• Copyright: 1997-12-29
• Publisher: Wiley-Interscience

Summary

The most accessible introduction to the theory and practice of multivariate analysis Multivariate Statistical Inference and Applications is a user-friendly introduction to basic multivariate analysis theory and practice for statistics majors as well as nonmajors with little or no background in theoretical statistics. Among the many special features of this extremely accessible first text on multivariate analysis are: * Clear, step-by-step explanations of all key concepts and procedures along with original, easy-to-follow proofs * Numerous problems, examples, and tables of distributions * Many real-world data sets drawn from a wide range of disciplines * Reviews of univariate procedures that give rise to multivariate techniques * An extensive survey of the world literature on multivariate analysis * An in-depth review of matrix theory * A disk including all the data sets and SAS command files for all examples and numerical problems found in the book These same features also make Multivariate Statistical Inference and Applications an excellent professional resource for scientists and clinicians who need to acquaint themselves with multivariate techniques. It can be used as a stand-alone introduction or in concert with its more methods-oriented sibling volume, the critically acclaimed Methods of Multivariate Analysis.

Author Biography

ALVIN C. RENCHER, PhD, is Professor of Statistics at Brigham Young University and a Fellow of the American Statistical Association. He is the author of Methods of Multivariate Analysis and has written articles for Biometrics, Technometrics, Biometrika, Communications in Statistics, and American Statistician.

Table of Contents

1. Some Properties of Random Vectors and Matrices	1	(36)
1.1 Introduction	1	(1)
1.2 Univariate and Bivariate Random Variables	2	(4)
1.2.1 Univariate Random Variables	2	(2)
1.2.2 Bivariate Random Variables	4	(2)
1.3 Mean Vectors and Covariance Matrices for Random Vectors	6	(5)
1.4 Correlation Matrices	11	(1)
1.5 Partitioned Mean Vectors and Covariance Matrices	12	(2)
1.6 Linear Functions of Random Variables	14	(6)
1.6.1 Sample Means, Variances, and Covariances	14	(5)
1.6.2 Population Means, Variances, and Covariances	19	(1)
1.7 Measuring Intercorrelation	20	(2)
1.8 Mahalanobis Distance	22	(1)
1.9 Missing Data	23	(4)
1.10 Robust Estimators of XXX and XXX	27	(10)
2. The Multivariate Normal Distribution	37	(23)
2.1 Univariate and Multivariate Normal Density Functions	37	(6)
2.1.1 Univariate Normal	37	(1)
2.1.2 Multivariate Normal	38	(2)
2.1.3 Constant Density Ellipsoids	40	(2)
2.1.4 Generating Multivariate Normal Data	42	(1)
2.1.5 Moments	42	(1)
2.2 Properties of Multivariate Normal Random Vectors	43	(6)
2.3 Estimation of Parameters in the Multivariate Normal Distribution	49	(7)
2.3.1 Maximum Likelihood Method	49	(3)
2.3.2 Properties of y and S	52	(1)
2.3.3 Wishart Distribution	53	(3)
2.4 Additional Topics	56	(4)
3. Hotelling's T(2)-Tests	60	(61)
3.1 Introduction	60	(1)
3.2 Test for H(0): XXX = XXX(0) with XXX Known	60	(1)
3.3 Hotelling's T(2)-test for H(0): XXX = XXX(0) with XXX Unknown	61	(13)
3.3.1 Univariate t-Test for H(0): XXX = XXX(0) with XXX(2) Unknown	61	(1)
3.3.2 Likelihood Ratio Method of Test Construction	62	(3)
3.3.3 One-Sample T(2)-Test	65	(1)
3.3.4 Formal Definition of T(2) and Relationship to F	66	(1)
3.3.5 Effect on T(2) of Adding a Variable	67	(3)
3.3.6 Properties of the T(2)-Test	70	(1)
3.3.7 Likelihood Ratio Test	71	(1)
3.3.8 Union-Intersection Test	72	(2)
3.4 Confidence Intervals and Tests for Linear Functions of XXX	74	(11)
3.4.1 Confidence Region for XXX	74	(1)
3.4.2 Confidence Interval for a Single Linear Combination a'XXX	74	(1)
3.4.3 Simultaneous Confidence Intervals for XXX(j) and a'XXX	74	(3)
3.4.4 Bonferroni Confidence Intervals for XXX(j) and a'XXX	77	(2)
3.4.5 Tests for H(0): a'XXX = a'XXX(0) and H(0): XXX(j) = XXX(0j)	79	(4)
3.4.6 Tests for H(0): CXXX = 0	83	(2)
3.5 Tests of H(0): XXX(1) = XXX(2) Assuming XXX(1) = XXX(2)	85	(7)
3.5.1. Review of Univariate Likelihood Ratio Test for H(0): XXX(1) = XXX(2) When XXX(2)(1) = XXX(2)(2)	85	(2)
3.5.2. Test for H(0): XXX(1) = XXX(2) When XXX(1) = XXX(2)	87	(1)
3.5.3. Effect on T(2) of Adding a Variable	87	(4)
3.5.4. Properties of the Two-Sample T(2)-Statistic	91	(1)
3.5.5. Likelihood Ratio and Union-Intersection Tests	91	(1)
3.6. Confidence Intervals and Tests for Linear Functions of Two Mean Vectors	92	(4)
3.6.1. Confidence Region for XXX(1) - XXX(2)	93	(1)
3.6.2. Simultaneous Confidence Intervals for a'(XXX(1) - XXX(2)) and XXX(1j) - XXX(2j)	93	(1)
3.6.3. Bonferroni Confidence Intervals for a'(XXX(1) - XXX(2)) and XXX(1j) - XXX(2j)	94	(1)
3.6.4. Tests for H(0): a'(XXX(1) - XXX(2)) = a'XXX(0) and H(0j): XXX(1j) - XXX(2j) = 0	94	(1)
3.6.5. Test for H(0): C(XXX(1) - XXX(2)) = 0	95	(1)
3.7. Robustness of the T(2) -test	96	(1)
3.7.1. Robustness to XXX(1) is not equal to XXX(2)	96	(1)
3.7.2. Robustness to Nonnormality	96	(1)
3.8. Paired Observation Test	97	(2)
3.9. Testing H(0): XXX(1) = XXX(2) When XXX(1) is not equal to XXX(2)	99	(5)
3.9.1. Univariate Case	99	(1)
3.9.2. Multivariate Case	100	(4)
3.10. Power and Sample Size	104	(4)
3.11. Tests on a Subvector	108	(4)
3.11.1. Two-Sample Case	108	(2)
3.11.2. Step-Down Test	110	(1)
3.11.3. Selection of Variables	111	(1)
3.11.4. One-Sample Case	112	(1)
3.12. Nonnormal Approaches to Hypothesis Testing	112	(2)
3.12.1. Elliptically Contoured Distributions	112	(1)
3.12.2. Nonparametric Tests	113	(1)
3.12.3. Robust Versions of T(2)	114	(1)
3.13. Application of T(2) In Multivariate Quality Control	114	(7)
4. Multivariate Analysis of Variance	121	(80)
4.1. One-Way Classification	121	(14)
4.1.1. Model for One-Way Multivariate Analysis of Variance	121	(1)
4.1.2. Wilks' Likelihood Ratio Test	122	(5)
4.1.3. Roy's Union-Intersection Test	127	(3)
4.1.4. The Pillai and Lawley-Hotelling Test Statistics	130	(1)
4.1.5. Summary of the Four Test Statistics	131	(1)
4.1.6. Effect of an Additional Variable on Wilks' XXX	132	(2)
4.1.7. Tests on Individual Variables	134	(1)
4.2. Power and Robustness Comparisons for the Four MANOVA Test Statistics	135	(3)
4.3. Tests for Equality of Covariance Matrices	138	(2)
4.4. Power and Sample Size for the Four MANOVA Tests	140	(2)
4.5. Contrasts Among Mean Vectors	142	(6)
4.5.1. Univariate Contrasts	142	(3)
4.5.2. Multivariate Contrasts	145	(3)
4.6. Two-Way Multivariate Analysis of Variance	148	(3)
4.7. Higher Order Models	151	(1)
4.8. Unbalanced Data	152	(22)
4.8.1. Introduction	152	(1)
4.8.2. Univariate One-Way Model	153	(2)
4.8.3. Multivariate One-Way Model	155	(5)
4.8.4. Univariate Two-Way Model	160	(8)
4.8.5. Multivariate Two-Way Model	168	(6)
4.9. Tests on a Subvector	174	(4)
4.9.1. Testing a Single Subvector	174	(3)
4.9.2. Step-Down Test	177	(1)
4.9.3. Stepwise Selection of Variables	177	(1)
4.10. Multivariate Analysis of Covariance	178	(16)
4.10.1. Introduction	178	(1)
4.10.2. Univariate Analysis of Covariance: One-Way Model with One Covariate	179	(4)
4.10.3. Univariate Anlaysis of Covariance: Two-Way Model with One Covariate	183	(3)
4.10.4. Additional Topics in Univariate Analysis of Covariance	186	(1)
4.10.5. Multivariate Analysis of Covariance	187	(7)
4.11. Alternative Approaches to Testing H(0): XXX(1) = XXX(2) = ... = XXX(k)	194	(7)
5. Discriminant Functions for Descriptive Group Separation	201	(29)
5.1. Introduction	201	(1)
5.2. Two Groups	201	(1)
5.3. Several Groups	202	(4)
5.3.1. Discriminant Functions	202	(3)
5.3.2. Assumptions	205	(1)
5.4. Standardized Coefficients	206	(1)
5.5. Tests of Hypotheses	207	(3)
5.5.1. Two Groups	207	(2)
5.5.2. Several Groups	209	(1)
5.6. Discriminant Analysis for Higher Order Designs	210	(1)
5.7. Interpretation of Discriminant Functions	210	(6)
5.7.1. Standardized Coefficients and Partial F-Values	211	(1)
5.7.2. Correlations between Variables and Discriminant Functions	211	(4)
5.7.3. Other Approaches	215	(1)
5.8. Confidence Intervals	216	(1)
5.9. Subset Selection	216	(3)
5.9.1. Discriminant Function Approach to Selection	217	(1)
5.9.2. Stepwise Selection	217	(1)
5.9.3. All Possible Subsets	218	(1)
5.9.4. Selection in Higher Order Designs	218	(1)
5.10. Bias in Subset Selection	219	(2)
5.11. Other Estimators of Discriminant Functions	221	(9)
5.11.1. Ridge Discriminant Analysis and Related Techniques	222	(1)
5.11.2. Robust Discriminant Analysis	223	(7)
6. Classification of Observations into Groups	230	(36)
6.1. Introduction	230	(1)
6.2. Two Groups	230	(6)
6.2.1. Equal Population Covariance Matrices	230	(2)
6.2.2. Unequal Population Covariance Matrices	232	(1)
6.2.3. Unequal Costs of Misclassification	233	(1)
6.2.4. Posterior Probability Approach	234	(1)
6.2.5. Robustness to Departures from the Assumptions	234	(1)
6.2.6. Robust Procedures	235	(1)
6.3. Several Groups	236	(4)
6.3.1. Equal Population Covariance Matrices	236	(1)
6.3.2. Unequal Population Covariance Matrices	237	(2)
6.3.3. Use of Linear Discriminant Functions for Classification	239	(1)
6.4. Estimation of Error Rates	240	(4)
6.5. Correcting for Bias in the Apparent Error Rate	244	(3)
6.5.1. Partitioning the Sample	244	(1)
6.5.2. Holdout Method	244	(1)
6.5.3. Bootstrap Estimator	245	(1)
6.5.4. Comparison of Error Estimators	245	(2)
6.6. Subset Selection	247	(4)
6.6.1. Selection Based on Separation of Groups	247	(2)
6.6.2. Selection Based on Allocation	249	(1)
6.6.3. Selection in the Heteroscedastic Case	250	(1)
6.7. Bias in Stepwise Classification Analysis	251	(3)
6.8. Logistic and Probit Classification	254	(7)
6.8.1. The Logistic Model for Two Groups with XXX(1) = XXX(2)	254	(2)
6.8.2. Comparison of Logistic Classification with Linear Classification Functions	256	(2)
6.8.3. Quadratic Logistic Functions When XXX(1) is not equal to XXX(2)	258	(1)
6.8.4. Logistic Classification for Several Groups	258	(1)
6.8.5. Additional Topics in Logistic Classification	259	(1)
6.8.6. Probit Classification	259	(2)
6.9. Additional Topics in Classification	261	(5)
7. Multivariate Regression	266	(46)
7.1. Introduction	266	(1)
7.2. Multiple Regression: Fixed x's	267	(9)
7.2.1. Least Squares Estimators and Properties	267	(1)
7.2.2. An Estimator for XXX(2)	268	(1)
7.2.3. The Model in Centered Form	269	(2)
7.2.4. Hypothesis Tests and Confidence Intervals	271	(4)
7.2.5. R(2) in Fixed-x Regression	275	(1)
7.2.6. Model Validation	275	(1)
7.3. Multiple Regression: Random x's	276	(3)
7.3.1. Model for Random x's	276	(1)
7.3.2. Estimation of XXX(0), XXX(1) and XXX(2)	277	(1)
7.3.3. R(2) in Random-x Regression	278	(1)
7.3.4. Tests and Confidence Intervals	279	(1)
7.4. Estimation in the Multivariate Multiple Regression Model: Fixed x's	279	(10)
7.4.1. The Multivariate Model	279	(2)
7.4.2. Least Squares Estimator for B	281	(2)
7.4.3. Properties of B	283	(2)
7.4.4. An Estimator for XXX	285	(1)
7.4.5. Normal Model for the Y(i)'s	286	(2)
7.4.6. The Multivariate Model in Centered Form	288	(1)
7.4.7. Measures of Multivariate Association	289	(1)
7.5. Hypothesis Tests in the Multivariate Multiple Regression Model: Fixed x's	289	(11)
7.5.1. Test for Significance of Regression	289	(4)
7.5.2. Test on a Subset of the Rows of B	293	(2)
7.5.3. General Linear Hypotheses CB = O and CBM = O	295	(2)
7.5.4. Tests and Confidence Intervals for a Single XXX(jk) and a Bilinear Function a'Bb	297	(1)
7.5.5. Simultaneous Tests and Confidence Intervals for the XXX(jk)'s and Bilinear Functions a'Bb	298	(1)
7.5.6. Tests in the Presence of Missing Data	299	(1)
7.6. Multivariate Model Validation: Fixed x's	300	(3)
7.6.1. Lack-of-Fit Tests	300	(1)
7.6.2. Residuals	300	(1)
7.6.3. Influence and Outliers	301	(2)
7.6.4. Measurement Errors	303	(1)
7.7. Multivariate Regression: Random x's	303	(2)
7.7.1. Multivariate Normal Model for Random x's	303	(1)
7.7.2. Estimation of XXX(0), B(1), and XXX	304	(1)
7.7.3. Tests and Confidence Intervals in the Multivariate Random-x Case	305	(1)
7.8. Additional Topics	305	(7)
7.8.1. Correlated Response Methods	305	(1)
7.8.2. Categorical Data	306	(1)
7.8.3. Subset Selection	307	(1)
7.8.4. Other Topics	307	(5)
8. Canonical Correlation	312	(25)
8.1. Introduction	312	(1)
8.2. Canonical Correlations and Canonical Variates	312	(6)
8.3. Properties of Canonical Correlations and Variates	318	(2)
8.3.1. Properties of Canonical Correlations	318	(2)
8.3.2. Properties of Canonical Variates	320	(1)
8.4. Tests of Significance for Canonical Correlations	320	(6)
8.4.1. Tests of Independence of y and x	320	(4)
8.4.2. Test of Dimension of Relationship between the y's and the x's	324	(2)
8.5. Validation	326	(2)
8.6. Interpretation of Canonical Variates	328	(3)
8.6.1. Standardized Coefficients	328	(1)
8.6.2. Rotation of Canonical Variate Coefficients	328	(1)
8.6.3. Correlations between Variables and Canonical Variates	329	(2)
8.7. Redundancy Analysis	331	(2)
8.8. Additional Topics	333	(4)
9. Principal Component Analysis	337	(40)
9.1. Introduction	337	(1)
9.2. Definition and Properties of Principal Components	338	(5)
9.2.1. Maximum Variance Property	338	(2)
9.2.2. Principal Components as Projections	340	(1)
9.2.3. Properties of Principal Components	341	(2)
9.3. Principal Components as a Rotation of Axes	343	(1)
9.4. Principal Components from the Correlation Matrix	344	(3)
9.5. Methods for Discarding Components	347	(5)
9.5.1. Percent of Variance	347	(1)
9.5.2. Average Eigenvalue	348	(1)
9.5.3. Scree Graph	348	(1)
9.5.4. Significance Tests	349	(3)
9.5.5. Other Methods	352	(1)
9.6. Information in the Last Few Principal Components	352	(1)
9.7. Interpretation of Principal Components	353	(10)
9.7.1. Special Patterns in S or R	353	(4)
9.7.2. Testing H(0): XXX = XXX(2) [(1 - p)I + pJ] and P(p) = (1 - p)I + pJ	357	(2)
9.7.3. Additional Rotation	359	(2)
9.7.4. Correlations between Variables and Principal Components	361	(2)
9.8. Relationship Between Principal Components and Regression	363	(8)
9.8.1. Principal Component Regression	364	(6)
9.8.2. Latent Root Regression	370	(1)
9.9. Principal Component Analysis with Grouped Data	371	(1)
9.10. Additional Topics	372	(5)
10. Factor Analysis	377	(22)
10.1. Introduction	377	(1)
10.2. Basic Factor Model	377	(4)
10.2.1. Model and Assumptions	377	(2)
10.2.2. Scale Invariance of the Model	379	(1)
10.2.3. Rotation of Factor Loadings in the Model	380	(1)
10.3. Estimation of Loadings and Communalities	381	(5)
10.3.1. Principal Component Method	381	(2)
10.3.2. Principal Factor Method	383	(1)
10.3.3. Iterated Principal Factor Method	384	(1)
10.3.4. Maximum Likelihood Method	384	(1)
10.3.5. Other Methods	385	(1)
10.3.6. Comparison of Methods	385	(1)
10.4. Determining the Number of Factors, m	386	(1)
10.5. Rotation of Factor Loadings	387	(4)
10.5.1. Introduction	387	(1)
10.5.2. Orthogonal Rotation	388	(1)
10.5.3. Oblique Rotations	389	(1)
10.5.4. Interpretation of the Factors	390	(1)
10.6. Factor Scores	391	(1)
10.7. Applicability of the Factor Analysis Model	392	(1)
10.8. Factor Analysis and Grouped Data	393	(1)
10.9. Additional Topics	394	(5)
Appendix A. Review of Matrix Algebra	399	(18)
A.1. Introduction	399	(2)
A.1.1. Basic Definitions	399	(1)
A.1.2. Matrices with Special Patterns	400	(1)
A.2. Properties of Matrix Addition and Multiplication	401	(3)
A.3. Partitioned Matrices	404	(2)
A.4. Rank of Matrices	406	(1)
A.5. Inverse Matrices	407	(1)
A.6. Positive Definite and Positive Semidefinite Matrices	408	(1)
A.7. Determinants	409	(1)
A.8. Trace of a Matrix	410	(1)
A.9. Orthogonal Vectors and Matrices	410	(1)
A.10. Eigenvalues and Eigenvectors	411	(1)
A.11. Eigenstructure of Symmetric and Positive Definite Matrices	412	(2)
A.12. Idempotent Matrices	414	(1)
A.13. Differentiation	414	(3)
Appendix B. Tables	417	(32)
Appendix C. Answers and Hints to Selected Problems	449	(56)
Appendix D. About the Diskette	505	(2)
Bibliography	507	(42)
Index	549