Linear Model Theory Univariate, Multivariate, and Mixed Models

by Muller, Keith E.; Stewart, Paul W.

• ISBN13: 9780471214885
• ISBN10: 0471214884
• Edition: 1st
• Format: Hardcover
• Copyright: 2006-08-04
• Publisher: Wiley-Interscience

Summary

A precise and accessible presentation of linear model theory, illustrated with data examplesStatisticians often use linear models for data analysis and for developing new statistical methods. Most books on the subject have historically discussed univariate, multivariate, and mixed linear models separately, whereas Linear Model Theory: Univariate, Multivariate, and Mixed Models presents a unified treatment in order to make clear the distinctions among the three classes of models.Linear Model Theory: Univariate, Multivariate, and Mixed Models begins with six chapters devoted to providing brief and clear mathematical statements of models, procedures, and notation. Data examples motivate and illustrate the models. Chapters 7-10 address distribution theory of multivariate Gaussian variables and quadratic forms. Chapters 11-19 detail methods for estimation, hypothesis testing, and confidence intervals. The final chapters, 20-23, concentrate on choosing a sample size. Substantial sets of excercises of varying difficulty serve instructors for their classes, as well as help students to test their own knowledge.The reader needs a basic knowledge of statistics, probability, and inference, as well as a solid background in matrix theory and applied univariate linear models from a matrix perspective. Topics covered include: A review of matrix algebra for linear models The general linear univariate model The general linear multivariate model Generalizations of the multivariate linear model The linear mixed model Multivariate distribution theory Estimation in linear models Tests in Gaussian linear models Choosing a sample size in Gaussian linear modelsFilling the need for a text that provides the necessary theoretical foundations for applying a wide range of methods in real situations, Linear Model Theory: Univariate, Multivariate, and Mixed Models centers on linear models of interval scale responses with finite second moments. Models with complex predictors, complex responses, or both, motivate the presentation.

Author Biography

KEITH E. MULLER, PhD, is Professor and Director of the Division of Biostatistics in the Department of Epidemiology and Health Policy Research in the College of Medicine at the University of Florida in Gainesville, as well as Professor Emeritus of Biostatistics at The University of North Carolina at Chapel Hill where the book was written.

PAUL W. STEWART, PhD, is Research Associate Professor of Biostatistics at The University of North Carolina at Chapel Hill.

Table of Contents

Preface	xiii
1 Matrix Algebra for Linear Models	1	(38)
1.1 Notation
1.2 Some Operators and Special Types of Matrices
1.3 Five Kinds of Multiplication
1.4 The Direct Sum
1.5 Rules of Operations
1.6 Other Special Types of Matrices
1.7 Quadratic and Bilinear Forms
1.8 Vector Spaces and Rank
1.9 Finding Rank
1.10 Determinants
1.11 The Inverse and Generalized Inverse
1.12 Eigenanalysis (Spectral Decomposition)
1.13 Some Factors of Symmetric Matrices
1.14 Singular Value Decomposition
1.15 Projections and Other Functions of a Design Matrix
1.16 Special Properties of Patterned Matrices
1.17 Function Optimization and Matrix Derivatives
1.18 Statistical Notation Involving Matrices
1.19 Statistical Formulas
1.20 Principal Components
1.21 Special Covariance Patterns
2 The General Linear Univariate Model	39	(16)
2.1 Motivation
2.2 Model Concepts
2.3 The General Linear Univariate Linear Model
2.4 The Univariate General Linear Hypothesis
2.5 Tests about Variances
2.6 The Role of the Intercept
2.7 Population Correlation and Strength of Relationship
2.8 Statistical Estimates
2.9 Testing the General Linear Hypothesis
2.10 Confidence Regions for theta
2.11 Sufficient Statistics for the Univariate Model Exercises
3 The General Linear Multivariate Model	55	(24)
3.1 Motivation
3.2 Definition of the Multivariate Model
3.3 The Multivariate General Linear Hypothesis
3.4 Tests About Covariance Matrices
3.5 Population Correlation
3.6 Statistical Estimates
3.7 Overview of Testing Multivariate Hypotheses
3.8 Computing MULTIREP Tests
3.9 Computing UNIREP Tests
3.10 Confidence Regions for Θ
3.11 Sufficient Statistics for the Multivariate Model
3.12 Allowing Missing Data in the Multivariate Model
Exercises
4 Generalizations of the Multivariate Linear Model	79	(12)
4.1 Motivation
4.2 The Generalized General Linear Univariate Model: Exact and Approximate Weighted Least Squares
4.3 Doubly Multivariate Models
4.4 Seemingly Unrelated Regressions
4.5 Growth Curve Models (GMANOVA)
4.6 The Relationship of the GCM to the Multivariate Model
4.7 Mixed, Hierarchical, and Related Models
5 The Linear Mixed Model	91	(10)
5.1 Motivation
5.2 Definition of the Mixed Model
5.3 Distribution-Free and Noniterative Estimates
5.4 Gaussian Likelihood and Iterative Estimates
5.5 Tests about β (Means, Fixed Effects)
5.6 Tests of Covariance Parameters, τ (Random Effects) Exercises
6 Choosing the Form of a Linear Model for Analysis	101	(14)
6.1 The Importance of Understanding Dependence
6.2 How Many Variables per Independent Sampling Unit?
6.3 What Types of Variables Play a Role?
6.4 What Repeated Sampling Scheme Was Used?
6.5 Analysis Strategies for Multivariate Data
6.6 Cautions and Recommendations
6.7 Review of Linear Model Notation
7 General Theory of Multivariate Distributions	115	(24)
7.1 Motivation
7.2 Notation and Concepts
7.3 Families of Distributions
7.4 Cumulative Distribution Function
7.5 Probability Density Function
7.6 Formulas for Probabilities and Moments
7.7 Characteristic Function
7.8 Moment Generating Function
7.9 Cumulant Generating Function
7.10 Transforming Random Variables
7.11 Marginal Distributions
7.12 Independence of Random Vectors
7.13 Conditional Distributions
7.14 (Joint) Moments of Multivariate Distributions
7.15 Conditional Moments of Distributions
7.16 Special Considerations for Random Matrices
8 Scalar, Vector, and Matrix Gaussian Distributions	139	(30)
8.1 Motivation
8.2 The Scalar Gaussian Distribution
8.3 The Vector ("Multivariate") Gaussian Distribution
8.4 Marginal Distributions
8.5 Independence
8.6 Conditional Distributions
8.7 Asymptotic Properties
8.8 The Matrix Gaussian Distribution
8.9 Assessing Multivariate Gaussian Distribution
8.10 Tests for Gaussian Distribution
Exercises
9 Univariate Quadratic Forms	169	(24)
9.1 Motivation
9.2 Chi-Square Distributions
9.3 General Properties of Quadratic Forms
9.4 Properties of Quadratic Forms in Gaussian Vectors
9.5 Independence among Linear and Quadratic Forms
9.6 The ANOVA Theorem
9.7 Ratios Involving Quadratic Forms
Exercises
10 Multivariate Quadratic Forms	193	(16)
10.1 The Wishart Distribution
10.2 The Characteristic Function of the Wishart
10.3 Properties of the Wishart
10.4 The Inverse Wishart
10.5 Related Distributions
Exercises
11 Estimation for Univariate and Weighted Linear Models	209	(34)
11.1 Motivation
11.2 Statement of the Problem
11.3 (Unrestricted) Linearly Equivalent Linear Models
11.4 Estimability and Criteria for Checking It
11.5 Coding Schemes and the Essence Matrix
11.6 Unrestricted Maximum Likelihood Estimation of β
11.7 Unrestricted BLUE Estimation of β
11.8 Unrestricted Least Squares Estimation of β
11.9 Unrestricted Maximum Likelihood Estimation of theta
11.10 Unrestricted BLUE Estimation of theta
11.11 Related Distributions
11.12 Formulations of Explicit Restrictions of β and θ
11.13 Restricted Estimation Via Equivalent Models
11.14 Fitting Piecewise Polynomial Models Via Splines
11.15 Estimation for the GGLM: Weighted Least Squares
Exercises
12 Estimation for Multivariate Linear Models	243	(20)
12.1 Alternate Formulations of the Model
12.2 Estimability in the Multivariate GLM
12.3 Unrestricted Likelihood Estimation
12.4 Estimation of Secondary Parameters
12.5 Estimation with Multivariate Restrictions
12.6 Unrestricted Estimation With Compound Symmetry: the "Univariate" Approach to Repeated Measures Exercises
13 Estimation for Generalizations of Multivariate Models	263	(16)
13.1 Motivation
13.2 Criteria and Algorithms
13.3 Weighted Estimation of Β and Σ
13.4 Transformations among Growth Curve Designs
13.5 Within-Individual Design Matrices
13.6 Estimation Methods
13.7 Relationships to the Univariate and Mixed Models Exercises
14 Estimation for Linear Mixed Models	279	(10)
14.1 Motivation
14.2 Statement of the General Linear Mixed Model
14.3 Estimation and Estimability
14.4 Some Special Types of Models
14.5 ML Estimation
14.6 REML Estimation
14.7 Small-Sample Properties of Estimators
14.8 Large-Sample Properties of Variance Estimators
14.9 Conditional Estimation of di and BLUP Prediction Exercises
15 Tests for Univariate Linear Models	289	(22)
15.1 Motivation
15.2 Testability of Univariate Hypotheses
15.3 Tests of a Priori Hypotheses
15.4 Related Distributions
15.5 Transformations and Invariance Properties
15.6 Confidence Regions for theta
Exercises
16 Tests for Multivariate Linear Models	311	(26)
16.1 Motivation
16.2 Testability of Multivariate Hypotheses
16.3 Tests of a Priori Hypotheses
16.4 Linear Invariance
16.5 Four Multivariate Test Statistics
16.6 Which Multivariate Test Is Best?
16.7 Univariate Approach to Repeated Measures: UNIREP
16.8 More on Invariance Properties
16.9 Tests of Hypotheses about Σ
16.10 Confidence Regions for Θ
Exercises
17 Tests for Generalizations of Multivariate Linear Models	337	(4)
17.1 Motivation
17.2 Doubly Multivariate Models
17.3 Missing Responses in Multivariate Linear Models
17.4 Exact and Approximate Weighted Least Squares
17.5 Seemingly Unrelated Regressions
17.6 Growth Curve Models (GMANOVA)
17.7 Testing Hypotheses in the GCM
17.8 Confidence Bands for Growth Curves
18 Tests for Linear Mixed Models	341	(8)
18.1 Overview
18.2 Estimability of theta = Cβ
18.3 Likelihood Ratio Tests of Cβ
18.4 Likelihood Ratio Tests Involving τ
18.5 Test Size of Wald-Type Tests of β Using REML
18.6 Using Wald-Type Tests of β with REML
18.7 Using Wald-Type Tests of {β, τ} with REML
19 A Review of Multivariate and Univariate Linear Models	349	(12)
19.1 Matrix Gaussian and Wishart Properties
19.2 Design Matrix Properties
19.3 Model Components
19.4 Primary Parameter and Related Estimators
19.5 Secondary Parameter Estimators
19.6 Added-Last and Added-in-Order Tests
20 Sample Size for Univariate Linear Models	361	(10)
20.1 Sample Size Consulting: Before You Begin
20.2 The Machinery of a Power Analysis
20.3 Independent t Example
20.4 Paired t Example
20.5 The Impact of Using σ² or β in Power Analysis
20.6 Random Predictors
20.7 Internal Pilot Designs
20.8 Other Criteria for Choosing a Sample Size
Exercises
21 Sample Size for Multivariate Linear Models	371	(12)
21.1 The Machinery of a Power Analysis
21.2 Paired t Example
21.3 Time by Treatment Example
21.4 Comparing between and within Designs
21.5 Some Invariance Properties
21.6 Random Predictors
21.7 Internal Pilot Designs
Exercises
22 Sample Size for Generalizations of Multivariate Models	383	(2)
22.1 Motivation
22.2 Sample Size Methods for Growth Curve Models
23 Sample Size for Linear Mixed Models	385	(2)
23.1 Motivation
23.2 Methods
23.3 Internal Pilot Designs
Appendix: Computing Resources	387	(6)
References	393	(12)
Index	405