Regression Estimators : A Comparative Study

by Gruber, Marvin H. J.

• ISBN13: 9780801894268
• ISBN10: 0801894263
• Edition: 2nd
• Format: Hardcover
• Copyright: 2010-05-25
• Publisher: Johns Hopkins Univ Pr

Summary

An examination of mathematical formulations of ridge regression type estimators points to a curious observation: estimators can be derived by both Bayesian and frequentist methods. In this updated and expanded edition of his 1990 treatise on the subject, Marvin H. J. Gruber presents, compares, and contrasts the development and properties of ridge-type estimators from these two philosophically different points of view.The book is organized into five sections. Part I gives a historical survey of the literature and summarizes basic ideas in matrix theory and statistical decision theory. Part II explores the mathematical relationships between estimators from both Bayesian and frequentist points of view. Part III considers the efficiency of estimators with and without averaging over a prior distribution. Part IV applies the methods and results discussed in the previous two sections to the Kalman filter, analysis of variance models, and penalized spines. Part V surveys recent developments in the field. These include efficiencies of ridge-type estimators for loss functions other than squared error loss functions and applications to information geometry. Gruber also includes an updated historical survey and bibliography.With more than 150 exercises, Regression Estimators is a valuable resource for graduate students and professional statisticians.Praise for the first edition"A comprehensive treatment... valuable to statisticians who would like to know more about the analytical properties of ridge-type estimators." -- Journal of the American Statistical Association"Highly recommended to anyone working on advanced applications or research in estimation in linear models." -- Technometrics

Author Biography

Marvin H. J. Gruber is a professor of mathematics and statistics at the Rochester Institute of Technology.

Table of Contents

Preface	p. ix
Introduction and Mathematical Preliminaries
Introduction	p. 3
The Purpose of This Book	p. 4
Least Square Estimators and the Need for Alternatives	p. 4
Historical Survey	p. 10
The Structure of the Book	p. 32
Mathematical and Statistical Preliminaries	p. 34
Introduction	p. 34
Matrix Theory Results	p. 35
The Bayes Estimator (BE)	p. 48
Admissible Estimators	p. 53
The Minimax Estimator	p. 56
Criterion for Comparing Estimators: Theobald's 1974 Result	p. 57
Some Useful Inequalities	p. 60
Some Miscellaneous Useful Matrix Results	p. 63
Summary	p. 65
The Estimators, Their Derivations, and Their Relationships
The Estimators	p. 69
Introduction	p. 69
The Least Square Estimator and Its Properties	p. 70
The Generalized Ridge Regression Estimator	p. 76
The Mixed Estimators	p. 79
The Linear Minimax Estimator	p. 85
The Bayes Estimator	p. 88
Summary	p. 92
How the Different Estimators Are Related	p. 94
Introduction	p. 94
Alternative Forms of the Bayes Estimator Full-Rank Case	p. 95
Alternative Forms of the Bayes Estimator Non-Full-Rank Case Estimable Parametric Functions	p. 98
Equivalence of the Generalized Ridge Estimator and the Bayes Estimator	p. 101
Equivalence of the Mixed Estimator and the Bayes Estimator	p. 103
Ridge Estimators in the Literature as Special Cases of the BE, Minimax Estimators, or Mixed Estimators	p. 109
An Extension of the Gauss-Markov Theorem	p. 116
Generalities	p. 117
Summary	p. 130
Comparing the Efficiency of the Estimators
Measures of Efficiency of the Estimators	p. 135
Introduction	p. 135
The Different Kinds of Mean Square Error	p. 136
Zellner's Balanced Loss Function	p. 141
The LINEX Loss Function	p. 143
Linear Admissibility	p. 144
Summary	p. 146
The Average Mean Square Error	p. 147
Introduction	p. 147
The Forms of the MSE for the Minimax, Bayes, and Mixed Estimators	p. 148
The Relationship between the Average Variance and the MSE	p. 151
The Average MSE of the Bayes Estimator	p. 153
Alternative Forms of the MSE of the Mixed Estimator	p. 155
Comparison of the MSE of Different BEs	p. 157
Comparison of the MSE of the Ridge and Contraction Estimators	p. 162
Comparison of the Average MSE of the Two-Parameter Liu Estimator and the Ordinary Ridge Regression Estimator	p. 165
Summary	p. 165
The MSE Neglecting the Prior Assumptions	p. 167
Introduction	p. 167
The MSE of the BE	p. 168
The MSE of the Mixed Estimators Neglecting Prior Assumptions	p. 171
Comparison of the Conditional MSE of the Bayes and Least Square Estimators and Comparison of the Conditional and Average MSE	p. 174
Comparison of the MSE of a Mixed Estimator with That of the LS Estimators	p. 187
Comparison of the MSE of Two Bayes Estimators	p. 192
Summary	p. 201
The MSE for Incorrect Prior Assumptions	p. 202
Introduction	p. 202
The Bayes Estimator and Its MSE	p. 203
The Minimax Estimator	p. 208
The Mixed Estimator	p. 210
Contaminated Priors	p. 213
Contaminated (Mixed) Bayes Estimators	p. 217
Summary	p. 220
Applications
The Kalman Filter	p. 223
Introduction	p. 223
The Kalman Filter as a Bayes Estimator	p. 225
The Kalman Filter as a Recursive Least Square Estimator, and the Connection with the Mixed Estimator	p. 228
The Minimax Estimator	p. 235
The Generalized Ridge Estimator	p. 237
The Average Mean Square Error	p. 239
The MSE for Incorrect Initial Prior Assumptions	p. 242
Applications	p. 244
Recursive Ridge Regression	p. 248
Summary	p. 251
Experimental Design Models	p. 252
Introduction	p. 252
The One-Way ANOVA Model	p. 253
The Bayes and Empirical Bayes Estimators	p. 263
The Two-Way Classification	p. 267
The Bayes and Empirical Bayes Estimators	p. 273
Summary	p. 278
Appendix to Section 10.2. Calculation of the MSE of Section 10.2	p. 278
How Penalized Splines and Ridge-Type Estimators Are Related	p. 283
Introduction	p. 283
Splines as a Special Kind of Regression Model	p. 284
Penalized Splines	p. 289
The Best Linear Unbiased Predictor (BLUP)	p. 290
Two Examples	p. 297
Summary	p. 299
Alternative Measures of Efficiency
Estimation Using Zellner's Balanced Loss Function	p. 303
Introduction	p. 303
Zellner's Balanced Loss Function	p. 304
The Estimators from Different Points of View	p. 305
The Average Mean Square Error	p. 312
The Risk without Averaging over a Prior Distribution	p. 315
Some Optimal Ridge Estimators	p. 318
Summary	p. 324
The LINEX and Other Asymmetric Loss Functions	p. 325
Introduction	p. 325
The LINEX Loss Function	p. 326
The Bayes Risk for a Regression Estimator	p. 336
The Frequentist Risk	p. 340
Summary	p. 352
Distances between Ridge-Type Estimators, and Information Geometry	p. 354
Introduction	p. 354
The Relevant Differential Geometry	p. 355
The Distance between Two Linear Bayes Estimators, Based on the Prior Distributions	p. 369
The Distance between Distributions of Ridge-Type Estimators from a Non-Bayesian Point of View	p. 382
Distances between the Mixed Estimators	p. 384
An Example Using the Kalman Filter	p. 387
Summary	p. 389
References	p. 391
Author Index	p. 403
Subject Index	p. 407
Table of Contents provided by Ingram. All Rights Reserved.