Regression Analysis : A Constructive Critique

by Richard A. Berk

• ISBN13: 9780761929048
• ISBN10: 0761929045
• Format: Hardcover
• Copyright: 2003-07-17
• Publisher: SAGE Publications, Inc

Summary

Regression Analysis: A Constructive Critique identifies a wide variety of problems with regression analysis as it is commonly used and then provides a number of ways in which practice could be improved. Regression is most useful for data reduction, leading to relatively simple but rich and precise descriptions of patterns in a data set. The emphasis on description provides readers with an insightful rethinking from the ground up of what regression analysis can do, so that readers can better match regression analysis with useful empirical questions and improved policy-related research."An interesting and lively text, rich in practical wisdom, written for people who do empirical work in the social sciences and their graduate students."--David A. Freedman, Professor of Statistics, University of California, Berkeley

Author Biography

Richard Berk is Professor in the Departments of Statistics and Sociology at the University of California, Los Angeles.

Table of Contents

Series Editor's Introduction	xi
Preface	xvii
1. Prologue: Regression Analysis as Problematic	1	(4)
2. A Grounded Introduction to Regression Analysis	5	(16)
2.1 Some Examples of Regression Analysis	5	(5)
2.1.1 Abortion and Subsequent Crime	5	(1)
2.1.2 Mandatory Basic Education for Welfare Recipients	6	(1)
2.1.3 Gender and Academic Salaries	7	(1)
2.1.4 Climate Change and Water Resources in India	8	(1)
2.1.5 Deforestation and Soil Erosion in the Yangtze River Valley	8	(1)
2.1.6 Epidemics of Hepatitis C	9	(1)
2.1.7 Onward and Upward	9	(1)
2.2 What Is Regression Analysis?	10	(8)
2.2.1 A Simple Illustration	10	(3)
2.2.2 Controlling for a Third Variable	13	(3)
2.2.3 Imposing a Smoother	16	(2)
2.3 Getting From Data to Stories	18	(3)
3. Simple Linear Regression	21	(18)
3.1 Introduction	21	(1)
3.2 Describing a Conditional Relationship With a Straight Line	22	(2)
3.3 Defining the "Best" Line	24	(3)
3.4 Some Useful Formulas	27	(1)
3.5 Standardized Slopes	28	(2)
3.6 Using Transformations for a Nonlinear Fit	30	(5)
3.7 What About the Variance Function?	35	(2)
3.8 Summary and Conclusions	37	(2)
4. Statistical Inference for Simple Linear Regression	39	(42)
4.1 The Role of Sampling	39	(19)
4.1.1 Random Sampling	39	(3)
4.1.2 Strategy I: Treating the Data as Population	42	(2)
4.1.3 Strategy II: Treating the Data as If They Were Generated by Random Sampling From a Population	44	(7)
4.1.4 Strategy III: Inventing an Imaginary Population	51	(2)
4.1.5 Strategy IV: Model-Based Sampling-Inventing a Friendly Natural Process Responsible for the Data	53	(3)
4.1.6 A Note on Randomization Inference	56	(2)
4.1.7 Summing Up	58	(1)
4.2 Simple Linear Regression Under Random Sampling	58	(11)
4.2.1 Estimating the Population Regression Line	58	(3)
4.2.2 Estimating the Standard Errors	61	(1)
4.2.3 Estimation Under Model-Based Sampling	62	(1)
4.2.4 Some Things That Can Go Wrong	62	(3)
4.2.5 Tests and Confidence Intervals	65	(4)
4.3 Statistical Power	69	(1)
4.4 Stochastic Predictors	69	(4)
4.5 Measurement Error	73	(1)
4.6 Can Resampling Techniques Help?	74	(5)
4.6.1 Percentile Confidence Intervals	76	(1)
4.6.2 Hypothesis Testing	77	(1)
4.6.3 Bootstrapping Regression	77	(1)
4.6.4 Possible Benefits From Resampling	78	(1)
4.7 Summary and Conclusions	79	(2)
5. Causal Inference for the Simple Linear Model	81	(22)
5.1 Introduction	81	(1)
5.2 Some Definitions: What's a Causal Effect?	82	(15)
5.2.1 The Neyman-Rubin Model	84	(4)
5.2.2 Thinking About Causal Effects as Response Schedules	88	(2)
5.2.3 What's an Intervention?	90	(7)
5.3 Studying Cause and Effect With Data	97	(4)
5.3.1 Using Nonstatistical Solutions for Making Causal Inferences	97	(1)
5.3.2 Using Statistical Solutions for Making Causal Inferences	98	(1)
5.3.3 Using the Simple Linear Model for Making Causal Inferences	99	(2)
5.4 Summary and Conclusions	101	(2)
6. The Formalities of Multiple Regression	103	(8)
6.1 Introduction	103	(1)
6.2 Terms and Predictors	103	(2)
6.3 Some Notation for Multiple Regression	105	(1)
6.4 Estimation	105	(2)
6.5 How Multiple Regression "Holds Constant"	107	(3)
6.6 Summary and Conclusions	110	(1)
7. Using and Interpreting Multiple Regression	111	(14)
7.1 Introduction	111	(1)
7.2 Another Formal Perspective on Holding Constant	111	(2)
7.3 When Does Holding Constant Make Sense?	113	(4)
7.4 Standardized Regression Coefficients: Once More With Feeling	117	(2)
7.5 Variances of the Coefficient Estimates	119	(3)
7.6 Summary and Conclusions	122	(3)
8. Some Popular Extensions of Multiple Regression	125	(26)
8.1 Introduction	125	(1)
8.2 Model Selection and Stepwise Regression	126	(9)
8.2.1 Model Selection by Removing Terms	127	(1)
8.2.2 Tests to Compare Models	128	(2)
8.2.3 Selecting Terms Without Testing	130	(2)
8.2.4 Stepwise Selection Methods	132	(1)
8.2.5 Some Implications	133	(2)
8.3 Using Categorical Terms: Analysis of Variance and Analysis of Covariance	135	(6)
8.3.1 An Extended Example	135	(6)
8.4 Back to the Variance Function: Weighted Least Squares	141	(6)
8.4.1 Visualizing Lack of Fit	141	(1)
8.4.2 Weighted Least Squares as a Possible Fix	142	(3)
8.4.3 Evaluating the Mean Function	145	(2)
8.5 Locally Weighted Regression Smoother	147	(1)
8.6 Summary and Conclusions	148	(3)
9. Some Regression Diagnostics	151	(20)
9.1 Introduction	151	(1)
9.2 Transformations of the Response Variable	152	(7)
9.2.1 Box-Cox Procedures	152	(1)
9.2.2 Inverse Fitted Value Response Plots	153	(6)
9.3 Leverage and Influence	159	(3)
9.3.1 Influential Cases and Cook's Distance	159	(3)
9.4 Cross-Validation	162	(1)
9.5 Misspecification Tests	163	(5)
9.5.1 Instrumental Variables	164	(3)
9.5.2 Tests for Exogeneity	167	(1)
9.6 Conclusions	168	(3)
10. Further Extensions of Regression Analysis	171	(32)
10.1 Regression Models for Longitudinal Data	171	(6)
10.1.1 Multiple Linear Regression for Time Series Data	172	(5)
10.2 Regression Analysis With Multiple Time Series Data	177	(3)
10.2.1 Fixed Effects Models	178	(1)
10.2.2 Random Effects Models	178	(2)
10.2.3 Estimation	180	(1)
10.3 Multilevel Models	180	(3)
10.4 The Generalized Linear Model	183	(5)
10.4.1 GLM Structure	183	(1)
10.4.2 Normal Models	184	(1)
10.4.3 Poisson Models	184	(2)
10.4.4 Poisson Models for Contingency Tables	186	(1)
10.4.5 Binomial Regression	186	(2)
10.5 Multiple Equation Models	188	(8)
10.5.1 Causal Inference Once Again	191	(5)
10.5.2 A Final Observation	196	(1)
10.6 Meta-Analysis	196	(4)
10.7 Conclusions	200	(3)
11. What to Do	203	(36)
11.1 How Did We Get Into This Mess?	203	(3)
11.2 Three Cheers for Description	206	(12)
11.2.1 What's Description?	207	(1)
11.2.2 Advocacy Settings	207	(2)
11.2.3 Descriptive Regressions as Part of a Broad Research Program	209	(1)
11.2.4 Spotting Provocative Associations	210	(2)
11.2.5 Some Other Benefits of Description	212	(6)
11.3 Two Cheers for Statistical Inference	218	(5)
11.3.1 Working With Near-Random Samples	220	(2)
11.3.2 Working With Data From Nature	222	(1)
11.3.3 Working With a Nearly Correct Model	222	(1)
11.4 One Cheer for Causal Inference	223	(11)
11.4.1 Special-Purpose Estimators	226	(4)
11.4.2 Propensity Scores	230	(1)
11.4.3 Sensitivity Analysis of the Selection Process	231	(1)
11.4.4 Bounding Treatment Effects	232	(2)
11.4.5 Some Forecasts	234	(1)
11.5 Some Final Observations	234	(5)
11.5.1 A Police Story	234	(3)
11.5.2 Regression Analysis as Too Little, Too Late	237	(2)
References	239	(12)
Index	251	(8)
About the Author	259