Nonparametric and Semiparametric Models

by Hardle, Wolfgang; Muller, Marlene; Sperlich, Stefan; werwatch, Axel

• ISBN13: 9783540207221
• ISBN10: 3540207228
• Format: Hardcover
• Copyright: 2004-05-14
• Publisher: Springer Verlag

Summary

The concept of nonparametric smoothing is a central idea in statistics that aims to simultaneously estimate and modes the underlying structure. The book considers high dimensional objects, as density functions and regression. The semiparametric modeling technique compromises the two aims, flexibility and simplicity of statistical procedures, by introducing partial parametric components. These components allow to match structural conditions like e.g. linearity in some variables and may be used to model the influence of discrete variables. The aim of this monograph is to present the statistical and mathematical principles of smoothing with a focus on applicable techniques. The necessary mathematical treatment is easily understandable and a wide variety of interactive smoothing examples are given. The book does naturally split into two parts: Nonparametric models (histogram, kernel density estimation, nonparametric regression) and semiparametric models (generalized regression, single index models, generalized partial linear models, additive and generalized additive models). The first part is intended for undergraduate students majoring in mathematics, statistics, econometrics or biometrics whereas the second part is intended to be used by master and PhD students or researchers. The material is easy to accomplish since the e-book character of the text gives a maximum of flexibility in learning (and teaching) intensity.

Table of Contents

Preface	v
Notation	xxi
1 Introduction	1	(20)
1.1 Density Estimation	1	(2)
1.2 Regression	3	(15)
1.2.1 Parametric Regression	5	(2)
1.2.2 Nonparametric Regression	7	(2)
1.2.3 Semiparametric Regression	9	(9)
Summary	18	(3)
Part I Nonparametric Models
2 Histogram	21	(18)
2.1 Motivation and Derivation	21	(3)
2.1.1 Construction	21	(2)
2.1.2 Derivation	23	(1)
2.1.3 Varying the Binwidth	23	(1)
2.2 Statistical Properties	24	(6)
2.2.1 Bias	25	(1)
2.2.2 Variance	26	(1)
2.2.3 Mean Squared Error	27	(2)
2.2.4 Mean Integrated Squared Error	29	(1)
2.2.5 Optimal Binwidth	29	(1)
2.3 Dependence of the Histogram on the Origin	30	(2)
2.4 Averaged Shifted Histogram	32	(3)
Bibliographic Notes	35	(1)
Exercises	36	(2)
Summary	38	(1)
3 Nonparametric Density Estimation	39	(46)
3.1 Motivation and Derivation	39	(7)
3.1.1 Introduction	39	(1)
3.1.2 Derivation	40	(3)
3.1.3 Varying the Bandwidth	43	(1)
3.1.4 Varying the Kernel Function	43	(2)
3.1.5 Kernel Density Estimation as a Sum of Bumps	45	(1)
3.2 Statistical Properties	46	(5)
3.2.1 Bias	46	(2)
3.2.2 Variance	48	(1)
3.2.3 Mean Squared Error	49	(1)
3.2.4 Mean Integrated Squared Error	50	(1)
3.3 Smoothing Parameter Selection	51	(6)
3.3.1 Silverman's Rule of Thumb	51	(2)
3.3.2 Cross-Validation	53	(2)
3.3.3 Refined Plug-in Methods	55	(1)
3.3.4 An Optimal Bandwidth Selector?!	56	(1)
3.4 Choosing the Kernel	57	(4)
3.4.1 Canonical Kernels and Bandwidths	57	(2)
3.4.2 Adjusting Bandwidths across Kernels	59	(1)
3.4.3 Optimizing the Kernel	60	(1)
3.5 Confidence Intervals and Confidence Bands	61	(5)
3.6 Multivariate Kernel Density Estimation	66	(13)
3.6.1 Bias, Variance and Asymptotics	70	(2)
3.6.2 Bandwidth Selection	72	(3)
3.6.3 Computation and Graphical Representation	75	(4)
Bibliographic Notes	79	(1)
Exercises	80	(2)
Summary	82	(3)
4 Nonparametric Regression	85	(60)
4.1 Univariate Kernel Regression	85	(13)
4.1.1 Introduction	85	(3)
4.1.2 Kernel Regression	88	(6)
4.1.3 Local Polynomial Regression and Derivative Estimation	94	(4)
4.2 Other Smoothers	98	(9)
4.2.1 Nearest-Neighbor Estimator	98	(3)
4.2.2 Median Smoothing	101	(1)
4.2.3 Spline Smoothing	101	(3)
4.2.4 Orthogonal Series	104	(3)
4.3 Smoothing Parameter Selection	107	(11)
4.3.1 A Closer Look at the Averaged Squared Error	110	(3)
4.3.2 Cross-Validation	113	(1)
4.3.3 Penalizing Functions	114	(4)
4.4 Confidence Regions and Tests	118	(10)
4.4.1 Pointwise Confidence Intervals	119	(1)
4.4.2 Confidence Bands	120	(4)
4.4.3 Hypothesis Testing	124	(4)
4.5 Multivariate Kernel Regression	128	(7)
4.5.1 Statistical Properties	130	(2)
4.5.2 Practical Aspects	132	(3)
Bibliographic Notes	135	(2)
Exercises	137	(2)
Summary	139	(6)
Part II Semiparametric Models
5 Semiparametric and Generalized Regression Models	145	(22)
5.1 Dimension Reduction	145	(6)
5.1.1 Variable Selection in Nonparametric Regression	148	(1)
5.1.2 Nonparametric Link Function	148	(1)
5.1.3 Semi- or Nonparametric Index	149	(2)
5.2 Generalized Linear Models	151	(11)
5.2.1 Exponential Families	151	(2)
5.2.2 Link Functions	153	(1)
5.2.3 Iteratively Reweighted Least Squares Algorithm	154	(8)
Bibliographic Notes	162	(2)
Exercises	164	(1)
Summary	165	(2)
6 Single Index Models	167	(22)
6.1 Identification	168	(2)
6.2 Estimation	170	(13)
6.2.1 Semiparametric Least Squares	172	(2)
6.2.2 Pseudo Likelihood Estimation	174	(4)
6.2.3 Weighted Average Derivative Estimation	178	(5)
6.3 Testing the SIM	183	(2)
Bibliographic Notes	185	(1)
Exercises	186	(1)
Summary	187	(2)
7 Generalized Partial Linear Models	189	(22)
7.1 Partial Linear Models	189	(2)
7.2 Estimation Algorithms for PLM and GPLM	191	(11)
7.2.1 Profile Likelihood	191	(4)
7.2.2 Generalized Speckman Estimator	195	(2)
7.2.3 Backfitting	197	(2)
7.2.4 Computational Issues	199	(3)
7.3 Testing the GPLM	202	(4)
7.3.1 Likelihood Ratio Test with Approximate Degrees of Freedom	202	(1)
7.3.2 Modified Likelihood Ratio Test	203	(3)
Bibliographic Notes	206	(1)
Exercises	207	(1)
Summary	208	(3)
8 Additive Models and Marginal Effects	211	(42)
8.1 Backfitting	212	(10)
8.1.1 Classical Backfitting	212	(7)
8.1.2 Modified Backfitting	219	(2)
8.1.3 Smoothed Backfitting	221	(1)
8.2 Marginal Integration Estimator	222	(12)
8.2.1 Estimation of Marginal Effects	224	(1)
8.2.2 Derivative Estimation for the Marginal Effects	225	(2)
8.2.3 Interaction Terms	227	(7)
8.3 Finite Sample Behavior	234	(13)
8.3.1 Bandwidth Choice	236	(3)
8.3.2 MASE in Finite Samples	239	(1)
8.3.3 Equivalent Kernel Weights	240	(7)
Bibliographic Notes	247	(1)
Exercises	248	(2)
Summary	250	(3)
9 Generalized Additive Models	253	(26)
9.1 Additive Partial Linear Models	254	(5)
9.2 Additive Models with Known Link	259	(5)
9.2.1 GAM using Backfitting	260	(2)
9.2.2 GAM using Marginal Integration	262	(2)
9.3 Generalized Additive Partial Linear Models	264	(4)
9.3.1 GAPLM using Backfitting	264	(1)
9.3.2 GAPLM using Marginal Integration	264	(4)
9.4 Testing in Additive Models, GAM, and GAPLM	268	(6)
Bibliographic Notes	274	(1)
Exercises	275	(1)
Summary	276	(3)
References	279	(12)
Author Index	291	(4)
Subject Index	295