Functional Data Analysis

by Ramsay, J. O.; Silverman, B. W.

• ISBN13: 9780387400808
• ISBN10: 038740080X
• Edition: 2nd
• Format: Hardcover
• Copyright: 2005-06-30
• Publisher: Springer

Summary

Scientists today collect samples of curves and other functional observations. This monograph presents many ideas and techniques for such data. Included are expressions in the functional domain of such classics as linear regression, principal components analysis, linear modelling, and canonical correlation analysis, as well as specifically functional techniques such as curve registration and principal differential analysis. Data arising in real applications are used throughout for both motivation and illustration, showing how functional approaches allow us to see new things, especially by exploiting the smoothness of the processes generating the data. The data sets exemplify the wide scope of functional data analysis; they are drwan from growth analysis, meterology, biomechanics, equine science, economics, and medicine.The book presents novel statistical technology while keeping the mathematical level widely accessible. It is designed to appeal to students, to applied data analysts, and to experienced researchers; it will have value both within statistics and across a broad spectrum of other fields. Much of the material is based on the authors' own work, some of which appears here for the first time.Jim Ramsay is Professor of Psychology at McGill University and is an international authority on many aspects of multivariate analysis. He draws on his collaboration with researchers in speech articulation, motor control, meteorology, psychology, and human physiology to illustrate his technical contributions to functional data analysis in a wide range of statistical and application journals.Bernard Silverman, author of the highly regarded "Density Estimation for Statistics and Data Analysis," and coauthor of "Nonparametric Regression and Generalized Linear Models: A Roughness Penalty Approach," is Professor of Statistics at Bristol University. His published work on smoothing methods and other aspects of applied, computational, and theoretical statistics has been recognized by the Presidents' Award of the Committee of Presidents of Statistical Societies, and the award of two Guy Medals by the Royal Statistical Society.

Author Biography

Bernard Silverman is Master of St. Peter's College and Professor of Statistics at Oxford University. Jim Ramsay is Professor of Psychology at McGill University.

Table of Contents

Preface to the Second Edition	v
1 Introduction	1	(18)
1.1 What are functional data?	1	(4)
1.2 Functional models for nonfunctional data	5	(1)
1.3 Some functional data analyses	5	(4)
1.4 The goals of functional data analysis	9	(2)
1.5 The first steps in a functional data analysis	11	(4)
1.5.1 Data representation: smoothing and interpolation	11	(1)
1.5.2 Data registration or feature alignment	12	(1)
1.5.3 Data display	13	(1)
1.5.4 Plotting pairs of derivatives	13	(2)
1.6 Exploring variability in functional data	15	(1)
1.6.1 Functional descriptive statistics	15	(1)
1.6.2 Functional principal components analysis	15	(1)
1.6.3 Functional canonical correlation	16	(1)
1.7 Functional linear models	16	(1)
1.8 Using derivatives in functional data analysis	17	(1)
1.9 Concluding remarks	18	(1)
2 Tools for exploring functional data	19	(18)
2.1 Introduction	19	(1)
2.2 Some notation	20	(2)
2.2.1 Scalars, vectors, functions and matrices	20	(1)
2.2.2 Derivatives and integrals	20	(1)
2.2.3 Inner products	21	(1)
2.2.4 Functions of functions	21	(1)
2.3 Summary statistics for functional data	22	(4)
2.3.1 Functional means and variances	22	(1)
2.3.2 Covariance and correlation functions	22	(2)
2.3.3 Cross-covariance and cross-correlation functions	24	(2)
2.4 The anatomy of a function	26	(3)
2.4.1 Functional features	26	(1)
2.4.2 Data resolution and functional dimensionality	27	(1)
2.4.3 The size of a function	28	(1)
2.5 Phase-plane plots of periodic effects	29	(5)
2.5.1 The log nondurable goods index	29	(1)
2.5.2 Phase-plane plots show energy transfer	30	(3)
2.5.3 The nondurable goods cycles	33	(1)
2.6 Further reading and notes	34	(3)
3 From functional data to smooth functions	37	(22)
3.1 Introduction	37	(1)
3.2 Some properties of functional data	38	(5)
3.2.1 What makes discrete data functional?	38	(1)
3.2.2 Samples of functional data	39	(1)
3.2.3 The interplay between smooth and noisy variation	39	(1)
3.2.4 The standard model for error and its limitations	40	(1)
3.2.5 The resolving power of data	41	(1)
3.2.6 Data resolution and derivative estimation	41	(2)
3.3 Representing functions by basis functions	43	(2)
3.4 The Fourier basis system for periodic data	45	(1)
3.5 The spline basis system for open-ended data	46	(7)
3.5.1 Spline functions and degrees of freedom	47	(2)
3.5.2 The B-spline basis for spline functions	49	(4)
3.6 Other useful basis systems	53	(3)
3.6.1 Wavelets	53	(1)
3.6.2 Exponential and power bases	54	(1)
3.6.3 Polynomial bases	54	(1)
3.6.4 The polygonal basis	55	(1)
3.6.5 The step-function basis	55	(1)
3.6.6 The constant basis	55	(1)
3.6.7 Empirical and designer bases	56	(1)
3.7 Choosing a scale for t	56	(1)
3.8 Further reading and notes	57	(2)
4 Smoothing functional data by least squares	59	(22)
4.1 Introduction	59	(1)
4.2 Fitting data using a basis system by least squares	59	(3)
4.2.1 Ordinary or unweighted least squares fits	60	(1)
4.2.2 Weighted least squares fits	61	(1)
4.3 A performance assessment of least squares smoothing	62	(1)
4.4 Least squares fits as linear transformations of the data	63	(4)
4.4.1 How linear smoothers work	64	(2)
4.4.2 The degrees of freedom of a linear smooth	66	(1)
4.5 Choosing the number K of basis functions	67	(3)
4.5.1 The bias/variance trade-off	67	(2)
4.5.2 Algorithms for choosing K	69	(1)
4.6 Computing sampling variances and confidence limits	70	(3)
4.6.1 Sampling variance estimates	70	(1)
4.6.2 Estimating Σe	71	(1)
4.6.3 Confidence limits	72	(1)
4.7 Fitting data by localized least squares	73	(6)
4.7.1 Kernel smoothing	74	(2)
4.7.2 Localized basis function estimators	76	(1)
4.7.3 Local polynomial smoothing	77	(1)
4.7.4 Choosing the bandwidth h	78	(1)
4.7.5 Summary of localized basis methods	78	(1)
4.8 Further reading and notes	79	(2)
5 Smoothing functional data with a roughness penalty	81	(30)
5.1 Introduction	81	(1)
5.2 Spline smoothing	82	(9)
5.2.1 Two competing objectives in function estimation	83	(1)
5.2.2 Quantifying roughness	84	(1)
5.2.3 The penalized sum of squared errors fitting criterion	84	(1)
5.2.4 The structure of a smoothing spline	85	(1)
5.2.5 How spline smooths are computed	86	(1)
5.2.6 Spline smoothing as a linear operation	87	(2)
5.2.7 Spline smoothing as an augmented least squares problem	89	(1)
5.2.8 Estimating derivatives by spline smoothing	90	(1)
5.3 Some extensions	91	(3)
5.3.1 Roughness penalties with fewer basis functions	91	(1)
5.3.2 More general measures of data fit	92	(1)
5.3.3 More general roughness penalties	92	(1)
5.3.4 Computing the roughness penalty matrix	93	(1)
5.4 Choosing the smoothing parameter	94	(6)
5.4.1 Some limits imposed by computational issues	94	(2)
5.4.2 The cross-validation or CV method	96	(1)
5.4.3 The generalized cross-validation or GCV method	97	(2)
5.4.4 Spline smoothing the simulated growth data	99	(1)
5.5 Confidence intervals for function values and functional probes	100	(4)
5.5.1 Linear functional probes	101	(1)
5.5.2 Two linear mappings defining a probe value	102	(1)
5.5.3 Computing confidence limits for function values	103	(1)
5.5.4 Confidence limits for growth acceleration	104	(1)
5.6 A bi-resolution analysis with smoothing splines	104	(5)
5.6.1 Complementary bases	105	(1)
5.6.2 Specifying the roughness penalty	106	(1)
5.6.3 Some properties of the estimates	107	(1)
5.6.4 Relationship to the roughness penalty approach	108	(1)
5.7 Further reading and notes	109	(2)
6 Constrained functions	111	(16)
6.1 Introduction	111	(1)
6.2 Fitting positive functions	111	(4)
6.2.1 A positive smoothing spline	113	(1)
6.2.2 Representing a positive function by a differential equation	114	(1)
6.3 Fitting strictly monotone functions	115	(2)
6.3.1 Fitting the growth of a baby's tibia	115	(1)
6.3.2 Expressing a strictly monotone function explicitly	115	(1)
6.3.3 Expressing a strictly monotone function as a differential equation	116	(1)
6.4 The performance of spline smoothing revisited	117	(1)
6.5 Fitting probability functions	118	(1)
6.6 Estimating probability density functions	119	(2)
6.7 Functional data analysis of point processes	121	(2)
6.8 Fitting a linear model with estimation of the density of residuals	123	(3)
6.9 Further notes and readings	126	(1)
7 The registration and display of functional data	127	(20)
7.1 Introduction	127	(2)
7.2 Shift registration	129	(3)
7.2.1 The least squares criterion for shift alignment	131	(1)
7.3 Feature or landmark registration	132	(5)
7.4 Using the warping function h to register x	137	(1)
7.5 A more general warping function h	137	(1)
7.6 A continuous fitting criterion for registration	138	(2)
7.7 Registering the height acceleration curves	140	(2)
7.8 Some practical advice	142	(1)
7.9 Computational details	142	(2)
7.9.1 Shift registration by the Newton-Raphson algorithm	142	(2)
7.10 Further reading and notes	144	(3)
8 Principal components analysis for functional data	147	(26)
8.1 Introduction	147	(1)
8.2 Defining functional PCA	148	(6)
8.2.1 PCA for multivariate data	148	(1)
8.2.2 Defining PCA for functional data	149	(2)
8.2.3 Defining an optimal empirical orthonormal basis	151	(1)
8.2.4 PCA and eigenanalysis	152	(2)
8.3 Visualizing the results	154	(6)
8.3.1 Plotting components as perturbations of the mean	154	(2)
8.3.2 Plotting principal component scores	156	(1)
8.3.3 Rotating principal components	156	(4)
8.4 Computational methods for functional PCA	160	(6)
8.4.1 Discretizing the functions	161	(1)
8.4.2 Basis function expansion of the functions	161	(3)
8.4.3 More general numerical quadrature	164	(2)
8.5 Bivariate and multivariate PCA	166	(5)
8.5.1 Defining multivariate functional PCA	167	(1)
8.5.2 Visualizing the results	168	(2)
8.5.3 Inner product notation: Concluding remarks	170	(1)
8.6 Further readings and notes	171	(2)
9 Regularized principal components analysis	173	(14)
9.1 Introduction	173	(2)
9.2 The results of smoothing the, PCA	175	(2)
9.3 The smoothing approach	177	(2)
9.3.1 Estimating the leading principal component	177	(1)
9.3.2 Estimating subsequent principal components	177	(1)
9.3.3 Choosing the smoothing parameter by CV	178	(1)
9.4 Finding the regularized PCA in practice	179	(3)
9.4.1 The periodic case	179	(2)
9.4.2 The nonperiodic case	181	(1)
9.5 Alternative approaches	182	(5)
9.5.1 Smoothing the data rather than the PCA	182	(2)
9.5.2 A stepwise roughness penalty procedure	184	(1)
9.5.3 A further approach	185	(2)
10 Principal components analysis of mixed data	187	(14)
10.1 Introduction	187	(2)
10.2 General approaches to mixed data	189	(1)
10.3 The PCA of hybrid data	190	(4)
10.3.1 Combining function and vector spaces	190	(1)
10.3.2 Finding the principal components in practice	191	(1)
10.3.3 Incorporating smoothing	192	(1)
10.3.4 Balance between functional and vector variation	192	(2)
10.4 Combining registration and PCA	194	(1)
10.4.1 Expressing the observations as mixed data	194	(1)
10.4.2 Balancing temperature and time shift effects	194	(1)
10.5 The temperature data reconsidered	195	(6)
10.5.1 Taking account of effects beyond phase shift	195	(3)
10.5.2 Separating out the vector component	198	(3)
11 Canonical correlation and discriminant analysis	201	(16)
11.1 Introduction	201	(3)
11.1.1 The basic problem	201	(3)
11.2 Principles of classical CCA	204	(1)
11.3 Functional canonical correlation analysis	204	(4)
11.3.1 Notation and assumptions	204	(1)
11.3.2 The naive approach does not give meaningful results	205	(1)
11.3.3 Choice of the smoothing parameter	206	(1)
11.3.4 The values of the correlations	207	(1)
11.4 Application to the study of lupus nephritis	208	(1)
11.5 Why is regularization necessary?	209	(1)
11.6 Algorithmic considerations	210	(3)
11.6.1 Discretization and basis approaches	210	(1)
11.6.2 The roughness of the canonical variates	211	(2)
11.7 Penalized optimal scoring and discriminant analysis	213	(2)
11.7.1 The optimal scoring problem	213	(1)
11.7.2 The discriminant problem	214	(1)
11.7.3 The relationship with CCA	214	(1)
11.7.4 Applications	215	(1)
11.8 Further readings and notes	215	(2)
12 Functional linear models	217	(6)
12.1 Introduction	217	(1)
12.2 A functional response and a categorical independent variable	218	(1)
12.3 A scalar response and a functional independent variable	219	(1)
12.4 A functional response and a functional independent variable	220	(1)
12.4.1 Concurrent	220	(1)
12.4.2 Annual or total	220	(1)
12.4.3 Short-term feed-forward	220	(1)
12.4.4 Local influence	221	(1)
12.5 What about predicting derivatives?	221	(1)
12.6 Overview	222	(1)
13 Modelling functional responses with multivariate covariates	223	(24)
13.1 Introduction	223	(1)
13.2 Predicting temperature curves from climate zones	223	(6)
13.2.1 Fitting the model	225	(1)
13.2.2 Assessing the fit	225	(4)
13.3 Force plate data for walking horses	229	(6)
13.3.1 Structure of the data	229	(2)
13.3.2 A functional linear model for the horse data	231	(2)
13.3.3 Effects and contrasts	233	(2)
13.4 Computational issues	235	(4)
13.4.1 The general model	235	(1)
13.4.2 Pointwise minimization	236	(1)
13.4.3 Functional linear modelling with regularized basis expansions	236	(2)
13.4.4 Using the Kronecker product to express B	238	(1)
13.4.5 Fitting the raw data	239	(1)
13.5 Confidence intervals for regression functions	239	(5)
13.5.1 How to compute confidence intervals	239	(2)
13.5.2 Confidence intervals for climate zone effects	241	(2)
13.5.3 Some cautions on interpreting confidence intervals	243	(1)
13.6 Further reading and notes	244	(3)
14 Functional responses, functional covariates and the con-current model	247	(14)
14.1 Introduction	247	(1)
14.2 Predicting precipitation profiles from temperature curves	248	(3)
14.2.1 The model for the daily logarithm of rainfall	248	(1)
14.2.2 Preliminary steps	248	(2)
14.2.3 Fitting the model and assessing fit	250	(1)
14.3 Long-term and seasonal trends in the nondurable goods index	251	(4)
14.4 Computational issues	255	(2)
14.5 Confidence intervals	257	(1)
14.6 Further reading and notes	258	(3)
15 Functional linear models for scalar responses	261	(18)
15.1 Introduction	261	(1)
15.2 A naive approach: Discretizing the covariate function	262	(2)
15.3 Regularization using restricted basis functions	264	(2)
15.4 Regularization with roughness penalties	266	(2)
15.5 Computational issues	268	(2)
15.5.1 Computing the regularized solution	269	(1)
15.5.2 Computing confidence limits	270	(1)
15.6 Cross-validation and regression diagnostics	270	(1)
15.7 The direct penalty method for computing β	271	(4)
15.7.1 Functional interpolation	272	(1)
15.7.2 The two-stage minimization process	272	(1)
15.7.3 Functional interpolation revisited	273	(2)
15.8 Functional regression and integral equations	275	(1)
15.9 Further reading and notes	276	(3)
16 Functional linear models for functional responses	279	(18)
16.1 Introduction: Predicting log precipitation from temperature	279	(3)
16.1.1 Fitting the model without regularization	280	(2)
16.2 Regularizing the fit by restricting the bases	282	(3)
16.2.1 Restricting the basis η(s)	282	(1)
16.2.2 Restricting the basis theta(t)	283	(1)
16.2.3 Restricting both bases	284	(1)
16.3 Assessing goodness of fit	285	(5)
16.4 Computational details	290	(3)
16.4.1 Fitting the model without regularization	291	(1)
16.4.2 Fitting the model with regularization	292	(1)
16.5 The general case	293	(2)
16.6 Further reading and notes	295	(2)
17 Derivatives and functional linear models	297	(10)
17.1 Introduction	297	(1)
17.2 The oil refinery data	298	(3)
17.3 The melanoma data	301	(4)
17.4 Some comparisons of the refinery and melanoma analyses	305	(2)
18 Differential equations and operators	307	(20)
18.1 Introduction	307	(1)
18.2 Exploring a simple linear differential equation	308	(2)
18.3 Beyond the constant coefficient first-order linear equation	310	(3)
18.3.1 Nonconstant coefficients	310	(1)
18.3.2 Higher order equations	311	(1)
18.3.3 Systems of equations	312	(1)
18.3.4 Beyond linearity	313	(1)
18.4 Some applications of linear differential equations and operators	313	(6)
18.4.1 Differential operators to produce new functional observations	313	(1)
18.4.2 The gross domestic product data	314	(2)
18.4.3 Differential operators to regularize or smooth models	316	(1)
18.4.4 Differential operators to partition variation	317	(2)
18.4.5 Operators to define solutions to problems	319	(1)
18.5 Some linear differential equation facts	319	(4)
18.5.1 Derivatives are rougher	319	(1)
18.5.2 Finding a linear differential operator that annihilates known functions	320	(2)
18.5.3 Finding the functions ξj satisfying Lξj = 0	322	(1)
18.6 Initial conditions, boundary conditions and other constraints	323	(2)
18.6.1 Why additional constraints are needed to define a solution	323	(1)
18.6.2 How L and B partition functions	324	(1)
18.6.3 The inner product defined by operators L and B	325	(1)
18.7 Further reading and notes	325	(2)
19 Principal differential analysis	327	(22)
19.1 Introduction	327	(1)
19.2 Defining the problem	328	(1)
19.3 A principal differential analysis of lip movement	329	(5)
19.3.1 The biomechanics of lip movement	330	(2)
19.3.2 Visualizing the PDA results	332	(2)
19.4 PDA of the pinch force data	334	(4)
19.5 Techniques for principal differential analysis	338	(5)
19.5.1 PDA by point-wise minimization	338	(1)
19.5.2 PDA using the concurrent functional linear model	339	(1)
19.5.3 PDA by iterating the concurrent linear model	340	(3)
19.5.4 Assessing fit in PDA	343	(1)
19.6 Comparing PDA and PCA	343	(5)
19.6.1 PDA and PCA both minimize sums of squared errors	343	(1)
19.6.2 PDA and PCA both involve finding linear operators	344	(1)
19.6.3 Differences between differential operators (PDA) and projection operators (PCA)	345	(3)
19.7 Further readings and notes	348	(1)
20 Green's functions and reproducing kernels	349	(10)
20.1 Introduction	349	(1)
20.2 The Green's function for solving a linear differential equation	350	(3)
20.2.1 The definition of the Green's function	351	(1)
20.2.2 A matrix analogue of the Green's function	352	(1)
20.2.3 A recipe for the Green's function	352	(1)
20.3 Reproducing kernels and Green's functions	353	(4)
20.3.1 What is a reproducing kernel?	354	(1)
20.3.2 The reproducing kernel for ker B	355	(1)
20.3.3 The reproducing kernel for ker L	356	(1)
20.4 Further reading and notes	357	(2)
21 More general roughness penalties	359	(20)
21.1 Introduction	359	(4)
21.1.1 The lip movement data	360	(1)
21.1.2 The weather data	361	(2)
21.2 The optimal basis for spline smoothing	363	(1)
21.3 An O(n) algorithm for L-spline smoothing	364	(5)
21.3.1 The need for a good algorithm	364	(2)
21.3.2 Setting up the smoothing procedure	366	(1)
21.3.3 The smoothing phase	367	(1)
21.3.4 The performance assessment phase	367	(2)
21.3.5 Other O(n) algorithms	369	(1)
21.4 A compact support basis for L-splines	369	(1)
21.5 Some case studies	370	(9)
21.5.1 The gross domestic product data	370	(1)
21.5.2 The melanoma data	371	(2)
21.5.3 The GDP data with seasonal effects	373	(1)
21.5.4 Smoothing simulated human growth data	374	(5)
22 Some perspectives on FDA	379	(6)
22.1 The context of functional data analysis	379	(3)
22.1.1 Replication and regularity	379	(1)
22.1.2 Some functional aspects elsewhere in statistics	380	(1)
22.1.3 Functional analytic treatments	381	(1)
22.2 Challenges for the future	382	(3)
22.2.1 Probability and inference	382	(1)
22.2.2 Asymptotic results	383	(1)
22.2.3 Multidimensional arguments	383	(1)
22.2.4 Practical methodology and applications	384	(1)
22.2.5 Back to the data!	384	(1)
Appendix: Some algebraic and functional techniques	385	(20)
A.1 Inner products (x, y)	385	(6)
A.1.1 Some specific examples	386	(1)
A.1.2 General properties	387	(2)
A.1.3 Descriptive statistics in inner product notation	389	(1)
A.1.4 Some extended uses of inner product notation	390	(1)
A.2 Further aspects of inner product spaces	391	(1)
A.2.1 Projections	391	(1)
A.2.2 Quadratic optimization	392	(1)
A.3 Matrix decompositions and generalized inverses	392	(2)
A.3.1 Singular value decompositions	392	(1)
A.3.2 Generalized inverses	393	(1)
A.3.3 The QR decomposition	393	(1)
A.4 Projections	394	(2)
A.4.1 Projection matrices	394	(1)
A.4.2 Finding an appropriate projection matrix	395	(1)
A.4.3 Projections in more general inner product spaces	395	(1)
A.5 Constrained maximization of a quadratic function	396	(2)
A.5.1 The finite-dimensional case	396	(1)
A.5.2 The problem in a more general space	396	(1)
A.5.3 Generalized eigenproblems	397	(1)
A.6 Kronecker Products	398	(1)
A.7 The multivariate linear model	399	(2)
A.7.1 Linear models from a transformation perspective	399	(1)
A.7.2 The least squares solution for B	400	(1)
A.8 Regularizing the multivariate linear model	401	(4)
A.8.1 Definition of regularization	401	(1)
A.8.2 Hard-edged constraints	401	(1)
A.8.3 Soft-edged constraints	402	(3)
References	405	(14)
Index	419