Statistical Methods in the Atmospheric Sciences: An Introduction

by Wilks, Daniel S.

• ISBN13: 9780127519654
• ISBN10: 0127519653
• Format: Hardcover
• Copyright: 1995-01-01
• Publisher: Academic Pr

Summary

This book introduces and explains the statistical methods used to describe, analyze, test, and forecast atmospheric data. It will be useful to students, scientists, and other professionals who seek to make sense of the scientific literature in meteorology, climatology, or other geophysical disciplines, or to understand and communicate what their atmospheric data sets have to say. The book includes chapters on exploratory data analysis, probability distributions, hypothesis testing, statistical weather forecasting, forecast verification, time(series analysis, and multivariate data analysis. Worked examples, exercises, and illustrations facilitate understanding of the material; an extensive and up-to-date list of references allows the reader to pursue selected topics in greater depth.

Table of Contents

Preface	xi
Introduction
What Is Statistics?	1	(1)
Descriptive and Inferential Statistics	2	(1)
Uncertainty about the Atmosphere	2	(2)
An Aside on Notation	4	(2)
Review of Probability
Background	6	(1)
The Elements of Probability	7	(2)
Events	7	(1)
The Sample Space	7	(1)
The Axioms of Probability	8	(1)
The Meaning of Probability	9	(1)
Frequency Interpretation	9	(1)
Bayesian (Subjective) Interpretation	9	(1)
Some Properties of Probability	10	(11)
Domain, Subsets, Complements, and Unions	10	(2)
Conditional Probability	12	(2)
Independence	14	(2)
Law of Total Probability	16	(1)
Bayes' Theorem	17	(2)
Exercises	19	(2)
Empirical Distributions and Exploratory Data Analysis
Background	21	(3)
Robustness and Resistance	21	(1)
Quantiles	22	(2)
Numerical Summary Measures	24	(3)
Location	24	(1)
Spread	25	(1)
Symmetry	26	(1)
Graphical Summary Techniques	27	(9)
Stem-and-Leaf Display	27	(2)
Boxplots	29	(1)
Schematic Plots	30	(3)
Other Boxplot Variants	33	(1)
Histograms	33	(1)
Cumulative Frequency Distributions	34	(2)
Reexpression	36	(8)
Power Transformations	37	(4)
Standardized Anomalies	41	(3)
Exploratory Techniques for Paired Data	44	(10)
Scatterplots	44	(1)
Pearson (``Ordinary'') Correlation	45	(5)
Rank Correlation	50	(1)
Serial Correlation	51	(2)
Autocorrelation Function	53	(1)
Exploratory Techniques for Higher-Dimensional Data	54	(10)
The Correlation Matrix	55	(2)
The Scatterplot Matrix	57	(2)
Correlation Maps	59	(3)
Exercises	62	(2)
Theoretical Probability Distributions
Background	64	(2)
What Is a Theoretical Distribution?	65	(1)
Parameters versus Statistics	65	(1)
Discrete versus Continuous Distributions	66	(1)
Discrete Distributions	66	(8)
Binomial Distribution	66	(4)
Geometric Distribution	70	(1)
Poisson Distribution	71	(3)
Statistical Expectations	74	(2)
Expected Value of a Random Variable	74	(1)
Expected Value of a Function of a Random Variable	74	(2)
Continuous Distributions	76	(23)
Distribution Functions and Expected Values	76	(3)
Gaussian Distribution	79	(7)
Gamma Distribution	86	(7)
Weibull Distribution	93	(2)
Lognormal Distribution	95	(1)
Beta Distribution	95	(2)
Gumbel Distribution	97	(2)
Multivariate Probability Distributions	99	(5)
Bivariate Normal Distribution	99	(3)
Multivariate Normal Distribution	102	(2)
Qualitative Assessments of the Goodness of Fit	104	(4)
Superposition of a Fitted Theoretical Distribution and Data Histogram	104	(2)
Probability Plots	106	(2)
Parameter Fitting Using Maximum Likelihood	108	(6)
Exercises	111	(3)
Hypothesis Testing
Background	114	(7)
Parametric versus Nonparametric Tests	114	(1)
The Sampling Distribution	115	(1)
The Elements of Any Hypothesis Test	115	(1)
Test Levels and p Values	116	(1)
Error Types and the Power of a Test	116	(1)
One-Sided versus Two-Sided Tests	117	(1)
Confidence Intervals: Inverting Hypothesis Tests	118	(3)
Some Parametric Tests	121	(16)
One-Sample t Test	121	(1)
Tests for Differences of Mean under Independence	122	(3)
Tests for Differences of Mean under Serial Dependence	125	(4)
Goodness-of-Fit Tests	129	(6)
Likelihood Ratio Test	135	(2)
Nonparametric Tests	137	(14)
``Classical'' Nonparametric Tests for Location	138	(7)
Resampling Tests	145	(6)
Field Significance and Multiplicity	151	(8)
The Multiplicity Problem for Independent Tests	151	(1)
Field Significance Given Spatial Correlation	152	(5)
Exercises	157	(2)
Statistical Weather Forecasting
Background	159	(1)
Review of Least-Squares Regression	160	(21)
Simple Linear Regression	160	(3)
Distribution of the Residuals	163	(2)
The Analysis-of-Variance Table	165	(1)
Goodness-of-Fit Measures	166	(2)
Sampling Distributions of the Regression Coefficients	168	(3)
Examining Residuals	171	(4)
Prediction Intervals	175	(6)
Objective Forecasts---Without NWP	181	(18)
Stratification and Compositing	182	(1)
When the Predictand Is a Probability	182	(3)
Predictor Selection	185	(3)
Screening Regression	188	(1)
Stopping Rules	189	(5)
Cross-Validation	194	(4)
Analog Forecasting	198	(1)
Objective Forecasts---With NWP	199	(11)
Perfect Prog Forecasts	200	(1)
Model Output Statistics (MOS) Forecasts	201	(1)
Comparisons between the Classical, Perfect Prog, and MOS Approaches	202	(7)
Operational MOS Forecasts	209	(1)
Probabilistic Field (Ensemble) Forecasts	210	(11)
Stochastic Dynamic Forecasts	211	(2)
Ensemble Forecasting	213	(2)
Choosing Initial Ensemble Members	215	(1)
Ensemble Averaging	216	(1)
Dispersion of the Ensemble	217	(4)
Subjective Probability Forecasts	221	(12)
The Subjective Distribution	223	(2)
Credible Interval Forecasts	225	(2)
Assessing Discrete Probabilities	227	(2)
Assessing Continuous Distributions	229	(1)
Exercises	230	(3)
Forecast Verification
Background	233	(5)
Purposes of Forecast Verification	233	(1)
Factorization of the Joint Distribution of Forecasts and Observations	234	(1)
Scalar Attributes of Forecast Performance	235	(2)
Forecast Skill	237	(1)
Categorical Forecasts of Discrete Predictands	238	(12)
The Contingency Table	238	(1)
Accuracy Measures for Binary Forecasts	239	(2)
Bias	241	(1)
Accuracy Measures for Multicategory Forecasts	242	(2)
Conversion of Probabilistic to Categorical Forecasts	244	(4)
Skill Measures for Contingency Tables	248	(2)
Categorical Forecasts of Continuous Predictands	250	(8)
Conditional Quantile Plots	251	(1)
Scalar Accuracy Measures	251	(4)
Skill Scores	255	(2)
Skill in Probability Space	257	(1)
Probability Forecasts	258	(14)
Dichotomous Events	259	(1)
Brier Score	259	(1)
Algebraic Decomposition of the Brier Score	260	(3)
The Attributes Diagram	263	(2)
The Reliability Diagram	265	(2)
Hedging and Strictly Proper Scoring Rules	267	(1)
Probability Forecasts for Multicategory Events	268	(1)
Ranked Probability Score	269	(3)
Categorical Forecasts of Fields	272	(12)
S1 Score	274	(1)
Mean-Squared Error	275	(2)
Anomaly Correlation	277	(4)
Exercises	281	(3)
Time Series
Background	284	(3)
Stationarity	284	(1)
Time-Series Models	285	(1)
Time-Domain versus Frequency-Domain Approaches	286	(1)
Time Domain. I. Discrete Data	287	(15)
Markov Chains	287	(1)
Two-State, First-Order Markov Chains	288	(4)
Test for Independence versus First-Order Serial Dependence	292	(3)
Some Applications of Two-State Markov Chains	295	(2)
Multistate Markov Chains	297	(1)
Higher-Order Markov Chains	298	(2)
Deciding among Alternative Orders of Markov Chains	300	(2)
Time Domain. II. Continuous Data	302	(23)
First-Order Autoregression	302	(6)
Higher-Order Autoregression	308	(5)
Order Selection Criteria	313	(2)
The Variance of a Time Average	315	(3)
Autoregressive Moving-Average Models	318	(1)
Simulation and Forecasting with Time-Domain Models	319	(4)
Multivariate Autoregressions	323	(2)
Frequency Domain. I. Harmonic Analysis	325	(16)
Cosine and Sine Functions	325	(1)
Representing a Simple Time Series with a Harmonic Function	326	(4)
Estimation of the Amplitude and Phase of a Single Harmonic	330	(3)
Higher Harmonics	333	(3)
Harmonic Dials	336	(3)
The Harmonic Functions as Uncorrelated Regression Predictors	339	(2)
Frequency Domain. II. Spectral Analysis	341	(18)
The Periodogram or Fourier Line Spectrum	341	(5)
Computing Spectra	346	(1)
Aliasing	347	(3)
Sampling Properties of Spectral Estimates	350	(2)
Theoretical Spectra of Autoregressive Models	352	(5)
Exercises	357	(2)
Methods for Multivariate Data
Background	359	(1)
Matrix Algebra Notation	360	(13)
Vectors	360	(2)
Matrices	362	(11)
Principal-Component (EOF) Analysis	373	(25)
Basics of PCA	374	(4)
Truncation of the Principal Components	378	(2)
How Many Principal Components Should Be Retained?	380	(3)
PCA Based on the Covariance Matrix versus the Correlation Matrix	383	(3)
Application of PCA to Fields	386	(7)
Rotation of the Eigenvectors	393	(1)
The Varied Terminology of PCA	394	(3)
Scaling Conventions in PCA	397	(1)
Canonical Correlation Analysis	398	(10)
Canonical Variates	399	(1)
Computation of CCA	400	(3)
CCA Applied to Fields	403	(1)
Combining PCA and CCA	403	(5)
Discriminant Analysis	408	(11)
Linear Discriminant Analysis	409	(6)
Multiple Discriminant Analysis	415	(4)
Cluster Analysis	419	(10)
Distance Measures and Clustering Methods	420	(3)
The Dendrogram, or Tree Diagram	423	(1)
How Many Clusters?	424	(3)
Exercises	427	(2)
Appendix A: Example Data Sets	429	(3)
Appendix B: Probability Tables	432	(7)
Appendix C: Answers to Exercises	439	(5)
References	444	(11)
Index	455	(10)
International Geophysics Series	465