Asymptotic Theory of Statistics and Probability

by Dasgupta, Anirban

• ISBN13: 9780387759708
• ISBN10: 0387759700
• Format: Hardcover
• Copyright: 2008-03-14
• Publisher: Springer Nature

Summary

This book is an encyclopedic treatment of classic as well as contemporary large sample theory, dealing with both statistical problems and probabilistic issues and tools. It is written in an extremely lucid style, with an emphasis on the conceptual discussion of the importance of a problem and the impact and relevance of the theorems. The book has 34 chapters over a wide range of topics, nearly 600 exercises for practice and instruction, and another 300 worked out examples. It also includes a large compendium of 300 useful inequalities on probability, linear algebra, and analysis that are collected together from numerous sources, as an invaluable reference for researchers in statistics, probability, and mathematics. It can be used as a graduate text, as a versatile research reference, as a source for independent reading on a wide assembly of topics, and as a window to learning the latest developments in contemporary topics. The book is unique in its detailed coverage of fundamental topics such as central limit theorems in numerous setups, likelihood based methods, goodness of fit, higher order asymptotics, as well as of the most modern topics such as the bootstrap, dependent data, Bayesian asymptotics, nonparametric density estimation, mixture models, and multiple testing and false discovery. It provides extensive bibliographic references on all topics that include very recent publications.

Author Biography

Anirban DasGupta is Professor of Statistics at Purdue University.

Table of Contents

Basic Convergence Concepts and Theorems	p. 1
Some Basic Notation and Convergence Theorems	p. 1
Three Series Theorem and Kolmogorov's Zero-One Law	p. 6
Central Limit Theorem and Law of the Iterated Logarithm	p. 7
Further Illustrative Examples	p. 10
Exercises	p. 12
References	p. 16
Metrics, Information Theory, Convergence, and Poisson Approximations	p. 19
Some Common Metrics and Their Usefulness	p. 20
Convergence in Total Variation and Further Useful Formulas	p. 22
Information-Theoretic Distances, de Bruijn's Identity, and Relations to Convergence	p. 24
Poisson Approximations	p. 28
Exercises	p. 31
References	p. 33
More General Weak and Strong Laws and the Delta Theorem	p. 35
General LLNs and Uniform Strong Law	p. 35
Median Centering and Kesten's Theorem	p. 38
The Ergodic Theorem	p. 39
Delta Theorem and Examples	p. 40
Approximation of Moments	p. 44
Exercises	p. 45
References	p. 47
Transformations	p. 49
Variance-Stabilizing Transformations	p. 50
Examples	p. 51
Bias Correction of the VST	p. 54
Symmetrizing Transformations	p. 57
VST or Symmetrizing Transform?	p. 59
Exercises	p. 59
References	p. 61
More General Central Limit Theorems	p. 63
The Independent Not IID Case and a Key Example	p. 63
CLT without a Variance	p. 66
Combinatorial CLT	p. 67
CLT for Exchangeable Sequences	p. 68
CLT for a Random Number of Summands	p. 70
Infinite Divisibility and Stable Laws	p. 71
Exercises	p. 77
References	p. 80
Moment Convergence and Uniform Integrability	p. 83
Basic Results	p. 83
The Moment Problem	p. 85
Exercises	p. 88
References	p. 89
Sample Percentiles and Order Statistics	p. 91
Asymptotic Distribution of One Order Statistic	p. 92
Joint Asymptotic Distribution of Several Order Statistics	p. 93
Bahadur Representations	p. 94
Confidence Intervals for Quantiles	p. 96
Regression Quantiles	p. 97
Exercises	p. 98
References	p. 100
Sample Extremes	p. 101
Sufficient Conditions	p. 101
Characterizations	p. 105
Limiting Distribution of the Sample Range	p. 107
Multiplicative Strong Law	p. 108
Additive Strong Law	p. 109
Dependent Sequences	p. 111
Exercises	p. 114
References	p. 116
Central Limit Theorems for Dependent Sequences	p. 119
Stationary m-dependence	p. 119
Sampling without Replacement	p. 121
Martingales and Examples	p. 123
The Martingale and Reverse Martingale CLTs	p. 126
Exercises	p. 127
References	p. 129
Central Limit Theorem for Markov Chains	p. 131
Notation and Basic Definitions	p. 131
Normal Limits	p. 132
Nonnormal Limits	p. 135
Convergence to Stationarity: Diaconis-Stroock-Fill Bound	p. 135
Exercises	p. 137
References	p. 139
Accuracy of Central Limit Theorems	p. 141
Uniform Bounds: Berry-Esseen Inequality	p. 142
Local Bounds	p. 144
The Multidimensional Berry-Esseen Theorems	p. 145
Other Statistics	p. 146
Exercises	p. 147
References	p. 149
Invariance Principles	p. 151
Motivating Examples	p. 152
Two Relevant Gaussian Processes	p. 153
The Erdös-Kac Invariance Principle	p. 156
Invariance Principles, Donsker's Theorem, and the KMT Construction	p. 157
Invariance Principle for Empirical Processes	p. 161
Extensions of Donsker's Principle and Vapnik-Chervonenkis Classes	p. 163
Glivenko-Cantelli Theorem for VC Classes	p. 164
CLTs for Empirical Measures and Applications	p. 167
Notation and Formulation	p. 168
Entropy Bounds and Specific CLTs	p. 169
Dependent Sequences: Martingales, Mixing, and Short-Range Dependence	p. 172
Weighted Empirical Processes and Approximations	p. 175
Exercises	p. 178
References	p. 180
Edgeworth Expansions and Cumulants	p. 185
Expansion for Means	p. 186
Using the Edgeworth Expansion	p. 188
Edgeworth Expansion for Sample Percentiles	p. 189
Edgeworth Expansion for the t-statistic	p. 190
Cornish-Fisher Expansions	p. 192
Cumulants and Fisher's k-statistics	p. 194
Exercises	p. 198
References	p. 200
Saddlepoint Approximations	p. 203
Approximate Evaluation of Integrals	p. 204
Density of Means and Exponential Tilting	p. 208
Derivation by Edgeworth Expansion and Exponential Tilting	p. 210
Some Examples	p. 211
Application to Exponential Family and the Magic Formula	p. 213
Tail Area Approximation and the Lugannani-Rice Formula	p. 213
Edgeworth vs. Saddlepoint vs. Chi-square Approximation	p. 217
Tail Areas for Sample Percentiles	p. 218
Quantile Approximation and Inverting the Lugannani-Rice Formula	p. 219
The Multidimensional Case	p. 221
Exercises	p. 222
References	p. 223
U-statistics	p. 225
Examples	p. 226
Asymptotic Distribution of U-statistics	p. 227
Moments of U-statistics and the Martingale Structure	p. 229
Edgeworth Expansions	p. 230
Nonnormal Limits	p. 232
Exercises	p. 232
References	p. 234
Maximum Likelihood Estimates	p. 235
Some Examples	p. 235
Inconsistent MLEs	p. 239
MLEs in the Exponential Family	p. 240
More General Cases and Asymptotic Normality	p. 242
Observed and Expected Fisher Information	p. 244
Edgeworth Expansions for MLEs	p. 245
Asymptotic Optimality of the MLE and Superefficiency	p. 247
Ha&jacute;ek-LeCam Convolution Theorem	p. 249
Loss of Information and Efron's Curvature	p. 251
Exercises	p. 253
References	p. 258
M Estimates	p. 259
Examples	p. 260
Consistency and Asymptotic Normality	p. 262
Bahadur Expansion of M Estimates	p. 265
Exercises	p. 267
References	p. 268
The Trimmed Mean	p. 271
Asymptotic Distribution and the Bahadur Representation	p. 271
Lower Bounds on Efficiencies	p. 273
Multivariate Trimmed Mean	p. 273
The 10-20-30-40 Rule	p. 275
Exercises	p. 277
References	p. 278
Multivariate Location Parameter and Multivariate Medians	p. 279
Notions of Symmetry of Multivariate Data	p. 279
Multivariate Medians	p. 280
Asymptotic Theory for Multivariate Medians	p. 282
The Asymptotic Covariance Matrix	p. 283
Asymptotic Covariance Matrix of the L1 Median	p. 284
Exercises	p. 287
References	p. 288
Bayes Procedures and Posterior Distributions	p. 289
Motivating Examples	p. 290
Bernstein-von Mises Theorem	p. 291
Posterior Expansions	p. 294
Expansions for Posterior Mean, Variance, and Percentiles	p. 298
The Tierney-Kadane Approximations	p. 300
Frequentist Approximation of Posterior Summaries	p. 302
Consistency of Posteriors	p. 304
The Difference between Bayes Estimates and the MLE	p. 305
Using the Brown Identity to Obtain Bayesian Asymptotics	p. 306
Testing	p. 311
Interval and Set Estimation	p. 312
Infinite-Dimensional Problems and the Diaconis-Freedman Results	p. 314
Exercises	p. 317
References	p. 320
Testing Problems	p. 323
Likelihood Ratio Tests	p. 323
Examples	p. 324
Asymptotic Theory of Likelihood Ratio Test Statistics	p. 334
Distribution under Alternatives	p. 336
Bartlett Correction	p. 338
The Wald and Rao Score Tests	p. 339
Likelihood Ratio Confidence Intervals	p. 340
Exercises	p. 342
References	p. 344
Asymptotic Efficiency in Testing	p. 347
Pitman Efficiencies	p. 348
Bahadur Slopes and Bahadur Efficiency	p. 353
Bahadur Slopes of U-statistics	p. 361
Exercises	p. 362
References	p. 363
Some General Large-Deviation Results	p. 365
Generalization of the Cramér-Chernoff Theorem	p. 365
The Gärtner-Ellis Theorem	p. 367
Large Deviation for Local Limit Theorems	p. 370
Exercises	p. 374
References	p. 375
Classical Nonparametrics	p. 377
Some Early Illustrative Examples	p. 378
Sign Test	p. 380
Consistency of the Sign Test	p. 381
Wilcoxon Signed-Rank Test	p. 383
Robustness of the t Confidence Interval	p. 388
The Bahadur-Savage Theorem	p. 393
Kolmogorov-Smirnov and Anderson Confidence Intervals	p. 394
Hodges-Lehmann Confidence Interval	p. 396
Power of the Wilcoxon Test	p. 397
Exercises	p. 398
References	p. 399
Two-Sample Problems	p. 401
Behrens-Fisher Problem	p. 402
Wilcoxon Rank Sum and Mann-Whitney Test	p. 405
Two-Sample U-statistics and Power Approximations	p. 408
Hettmansperger's Generalization	p. 410
The Nonparametric Behrens-Fisher Problem	p. 412
Robustness of the Mann-Whitney Test	p. 415
Exercises	p. 417
References	p. 418
Goodness of Fit	p. 421
Kolmogorov-Smirnov and Other Tests Based on Fn	p. 422
Computational Formulas	p. 422
Some Heuristics	p. 423
Asymptotic Null Distributions of Dn, Cn, An, and Vn	p. 424
Consistency and Distributions under Alternatives	p. 425
Finite Sample Distributions and Other EDF-Based Tests	p. 426
The Berk-Jones Procedure	p. 428
ϕ-Divergences and the Jager-Wellner Tests	p. 429
The Two-Sample Case	p. 431
Tests for Normality	p. 434
Exercises	p. 436
References	p. 438
Cm-square Tests for Goodness of Fit	p. 441
The Pearson ¿2 Test	p. 441
Asymptotic Distribution of Pearson's Chi-square	p. 442
Asymptotic Distribution under Alternatives and Consistency	p. 442
Choice of k	p. 443
Recommendation of Mann and Wald	p. 445
Power at Local Alternatives and Choice of k	p. 445
Exercises	p. 448
References	p. 449
Goodness of Fit with Estimated Parameters	p. 451
Preliminary Analysis by Stochastic Expansion	p. 452
Asymptotic Distribution of EDF-Based Statistics for Composite Nulls	p. 453
Chi-square Tests with Estimated Parameters and the Chernoff-Lehmann Result	p. 455
Chi-square Tests with Random Cells	p. 457
Exercises	p. 457
References	p. 458
The Bootstrap	p. 461
Bootstrap Distribution and the Meaning of Consistency	p. 462
Consistency in the Kolmogorov and Wasserstein Metrics	p. 464
Delta Theorem for the Bootstrap	p. 468
Second-Order Accuracy of the Bootstrap	p. 468
Other Statistics	p. 471
Some Numerical Examples	p. 473
Failure of the Bootstrap	p. 475
m out of n Bootstrap	p. 476
Bootstrap Confidence Intervals	p. 478
Some Numerical Examples	p. 482
Bootstrap Confidence Intervals for Quantiles	p. 483
Bootstrap in Regression	p. 483
Residual Bootstrap	p. 484
Confidence Intervals	p. 485
Distribution Estimates in Regression	p. 486
Bootstrap for Dependent Data	p. 487
Consistent Bootstrap for Stationary Autoregression	p. 488
Block Bootstrap Methods	p. 489
Optimal Block Length	p. 491
Exercises	p. 492
References	p. 495
Jackknife	p. 499
Notation and Motivating Examples	p. 499
Bias Correction by the Jackknife	p. 502
Variance Estimation	p. 503
Delete-d Jackknife and von Mises Functionals	p. 504
A Numerical Example	p. 507
Jackknife Histogram	p. 508
Exercises	p. 511
References	p. 512
Permutation Tests	p. 513
General Permutation Tests and Basic Group Theory	p. 514
Exact Similarity of Permutation Tests	p. 516
Power of Permutation Tests	p. 519
Exercises	p. 520
References	p. 521
Density Estimation	p. 523
Basic Terminology and Some Popular Methods	p. 523
Measures of the Quality of Density Estimates	p. 526
Certain Negative Results	p. 526
Minimaxity Criterion	p. 529
Performance of Some Popular Methods: A Preview	p. 530
Rate of Convergence of Histograms	p. 531
Consistency of Kernel Estimates	p. 533
Order of Optimal Bandwidth and Superkernels	p. 535
The Epanechnikov Kernel	p. 538
Choice of Bandwidth by Cross Validation	p. 539
Maximum Likelihood CV	p. 540
Least Squares CV	p. 542
Stone's Result	p. 544
Comparison of Bandwidth Selectors and Recommendations	p. 545
L1 Optimal Bandwidths	p. 547
Variable Bandwidths	p. 548
Strong Uniform Consistency and Confidence Bands	p. 550
Multivariate Density Estimation and Curse of Dimensionality	p. 552
Kernel Estimates and Optimal Bandwidths	p. 556
Estimating a Unimodal Density and the Grenander Estimate	p. 558
The Grenander Estimate	p. 558
Mode Estimation and Chernoff's Distribution	p. 561
Exercises	p. 564
References	p. 568
Mixture Models and Nonparametric Deconvolution	p. 571
Mixtures as Dense Families	p. 572
z Distributions and Other Gaussian Mixtures as Useful Models	p. 573
Estimation Methods and Their Properties: Finite Mixtures	p. 577
Maximum Likelihood	p. 577
Minimum Distance Method	p. 578
Moment Estimates	p. 579
Estimation in General Mixtures	p. 580
Strong Consistency and Weak Convergence of the MLE	p. 582
Convergence Rates for Finite Mixtures and Nonparametric Deconvolution	p. 584
Nonparametric Deconvolution	p. 585
Exercises	p. 587
References	p. 589
High-Dimensional Inference and False Discovery	p. 593
Chi-square Tests with Many Cells and Sparse Multinomials	p. 594
Regression Models with Many Parameters: The Portnoy Paradigm	p. 597
Multiple Testing and False Discovery: Early Developments	p. 599
False Discovery: Definitions, Control, and the Benjamini-Hochberg Rule	p. 601
Distribution Theory for False Discoveries and Poisson and First-Passage Asymptotics	p. 604
Newer FDR Controlling Procedures	p. 606
Storey-Taylor-Siegmund Rule	p. 606
Higher Criticism and the Donoho-Jin Developments	p. 608
False Nondiscovery and Decision Theory Formulation	p. 611
Genovese-Wasserman Procedure	p. 612
Asymptotic Expansions	p. 614
Lower Bounds on the Number of False Hypotheses	p. 616
Bühlmann-Meinshausen-Rice Method	p. 617
The Dependent Case and the Hall-Jin Results	p. 620
Increasing and Multivariate Totally Positive Distributions	p. 620
Higher Criticism under Dependence: Hall-Jin Results	p. 623
Exercises	p. 625
References	p. 628
A Collection of Inequalities in Probability, Linear Algebra, and Analysis	p. 633
Probability Inequalities	p. 633
Improved Bonferroni Inequalities	p. 633
Concentration Inequalities	p. 634
Tail Inequalities for Specific Distributions	p. 639
Inequalities under Unimodality	p. 641
Moment and Monotonicity Inequalities	p. 643
Inequalities in Order Statistics	p. 652
Inequalities for Normal Distributions	p. 655
Inequalities for Binomial and Poisson Distributions	p. 656
Inequalities in the Central Limit Theorem	p. 658
Martingale Inequalities	p. 661
Matrix Inequalities	p. 663
Rank, Determinant, and Trace Inequalities	p. 663
Eigenvalue and Quadratic Form Inequalities	p. 667
Series and Polynomial Inequalities	p. 671
Integral and Derivative Inequalities	p. 675
Glossary of Symbols	p. 689
Index	p. 693
Table of Contents provided by Publisher. All Rights Reserved.