Fat-Tailed Distributions Data, Diagnostics and Dependence, Volume 1

by Cooke, Roger M.; Nieboer, Daan; Misiewicz, Jolanta

• ISBN13: 9781848217928
• ISBN10: 1848217927
• Edition: 1st
• Format: Hardcover
• Copyright: 2014-12-03
• Publisher: Wiley-ISTE

Summary

This title is written for the numerate nonspecialist, and hopes to serve three purposes. First it gathers mathematical material from diverse but related fields of order statistics, records, extreme value theory, majorization, regular variation and subexponentiality. All of these are relevant for understanding fat tails, but they are not, to our knowledge, brought together in a single source for the target readership. Proofs that give insight are included, but for most fussy calculations the reader is referred to the excellent sources referenced in the text. Multivariate extremes are not treated. This allows us to present material spread over hundreds of pages in specialist texts in twenty pages. Chapter 5 develops new material on heavy tail diagnostics and gives more mathematical detail. Since variances and covariances may not exist for heavy tailed joint distributions, Chapter 6 reviews dependence concepts for certain classes of heavy tailed joint distributions, with a view to regressing heavy tailed variables.

Second, it presents a new measure of obesity. The most popular definitions in terms of regular variation and subexponentiality invoke putative properties that hold at infinity, and this complicates any empirical estimate. Each definition captures some but not all of the intuitions associated with tail heaviness. Chapter 5 studies two candidate indices of tail heaviness based on the tendency of the mean excess plot to collapse as data are aggregated. The probability that the largest value is more than twice the second largest has intuitive appeal but its estimator has very poor accuracy. The Obesity index is defined for a positive random variable X as:

Ob(X) = P (X1 +X4 > X2 +X3|X1 ≤ X2 ≤ X3 ≤ X4), Xi independent copies of X.

For empirical distributions, obesity is defined by bootstrapping. This index reasonably captures intuitions of tail heaviness. Among its properties, if α > 1 then Ob(X) < Ob(Xα). However, it does not completely mimic the tail index of regularly varying distributions, or the extreme value index. A Weibull distribution with shape 1/4 is more obese than a Pareto distribution with tail index 1, even though this Pareto has infinite mean and the Weibull’s moments are all finite. Chapter 5 explores properties of the Obesity index.

Third and most important, we hope to convince the reader that fat tail phenomena pose real problems; they are really out there and they seriously challenge our usual ways of thinking about historical averages, outliers, trends, regression coefficients and confidence bounds among many other things. Data on flood insurance claims, crop loss claims, hospital discharge bills, precipitation and damages and fatalities from natural catastrophes drive this point home. While most fat tailed distributions are ”bad”, research in fat tails is one distribution whose tail will hopefully get fatter.

Author Biography

Roger M. Cooke, Chauncey Starr Chair for Risk Analysis Resources for the Future, USA and Dept. Math. TU Delft, Netherlands 

Daan Nieboer, Erasmus Universiteit Rotterdam, Department of Public Health (MGZ), Netherlands 

Jolanta Misiewicz, Professor (Full), Warsaw University of Technology, Faculty of Mathematics and Information Science, Mazowieckie, Poland

Table of Contents

INTRODUCTION ix

CHAPTER 1. FATNESS OF TAIL 1

1.1. Fat tail heuristics 1

1.2. History and data 4

1.2.1. US flood insurance claims 4

1.2.2. US crop loss 5

1.2.3. US damages and fatalities from natural disasters 5

1.2.4. US hospital discharge bills 6

1.2.5. G-Econ data 6

1.3. Diagnostics for heavy-tailed phenomena 6

1.3.1. Historical averages 7

1.3.2. Records 8

1.3.3. Mean excess 11

1.3.4. Sum convergence: self-similar or normal 12

1.3.5. Estimating the tail index 15

1.3.6. The obesity index 20

1.4. Relation to reliability theory 24

1.5. Conclusion and overview of the technical chapters 25

CHAPTER 2. ORDER STATISTICS 27

2.1. Distribution of order statistics 27

2.2. Conditional distribution 32

2.3. Representations for order statistics 33

2.4. Functions of order statistics 36

2.4.1. Partial sums 36

2.4.2. Ratio between order statistics 37

CHAPTER 3. RECORDS 41

3.1. Standard record value processes 41

3.2. Distribution of record values 42

3.3. Record times and related statistics 44

3.4. k-records 46

CHAPTER 4. REGULARLY VARYING AND SUBEXPONENTIAL DISTRIBUTIONS 49

4.1. Classes of heavy-tailed distributions 50

4.1.1. Regularly varying distribution functions 50

4.1.2. Subexponential distribution functions 55

4.1.3. Related classes of heavy-tailed distributions 58

4.2. Mean excess function 59

4.2.1. Properties of the mean excess function 60

CHAPTER 5. INDICES AND DIAGNOSTICS OF TAIL HEAVINESS 65

5.1. Self-similarity 66

5.1.1. Distribution of the ratio between order statistics 69

5.2. The ratio as index 76

5.3. The obesity index 80

5.3.1. Theory of majorization 85

5.3.2. The obesity index of selected data sets 91

CHAPTER 6. DEPENDENCE 95

6.1. Definition and main properties 95

6.2. Isotropic distributions 96

6.3. Pseudo-isotropic distributions 100

6.3.1. Covariation as a measure of dependence for essentially heavy-tail jointly pseudo-isotropic variables 104

6.3.2. Codifference 109

6.3.3. The linear regression model for essentially heavy-tail distribution 110

CONCLUSIONS AND PERSPECTIVES 115

BIBLIOGRAPHY 119

INDEX 123