Data Analysis What Can Be Learned From the Past 50 Years

by Huber, Peter J.

• ISBN13: 9781118010648
• ISBN10: 1118010647
• Edition: 1st
• Format: Hardcover
• Copyright: 2011-04-11
• Publisher: Wiley

Summary

This book explores the many provocative questions concerning the fundamentals of data analysis. It is based on the time-tested experience of one of the gurus of the subject matter. Why should one study data analysis? How should it be taught? What techniques work best, and for whom? How valid are the results? How much data should be tested? Which machine languages should be used, if used at all? Emphasis on apprenticeship (through hands-on case studies) and anecdotes (through real-life applications) are the tools that Peter J. Huber uses in this volume. Concern with specific statistical techniques is not of immediate value; rather, questions of strategy when to use which technique are employed. Central to the discussion is an understanding of the significance of massive (or robust) data sets, the implementation of languages, and the use of models. Each is sprinkled with an ample number of examples and case studies. Personal practices, various pitfalls, and existing controversies are presented when applicable. The book serves as an excellent philosophical and historical companion to any present-day text in data analysis, robust statistics, data mining, statistical learning, or computational statistics.

Author Biography

Peter J. Huber, PhD, is a world-renowned statistician who has published four books and more than seventy journal articles in the areas of statistics and data analysis. He has held academic positions at Harvard University, Massachusetts Institute of. Technology, Cornell University, and ETH Zurich (Switzerland), and has made significant research contributions in the areas of robust statistics, computational statistics, and strategies in data analysis. A Fellow of the Institute of Mathematical Statistics and the American Academy of Arts and Sciences, Dr. Huber is the coauthor of Robust Statistics, Second Edition, also published by Wiley.

Table of Contents

Preface	p. xi
What is Data Analysis?	p. 1
Tukey's 1962 paper	p. 3
The Path of Statistics	p. 5
Strategy Issues in Data Analysis	p. 11
Strategy in Data Analysis	p. 11
Philosophical issues	p. 13
On the theory of data analysis and its teaching	p. 14
Science and data analysis	p. 15
Economy of forces	p. 16
Issues of size	p. 17
Strategic planning	p. 21
Planning the data collection	p. 21
Choice of data and methods.	p. 22
Systematic and random errors	p. 23
Strategic reserves	p. 24
Human factors	p. 25
The stages of data analysis	p. 26
Inspection	p. 26
Error checking	p. 27
Modification	p. 30
Comparison	p. 30
Modeling and Model fitting	p. 30
Simulation	p. 31
What-if analyses	p. 32
Interpretation	p. 32
Presentation of conclusions	p. 32
Tools required for strategy reasons	p. 33
Ad hoc programming	p. 33
Graphics	p. 34
Record keeping	p. 35
Creating and keeping order	p. 35
Massive Data Sets	p. 37
Introduction	p. 38
Disclosure: Personal experiences	p. 39
What is massive? A classification of size	p. 39
Obstacles to scaling	p. 40
Human limitations: visualization	p. 40
Human - machine interactions	p. 41
Storage requirements	p. 41
Computational complexity	p. 42
Conclusions	p. 43
On the structure of large data sets	p. 43
Types of data	p. 43
How do data sets grow?	p. 44
On data organization	p. 44
Derived data sets	p. 45
Data base management and related issues	p. 46
Data archiving	p. 48
The stages of a data analysis	p. 49
Planning the data collection	p. 49
Actual collection	p. 50
Data access	p. 50
Initial data checking	p. 50
Data analysis proper	p. 51
The final product: presentation of arguments and conclusions	p. 51
Examples and some thoughts on strategy	p. 52
Volume reduction	p. 55
Supercomputers and software challenges	p. 56
When do we need a Concorde?	p. 57
General Purpose Data Analysis and Supercomputers	p. 57
Languages, Programming Environments and Data-based Prototyping	p. 58
Summary of conclusions	p. 59
Languages for Data Analysis	p. 61
Goals and purposes	p. 62
Natural languages and computing languages	p. 64
Natural languages	p. 64
Batch languages	p. 65
Immediate languages	p. 67
Language and literature	p. 68
Object orientation and related structural issues	p. 69
Extremism and compromises, slogans and reality	p. 71
Some conclusions	p. 73
Interface issues	p. 74
The command line interface	p. 75
The menu interface	p. 78
The batch interface and programming environments	p. 80
Some personal experiences	p. 81
Miscellaneous issues	p. 82
On building blocks	p. 82
On the scope of names	p. 83
On notation	p. 83
Book-keeping problems	p. 84
Requirements for a general purpose immediate language	p. 85
Approximate Models	p. 89
Models	p. 89
Bayesian modeling	p. 92
Mathematical statistics and approximate models	p. 94
Statistical significance and physical relevance	p. 96
Judicious use of a wrong model	p. 97
Composite models	p. 98
Modeling the length of day	p. 99
The role of simulation	p. 111
Summary of conclusions	p. 112
Pitfalls	p. 113
Simpson's paradox	p. 114
Missing data	p. 116
The Case of the Babylonian Lunar Six	p. 118
X-ray crystallography	p. 126
Regression of Y on X or of X on Y?	p. 129
Create order in data	p. 133
General considerations	p. 134
Principal component methods	p. 135
Principal component methods: Jury data	p. 137
Multidimensional scaling	p. 145
Multidimensional scaling: the method	p. 145
Multidimensional scaling: a synthetic example	p. 145
Multidimensional scaling: map reconstruction	p. 147
Correspondence analysis	p. 147
Correspondence analysis: the method	p. 147
Kültepe eponyms	p. 148
Further examples: marketing and Shakespearean plays	p. 156
Multidimensional scaling vs. Correspondence analysis	p. 160
Hodson's grave data	p. 162
Plato data	p. 168
More case studies	p. 177
A nutshell example	p. 178
Shape invariant modeling	p. 182
Comparison of point configurations	p. 184
The cyclodecane conformation	p. 186
The Thomson problem	p. 189
Notes on numerical optimization	p. 190
References	p. 195
Index	p. 205
Table of Contents provided by Ingram. All Rights Reserved.