Now re-issued in a new style and format, but with the same reliable content of the Third Edition, Probability and Measure offers advanced students, scientists, and engineers an integrated introduction to measure theory and probability. Retaining the unique approach of the previous editions, this text interweaves material on probability and measure so that probability problems generate an interest in measure theory and measure theory is, then, developed and applied to probability. Probability and Measure provides a thorough coverage of probability, measure, integration, random variables and expected values, convergence of distributions, derivatives and conditional probability, and stochastic processes. In recognition of over thirty years of publication and having impacted tens of thousands of readers, the 2012 Edition features an open and user-friendly design, updated bibliography, improved treatment of Brownian motion, and replacement of queuing theory with ergodic theory. Like its previous editions, this re-issue will be well received by students of mathematics, statistics, economics, and a wide variety of disciplines that require a solid understanding of probability theory. An extensive supplement of additional notes on the problems and chapter commentaries is available at further cost either independently or as a set with the book.

FOREWORD xi

PREFACE xiii

*Patrick Billingsley* 1925–2011 xv

**Chapter1 PROBABILITY 1**

**1. BOREL’S NORMAL NUMBER THEOREM, 1**

The Unit Interval

The Weak Law of Large Numbers

The Strong Law of Large Numbers

Strong Law Versus Weak

Length

The Measure Theory of Diophantine Approximation*

**2. PROBABILITY MEASURES**, **18**

Spaces

Assigning Probabilities

Classes of Sets

Probability Measures

Lebesgue Measure on the Unit Interval

Sequence Space*

Constructing *s*-Fields*

**3. EXISTENCE AND EXTENSION**, **39**

Construction of the Extension

Uniqueness and the *p***–***?* Theorem

Monotone Classes

Lebesgue Measure on the Unit Interval

Completeness

Nonmeasurable Sets

Two Impossibility Theorems*

**4. DENUMERABLE PROBABILITIES**, **53**

General Formulas

Limit Sets

Independent Events

Subfields

The Borel-Cantelli Lemmas

The Zero-One Law

**5. SIMPLE RANDOM VARIABLES**, **72**

Definition

Convergence of Random Variables

Independence

Existence of Independent Sequences

Expected Value

Inequalities

**6. THE LAW OF LARGE NUMBERS**, **90**

The Strong Law

The Weak Law

Bernstein's Theorem

A Refinement of the Second Borel-Cantelli Lemma

**7. GAMBLING SYSTEMS**, **98**

Gambler's Ruin

Selection Systems

Gambling Policies

Bold Play*

Timid Play*

**8. MARKOV CHAINS**, **117**

Definitions

Higher-Order Transitions

An Existence Theorem

Transience and Persistence

Another Criterion for Persistence

Stationary Distributions

Exponential Convergence*

Optimal Stopping*

**9. LARGE DEVIATIONS AND THE LAW OF THE ITERATED LOGARITHM**, **154**

Moment Generating Functions

Large Deviations

Chernoff's Theorem*

The Law of the Iterated Logarithm

**Chapter2 MEASURE 167**

**10. GENERAL MEASURES, 167**

Classes of Sets

Conventions Involving 8

Measures

Uniqueness

**11. OUTER MEASURE**, **174**

Outer Measure

Extension

An Approximation Theorem

**12. MEASURES IN EUCLIDEAN SPACE**, **181**

Lebesgue Measure

Regularity

Specifying Measures on the Line

Specifying Measures in *Rk*

Strange Euclidean Sets*

**13. MEASURABLE FUNCTIONS AND MAPPINGS**, **192**

Measurable Mappings

Mappings into *Rk*

Limits and Measurability

Transformations of Measures

**14. DISTRIBUTION FUNCTIONS**, **198**

Distribution Functions

Exponential Distributions

Weak Convergence

Convergence of Types*

Extremal Distributions*

**Chapter3 INTEGRATION 211**

**15. THE INTEGRAL, 211**

Definition

Nonnegative Functions

Uniqueness

**16. PROPERTIES OF THE INTEGRAL**, **218**

Equalities and Inequalities

Integration to the Limit

Integration over Sets

Densities

Change of Variable

Uniform Integrability

Complex Functions

**17. THE INTEGRAL WITH RESPECT TO LEBESGUE MEASURE**, **234**

The Lebesgue Integral on the Line

The Riemann Integral

The Fundamental Theorem of Calculus

Change of Variable

The Lebesgue Integral in *Rk*

Stieltjes Integrals

**18. PRODUCT MEASURE AND FUBINI’S THEOREM**, **245**

Product Spaces

Product Measure

Fubini's Theorem

Integration by Parts

Products of Higher Order

**19. THE ***Lp* SPACES*, **256**

Definitions

Completeness and Separability

Conjugate Spaces

Weak Compactness

Some Decision Theory

The Space *L*2

An Estimation Problem

**Chapter4 RANDOM VARIABLES AND EXPECTED VALUES 271**

**20. RANDOM VARIABLES AND DISTRIBUTIONS, 271**

Random Variables and Vectors

Subfields

Distributions

Multidimensional Distributions

Independence

Sequences of Random Variables

Convolution

Convergence in Probability

The Glivenko-Cantelli Theorem*

**21. EXPECTED VALUES**, **291**

Expected Value as Integral

Expected Values and Limits

Expected Values and Distributions

Moments

Inequalities

Joint Integrals

Independence and Expected Value

Moment Generating Functions

**22. SUMS OF INDEPENDENT RANDOM VARIABLES**, **300**

The Strong Law of Large Numbers

The Weak Law and Moment Generating Functions

Kolmogorov's Zero-One Law

Maximal Inequalities

Convergence of Random Series

Random Taylor Series*

**23. THE POISSON PROCESS**, **316**

Characterization of the Exponential Distribution

The Poisson Process

The Poisson Approximation

Other Characterizations of the Poisson Process

Stochastic Processes

**24. THE ERGODIC THEOREM***, **330**

Measure-Preserving Transformations

Ergodicity

Ergodicity of Rotations

Proof of the Ergodic Theorem

The Continued-Fraction Transformation

Diophantine Approximation

**Chapter5 CONVERGENCE OF DISTRIBUTIONS 349**

**25. WEAK CONVERGENCE, 349**

Definitions

Uniform Distribution Modulo 1*

Convergence in Distribution

Convergence in Probability

Fundamental Theorems

Helly's Theorem

Integration to the Limit

**26. CHARACTERISTIC FUNCTIONS**, **365**

Definition

Moments and Derivatives

Independence

Inversion and the Uniqueness Theorem

The Continuity Theorem

Fourier Series*

**27. THE CENTRAL LIMIT THEOREM**, **380**

Identically Distributed Summands

The Lindeberg and Lyapounov Theorems

Dependent Variables*

**28. INFINITELY DIVISIBLE DISTRIBUTIONS***, **394**

Vague Convergence

The Possible Limits

Characterizing the Limit

**29. LIMIT THEOREMS IN ***Rk*, **402**

The Basic Theorems

Characteristic Functions

Normal Distributions in *Rk*

The Central Limit Theorem

**30. THE METHOD OF MOMENTS***, **412**

The Moment Problem

Moment Generating Functions

Central Limit Theorem by Moments

Application to Sampling Theory

Application to Number Theory

**Chapter6 DERIVATIVES AND CONDITIONAL PROBABILITY 425**

**31. DERIVATIVES ON THE LINE*, 425**

The Fundamental Theorem of Calculus

Derivatives of Integrals

Singular Functions

Integrals of Derivatives

Functions of Bounded Variation

**32. THE RADON–NIKODYM THEOREM**, **446**

Additive Set Functions

The Hahn Decomposition

Absolute Continuity and Singularity

The Main Theorem

**33. CONDITIONAL PROBABILITY**, **454**

The Discrete Case

The General Case

Properties of Conditional Probability

Difficulties and Curiosities

Conditional Probability Distributions

**34. CONDITIONAL EXPECTATION**, **472**

Definition

Properties of Conditional Expectation

Conditional Distributions and Expectations

Sufficient Subfields*

Minimum-Variance Estimation*

**35. MARTINGALES**, **487**

Definition

Submartingales

Gambling

Functions of Martingales

Stopping Times

Inequalities

Convergence Theorems

Applications: Derivatives

Likelihood Ratios

Reversed Martingales

Applications: de Finetti's Theorem

Bayes Estimation

A Central Limit Theorem*

**Chapter7 STOCHASTIC PROCESSES 513**

**36. KOLMOGOROV'S EXISTENCE THEOREM, 513**

Stochastic Processes

Finite-Dimensional Distributions

Product Spaces

Kolmogorov's Existence Theorem

The Inadequacy of *RT*

A Return to Ergodic Theory

The Hewitt**–**Savage Theorem*

**37. BROWNIAN MOTION**, **530**

Definition

Continuity of Paths

Measurable Processes

Irregularity of Brownian Motion Paths

The Strong Markov Property

The Reflection Principle

Skorohod Embedding

Invariance*

**38. NONDENUMERABLE PROBABILITIES**, **558**

Introduction

Definitions

Existence Theorems

Consequences of Separability*

APPENDIX 571

NOTES ON THE PROBLEMS 587

BIBLIOGRAPHY 617

INDEX 619