A first course in Probability and Markov Chains presents an introduction to the basic elements in statistics and focuses in two main areas. The first part of the book looks at notions and structures in probability, including Combinatorics, probability measures, probability distributions, conditional probability, inclusion-exclusion formulas, random variables, dispersion indexes, independent random variables as well as Weak and Strong Laws of Large Numbers and Central Limit Theorem. A list of classical probability distributions, both discrete and continuous, is also included. In the second part of the book explores Discrete Time Discrete Markov Chains (DTDMC) which are discussed together with an introduction to Poisson processes and Continuous Time Discrete Markov Chains (CTDMC). The books main focus is in making use of measure theory notations that unify all the presentation, in particular avoiding the separate treatment of continuous and discrete distributions.

Giuseppe Modica and Laura Poggiolini,
Dipartimento di Sistemi e Informatica, Università di Firenze, Italy

Preface

1 Combinatorics

1.1 Binomial Coefficients

1.1.1 Pascal triangle

1.1.2 Some properties of binomial coefficients

1.1.3 Generalized binomial coefficients and binomial series

1.1.4 Inversion formulas

1.1.5 Exercises

1.2 Sets, Permutations and Functions

1.2.1 Sets

1.2.2 Permutations

1.2.3 Multisets

1.2.4 Lists and functions

1.2.5 Injective functions

1.2.6 Monotone increasing functions

1.2.7 Monotone nondecreasing functions

1.2.8 Surjective functions

1.2.9 Exercises

1.3 Drawings

1.3.1 Ordered drawings

1.3.2 Simple drawings

1.3.3 Multiplicative property of drawings

1.3.4 Exercises

1.4 Grouping

1.4.1 Collocations of pairwise different objects

1.4.2 Collocations of identical objects

1.4.3 Multiplicative property

1.4.4 Collocations in Statistical Physics

1.4.5 Exercises

2 Probability Measures

2.1 Elementary Probability

2.1.1 Exercises

2.2 Probability Measures

2.2.1 Events

2.2.2 Probability measures

2.2.3 Continuity of measures

2.2.4 Integral with respect to a measure

2.2.5 Probabilities on finite and denumerable sets

2.2.6 Probabilities on denumerable sets

2.2.7 Probabilities on uncountable sets

2.2.8 Exercises

2.3 Conditional Probability

2.3.1 Conditional probability

2.3.2 Bayes’ formula

2.3.3 Exercises

2.4 Inclusion-Exclusion Principle

2.4.1 Exercises

3 Random Variables

3.1 Random Variables

3.1.1 Definitions

3.1.2 Expected value

3.1.3 Functions of random variables

3.1.4 Cavalieri formula

3.1.5 Variance

3.1.6 Markov and Chebyshev inequalities

3.1.7 Variational characterization of the median and

of the expected value

3.1.8 Exercises

3.2 A Few Discrete Distributions

3.2.1 Bernoulli distribution

3.2.2 Binomial distribution

3.2.3 Hypergeometric distribution

3.2.4 Negative binomial distribution

3.2.5 Poisson distribution

3.2.6 Geometric distribution

3.2.7 Exercises

3.3 Some Absolutely Continuous Distributions

3.3.1 Uniform distribution

3.3.2 Normal distribution

3.3.3 Exponential distribution

3.3.4 Gamma distributions

3.3.5 Failure rate

3.3.6 Exercises

4 Vector Valued Random Variables

4.1 Joint Distribution

4.1.1 Joint and marginal distributions

4.1.2 Exercises

4.2 Covariance

4.2.1 Random variables with finite expected value and variance

4.2.2 Correlation coefficient

4.2.3 Exercises

4.3 Independent Random Variables

4.3.1 Independent events

4.3.2 Independent random variables

4.3.3 Independence of many random variables

4.3.4 Sum of independent random variables

4.3.5 Exercise

4.4 Sequences of Independent Random Variables

4.4.1 Weak law of large numbers

4.4.2 Borel–Cantelli lemma

4.4.3 Convergences of random variables

4.4.4 Strong law of large numbers

4.4.5 A few applications of the law of large numbers

4.4.6 Central limit theorem

4.4.7 Exercises

5 Discrete Time Markov Chains

5.1 Stochastic Matrices

5.1.1 Definitions

5.1.2 Oriented graphs

5.1.3 Exercises

5.2 Markov Chains

5.2.1 Stochastic processes

5.2.2 Transition matrices

5.2.3 Homogeneous processes

5.2.4 Markov chains

5.2.5 Canonical representation of Markov chains

5.2.6 Exercises

5.3 Some Characteristic Parameters

5.3.1 Steps for a first visit

5.3.2 Probability of (at least) r visits

5.3.3 Recurrent and transient states

5.3.4 Mean first passage time

5.3.5 Hitting time and hitting probabilities

5.3.6 Exercises

5.4 Finite Stochastic Matrices

5.4.1 Canonical representation

5.4.2 States classification

5.4.3 Exercises

5.5 Regular Stochastic Matrices

5.5.1 Iterated maps

5.5.2 Existence of fixed points

5.5.3 Regular stochastic matrices

5.5.4 Characteristic parameters

5.5.5 Exercises

5.6 Ergodic Property

5.6.1 Number of steps between consecutive visits

5.6.2 Ergodic theorem

5.6.3 Powers of irreducible stochastic matrices

5.6.4 Markov chain Monte Carlo

5.7 Renewal Theorem

5.7.1 Periodicity

5.7.2 Renewal theorem

5.7.3 Exercises

6 An Introduction to Continuous Time Markov Chains

6.1 Poisson Process

6.2 Continuous Time Markov Chains

6.2.1 Continuous semigroups of stochastic matrices

6.2.2 Examples of right-continuous Markov chains

6.2.3 Holding times

A Power Series

A.1 Basic Properties

A.2 Product of Series

A.3 Banach Space Valued Power Series

A.3.1 Exercises

B Measure and Integration

B.1 Measures

B.1.1 Basic properties

B.1.2 Construction of measures

B.1.3 Exercises

B.2 Measurable Functions and Integration

B.2.1 Measurable functions

B.2.2 The integral

B.2.3 Properties of the integral

B.2.4 Cavalieri formula

B.2.5 Markov inequality

B.2.6 Null sets and the integral

B.2.7 Push-forward of a measure

B.2.8 Exercises

B.3 Product Measures and Iterated Integrals

B.3.1 Product measures

B.3.2 Reduction formulas

B.3.3 Exercises

B.4 Convergence Theorems

B.4.1 Almost everywhere convergence

B.4.2 Strong convergence

B.4.3 Fatou lemma

B.4.4 Dominated convergence theorem

B.4.5 Absolute continuity of integrals

B.4.6 Differentiation of the integral

B.4.7 Weak convergence of measures

B.4.8 Exercises

C Systems of Linear Ordinary Differential Equations

C.1 Cauchy Problem

C.1.1 Uniqueness

C.1.2 Existence

C.2 Efficient Computation of eQt

C.2.1 Similarity methods

C.2.2 Putzer method

C.3 Semigroups

References 3

Index

What is included with this book?

The New copy of this book will include any supplemental materials advertised. Please check the title of the book to determine if it should include any access cards, study guides, lab manuals, CDs, etc.

The Used, Rental and eBook copies of this book are not guaranteed to include any supplemental materials. Typically, only the book itself is included. This is true even if the title states it includes any access cards, study guides, lab manuals, CDs, etc.