Meyn & Tweedie is back! The bible on Markov chains in general state spaces has been brought up to date to reflect developments in the field since 1996 - many of them sparked by publication of the first edition. The pursuit of more efficient simulation algorithms for complex Markovian models, or algorithms for computation of optimal policies for controlled Markov models, has opened new directions for research on Markov chains. As a result, new applications have emerged across a wide range of topics including optimisation, statistics, and economics. New commentary and an epilogue by Sean Meyn summarise recent developments and references have been fully updated. This second edition reflects the same discipline and style that marked out the original and helped it to become a classic: proofs are rigorous and concise, the range of applications is broad and knowledgeable, and key ideas are accessible to practitioners with limited mathematical background.

List of figures	p. xi
Prologue to the Second edition, Peter W. Glynn	p. xiii
Preface to the second edition, Sean Meyn	p. xvii
Preface to the first edition	p. xxi
Communication and Regeneration	p. 1
Heuristics	p. 3
A range of Markovian environments	p. 3
Basic models in practice	p. 6
Stochastic stability for Markov models	p. 13
Commentary	p. 19
Markov models	p. 21
Markov models in time series	p. 22
Nonlinear state space models	p. *26
Models in control and systems theory	p. 33
Markov models with regeneration times	p. 38
Commentary	p. *46
Transition probabilities	p. 48
Defining a Markovian Process	p. 49
Foundations on a countable space	p. 51
Specific transition matrices	p. 54
Foundations for general state space chains	p. 59
Building transition kernels for specific models	p. 67
Commentary	p. 72
Irreducibility	p. 75
Communication and irreducibility: Countable spaces	p. 76
¿-Irreducibility	p. 81
¿-Irreducibility for random walk models	p. 87
¿-Irreducible linear models	p. 89
Commentary	p. 93
Pseudo-atoms	p. 96
Splitting ¿-irreducible chains	p. 97
Small sets	p. 102
Small sets for specific models	p. 106
Cyclic behavior	p. 110
Petite sets and sampled chains	p. 115
Commentary	p. 121
Topology and continuity	p. 123
Feller properties and forms of stability	p. 125
T-chains	p. 130
Continuous components for specific models	p. 134
e-Chains	p. 139
Commentary	p. 144
The nonlinear state space model	p. 146
Forward accessibility and continuous components	p. 147
Minimal sets and irreducibility	p. 154
Periodicity for nonlinear state space models	p. 157
Forward accessible examples	p. 161
Equicontinuity and the nonlinear state space model	p. 163
Commentary	p. *165
Stability Structures	p. 169
Transience and recurrence	p. 171
Classifying chains on countable spaces	p. 173
Classifying ¿-irreducible chains	p. 177
Recurrence and transience relationships	p. 182
Classification using drift criteria	p. 187
Classifying random walk on R+	p. 193
Commentary	p. *197
Harris and topological recurrence	p. 199
Harris recurrence	p. 201
Non-evanescent and recurrent chains	p. 206
Topologically recurrent and transient states	p. 208
Criteria for stability on a topological space	p. 213
Stochastic comparison and increment analysis	p. 218
Commentary	p. 228
The existence of ¿	p. 229
Stationarity and invariance	p. 230
The existence of ¿: chains with atoms	p. 234
Invariant measures for countable space models	p. *236
The existence of ¿: ¿-irreducible chains	p. 241
Invariant measures for general models	p. 247
Commentary	p. 253
Drift and regularity	p. 256
Regular chains	p. 258
Drift, hitting times and deterministic models	p. 261
Drift, criteria for regularity	p. 263
Using the regularity criteria	p. 272
Evaluating non-positivity	p. 278
Commentary	p. 285
Invariance and tightness	p. 288
Chains bounded in probability	p. 289
Generalized sampling and invariant measures	p. 292
The existence of a ¿-finite invariant measure	p. 298
Invariant measures for e-chains	p. 300
Establishing boundedness in probability	p. 305
Commentary	p. 308
Convergence	p. 311
Ergodicity	p. 313
Ergodic chains on countable spaces	p. 316
Renewal and regeneration	p. 320
Ergodicity of positive Harris chains	p. 326
Sums of transition probabilities	p. 329
Commentary	p. *334
f-Ergodicity and f-regularity	p. 336
f-Properties: chains with atoms	p. 338
f-Regularity and drift	p. 342
f-Ergodicity for general chains	p. 349
f-Ergodicity of specific models	p. 352
A key renewal theorem	p. 354
Commentary	p. 359
Geometric ergodicity	p. 362
Geometric properties: chains with atoms	p. 364
Kendall sets and drift criteria	p. 372
f-Geometric regularity of ¿ and its skeleton	p. 380
f-Geometric ergodicity for general chains	p. 384
Simple random walk and linear models	p. 388
Commentary	p. *390
V-Uniform ergodicity	p. 392
Operator norm convergence	p. 395
Uniform ergodicity	p. 400
Geometric ergodicity and increment analysis	p. 407
Models from queueing theory	p. 411
Autoregressive and state space models	p. 414
Commentary	p. *418
Sample paths and limit theorems	p. 421
Invariant ¿-fields and the LLN	p. 423
Ergodic theorems for chains possessing an atom	p. 428
General Harris chains	p. 433
The functional CLT	p. 443
Criteria for the CLT and the LIL	p. 450
Applications	p. 454
Commentary	p. *456
Positivity	p. 462
Null recurrent chains	p. 464
Characterizing positivity using Pn	p. 469
Positivity and T-chains	p. 471
Positivity and e-chains	p. 473
The LLN for e-chains	p. 477
Commentary	p. *480
Generalized classification criteria	p. 482
State-dependent drifts	p. 483
History-dependent drift criteria	p. 491
Mixed drift conditions	p. 498
Commentary	p. *508
Epilogue to the second edition	p. 510
Geometric ergodicity and spectral theory	p. 510
Simulation and MCMC	p. 521
Continuous time models	p. 523
Appendices	p. 529
Mud maps	p. 532
Recurrence versus transience	p. 532
Positivity versus nullity	p. 534
Convergence properties	p. 536
Testing for stability	p. 538
Glossary of drift conditions	p. 538
The Scalar SETAR model: a complete classification	p. 540
Glossary of models assumptions	p. 543
Regenerative models	p. 543
State space models	p. 546
Some mathematical background	p. 552
Some measure theory	p. 552
Some probability theory	p. 555
Some topology	p. 556
Some real analysis	p. 557
Convergence concepts for measures	p. 558
Some martingale theory	p. 561
Some results on sequences and numbers	p. 563
Bibliography	p. 567
Indexes	p. 587
General index	p. 587
Symbols	p. 593
Table of Contents provided by Ingram. All Rights Reserved.

What is included with this book?

The New copy of this book will include any supplemental materials advertised. Please check the title of the book to determine if it should include any access cards, study guides, lab manuals, CDs, etc.

The Used, Rental and eBook copies of this book are not guaranteed to include any supplemental materials. Typically, only the book itself is included. This is true even if the title states it includes any access cards, study guides, lab manuals, CDs, etc.