Markov Processes and Applications: Algorithms, Networks, Genome and Finance

by Etienne Pardoux (Universite de Provence, Marseille)

• ISBN13: 9780470721865
• ISBN10: 0470721863
• Format: eBook
• Copyright: 2008-11-01
• Publisher: Wiley

Summary

"This well-written book provides a clear and accessible treatment of the theory of discrete and continuous-time Markov chains, with an emphasis towards applications. The mathematical treatment is precise and rigorous without superfluous details, and the results are immediately illustrated in illuminating examples. This book will be extremely useful to anybody teaching a course on Markov processes." Jean-FranYois Le Gall, Professor at Universit_ de Paris-Orsay, France.Markov processes is the class of stochastic processes whose past and future are conditionally independent, given their present state. They constitute important models in many applied fields.After an introduction to the Monte Carlo method, this book describes discrete time Markov chains, the Poisson process and continuous time Markov chains. It also presents numerous applications including Markov Chain Monte Carlo, Simulated Annealing, Hidden Markov Models, Annotation and Alignment of Genomic sequences, Control and Filtering, Phylogenetic tree reconstruction and Queuing networks. The last chapter is an introduction to stochastic calculus and mathematical finance.Features include:The Monte Carlo method, discrete time Markov chains, the Poisson process and continuous time jump Markov processes. An introduction to diffusion processes, mathematical finance and stochastic calculus. Applications of Markov processes to various fields, ranging from mathematical biology, to financial engineering and computer science. Numerous exercises and problems with solutions to most of them

Table of Contents

Preface.

Simulations and the Monte Carlo method.

Description of the method

Convergence theorems

Simulation of random variables

Variance reduction techniques

Exercises

Markov chains.

Definitions and elementary properties

Examples

Strong Markov property

Recurrent and transient states

The irreducible and recurrent case

The aperiodic case

Reversible Markov chain

Rate of convergence to equilibrium

Statistics of Markov chains

Exercises

Stochastic algorithms.

Markov chain Monte Carlo

Simulation of the invariant probability

Rate of convergence towards the invariant probability

Simulated annealing

Exercises

Markov chains and the genome.

Reading DNA

The i.i.d. model

The Markov model

Hidden Markov models

Hidden semi-Markov model

Alignment of two sequences

A multiple alignment algorithm

Exercises

Control and filtering of Markov chains.

Deterministic optimal control

Control of Markov chains

Linear quadratic optimal control

Filtering of Markov chains

The Kalman-Bucy filter

Linear-quadratic control with partial observation

Exercises

The Poisson process.

Point processes and counting processes

The Poisson process

The Markov property

Large time behaviour

Exercises

Jump Markov processes.

General facts

Infinitesimal generator

The strong Markov property

Embedded Markov chain

Recurrent and transient states

The irreducible recurrent case

Reversibility

Markov models of evolution and phylogeny

Application to discretized partial differential equations

Simulated annealing

Exercises

Queues and networks.

M/M/1 queue

M/M/1/K queue

M/M/s queue

M/M/s/s queue

Repair shop

Queues in series

M/G/∞ queue

M/G/1 queue

Open Jackson network

Closed Jackson network

Telephone network

Kelly networks

Exercises

Introduction to mathematical finance.

Fundamental concepts

European options in the discrete model

The Black-Scholes model and formula

American options in the discrete model

American options in the Black-Scholes model

Interest rate and bonds

Exercises

Solutions to selected exercises.

Chapter 1

Chapter 2

Chapter 3

Chapter 4

Chapter 5

Chapter 6

Chapter 7

Chapter 8

Chapter 9

References

Index

Table of Contents provided by Publisher. All Rights Reserved.