An Introduction to Stochastic Modeling

by Pinsky; Karlin

• ISBN13: 9780123814166
• ISBN10: 0123814162
• Edition: 4th
• Format: Paperback
• Copyright: 2010-12-10
• Publisher: Elsevier Science

Summary

Serving as the foundation for a one-semester course in stochastic processes for students familiar with elementary probability theory and calculus, Introduction to Stochastic Modeling, Third Edition, bridges the gap between basic probability and an intermediate level course in stochastic processes. The objectives of the text are to introduce students to the standard concepts and methods of stochastic modeling, to illustrate the rich diversity of applications of stochastic processes in the applied sciences, and to provide exercises in the application of simple stochastic analysis to realistic problems.

Table of Contents

Preface to the Fourth Edition	p. xi
Preface to the Third Edition	p. xiii
Preface to the First Edition	p. xv
To the Instructor	p. xvii
Acknowledgments	p. xix
Introduction	p. 1
Stochastic Modeling	p. 1
Stochastic Processes	p. 4
Probability Review	p. 4
Events and Probabilities	p. 4
Random Variables	p. 5
Moments and Expected Values	p. 7
Joint Distribution Functions	p. 8
Sums and Convolutions	p. 10
Change of Variable	p. 10
Conditional Probability	p. 11
Review of Axiomatic Probability Theory	p. 12
The Major Discrete Distributions	p. 19
Bernoulli Distribution	p. 20
Binomial Distribution	p. 20
Geometric and Negative Binominal Distributions	p. 21
The Poisson Distribution	p. 22
The Multinomial Distribution	p. 24
Important Continuous Distributions	p. 27
The Normal Distribution	p. 27
The Exponential Distribution	p. 28
The Uniform Distribution	p. 30
The Gamma Distribution	p. 30
The Beta Distribution	p. 31
The Joint Normal Distribution	p. 31
Some Elementary Exercises	p. 34
Tail Probabilities	p. 34
The Exponential Distribution	p. 37
Useful Functions, Integrals, and Sums	p. 42
Conditional Probability and Conditional Expectation	p. 47
The Discrete Case	p. 47
The Dice Game Craps	p. 52
Random Sums	p. 57
Conditional Distributions: The Mixed Case	p. 58
The Moments of a Random Sum	p. 59
The Distribution of a Random Sum	p. 61
Conditioning on a Continuous Random Variable	p. 65
Martingales	p. 71
The Definition	p. 72
The Markov Inequality	p. 73
The Maximal Inequality for Nonnegative Martingales	p. 73
Markov Chains: Introduction	p. 79
Definitions	p. 79
Transition Probability Matrices of a Markov Chain	p. 83
Some Markov Chain Models	p. 87
An Inventory Model	p. 87
The Ehrenfest Urn Model	p. 89
Markov Chains in Genetics	p. 90
A Discrete Queueing Markov Chain	p. 92
First Step Analysis	p. 95
Simple First Step Analyses	p. 95
The General Absorbing Markov Chain	p. 102
Some Special Markov Chains	p. 111
The Two-State Markov Chain	p. 112
Markov Chains Defined by Independent Random Variables	p. 114
One-Dimensional Random Walks	p. 116
Success Runs	p. 120
Functionals of Random Walks and Success Runs	p. 124
The General Random Walk	p. 128
Cash Management	p. 132
The Success Runs Markov Chain	p. 134
Another Look at First Step Analysis	p. 139
Branching Processes	p. 146
Examples of Branching Processes	p. 147
The Mean and Variance of a Branching Process	p. 148
Extinction Probabilities	p. 149
Branching Processes and Generating Functions	p. 152
Generating Functions and Extinction Probabilities	p. 154
Probability Generating Functions and Sums of Independent Random Variables	p. 157
Multiple Branching Processes	p. 159
The Long Run Behavior of Markov Chains	p. 165
Regular Transition Probability Matrices	p. 165
Doubly Stochastic Matrices	p. 170
Interpretation of the Limiting Distribution	p. 171
Examples	p. 178
Including History in the State Description	p. 178
Reliability and Redundancy	p. 179
A Continuous Sampling Plan	p. 181
Age Replacement Policies	p. 183
Optimal Replacement Rules	p. 185
The Classification of States	p. 194
Irreducible Markov Chains	p. 195
Periodicity of a Markov Chain	p. 196
Recurrent and Transient States	p. 198
The Basic Limit Theorem of Markov Chains	p. 203
Reducible Markov Chains	p. 215
Poisson Processes	p. 223
The Poisson Distribution and the Poisson Process	p. 223
The Poisson Distribution	p. 223
The Poisson Process	p. 225
Nonhomogeneous Processes	p. 226
Cox Processes	p. 227
The Law of Rare Events	p. 232
The Law of Rare Events and the Poisson Process	p. 234
Proof of Theorem 5.3	p. 237
Distributions Associated with the Poisson Process	p. 241
The Uniform Distribution and Poisson Processes	p. 247
Shot Noise	p. 253
Sum Quota Sampling	p. 255
Spatial Poisson Processes	p. 259
Compound and Marked Poisson Processes	p. 264
Compound Poisson Processes	p. 264
Marked Poisson Processes	p. 267
Continuous Time Markov Chains	p. 277
Pure Birth Processes	p. 277
Postulates for the Poisson Process	p. 277
Pure Birth Process	p. 278
The Yule Process	p. 282
Pure Death Processes	p. 286
The Linear Death Process	p. 287
Cable Failure Under Static Fatigue	p. 290
Birth and Death Processes	p. 295
Postulates	p. 295
Sojourn Times	p. 296
Differential Equations of Birth and Death Processes	p. 299
The Limiting Behavior of Birth and Death Processes	p. 304
Birth and Death Processes with Absorbing States	p. 316
Probability of Absorption into State 0	p. 316
Mean Time Until Absorption	p. 318
Finite-State Continuous Time Markov Chains	p. 327
A Poisson Process with a Markov Intensity	p. 338
Renewal Phenomena	p. 347
Definition of a Renewal Process and Related Concepts	p. 347
Some Examples of Renewal Processes	p. 353
Brief Sketches of Renewal Situations	p. 353
Block Replacement	p. 354
The Poisson Process Viewed as a Renewal Process	p. 358
The Asymptotic Behavior of Renewal Processes	p. 362
The Elementary Renewal Theorem	p. 363
The Renewal Theorem for Continuous Lifetimes	p. 365
The Asymptotic Distribution of N(t)	p. 367
The Limiting Distribution of Age and Excess Life	p. 368
Generalizations and Variations on Renewal Processes	p. 371
Delayed Renewal Processes	p. 371
Stationary Renewal Processes	p. 372
Cumulative and Related Processes	p. 372
Discrete Renewal Theory	p. 379
The Discrete Renewal Theorem	p. 383
Deterministic Population Growth with Age Distribution	p. 384
Brownian Motion and Related Processes	p. 391
Brownian Motion and Gaussian Processes	p. 391
A Little History	p. 391
The Brownian Motion Stochastic Process	p. 392
The Central Limit Theorem and the Invariance Principle	p. 396
Gaussian Processes	p. 398
The Maximum Variable and the Reflection Principle	p. 405
The Reflection Principle	p. 406
The Time to First Reach a Level	p. 407
The Zeros of Brownian Motion	p. 408
Variations and Extensions	p. 411
Reflected Brownian Motion	p. 411
Absorbed Brownian Motion	p. 412
The Brownian Bridge	p. 414
Brownian Meander	p. 416
Brownian Motion with Drift	p. 419
The Gambler's Ruin Problem	p. 420
Geometric Brownian Motion	p. 424
The Ornstein-Uhlenbeck Process	p. 432
A Second Approach to Physical Brownian Motion	p. 434
The Position Process	p. 437
The Long Run Behavior	p. 439
Brownian Measure and Integration	p. 441
Queueing Systems	p. 447
Queueing Processes	p. 447
The Queueing Formula L = X W	p. 448
A Sampling of Queueing Models	p. 449
Poisson Arrivals, Exponential Service Times	p. 451
The M/M/1 System	p. 452
The M/M/$ System	p. 456
The M/M/s System	p. 457
General Service Time Distributions	p. 460
The M/G/1 System	p. 460
The M/G/$ System	p. 465
Variations and Extensions	p. 468
Systems with Balking	p. 468
Variable Service Rates	p. 469
A System with Feedback	p. 470
A Two-Server Overflow Queue	p. 470
Preemptive Priority Queues	p. 472
Open Acyclic Queueing Networks	p. 480
The Basic Theorem	p. 480
Two Queues in Tandem	p. 481
Open Acyclic Networks	p. 482
Appendix: Time Reversibility	p. 485
Proof of Theorem 9.1	p. 487
General Open Networks	p. 488
The General Open Network	p. 492
Random Evolutions	p. 495
Two-State Velocity Model	p. 495
Two-State Random Evolution	p. 498
The Telegraph Equation	p. 500
Distribution Functions and Densities in the Two-State Model	p. 501
Passage Time Distributions	p. 505
JV-State Random Evolution	p. 507
Finite Markov Chains and Random Velocity Models	p. 507
Constructive Approach of Random Velocity Models	p. 507
Random Evolution Processes	p. 508
Existence-Uniqueness of the First-Order System (10.26)	p. 509
Single Hyperbolic Equation	p. 510
Spectral Properties of the Transition Matrix	p. 512
Recurrence Properties of Random Evolution	p. 515
Weak Law and Central Limit Theorem	p. 516
Isotropic Transport in Higher Dimensions	p. 521
The Rayleigh Problem of Random Flights	p. 521
Three-Dimensional Rayleigh Model	p. 523
Characteristic Functions and Their Applications	p. 525
Definition of the Characteristic Function	p. 525
Two Basic Properties of the Characteristic Function	p. 526
Inversion Formulas for Characteristic Functions	p. 527
Fourier Reciprocity/Local Non-Uniqueness	p. 530
Fourier Inversion and Parseval's Identity Inversion	p. 531
Formula for General Random Variables	p. 532
The Continuity Theorem	p. 533
Proof of the Continuity Theorem	p. 534
Proof of the Central Limit Theorem	p. 535
Stirling's Formula and Applications	p. 536
Poisson Representation of n!	p. 537
Proof of Stirling's Formula	p. 538
Local deMoivre-Laplace Theorem	p. 539
Further Reading	p. 541
Answers to Exercises	p. 543
Index	p. 557
Table of Contents provided by Ingram. All Rights Reserved.