More New and Used

from Private Sellers

**Note:**Supplemental materials are not guaranteed with Rental or Used book purchases.

# An Introduction to Stochastic Modeling

**by**Pinsky, Mark A.; Karlin, Samuel

4th

### 9780123814166

0123814162

Paperback

12/10/2010

Elsevier Science Ltd

## Questions About This Book?

Why should I rent this book?

Renting is easy, fast, and cheap! Renting from eCampus.com can save you hundreds of dollars compared to the cost of new or used books each semester. At the end of the semester, simply ship the book back to us with a free UPS shipping label! No need to worry about selling it back.

How do rental returns work?

Returning books is as easy as possible. As your rental due date approaches, we will email you several courtesy reminders. When you are ready to return, you can print a free UPS shipping label from our website at any time. Then, just return the book to your UPS driver or any staffed UPS location. You can even use the same box we shipped it in!

What version or edition is this?

This is the 4th edition with a publication date of 12/10/2010.

What is included with this book?

- The
**New**copy of this book will include any supplemental materials advertised. Please check the title of the book to determine if it should include any CDs, lab manuals, study guides, etc. - The
**Used**copy of this book is not guaranteed to include any supplemental materials. Typically, only the book itself is included. - The
**Rental**copy of this book is not guaranteed to include any supplemental materials. You may receive a brand new copy, but typically, only the book itself. - The
**eBook**copy of this book is not guaranteed to include any supplemental materials. Typically only the book itself is included.

## Summary

Serving as the foundation for a one-semester course in stochastic processes for students familiar with elementary probability theory and calculus, Introduction to Stochastic Modeling, Third Edition, bridges the gap between basic probability and an intermediate level course in stochastic processes. The objectives of the text are to introduce students to the standard concepts and methods of stochastic modeling, to illustrate the rich diversity of applications of stochastic processes in the applied sciences, and to provide exercises in the application of simple stochastic analysis to realistic problems.

## Table of Contents

Preface to the Fourth Edition | p. xi |

Preface to the Third Edition | p. xiii |

Preface to the First Edition | p. xv |

To the Instructor | p. xvii |

Acknowledgments | p. xix |

Introduction | p. 1 |

Stochastic Modeling | p. 1 |

Stochastic Processes | p. 4 |

Probability Review | p. 4 |

Events and Probabilities | p. 4 |

Random Variables | p. 5 |

Moments and Expected Values | p. 7 |

Joint Distribution Functions | p. 8 |

Sums and Convolutions | p. 10 |

Change of Variable | p. 10 |

Conditional Probability | p. 11 |

Review of Axiomatic Probability Theory | p. 12 |

The Major Discrete Distributions | p. 19 |

Bernoulli Distribution | p. 20 |

Binomial Distribution | p. 20 |

Geometric and Negative Binominal Distributions | p. 21 |

The Poisson Distribution | p. 22 |

The Multinomial Distribution | p. 24 |

Important Continuous Distributions | p. 27 |

The Normal Distribution | p. 27 |

The Exponential Distribution | p. 28 |

The Uniform Distribution | p. 30 |

The Gamma Distribution | p. 30 |

The Beta Distribution | p. 31 |

The Joint Normal Distribution | p. 31 |

Some Elementary Exercises | p. 34 |

Tail Probabilities | p. 34 |

The Exponential Distribution | p. 37 |

Useful Functions, Integrals, and Sums | p. 42 |

Conditional Probability and Conditional Expectation | p. 47 |

The Discrete Case | p. 47 |

The Dice Game Craps | p. 52 |

Random Sums | p. 57 |

Conditional Distributions: The Mixed Case | p. 58 |

The Moments of a Random Sum | p. 59 |

The Distribution of a Random Sum | p. 61 |

Conditioning on a Continuous Random Variable | p. 65 |

Martingales | p. 71 |

The Definition | p. 72 |

The Markov Inequality | p. 73 |

The Maximal Inequality for Nonnegative Martingales | p. 73 |

Markov Chains: Introduction | p. 79 |

Definitions | p. 79 |

Transition Probability Matrices of a Markov Chain | p. 83 |

Some Markov Chain Models | p. 87 |

An Inventory Model | p. 87 |

The Ehrenfest Urn Model | p. 89 |

Markov Chains in Genetics | p. 90 |

A Discrete Queueing Markov Chain | p. 92 |

First Step Analysis | p. 95 |

Simple First Step Analyses | p. 95 |

The General Absorbing Markov Chain | p. 102 |

Some Special Markov Chains | p. 111 |

The Two-State Markov Chain | p. 112 |

Markov Chains Defined by Independent Random Variables | p. 114 |

One-Dimensional Random Walks | p. 116 |

Success Runs | p. 120 |

Functionals of Random Walks and Success Runs | p. 124 |

The General Random Walk | p. 128 |

Cash Management | p. 132 |

The Success Runs Markov Chain | p. 134 |

Another Look at First Step Analysis | p. 139 |

Branching Processes | p. 146 |

Examples of Branching Processes | p. 147 |

The Mean and Variance of a Branching Process | p. 148 |

Extinction Probabilities | p. 149 |

Branching Processes and Generating Functions | p. 152 |

Generating Functions and Extinction Probabilities | p. 154 |

Probability Generating Functions and Sums of Independent Random Variables | p. 157 |

Multiple Branching Processes | p. 159 |

The Long Run Behavior of Markov Chains | p. 165 |

Regular Transition Probability Matrices | p. 165 |

Doubly Stochastic Matrices | p. 170 |

Interpretation of the Limiting Distribution | p. 171 |

Examples | p. 178 |

Including History in the State Description | p. 178 |

Reliability and Redundancy | p. 179 |

A Continuous Sampling Plan | p. 181 |

Age Replacement Policies | p. 183 |

Optimal Replacement Rules | p. 185 |

The Classification of States | p. 194 |

Irreducible Markov Chains | p. 195 |

Periodicity of a Markov Chain | p. 196 |

Recurrent and Transient States | p. 198 |

The Basic Limit Theorem of Markov Chains | p. 203 |

Reducible Markov Chains | p. 215 |

Poisson Processes | p. 223 |

The Poisson Distribution and the Poisson Process | p. 223 |

The Poisson Distribution | p. 223 |

The Poisson Process | p. 225 |

Nonhomogeneous Processes | p. 226 |

Cox Processes | p. 227 |

The Law of Rare Events | p. 232 |

The Law of Rare Events and the Poisson Process | p. 234 |

Proof of Theorem 5.3 | p. 237 |

Distributions Associated with the Poisson Process | p. 241 |

The Uniform Distribution and Poisson Processes | p. 247 |

Shot Noise | p. 253 |

Sum Quota Sampling | p. 255 |

Spatial Poisson Processes | p. 259 |

Compound and Marked Poisson Processes | p. 264 |

Compound Poisson Processes | p. 264 |

Marked Poisson Processes | p. 267 |

Continuous Time Markov Chains | p. 277 |

Pure Birth Processes | p. 277 |

Postulates for the Poisson Process | p. 277 |

Pure Birth Process | p. 278 |

The Yule Process | p. 282 |

Pure Death Processes | p. 286 |

The Linear Death Process | p. 287 |

Cable Failure Under Static Fatigue | p. 290 |

Birth and Death Processes | p. 295 |

Postulates | p. 295 |

Sojourn Times | p. 296 |

Differential Equations of Birth and Death Processes | p. 299 |

The Limiting Behavior of Birth and Death Processes | p. 304 |

Birth and Death Processes with Absorbing States | p. 316 |

Probability of Absorption into State 0 | p. 316 |

Mean Time Until Absorption | p. 318 |

Finite-State Continuous Time Markov Chains | p. 327 |

A Poisson Process with a Markov Intensity | p. 338 |

Renewal Phenomena | p. 347 |

Definition of a Renewal Process and Related Concepts | p. 347 |

Some Examples of Renewal Processes | p. 353 |

Brief Sketches of Renewal Situations | p. 353 |

Block Replacement | p. 354 |

The Poisson Process Viewed as a Renewal Process | p. 358 |

The Asymptotic Behavior of Renewal Processes | p. 362 |

The Elementary Renewal Theorem | p. 363 |

The Renewal Theorem for Continuous Lifetimes | p. 365 |

The Asymptotic Distribution of N(t) | p. 367 |

The Limiting Distribution of Age and Excess Life | p. 368 |

Generalizations and Variations on Renewal Processes | p. 371 |

Delayed Renewal Processes | p. 371 |

Stationary Renewal Processes | p. 372 |

Cumulative and Related Processes | p. 372 |

Discrete Renewal Theory | p. 379 |

The Discrete Renewal Theorem | p. 383 |

Deterministic Population Growth with Age Distribution | p. 384 |

Brownian Motion and Related Processes | p. 391 |

Brownian Motion and Gaussian Processes | p. 391 |

A Little History | p. 391 |

The Brownian Motion Stochastic Process | p. 392 |

The Central Limit Theorem and the Invariance Principle | p. 396 |

Gaussian Processes | p. 398 |

The Maximum Variable and the Reflection Principle | p. 405 |

The Reflection Principle | p. 406 |

The Time to First Reach a Level | p. 407 |

The Zeros of Brownian Motion | p. 408 |

Variations and Extensions | p. 411 |

Reflected Brownian Motion | p. 411 |

Absorbed Brownian Motion | p. 412 |

The Brownian Bridge | p. 414 |

Brownian Meander | p. 416 |

Brownian Motion with Drift | p. 419 |

The Gambler's Ruin Problem | p. 420 |

Geometric Brownian Motion | p. 424 |

The Ornstein-Uhlenbeck Process | p. 432 |

A Second Approach to Physical Brownian Motion | p. 434 |

The Position Process | p. 437 |

The Long Run Behavior | p. 439 |

Brownian Measure and Integration | p. 441 |

Queueing Systems | p. 447 |

Queueing Processes | p. 447 |

The Queueing Formula L = X W | p. 448 |

A Sampling of Queueing Models | p. 449 |

Poisson Arrivals, Exponential Service Times | p. 451 |

The M/M/1 System | p. 452 |

The M/M/$ System | p. 456 |

The M/M/s System | p. 457 |

General Service Time Distributions | p. 460 |

The M/G/1 System | p. 460 |

The M/G/$ System | p. 465 |

Variations and Extensions | p. 468 |

Systems with Balking | p. 468 |

Variable Service Rates | p. 469 |

A System with Feedback | p. 470 |

A Two-Server Overflow Queue | p. 470 |

Preemptive Priority Queues | p. 472 |

Open Acyclic Queueing Networks | p. 480 |

The Basic Theorem | p. 480 |

Two Queues in Tandem | p. 481 |

Open Acyclic Networks | p. 482 |

Appendix: Time Reversibility | p. 485 |

Proof of Theorem 9.1 | p. 487 |

General Open Networks | p. 488 |

The General Open Network | p. 492 |

Random Evolutions | p. 495 |

Two-State Velocity Model | p. 495 |

Two-State Random Evolution | p. 498 |

The Telegraph Equation | p. 500 |

Distribution Functions and Densities in the Two-State Model | p. 501 |

Passage Time Distributions | p. 505 |

JV-State Random Evolution | p. 507 |

Finite Markov Chains and Random Velocity Models | p. 507 |

Constructive Approach of Random Velocity Models | p. 507 |

Random Evolution Processes | p. 508 |

Existence-Uniqueness of the First-Order System (10.26) | p. 509 |

Single Hyperbolic Equation | p. 510 |

Spectral Properties of the Transition Matrix | p. 512 |

Recurrence Properties of Random Evolution | p. 515 |

Weak Law and Central Limit Theorem | p. 516 |

Isotropic Transport in Higher Dimensions | p. 521 |

The Rayleigh Problem of Random Flights | p. 521 |

Three-Dimensional Rayleigh Model | p. 523 |

Characteristic Functions and Their Applications | p. 525 |

Definition of the Characteristic Function | p. 525 |

Two Basic Properties of the Characteristic Function | p. 526 |

Inversion Formulas for Characteristic Functions | p. 527 |

Fourier Reciprocity/Local Non-Uniqueness | p. 530 |

Fourier Inversion and Parseval's Identity Inversion | p. 531 |

Formula for General Random Variables | p. 532 |

The Continuity Theorem | p. 533 |

Proof of the Continuity Theorem | p. 534 |

Proof of the Central Limit Theorem | p. 535 |

Stirling's Formula and Applications | p. 536 |

Poisson Representation of n! | p. 537 |

Proof of Stirling's Formula | p. 538 |

Local deMoivre-Laplace Theorem | p. 539 |

Further Reading | p. 541 |

Answers to Exercises | p. 543 |

Index | p. 557 |

Table of Contents provided by Ingram. All Rights Reserved. |