Bayesian Networks An Introduction

by Koski, Timo; Noble, John

• ISBN13: 9780470743041
• ISBN10: 0470743042
• Edition: 1st
• Format: Hardcover
• Copyright: 2009-11-02
• Publisher: Wiley

Summary

A self-contained introduction to the theory and applications of Bayesian NetworksBayesian networks are a topic of interest and importance for statisticians, computer scientists and those involved in modelling and the learning of complex data sets. The material included in this introductory guide has been extensively tested in classroom teaching and assumes a basic knowledge of probability and statistics course and basic mathematics. All notions are explained carefully with an extensive set of exercises throughout the book as well as computer exercises. A solutions manual is also provided online.

Author Biography

Timo Koski Institutionen fr Matematik, Kungliga Teknisha Hgskolan, Stockholm, Sweden John M. Noble Matematiska Institutionen, Linkpings Tekniska Hgskola, Linkpings universiter, Linkping, Sweden

Table of Contents

Preface	p. ix
Graphical models and probabilistic reasoning	p. 1
Introduction	p. 1
Axioms of probability and basic notations	p. 4
The Bayes update of probability	p. 9
Inductive learning	p. 11
Bayes' rule	p. 12
Jeffrey's rule	p. 13
Pearl's method of virtual evidence	p. 13
Interpretations of probability and Bayesian networks	p. 14
Learning as inference about parameters	p. 15
Bayesian statistical inference	p. 17
Tossing a thumb-tack	p. 20
Multinomial sampling and the Dirichlet integral	p. 24
Notes	p. 28
Exercises: Probabilistic theories of causality, Bayes' rule, multinomial sampling and the Dirichlet density	p. 31
Conditional independence, graphs and d-separation	p. 37
Joint probabilities	p. 37
Conditional independence	p. 38
Directed acyclic graphs and d-separation	p. 41
Graphs	p. 41
Directed acyclic graphs and probability distributions	p. 45
The Bayes ball	p. 50
Illustrations	p. 51
Potentials	p. 53
Bayesian networks	p. 58
Object oriented Bayesian networks	p. 63
d-Separation and conditional independence	p. 66
Markov models and Bayesian networks	p. 67
I-maps and Markov equivalence	p. 69
The trek and a distribution without a faithful graph	p. 72
Notes	p. 73
Exercises: Conditional independence and d-separation	p. 75
Evidence, sufficiency and Monte Carlo methods	p. 81
Hard evidence	p. 82
Soft evidence and virtual evidence	p. 85
Jeffrey's rule	p. 86
Pearl's method of virtual evidence	p. 87
Queries in probabilistic inference	p. 88
The chest clinic problem	p. 89
Bucket elimination	p. 89
Bayesian sufficient statistics and prediction sufficiency	p. 92
Bayesian sufficient statistics	p. 92
Prediction sufficiency	p. 92
Prediction sufficiency for a Bayesian network	p. 95
Time variables	p. 98
A brief introduction to Markov chain Monte Carlo methods	p. 100
Simulating a Markov chain	p. 103
Irreducibility, aperiodicity and time reversibility	p. 104
The Metropolis-Hastings algorithm	p. 108
The one-dimensional discrete Metropolis algorithm	p. 111
Notes	p. 112
Exercises: Evidence, sufficiency and Monte Carlo methods	p. 113
Decomposable graphs and chain graphs	p. 123
Definitions and notations	p. 124
Decomposable graphs and triangulation of graphs	p. 127
Junction trees	p. 131
Markov equivalence	p. 133
Markov equivalence, the essential graph and chain graphs	p. 138
Notes	p. 144
Exercises: Decomposable graphs and chain graphs	p. 145
Learning the conditional probability potentials	p. 149
Initial illustration: maximum likelihood estimate for a fork connection	p. 149
The maximum likelihood estimator for multinomial sampling	p. 151
MLE for the parameters in a DAG: the general setting	p. 155
Updating, missing data, fractional updating	p. 160
Notes	p. 161
Exercises: Learning the conditional probability potentials	p. 162
Learning the graph structure	p. 167
Assigning a probability distribution to the graph structure	p. 168
Markov equivalence and consistency	p. 171
Establishing the DAG isomorphic property	p. 173
Reducing the size of the search	p. 176
The Chow-Liu tree	p. 177
The Chow-Liu tree: A predictive approach	p. 179
The K2 structural learning algorithm	p. 183
The MMHC algorithm	p. 184
Monte Carlo methods for locating the graph structure	p. 186
Women in mathematics	p. 189
Notes	p. 191
Exercises: Learning the graph structure	p. 192
Parameters and sensitivity	p. 197
Changing parameters in a network	p. 198
Measures of divergence between probability distributions	p. 201
The Chan-Darwiche distance measure	p. 202
Comparison with the Kullback-Leibler divergence and euclidean distance	p. 209
Global bounds for queries	p. 210
Applications to updating	p. 212
Parameter changes to satisfy query constraints	p. 216
Binary variables	p. 218
The sensitivity of queries to parameter changes	p. 220
Notes	p. 224
Exercises: Parameters and sensitivity	p. 225
Graphical models and exponential families	p. 229
Introduction to exponential families	p. 229
Standard examples of exponential families	p. 231
Graphical models and exponential families	p. 233
Noisy 'or' as an exponential family	p. 234
Properties of the log partition function	p. 237
Fenchel Legendre conjugate	p. 239
Kullback-Leibler divergence	p. 241
Mean field theory	p. 243
Conditional Gaussian distributions	p. 246
CG potentials	p. 249
Some results on marginalization	p. 249
CG regression	p. 250
Notes	p. 251
Exercises: Graphical models and exponential families	p. 252
Causality and intervention calculus	p. 255
Introduction	p. 255
Conditioning by observation and by intervention	p. 257
The intervention calculus for a Bayesian network	p. 258
Establishing the model via a controlled experiment	p. 262
Properties of intervention calculus	p. 262
Transformations of probability	p. 265
A note on the order of 'see' and 'do' conditioning	p. 267
The 'Sure Thing' principle	p. 268
Back door criterion, confounding and identifiability	p. 270
Notes	p. 273
Exercises: Causality and intervention calculus	p. 275
The junction tree and probability updating	p. 279
Probability updating using a junction tree	p. 279
Potentials and the distributive law	p. 280
Marginalization and the distributive law	p. 283
Elimination and domain graphs	p. 284
Factorization along an undirected graph	p. 288
Factorizing along a junction tree	p. 290
Flow of messages initial illustration	p. 292
Local computation on junction trees	p. 294
Schedules	p. 296
Local and global consistency	p. 302
Message passing for conditional Gaussian distributions	p. 305
Using a junction tree with virtual evidence and soft evidence	p. 311
Notes	p. 313
Exercises: The junction tree and probability updating	p. 314
Factor graphs and the sum product algorithm	p. 319
Factorization and local potentials	p. 319
Examples of factor graphs	p. 320
The sum product algorithm	p. 323
Detailed illustration of the algorithm	p. 329
Notes	p. 332
Exercise: Factor graphs and the sum product algorithm	p. 333
References	p. 335
Index	p. 343
Table of Contents provided by Ingram. All Rights Reserved.