Carlos Daniel Paulino
Peter Müller
1 Bayesian Inference 1
1.1 The Classical Paradigm 2
1.2 The Bayesian Paradigm 5
1.3 Bayesian Inference 8
1.3.1 Parametric Inference 8
1.3.2 Predictive Inference 12
1.4 Conclusion 13
Problems 14
2 Representation of Prior Information 17
2.1 Non-Informative Priors 18
2.2 Natural Conjugate Priors 23
Problems 26
3 Bayesian Inference in Basic Problems 29
3.1 The Binomial ∧ Beta Model 29
3.2 The Poisson ∧ Gamma Model 31
3.3 Normal (Known μ) ∧ Inverse Gamma Model 32
3.4 Normal (Unknown μ,σ 2 ) ∧ Jeffreys’ Prior 33
3.5 Two Independent Normal Models ∧ Marginal Jeffreys’ Priors 34
3.6 Two Independent Binomials ∧ Beta Distributions 35
3.7 Multinomial ∧ Dirichlet Model 37
3.8 Inference in Finite Populations 40
Problems 41
4 Inference by Monte Carlo Methods 45
4.1 Simple Monte Carlo 45
4.1.1 Posterior Probabilities 48
4.1.2 Credible Intervals 49
vi Contents
4.1.3 Marginal Posterior Distributions 50
4.1.4 Predictive Summaries 52
4.2 Monte Carlo with Importance Sampling 52
4.2.1 Credible Intervals 56
4.2.2 Bayes Factors 58
4.2.3 Marginal Posterior Densities 59
4.3 Sequential Monte Carlo 61
4.3.1 Dynamic State Space Models 61
4.3.2 Particle Filter 62
4.3.3 Adapted Particle Filter 64
4.3.4 Parameter Learning 65
Problems 66
5 Model Assessment 72
5.1 Model Criticism and Adequacy 72
5.2 Model Selection and Comparison 78
5.2.1 Measures of Predictive Performance 79
5.2.2 Selection by Posterior Predictive Performance 83
5.2.3 Model Selection Using Bayes Factors 85
5.3 Further Notes on Simulation in Model Assessment 87
5.3.1 Evaluating Posterior Predictive Distributions 87
5.3.2 Prior Predictive Density Estimation 88
5.3.3 Sampling from Predictive Distributions 89
Problems 90
6 Markov Chain Monte Carlo Methods 92
6.1 Definitions and Basic Results for Markov Chains 93
6.2 Metropolis–Hastings Algorithm 96
6.3 Gibbs Sampler 100
6.4 Slice Sampler 107
6.5 Hamiltonian Monte Carlo 109
6.5.1 Hamiltonian Dynamics 109
6.5.2 Hamiltonian Monte Carlo Transition Probabilities 113
6.6 Implementation Details 115
Problems 118
7 Model Selection and Trans-dimensional MCMC 131
7.1 MC Simulation over the Parameter Space 132
7.2 MC Simulation over the Model Space 133
7.3 MC Simulation over Model and Parameter Space 138
7.4 Reversible Jump MCMC 140
Problems 145
Contents vii
8 Methods Based on Analytic Approximations 152
8.1 Analytical Methods 153
8.1.1 Multivariate Normal Posterior Approximation 153
8.1.2 The Classical Laplace Method 156
8.2 Latent Gaussian Models (LGM) 161
8.3 Integrated Nested Laplace Approximation 163
8.4 Variational Bayesian Inference 166
8.4.1 Posterior Approximation 166
8.4.2 Coordinate Ascent Algorithm 167
8.4.3 Automatic Differentiation Variational Inference 170
Problems 171
9 Software 174
9.1 Application Example 175
9.2 The BUGS Project: WinBUGS and OpenBUGS 176
9.2.1 Application Example: Using R2OpenBUGS 177
9.3 JAGS 183
9.3.1 Application Example: Using R2jags 183
9.4 Stan 187
9.4.1 Application Example: Using RStan 188
9.5 BayesX 194
9.5.1 Application Example: Using R2BayesX 196
9.6 Convergence Diagnostics: the Programs CODA and BOA 200
9.6.1 Convergence Diagnostics 201
9.6.2 The CODA and BOA Packages 203
9.6.3 Application Example: CODA and BOA 205
9.7 R-INLA and the Application Example 215
9.7.1 Application Example 217
Problems 224
Appendix A. Probability Distributions 224
Appendix B. Programming Notes 229
References 234
Index