PART II ADVANCED TOPICS
6. Non-parametric estimation 129
6.1 Introduction: free-form solutions 129
6.1.1 Singular value decomposition 130
6.1.2 A parametric free-form solution? 135
6.2 MaxEnt: images, monkeys and a non-uniform prior 136
6.2.1 Regularization 138
6.3 Smoothness: fuzzy pixels and spatial correlations 140
6.3.1 Interpolation 141
6.4 Generalizations: some extensions and comments 142
6.4.1 Summary of the basic strategy 144
6.4.2 Inference or inversion? 145
6.4.3 Advanced examples 148
7. Experimental design 149
7.1 Introduction: general issues 149
7.2 Example 7: optimizing resolution functions 151
7.2.1 An isolated sharp peak 152
7.2.2 A free-form solution 156
7.3 Calibration, model selection and binning 161
7.4 Information gain: quantifying the worth of an experiment 163
8. Least-squares extensions 165
8.1 Introduction: constraints and restraints 165
8.2 Noise scaling: a simple global adjustment 166
8.3 Outliers: dealing with erratic data 167
8.3.1 A conservative formulation 168
8.3.2 The good-and-bad data model 171
8.3.3 The Cauchy formulation 172
8.4 Background removal 173
8.5 Correlated noise: avoiding over-counting 174
8.5.1 Nearest-neighbour correlations 175
8.5.2 An elementary example 176
8.5.3 Time series 177
8.6 Log-normal: least-squares for magnitude data 179
9. Nested sampling 181
9.1 Introduction: the computational problem 181
9.1.1 Evidence and posterior 182
9.2 Nested sampling: the basic idea 184
9.2.1 Iterating a sequence of objects 185
9.2.2 Terminating the iterations 186
9.2.3 Numerical uncertainty of computed results 187
9.2.4 Programming nested sampling in ‘C’ 188
9.3 Generating a new object by random sampling 190
9.3.1 Markov chain Monte Carlo (MCMC) exploration 191
9.3.2 Programming the lighthouse problem in ‘C’ 192
9.4 Monte Carlo sampling of the posterior 195
9.4.1 Posterior distribution 196
9.4.2 Equally-weighted posterior samples: staircase sampling 197
9.4.3 The lighthouse posterior 198
9.4.4 Metropolis exploration of the posterior 199
9.5 How many objects are needed? 200
9.5.1 Bi-modal likelihood with a single ‘gate’ 200
9.5.2 Multi-modal likelihoods with several ‘gates’ 201
9.6 Simulated annealing 203
9.6.1 The problem of phase changes 203
9.6.2 Example: order/disorder in a pseudo-crystal 204
9.6.3 Programming the pseudo-crystal in ‘C’ 206
10. Quantification 209
10.1 Exploring an intrinsically non-uniform prior 209
10.1.1 Binary trees for controlling MCMC transitions 210
10.2 Example: ON/OFF switching 212
10.2.1 The master engine: flipping switches individually 212
10.2.2 Programming which components are present 212
10.2.3 Another engine: exchanging neighbouring switches 215
10.2.4 The control of multiple engines 216
10.3 Estimating quantities 216
10.3.1 Programming the estimation of quantities in ‘C’ 218
10.4 Final remarks 223
A. Gaussian integrals 224
A.1 The univariate case 224
A.2 The bivariate extension 225
A.3 The multivariate generalization 226
B. Cox’s derivation of probability 229
B.1 Lemma 1: associativity equation 232
B.2 Lemma 2: negation 235
Bibliography 237
Index 241