发帖
雷达卡
Stat 535C - CPSC 535D
Course scheduleTwo lectures per week (Tuesday and Thursday from
10.00 to 11.30) in LSK 301.
If you want to arrange a meeting, just send me an email at the following address arnaud at cs dot ubc dot ca
Announcements
- I have posted the slides of Lecture 21.
Assignements
- Assignment #1 Pdf Check here the results you should have obtained
- Assignement #2 Pdf Dataset (matlab format) Dataset (txt file)
Handouts
- 10/01/06: Lecture 2 - Sufficiency, Likelihood and Conditionality Principles Revised version 10/01/06 Pdf Ps Ps-4pages
- 12/01/06: No lecture
- 19/01/06: Lecture 4 - More Bayesian Statistics (Examples, Testing hypothesis, Bayes factors) Revised version 23/01/06 Pdf Ps Ps-4pages
- h6 by B. Vidakovic
- R. Kass and A. Raftery, Bayes Factors, JASA, 1995 paper
- R. Kass, Bayes Factors in Practice, The Statistician, 1992 here
- M. Lavine and M.J. Schervish, Bayes Factors: What they are and what they are not, The American Statistician, 1999 here
- 24/01/06: Lecture 5 - And more Bayesian Statistics (Bayesian model selection) Revised version 24/01/06 Pdf Ps Ps-4pages
- Chapter 7 of the Bayesian Choice by C.P. Robert
- J. Hoeting, D. Madigan, A. Raftery and C. Volinsky, Bayesian model averaging: A tutorial, Statistical Science, 1999 here
- A. Raftery, D. Madigan and J. Hoeting, Bayesian model averaging for linear regression models, JASA, 1997 here
- P. Brown, T. Fearn and M. Vannucci, Bayesian wavelet regression on curves with application to a spectroscopic calibration problem, JASA, 2001 here
- 26/01/06: Lecture 6 - And more Bayesian Statistics (From prior information to prior distribution) Pdf Ps Ps-4pages
- Section 3.1 and 3.2 of Monte Carlo Statistical Methods.
Additional reading:
- Chapter 2 of Monte Carlo Statistical Methods.
- Scale mixture of Gaussians, JRSS B, 1974 here: very useful representation of non-Gaussian distributions as infinite mixture of Gaussians
- W. Gilks and P. Wild, Adaptive rejection sampling for Gibbs sampling, Applied Statistics, 1992 here
- B.D. Flury, Rejection sampling made easy, SIAM Review, 1990 here
More advanced
- A. Peterson and R. Kronmal, On mixture methods for the computer generation of random variables, The American Statistician, 1982 here
- J. Halton, Reject the rejection technique, J. Scientific Computing, 1992. (by the way please don't reject it)
- A. Beskos and G. Roberts, Exact simulation of diffusions, Annals of Applied Proba, 2005. here
Check Proposition 1 and its proof for a very clever and useful remark about rejection sampling. Additional reading:
- Chapter 3 of Monte Carlo Statistical Methods.
- Y. Chen, Another look at rejection sampling through importance sampling, Stat. Proba. Lett., 2005 here
- J. Geweke, Bayesian inference in econometric models using Monte Carlo integration, Econometrica, 1989 here
- H. Van Dijk, J. Hop, A. Louter, An Algorithm for the Computation of Posterior Moments and Densities Using Simple Importance Sampling, The Statistician, 1987 here
Optional reading
- A. Owen and Y. Zhou, Safe and effective importance sampling, JASA, 2000 here
Additional reading:
- D. Mackay, Introduction to Monte Carlo methods, here
- R. Neal, Probabilistic Inference Using Markov Chain Monte Carlo Methods, Technical report, 1993 here
- C. Andrieu, A. Doucet, N. De Freitas and M. Jordan, Markov chain Monte Carlo for Machine Learning, Machine Learning, 2003 here
- S. Brooks, Markov chain Monte Carlo Methods and Its Application, The Statistician, 1998 here
- G. Casella and E.I. George, Explaining the Gibbs sampler. The American Statistician, 1992 here
- S. Chib and E. Greenberg, Understanding the Metropolis-Hastings algorithm, The American Statistician, 1995 here
- 21/02/06: Lecture 11 - Gibbs samplers - Case Studies 1: Variable Selection and Mixture Models
Pdf
Ps
Ps-4pages
- 23/02/06: Lecture 12 - Gibbs samplers - Case Studies 2: Time Series Models Revised version 27/02/06
Pdf
Ps
Ps-4pages
- B. Carlin, N. Polson and D. Stoffer, A Monte Carlo Approach to Nonnormal and Nonlinear State-Space Modeling, JASA, 1992 here
- C. Carter and R. Kohn, On Gibbs Sampling for State-Space Models, Biometrika, 1994 [url=http://www.jstor.org/cgi-bin/jstor/printpage/00063444/di992425/99p0485a/0.pdf?backcontext=results&dowhat=Acrobat&config=&userID=8e670872@ubc.ca/01cce44035244fd109897ca0ea&0.pdfhttp://www.jstor.org/cgi-bin/jst ... ext=results&dowhat=Acrobat&config=&userID=8e670872@ubc.ca/01cce44035244fd109897ca0ea&0.pdf]here[/url]
- S. Chib, Calculating Posterior Distributions and Modal Estimates in Markov Mixture Models, J. Econometrics, 1996 here
- E. Jacquier, N. Polson and P. Rossi, Bayesian Analysis of Stochastic Volatility Models, J. Bus. Econ. Statist., 1994 here
- S. Chib and E. Greenberg, Understanding the Metropolis-Hastings algorithm, The American Statistician, 1995 here
- Chapter 7 of Robert & Casella.
- You can play with the following java applets.
- 02/03/06: Lecture 14 - More about the Metropolis-Hastings Algorithm: mixture, composition, hybrid algorithms Pdf Ps Ps-4pages
- 07/03/06: Lecture 15 - MH algorithm - Case Studies 3: Generalized Linear Models Revised version 08/02/06 Pdf Ps Ps-4pages
- As an exercise, you could fit the logistic model p. 15 of Robert & Casella.
- The bank dataset is here
- Another less trivial but interesting example is here (start with the number of sinusoids fixed)
- L. Held, Conditional Prior Proposals in Dynamic Models, Scand. J. Statist., 1999 Pdf file here
- M.K. Pitt & N. Shephard, Likelihood Analysis of Non-Gaussian Measurement Time Series, Biometrika, 1996 Pdf file here - Chapter 11 of Robert & Casella
- P.J. Green, Transdimensional Markov chain Monte Carlo, Highly Structured Stochastic Systems, OUP, 2003 Pdf file here
- S. Sisson, [size=-1]Trans-dimensional Markov chains: A decade of progress and future perspectives., JASA, 2005 Pdf file here
- Chapter 11 of Robert & Casella
- P.J. Green, Transdimensional Markov chain Monte Carlo, Highly Structured Stochastic Systems, OUP, 2003 Pdf file here
- S. Sisson, [size=-1]Trans-dimensional Markov chains: A decade of progress and future perspectives, JASA, 2005 Pdf file here
- Chapter 8 of Robert & Casella
- C. Andrieu, L. Breyer & A. Doucet, Convergence of Simulated Annealing using Foster-Lyapunov Criteria, Journal Applied Probability, 2001. Pdf file here
- Paul Damien, Jon Wakefield, Stephen Walker, Gibbs Sampling for Bayesian Non-Conjugate and Hierarchical Models by Using Auxiliary Variables, JRSS B, 1999 Pdf file here
- R. Neal, Sampling from Multimodal Distributions using Tempered Transitions, Statistics and Computing, 1996 Pdf file here
- C. Geyer & E. Thompson, Annealing Markov Chain Monte Carlo with Applications to Ancestral Inference, JASA, 1995 [url=http://www.jstor.org/cgi-bin/jstor/printpage/01621459/di986005/98p0228m/0?frame=noframe&dpi=3&userID=8e670872@ubc.ca/01cc99333c4a7310a18efee15&backcontext=page&backurl=/cgi-bin/jstor/viewitem/01621459/di986005/98p0228m/0%3fframe%3dnoframe%26dpi%3d3%26userID%3d8e670872@ubc.ca/01cc99333c4a7310a18efee15%26config%3d%26PAGE%3d0&action=download&config=jstor]Pdf file here[/url]
- Chapter 11 of Robert & Casella
- A. Doucet, N. De Freitas and N.J. Gordon, An introduction to Sequential Monte Carlo, Ps file here
- J. Carpenter, P. Clifford and P. Fearnhead, An Improved Particle Filter for Non-linear Problems, Pdf file here
- A. Doucet, S.J. Godsill and C. Andrieu, On Sequential Monte Carlo sampling methods for Bayesian filtering, Stat. Comp., 2000 (reprinted 2005) Pdf file here
- M.K. Pitt and N. Shephard, Filtering via Simulation: Auxiliary Particle Filter, JASA, 1999 Pdf file here
- 30/03/06: Lecture 22 - More Sequential Monte Carlo: Beyond standard optimal filtering Pdf Ps Ps-4pages
- A. Kong, J.S. Liu and W.H. Wong, Sequential Imputations and Bayesian Missing Data Problems, JASA, 1994 Pdf file here
- R. Chen and J.S. Liu, Predictive Updating Methods with Application to Bayesian Classification, JRSS B, 1996 Pdf file here
- J.S. Liu and R. Chen, Sequential Monte Carlo methods for dynamic systems, JASA, 1998 Pdf file here
- P. Del Moral, A. Doucet and A. Jasra, Sequential Monte Carlo samplers, JRSSB, 2006 Pdf file here
- P. Del Moral, A. Doucet and A. Jasra, Sequential Monte Carlo for Bayesian Computation, Bayesian Statistics, 2006 Pdf file here (first draft! do not distribute!)
ObjectivesTo provide students an introduction to modern computational methods used in (Bayesian) statistics. The computational methods
presented here will be illustrated by a large number of complex statistical models: (dynamic) generalised linear models, mixture
and hidden Markov models, Dirichlet processes, nonlinear regression and classification models, stochastic volatility models etc.
Course contents
- Introduction to Bayesian Statistics.
- Probability as measure of uncertainty.
- Posterior distribution as compromise between data and prior information.
- Prior distributions: conjugacy and noninformative priors.
- Bayes factors.
- Large sample inference.
- Introduction to Monte Carlo Methods
- Limitations of deterministic numerical methods.
- Monte Carlo integration and Non-Uniform random variable generation (inverse method, accept/reject)
- Importance sampling.
- Variance reduction techniques (Rao-Blackwellisation, antithetic variables).
- Markov Chain Monte Carlo Methods - Basics
- Introduction to general state-space Markov chain theory.
- Metropolis-Hastings algorithm.
- Gibbs sampler.
- Hybrid algorithms.
- Case studies: Capture-Recapture experiments, Regression and Variable selection, Generalised linear models, Models for Robust inference
- Case studies: Mixture models and Hidden Markov models, Nonparametric Bayes, Markov random fields.
- Markov Chain Monte Carlo Methods - Advanced Topics
- Variable dimension algorithms (Reversible jump MCMC).
- Simulated tempering.
- Monte Carlo optimization (MCEM, simulated annealing).
- Perfect simulation.
- Case studies: Nonlinear Regression and Variable selection, Mixture models, Hidden Markov models, Bayes CART.
- Sequential Monte Carlo Methods & Particle Filtering Methods
- Dynamic generalized linear models, hidden Markov models, nonlinear non-Gaussian state-space models.
- Sequential importance sampling and resampling.
- Filtering/smoothing and parameter estimation.
- Sequential Monte Carlo for static problems and extensions.
- Case studies: Switching State-Space models, Stochastic Volatility models, Contingency tables, Linkage analysis.
- Christian P. Robert and George Casella, Monte Carlo Statistical Methods, Springer, 2nd edition
- Jean-Michel Marin and Christian P. Robert, Bayesian Core: A Practical Approach to Computational Bayesian Statistics, Springer, to appear.
- Denis G.T. Denison, Chris C. Holmes, Bani K. Mallick and Adrian F.M. Smith, Bayesian Methods for Nonlinear Classification and Regression, Wiley.
- Arnaud Doucet, Nando De Freitas and Neil J. Gordon (eds), Sequential Monte Carlo in Practice, Springer.
- Andrew Gelman, John B. Carlin, Hal Stern and Donald B. Rubin, Bayesian Data Analysis, Chapman&Hall/CRC, 2nd edition.
- Christian P. Robert, The Bayesian Choice, Springer, 2nd edition.
Grading
This will be based on several assignments, a midterm exam and a final project (exact weighting yet to be decided). The computational part of the
assignments will be done using the R statistical language or Matlab. If you don't know what these are, I urge you to familiarize yourself with them.
Note that R is open source and can be downloaded for free.
Some interesting links - other Bayesian computational courses
- Francesca Dominici's course at Johns Hopkins
- Sujit Ghosh's course at NCSU
- Paula Sebastiani's course at UMass
- Lawrence Joseph's course at McGill
- Gary Rosner's course at Rice
- Kate Cowles's course at Iowa
- Brani Vidakovic's course at Georgia Tech
- David Madigan's course at Rutgers
- Bayesian Java applications
- There are R Tutorials here
- The BUGS Project
- R
- Figueiredo's Bayesian Tutorial
- Wasserman's Decision Theory Notes
- Stuart Cole's Bayesian Notes
京公网安备 11010802022788号
