APSTA-GE 2123: Bayesian Inference
This is the home of the Spring 2025 Bayesian Inference class at NYU Steinhardt. If you are enrolled in the course, the assignment will be posted on Brightspace.
This is the current version of the syllabus. Please check the date, as it is subject to change.
Lectures
The following is a preliminary lecture plan. We will add the links after each lecture.
- Lecture 01: Introduction and Bayesian Workflow
- Lecture 02: Conjugate Models and Beta-Binomial
- Lecture 03: More Conjugate Models and Introduction to Posterior Sampling
- Lecture 04: MCMC, Posterior Inference, and Prediction
- Lecture 05: Linear Regression and Model Evaluation
- Lecture 06: Expanding the Linear Model and Modeling Counts
- Lecture 07: Logistic regression and introduction to hierarchical models
Background Resources
The following resources may be helpful to those who need a refresher on the prerequisites.
R for Data Science 2e, Wickham et al.
R is excellent for data manipulation, exploratory data analysis, and statistical graphics. This book covers it all. Note that their definition of workflow is different from Bayesian workflow, as the latter is focused a lot more on model building and model checking.
Calculus Made Easy, Thompson
This is a funny and timeless classic, originally published in 1910, which demystifies calculus and includes gems like the following.
Thus \(\int \text{d}x\) means the sum of all the little bits of \(x\) ; or \(\int \text{d}t\) means the sum of all the little bits of \(t\). Ordinary mathematicians call this symbol “the integral of.” Now any fool can see that if \(x\) is considered as made up of a lot of little bits, each of which is called \(\text{d}x\), if you add them all up together you get the sum of all the \(\text{d}x\)’s, (which is the same thing as the whole of \(x\)). The word “integral” simply means “the whole.”
Calculus, Strang et al.
This is a traditional calculus textbook written by pedagogical masters like Gilbert Strang of the linear algebra fame and is offered for free by OpenStax.
YouTube: Essence of Calculus, Sanderson
Grant Sanderson is the owner and operator of the
3blue1brownYouTube channel, which contains very pretty visualizations of complex mathematical concepts. Even if you are familiar with Calculus, Essense of Calculus will likely give you a more enlightening perspective on the subject.Intoduction to Probability, Blitzstein et al.
Introduction to Probability is a pedagogical masterpiece, and the corresponding YouTube videos are great as well. Bayesian inference takes probability seriously, and the students should be familiar with chapters 1 through 10.
Introduction to Probability Cheatsheet v2, Chen et al.
A 10-page summary of a 600-page book.
Additional Bayesian Resources
Statistical Rethinking, McElreath
We recommend this book to people who want to learn statistics the right way, which is to say by using the rules of conditional probability without confusing and often misused constructs like p-values and hypothesis tests. Most examples are drawn from evolutionary anthropology, Richard’s field of study. It relies heavily on the R language and Richard’s rethinking R package.
A Student’s Guide to Bayesian Statistics, Lambert
Ben Lambert’s is a depth-first approach to Bayes. He doesn’t rely on high-level interface packages like
rethinking,rstanarm, orbrms, but instead goes right into Stan, which has the benefit of understanding what the model is doing. Ben also provides a helpful comparison to frequentist inference for students who first learned statistics the wrong way1. His corresponding video course is excellent as well.Bayesian Data Analysis, Gelman et al.
BDA3, as the book is called among friends, is an advanced treatment of the subject, not because of the technical content per se, but because of what it assumes the reader knows prior to opening the book. So if your \(p(\theta)\) is moderately informative, your \(p(\theta \mid y)\) should be pretty tight. See what I did there?
Stan User’s Guide, Stan Development Team
Stan User’s Guide is a modest name for the best instruction manual to coding statistical models in Stan starting with a simple linear regression and covering advanced topics like Ordinary Differential Equation (ODE) models, model evaluation, poststratification, decision analysis, and many others. The latter topic is a particularly natural consequence of Bayesian inference.
Bayesian Data Analysis Course, Vehtari
Aki Vehtari teaches a full-semester BDA course at Aalto University in Finland and generously puts all the content online. This is an advanced class for people who like to dig into the details of Bayesian computation. The content is excellent and is highly recommended.
Ten Great Ideas About Chance, Diaconis and Skyrms
Ten Great Ideas About Chance is about the history and philosophy of Bayesian thinking. It is targeted at scientists and students familiar with the basic mathematics of probability.