Andrew G. Roberts

Metropolis-Hastings Kernels for General State Spaces

An introduction to Tierney’s general formulation of Metropolis-Hastings algorithms.

MCMC

Sampling

Computational Statistics

\[ \newcommand{\R}{\mathbb{R}} \newcommand{\E}{\mathbb{E}} \newcommand{\Pr}{\mathbb{P}} \newcommand{\given}{\mid} \newcommand{\Def}{:=} \newcommand{\stateSpace}{\mathsf{X}} \…

Annealed Importance Sampling

MCMC

Sampling

Computational Statistics

In this post we walk through the annealed importance sampling (AIS) scheme introduced in the classic paper Neal (1998). The algorithm seeks to improve upon standard IS by…

Doubly Intractable MCMC

Doubly intractable MCMC, auxiliary variable methods, and the exchange algorithm.

MCMC

Sampling

Computational Statistics

Exact MCMC with an intractable likelihood.

Transforming Simplex-values Parameters

Statistics

Walking through Stan’s parameter transformation for parameters that sum to one.

The Reversed Cholesky Decomposition

Linear-Algebra

Cholesky-like decomposition with upper-triangular matrices.

Regularized Least Squares with Singular Prior

Statistics

Data-Assimilation

Optimization

Inverse-Problem

Solving the regularized least squares optimization problem when the prior covariance matrix is not positive definite.

Nonlinear Least Squares

Statistics

Optimization

Inverse-Problem

Gauss-Newton, Levenberg-Marquardt

Ensemble Kalman Methods for Solving Inverse Problems: An Introduction

Inverse-Problem

Data-Assimilation

Optimization

Sampling

Computational Statistics

EKI

The first post in a series on using Ensemble Kalman methods to approximately solve Bayesian inverse problems.

Pseudo-Marginal MCMC

Pseudo-Marginal MCMC

MCMC

Sampling

Computational Statistics

MCMC with an unbiased density approximation.

Linear Gaussian Inverse Problems

Derivations and discussion of linear Gaussian inverse problems.

statistics

inverse-problem

This post focuses on Bayesian inverse problems with the following features: - Linear forward model. - Additive Gaussian observation noise. - Gaussian prior distribution. -…

Basis Expansions for Black-Box Function Emulation

Gaussian-Process

A discussion of the popular output dimensionality strategy for emulating multi-output functions.

Approximating Nonlinear Functions of Gaussians

And the Extended Kalman Filter

Data-Assimilation

I discuss the generic problem of approximating the distribution resulting from a non-linear transformation of a Gaussian random variable, and then show how this leads to extensions of the Kalman filter which yield approximate filtering algorithms in the non-linear setting.

Introduction to Gaussian Process Priors and Hyperparameter Estimation

Statistics

Gaussian-Process

kernel-methods

A deep dive into hyperparameter specifications for GP mean and covariance functions, including both frequentist and Bayesian methods for hyperparameter estimation.

Deriving the Metropolis-Hastings Update from the Transition Kernel

MCMC

Probability

Computational Statistics

The Metropolis-Hastings (MH) Markov Chain Monte Carlo (MCMC) method is typically introduced in the form of a practical algorithm. In a more theoretically-oriented context…

Principal Components Analysis

Statistics

Linear-Algebra

A deep dive into PCA.