Andrew G. Roberts

Constructing a Covariance Matrix in a Non-Standard Basis

linear-algebra

probability

A short note on constructing covariances using matrix decomposition.

Several Perspectives on the Ensemble Kalman Filter

A deep dive into the EnKF analysis update

Inverse-Problem

Data-Assimilation

Optimization

Sampling

Computational Statistics

This post motivates and explores the posterior approximation utilized by a single step of the celebrated EnKF algorithm.

Metropolis-Hastings Kernels for General State Spaces

MCMC

Sampling

Computational Statistics

An introduction to Tierney’s general formulation of Metropolis-Hastings algorithms.

Annealed Importance Sampling

MCMC

Sampling

Computational Statistics

An introduction to annealed importance sampling.

Doubly Intractable MCMC

Doubly intractable MCMC, auxiliary variable methods, and the exchange algorithm.

MCMC

Sampling

Computational Statistics

Exact MCMC with an intractable likelihood.

Transforming Simplex-values Parameters

Statistics

Walking through Stan’s parameter transformation for parameters that sum to one.

The Reversed Cholesky Decomposition

Linear-Algebra

Cholesky-like decomposition with upper-triangular matrices.

Regularized Least Squares with Singular Prior

Statistics

Data-Assimilation

Optimization

Inverse-Problem

Solving the regularized least squares optimization problem when the prior covariance matrix is not positive definite.

Nonlinear Least Squares

Statistics

Optimization

Inverse-Problem

Gauss-Newton, Levenberg-Marquardt

Ensemble Kalman Methods for Solving Inverse Problems: An Introduction

Inverse-Problem

Data-Assimilation

Optimization

Sampling

Computational Statistics

EKI

The first post in a series on using Ensemble Kalman methods to approximately solve Bayesian inverse problems.

Pseudo-Marginal MCMC

Pseudo-Marginal MCMC

MCMC

Sampling

Computational Statistics

MCMC with an unbiased density approximation.

Linear Gaussian Inverse Problems

Derivations and discussion of linear Gaussian inverse problems.

statistics

inverse-problem

\[ \newcommand{\R}{\mathbb{R}} \newcommand{\E}{\mathbb{E}} \newcommand{\Pr}{\mathbb{P}} \newcommand{\given}{\mid} \newcommand{\Def}{:=} \newcommand{\Cov}{\mathrm{Cov}} \newco…

Basis Expansions for Black-Box Function Emulation

Gaussian-Process

A discussion of the popular output dimensionality strategy for emulating multi-output functions.

A Few Different Adaptive Metropolis Schemes

Adaptively updating a Gaussian proposal covariance for Random Walk Metropolis-Hastings samplers.

MCMC

Computational Statistics

Random Walk Metropolis-Hastings (RWMH) is a workhorse Markov Chain Monte Carlo (MCMC) algorithm for sampling from arbitrary probability distributions. The key to getting it…

Gaussian Measures, Part 2 - The Multivariate Case

Gaussian measures in finite dimensions; i.e., multivariate Gaussians/Gaussian vectors.

{% katexmm %} Let \(x, y \in \mathbb{R}^n\). Throughout this post, we write \(\langle x, y \rangle = x^\top y\) for the standard inner product on \(\mathbb{R}^n\), and \(\lVe…

Gaussian Measures, Part 1 - The Univariate Case

A brief introduction to Gaussian measures in one dimension.

GP

Prob-Theory

I intend for this to be part one of a (at least) three part series on Gaussian measures, with the ultimate goal being to understand Gaussian processes as random elements in…

Approximating Nonlinear Functions of Gaussians

And the Extended Kalman Filter

Data-Assimilation

I discuss the generic problem of approximating the distribution resulting from a non-linear transformation of a Gaussian random variable, and then show how this leads to extensions of the Kalman filter which yield approximate filtering algorithms in the non-linear setting.

Introduction to Gaussian Process Priors and Hyperparameter Estimation

Statistics

Gaussian-Process

kernel-methods

A deep dive into hyperparameter specifications for GP mean and covariance functions, including both frequentist and Bayesian methods for hyperparameter estimation.

Deriving the Metropolis-Hastings Update from the Transition Kernel

MCMC

Probability

Computational Statistics

The Metropolis-Hastings (MH) Markov Chain Monte Carlo (MCMC) method is typically introduced in the form of a practical algorithm. In a more theoretically-oriented context…

Principal Components Analysis

Statistics

Linear-Algebra

A deep dive into PCA.