MCMC

When high-dimensional spaces are overwhelming, Gibbs sampling adopts a ‘divide and conquer’ strategy. By utilizing full conditional distributions, it breaks down complex N-dimensional joint sampling into N simple 1-dimensional sampling steps. This article explains its intuition, mathematical proof (Brook’s Lemma), and Python implementation. [Read More]

The original Metropolis is limited by symmetric proposals, often ‘hitting walls’ at boundaries or getting lost in high dimensions. The MH algorithm introduces the ‘Hastings Correction’, allowing asymmetric proposals (like Langevin dynamics) while maintaining detailed balance, significantly improving efficiency. [Read More]

The Metropolis algorithm is the cornerstone of MCMC. We delve into its strategy for handling unnormalized densities, from the random walk mechanism to sampling 2D correlated Gaussians, complete with Python implementation and visual diagnostics. [Read More]