Gibbs Sampling Explained: The Wisdom of Divide and Conquer

 Posted on January 30, 2026  |  2103 words  • Other languages: CH

When high-dimensional spaces are overwhelming, Gibbs sampling adopts a ‘divide and conquer’ strategy. By utilizing full conditional distributions, it breaks down complex N-dimensional joint sampling into N simple 1-dimensional sampling steps. This article explains its intuition, mathematical proof (Brook’s Lemma), and Python implementation. [Read More]
Gibbs Sampling  MCMC  Conditional Distribution  Bayesian Inference  Dimensionality Reduction  Python 

The Metropolis-Hastings Algorithm: Breaking the Symmetry

 Posted on January 29, 2026  |  3439 words  • Other languages: CH

The original Metropolis is limited by symmetric proposals, often ‘hitting walls’ at boundaries or getting lost in high dimensions. The MH algorithm introduces the ‘Hastings Correction’, allowing asymmetric proposals (like Langevin dynamics) while maintaining detailed balance, significantly improving efficiency. [Read More]
MCMC  MH Algorithm  Hastings Correction  Detailed Balance  Python  Bayesian Statistics 

Monte Carlo Sampling

 Posted on August 30, 2025  |  2912 words  • Other languages: CH

Understand the core concepts of Monte Carlo: Law of Large Numbers, rejection sampling, importance sampling, variance reduction techniques (antithetic variates, control variates, stratified sampling). [Read More]
Monte Carlo  Sampling  Mathematics  python 

Introduction to MCMC

 Posted on August 22, 2025  |  1757 words  • Other languages: CH

The reason we need MCMC is that many distributions are only known in their unnormalized form, making traditional sampling/integration methods ineffective. By constructing a ‘correct Markov chain’, we can obtain the target distribution from its stationary distribution, meaning the long-term distribution of the trajectory ≈ target distribution. [Read More]
Monte Carlo  Markov Chain  Sampling  Mathematics  python