Qiongjie's Notes

This article introduces the combination of Bayesian methods and Markov Random Fields (MRF) through an image segmentation example. By introducing ‘Homogeneity Prior’ and ‘Gibbs Sampling’, it demonstrates how to use MRF to solve image denoising and segmentation problems, and compares the results of traditional Maximum Likelihood, MRF Simulated Annealing, and the Simple Relaxation (ICM) method commonly used in industry. [Read More]

This article introduces the core concepts of Markov Random Fields (MRF), extending from Markov Chains in time series to MRFs on spatial lattices. It defines local dependencies through ’neighborhoods’ and ‘cliques,’ and details how the Hammersley-Clifford theorem connects MRFs with Gibbs distributions. Finally, it demonstrates a Python implementation of image denoising using MRF combined with Simulated Annealing and Gibbs Sampling. [Read More]

When optimization problems are trapped in the maze of local optima, deterministic algorithms are often helpless. This article takes you into the world of stochastic optimization, exploring how to transform the problem of finding minimum energy into finding maximum probability. We will delve into the physical intuition and mathematical principles of the Simulated Annealing algorithm, demonstrate its elegant mechanism of ‘high-temperature exploration, low-temperature locking’ through dynamic visualization, and derive the Pincus Theorem in detail, mathematically proving why the annealing algorithm can find the global optimal solution. [Read More]

Deterministic optimization is the cornerstone for understanding modern MCMC algorithms (like HMC, Langevin). This article delves into three classic deterministic optimization strategies: Newton’s Method (second-order perspective using curvature), Coordinate Descent (the divide-and-conquer predecessor to Gibbs), and Steepest Descent (greedy first-order exploration). Through mathematical derivation and Python visualization, we compare their behavioral patterns and convergence characteristics across different terrains (convex surfaces, narrow valleys, strong coupling). [Read More]

When high-dimensional spaces are overwhelming, Gibbs sampling adopts a ‘divide and conquer’ strategy. By utilizing full conditional distributions, it breaks down complex N-dimensional joint sampling into N simple 1-dimensional sampling steps. This article explains its intuition, mathematical proof (Brook’s Lemma), and Python implementation. [Read More]

The original Metropolis is limited by symmetric proposals, often ‘hitting walls’ at boundaries or getting lost in high dimensions. The MH algorithm introduces the ‘Hastings Correction’, allowing asymmetric proposals (like Langevin dynamics) while maintaining detailed balance, significantly improving efficiency. [Read More]

The Metropolis algorithm is the cornerstone of MCMC. We delve into its strategy for handling unnormalized densities, from the random walk mechanism to sampling 2D correlated Gaussians, complete with Python implementation and visual diagnostics. [Read More]

Learn about Markov processes, stationary distributions, and convergence of Markov chains. [Read More]

A non-generative, self-supervised framework predicting high-level feature representations of masked regions from visible context, enabling scalable and efficient visual pretraining. [Read More]

Predict handcrafted features (e.g., HOG) of masked regions instead of raw pixels. [Read More]