Authors: Jonathan Ho, Ajay Jain, Pieter Abbeel (UC Berkeley) · arXiv: 2006.11239
Established that optimizing the variational lower bound (ELBO) of a T-step Gaussian diffusion chain simplifies to a reweighted Mean Squared Error objective predicting the added noise epsilon, linking thermodynamic diffusion with denoising score matching.
Key Quantitative & Theoretical Takeaway: The simplified surrogate objective L_simple(theta) discards variational weighting terms for small t, directing model capacity toward perceptual visual quality rather than imperceptible high-frequency bits.
We present high quality image synthesis results using diffusion probabilistic models, a class of latent variable models inspired by considerations from non-equilibrium thermodynamics. Our best results are obtained by training on a weighted variational bound designed according to a novel connection between diffusion models and denoising score matching with Langevin dynamics.
By conditioning the forward Gaussian process on x_0 using Bayes rule, each KL divergence term L_{t-1} compares two Gaussians whose means can be parameterized in terms of the added noise epsilon. Dropping the time-dependent variance weights yields the unweighted MSE noise-prediction objective L_simple.
Because the forward process q(x_{1:T}|x_0) has no learnable parameters and L_T is a constant KL divergence to a standard Gaussian when beta_t follows a linear schedule from 10^-4 to 0.02.
Replace arxiv.org with nujan.app on any arXiv URL (for example, https://nujan.app/abs/2006.11239) to open the PDF and AI research partner side-by-side.