The forward process, and why it is Gaussian

Three requirements have to hold at once.

1. Closed-form marginals. Composing Gaussians gives a Gaussian, so $q(x_t \mid x_0)$ is known analytically and training samples $t$ directly. With almost any other noise family you would have to simulate the chain per example — hundreds of forward passes per gradient step.
2. Gaussian reverse steps. For a small enough step, the reverse conditional $q(x_{t-1} \mid x_t)$ is approximately Gaussian. This is what licenses a model that outputs a mean (and optionally a variance) rather than an arbitrary distribution. If reverse steps were not approximately Gaussian you would need a far more expressive per-step model, and the step count could not be traded against quality the way it is.
3. Tweedie. The score is recoverable from the posterior mean, so an MSE-trained denoiser is a score model. This is a property of the exponential family, not a general fact.

Discrete corruption processes — masking, absorbing states, multinomial diffusion — drop the second and third and replace them with a categorical machinery. They work, and they are the right choice for genuinely discrete data, but they lose the ODE view in Module 4 and with it most of the fast samplers.

It collapses two apparently different research programmes into one, and it tells you what your network is really computing.

Concretely: train $\varepsilon_\theta(x_t, t)$ under $\mathbb{E}\lVert\varepsilon - \varepsilon_\theta\rVert^2$. The Bayes-optimal predictor of an MSE objective is the conditional expectation, so at convergence $\varepsilon_\theta = \mathbb{E}[\varepsilon \mid x_t]$. Substituting the forward marginal and rearranging,

So the noise prediction is the score up to a known scalar. Three consequences follow immediately. The reverse SDE and the probability-flow ODE (Module 4) can be written directly in terms of the network output, so every ODE solver in the numerical analysis literature becomes available. Guidance (Module 6) is score arithmetic — you add scaled score terms, which is only meaningful because the output is a score. And you can convert freely between $\varepsilon$-, $x_0$-, $v$- and score-parameterisations at inference, because they are affine reparameterisations of one quantity (Module 2).

The mental shift worth making: the network is not learning “how to remove noise”. It is learning a vector field — the gradient of the log-density of the data blurred at scale $t$ — and denoising is simply the cheapest supervised way to fit it.

No. They are the same family under a change of variables, and treating them as rival designs is a historical artefact of the two lineages meeting late.

Set $\sigma_t^2 = (1-\bar\alpha_t)/\bar\alpha_t$. Then the VP sample is exactly the VE sample rescaled: $x^{VP}_t = x^{VE}_t / \sqrt{1+\sigma_t^2}$, and the scores relate by the same factor. Karras et al. made this explicit by writing all of them as one ODE with a choice of scaling $s(t)$ and noise $\sigma(t)$, then arguing those choices should be made on numerical grounds rather than probabilistic ones.

What genuinely differs is conditioning of the numerics, and that is not nothing. VP keeps the network’s input variance around 1 at every noise level, which is friendly to normalisation layers. VE inputs span orders of magnitude, so it needs explicit input scaling and a $\sigma$-dependent output skip connection to stay well-conditioned — the preconditioning that the EDM formulation makes explicit.

So the right question is never “VP or VE” but “what scaling keeps the network’s inputs, targets and loss weighting well-conditioned across noise levels”, which is Module 2.

Because the terminal signal-to-noise ratio is not actually zero, so training and sampling start from different distributions.

With the standard linear schedule, $\bar\alpha_T \approx 4\times10^{-5}$, giving $\sqrt{\bar\alpha_T} \approx 0.0063$. Tiny — but the mean brightness of an image is a very low-frequency, high-variance-across-dataset quantity, so that small surviving component is informative. During training the model always sees an $x_T$ that quietly encodes the image’s mean level, and it learns to rely on it. At sampling time you hand it pure $\mathcal{N}(0, I)$, whose mean is zero. The model duly produces images with roughly average brightness, and cannot be prompted out of it.

The fix is to rescale the schedule so $\bar\alpha_T = 0$ exactly, which requires also switching to a $v$-parameterisation (Module 2), because $\varepsilon$-prediction is degenerate at zero SNR — if there is no signal, predicting the noise is trivial and gives no gradient toward $x_0$.

The general lesson is worth more than the specific bug: any mismatch between the training distribution of $x_T$ and the sampling prior shows up as a systematic bias in the output, and because it lives at the highest noise level it affects the most global attributes — brightness, colour balance, overall composition.