Schedules, SNR, and why resolution changes everything

Because natural images are spatially redundant and the noise is not.

Add i.i.d. noise of variance $\sigma^2$ per pixel. Now average over a $k \times k$ block. The signal survives almost intact — neighbouring pixels are highly correlated, so their mean is close to the true local mean — while the noise variance falls to $\sigma^2/k^2$. The effective SNR of the low-frequency content therefore improves by $k^2$ simply because more pixels are available to average.

Concretely, doubling the linear resolution gives four times as many pixels per unit of image content, so the recoverable low-frequency signal gains a factor of about 4 in SNR — roughly 1.4 nats of log-SNR per doubling, or about 5.5 nats going from 64 to 1024.

Consequences worth stating:

• A fixed schedule becomes effectively less noisy as resolution grows, so the model spends its capacity in the wrong place.
• The fix is a shift, not a rescale: subtract about $2\log(n/n_{\text{ref}})$ from log-SNR so that the same information is destroyed at the same point in the schedule.
• This also explains why latent diffusion (Module 8) needs less shifting — the latent has already removed much of the spatial redundancy, so its effective resolution is much smaller than the pixel resolution suggests.

Because uniform in $t$ is not uniform in anything meaningful, and $t$ is not the axis on which the learning problem is defined.

The mapping $t \mapsto \lambda$ is highly non-linear, so uniform $t$ concentrates samples wherever the schedule happens to move slowly through log-SNR. With the linear schedule that is the high-noise end, where the target is nearly degenerate and there is little to learn — you spend a large share of your compute on the easiest region.

Three better options:

1. Uniform in log-SNR. Sample $\lambda$ uniformly on its useful range and invert the schedule to get $t$. Simple, principled, and usually an improvement on its own.
2. A distribution over $\log\sigma$. EDM samples $\log\sigma \sim \mathcal{N}(-1.2, 1.2)$, deliberately concentrating on the middle band where the model is most uncertain and the sampler will later spend most of its steps.
3. Importance sampling by observed loss. Maintain a running estimate of loss per $\lambda$ bin and sample proportionally, which is a variance-reduction technique for the gradient estimator and adapts automatically as training progresses.

Note the interaction with Module 2: changing the $t$ distribution and changing the loss weighting have overlapping effects, since both scale the contribution of a noise level to the gradient. Do one deliberately rather than both accidentally, or you will not be able to attribute the result.

Where the trajectory has the most curvature, which is not where the noise is highest.

The probability-flow ODE (Module 4) is close to linear at very high noise — the score points more or less straight at the data mean, and large steps are accurate. It is also close to linear at very low noise, where the remaining update is a small refinement. The curvature is concentrated in the middle band, near $\lambda \approx 0$, which is exactly where the sample commits to what it is going to be.

This is why practical step schedules are not uniform in $t$:

• Karras spacing distributes $\sigma$ according to $\sigma_i \propto (\sigma_{\max}^{1/\rho} + \frac{i}{N-1}(\sigma_{\min}^{1/\rho} - \sigma_{\max}^{1/\rho}))^{\rho}$ with $\rho = 7$, which concentrates steps at low $\sigma$ while still taking a few large ones early.
• Uniform-in-$\lambda$ spacing is a reasonable default and much better than uniform in $t$.
• Very few steps at the very start is safe; very few in the middle is what produces the characteristic “the composition changed between step counts” instability.

The general framing is Module 4’s: this is a numerical integration problem, and step placement is adaptive quadrature. You put nodes where the integrand varies.

The schedule, before anything about architecture or data.

The signature is diagnostic. Global incoherence with good local detail means the model was never trained in the regime where global structure is uncertain — it has effectively only ever seen “refine this mostly-decided image” problems. That is exactly what happens when a schedule tuned at low resolution is reused at high resolution, because the same per-pixel noise destroys far less information when there are more pixels to average over.

Check: plot the effective log-SNR of your training distribution at this resolution. Better, look at samples of $x_t$ at several $t$ and ask a human at what point the composition becomes unrecoverable. If that point is at the extreme end of your schedule, almost all your training budget is downstream of the decision you care about.

Fix: shift the schedule toward higher noise by roughly $2\log(n/n_{\text{ref}})$ in log-SNR. Related mitigations that address the same problem from other directions: cascaded generation (make global decisions at low resolution and upsample), and training at multiple resolutions with the shift applied per resolution.

The general principle worth keeping: when a model fails at one spatial scale and not another, suspect the noise schedule before the architecture, because the schedule is what determines which scale each training example is asking about.