Fixed-point and rounding

Order matters here because the cheap checks eliminate most cases.

1. Per-tensor versus per-channel scale. One channel with a 100× larger range forces a scale that crushes every other channel to a handful of levels. Per-output-channel scaling for weights is nearly free and usually the whole problem.
2. Are you saturating or wrapping? If wrapping, a single outlier flips sign and the layer outputs garbage. Check the overflow behaviour before anything statistical.
3. Accumulator width. $W + \lceil \log_2 N \rceil$. If someone accumulated int8 products in int16, a 512-term dot product overflows routinely.
4. Symmetric versus asymmetric. Post-ReLU activations are one-sided; a symmetric (zero-point free) scheme wastes half its range on values that never occur, costing a full bit.
5. Calibration data. An unrepresentative calibration set gives ranges that clip real inputs.
6. Activation outliers. In transformers a few channels run 100× larger than the rest, which destroys per-tensor activation scales specifically. The fixes are per-channel or per-group scaling, or keeping the outlier channels in higher precision.

Worth noting the counter-intuitive one: clipping outliers often improves accuracy. Clipping at a percentile instead of the max sacrifices a few saturated values and buys resolution everywhere else, and the trade is usually strongly favourable.

Because it is biased, and bias accumulates coherently while random error accumulates in quadrature.

Truncation of a two’s-complement value rounds toward $-\infty$, so its error is uniform on $[-1, 0]$ LSB with mean $-0.5$. Over $N$ operations the expected total is $-0.5N$: for $N = 512$, an offset of 256 LSB. Round-to-nearest-even has zero mean and error uniform on $[-0.5, +0.5]$, standard deviation $\approx 0.29$ LSB, so over $N$ operations the deviation is $0.29\sqrt{N} \approx$ 6.5 LSB.

That is a factor of forty at $N = 512$, and the gap widens with $N$ because one grows linearly and the other as a square root. Then it compounds through layers: a systematic offset entering a nonlinearity shifts its operating point, and the next layer amplifies a bias it treats as signal. Random error largely cancels in the same situation.

The practical statement: the choice between truncate and round-to-nearest-even is not a rounding detail, it is the difference between error that grows like $N$ and error that grows like $\sqrt{N}$. Round-half-up is also biased (toward $+\infty$ on ties) which is why the standard is specifically round-to-nearest-even — ties go both ways, so even the tie-breaking is unbiased. Stochastic rounding takes this further and is what allows small updates to survive in low-precision training rather than being rounded away every step.

In floating point, considerably more than people expect:

• Fix the reduction order. Floating-point addition is not associative, so any change in how a sum is decomposed changes the result. That means no atomics, no run-to-run-varying block or tile sizes, no library kernel that picks a different split by heuristic, and identical thread counts.
• Control FMA contraction. a*b + c as a fused multiply-add rounds once; as separate operations it rounds twice. The compiler is free to choose, so it must be pinned.
• Disable fast math. Any reassociation permission voids the guarantee.
• Pin library versions. IEEE-754 mandates correct rounding for $+, -, \times, \div, \sqrt{}$, but not for transcendentals. Two libm implementations may both be excellent and differ in the last bit of exp, so reproducibility across machines requires the same math library, not merely a compliant one.

In fixed point it is free. Integer addition and multiplication are exact and associative, so any reduction order gives the identical result, no compiler flag can change it, and no library is involved. The only thing that must match is where saturation clamps.

That asymmetry is a substantive architectural argument, not a footnote. If you need an auditable, replayable, provably identical computation — a risk system, a settlement path, anything you may have to reconstruct months later — integer arithmetic gives it by construction, and floating point gives it only through discipline that one careless library upgrade destroys.

Because e4m3 has three mantissa bits, so its machine epsilon is about $2^{-4} = 0.0625$.

Consider accumulating values around 1.0. Once the running sum reaches roughly 16, the smallest representable increment near that magnitude is about 1. Adding a value of 0.5 rounds to no change at all: the addition is a no-op, and it stays a no-op for every subsequent term. The accumulator stops moving while the true sum keeps growing. This is swamping, and in fp8 it happens after a handful of additions rather than the millions it takes in fp32.

So the rule is not a safety margin, it is what makes the reduction converge at all. Store and transport in fp8; accumulate in fp16 or fp32; requantise once at the end. Every tensor-core FP8 path does exactly this in hardware, and the FP8 “throughput” figures quoted for accelerators are always fp8 inputs with fp32 accumulation.

The general principle, which transfers to the fixed-point case directly: the precision needed for storage and the precision needed for accumulation are different problems with different answers. Narrow the first aggressively, because it buys bandwidth and multiplier area. Never narrow the second, because it buys almost nothing — accumulator width costs adder area, which grows linearly, while the multipliers you already shrank cost quadratically.

Work it in three steps, and state the accumulator explicitly because that is the step people skip.

Range. A maximum of 6.2 needs 3 integer bits plus a sign to represent up to 8. So Q4.11 in 16 bits gives range $[-8, 8)$ and resolution $2^{-11} \approx 4.9\times10^{-4}$. If the distribution is heavy-tailed, clipping at the 99.9th percentile — say 3.1 — buys a whole extra fraction bit (Q3.12), and saturating the rare outliers is usually a net accuracy gain.

Product scale. Q4.11 activations times Q1.6 int8 weights lands in Q5.17, needing 23 bits.

Accumulator. $23 + \lceil \log_2 256 \rceil = 23 + 8 =$ 31 bits. A 32-bit accumulator fits with one bit to spare; a 24-bit accumulator overflows and, if it wraps rather than saturates, does so silently.

Requantise. Shift right by the difference in fractional bits, round to nearest even, saturate to the next layer’s format.

The habit worth building: every fixed-point layer is specified by four numbers — input format, weight format, accumulator width, output format — and the accumulator is derived rather than chosen. If someone cannot tell you their accumulator width, they do not yet know whether their pipeline is correct.