14 modules · 4 parts · ~253 min

ML Hardware/Software Codesign

Fourteen modules on designing the chip and the compiler as one problem. Three resource abstractions — compute, memory, interconnect — each exposing capacity, bandwidth, latency and granularity, plus the field that makes it codesign: who decides placement. Then mapping, the act that binds them, and the four things that break a static mapping: sparsity, precision, dynamism and scale.

Part OThe three abstractions

What each resource exposes.

01The codesign contractWhy the hardware/software interface leaks for ML and not for most things, the three-resource abstraction that follows, and what "balanced" actually means when a single workload is only ever bottlenecked on one resource.
02Compute: the shape of the multiplyPeak FLOP/s is the least useful number about a compute unit. What matters is the native operation shape, the granularity at which it must be fed, and the two independent ways utilization dies.
03Memory: capacity, bandwidth, and who decides placementThe hierarchy as a ladder of (capacity, bandwidth, latency, granularity, who-manages) tuples. Cache versus scratchpad as a codesign decision, why bandwidth costs area, and why granularity is what actually punishes you.
04Interconnect: memory at a longer distanceAn interconnect is a memory level you cannot store anything in. The same four numbers, topology in place of capacity, collectives as the access primitives — and the beachfront limit that drives everything else in the series.

Part PMapping

The act that binds them.

05The mapping problem, stated preciselyA mapping is four decisions: tiling, permutation, spatial assignment, binding. Which makes arithmetic intensity a property of the mapping rather than the algorithm — and makes the search space astronomically large.
06Dataflow: what stays stillWeight-, output-, input- and row-stationary as four answers to one question — which tensor do you refuse to re-fetch. Why partial sums decide it, and why fixing a dataflow in RTL fixes one dimension of the mapspace for the life of the chip.
07Tiling the hierarchyThe one closed form worth memorizing: traffic falls as the square root of capacity, so doubling SRAM buys 1.41×. Multi-level tiling, double buffering as capacity spent on latency, and why the levels cannot be tiled independently.
08Cost models and searching the mapspaceHow you evaluate a million mappings on hardware that does not exist yet. Analytic models versus autotuning, what each is systematically wrong about, and why for codesign the model’s accuracy is the chip’s accuracy.

Part QWhere mappings break

Irregularity, precision, scale.

09Sparsity and irregularityEvery static mapping assumes a dense rectangular iteration space. Three ways sparsity breaks that, why 90% unstructured rarely gives 10×, and the rule that hardware exploits sparsity only when the pattern is shaped to the array.
10Numerics as a codesign knobPrecision is the only knob that moves compute, memory and interconnect at once. Block scaling as the modern answer, why the block size must divide the array’s K-tile, and why "what precision" is the wrong question.
11Parallelism is mapping onto the interconnectDP, TP, PP, EP and SP are Module 5’s four decisions applied one level up, with the network as the memory level. Each strategy is priced by the collective it forces, and the placement rule falls out of the arithmetic.

Part RIn practice

Real workloads, real chips, the real loop.

12Case study: attention, mappedFlashAttention is a mapping result, not an algorithm result — identical FLOPs, one tensor re-bound. Prefill and decode as two points in the resource space with the same math, and GQA/MLA as model architecture bending to a memory constraint.
13Three architectures, read as three mappingsGPU, TPU and the static-SRAM machines differ mostly in one thing: who absorbs uncertainty at runtime. Read each on the three axes, and the flexibility/efficiency curve stops being a mystery.
14The codesign loop, and where it failsHardware is frozen two to three years before the workload it runs, so the compiler is the only thing that can adapt — and it is on the critical path of the chip’s value while being consistently underfunded relative to it.