Series
Multi-part technical writing, each built the same way: a mental model, the numbers worth memorizing, critical-thinking probes, and self-checks with the answers hidden.
- ML Hardware & Low-Latency SystemsFrom roofline to nanosecond inferenceTwelve modules, written for someone who already knows the basics. Starts from roofline and GPU throughput instincts, then inverts them for the batch-of-one regime: tail latency, CPU microarchitecture, PCIe and the wire, fixed-point numerics, and the workloads that answer in nanoseconds.
- DiffusionFrom score matching to fast samplersNine modules on diffusion models for people who have trained one and want the derivations to actually connect. The forward process and why it is designed the way it is, three equivalent views of the objective, the sampler as an ODE solver, guidance, latent and flow-matching formulations, and where inference time actually goes.
- ML Software & CompilersHow PyTorch becomes kernelsFourteen modules on the stack between model code and the GPU. Why eager execution won and what it costs per op, how torch.compile captures Python at runtime, what Inductor and Triton actually generate, how the caching allocator and CUDA Graphs attack the two runtime overheads, and the three places where none of it works cleanly yet: dynamic shapes, MoE routing, and top-k.
- ML Hardware/Software CodesignCompute, memory, interconnect — and the mapping between themFourteen modules on designing the chip and the compiler as one problem. Three resource abstractions — compute, memory, interconnect — each exposing capacity, bandwidth, latency and granularity, plus the field that makes it codesign: who decides placement. Then mapping, the act that binds them, and the four things that break a static mapping: sparsity, precision, dynamism and scale.
- Serving Kimi K3A 2.8T hybrid-attention MoE, and what it takes to keep it fedEleven modules on putting a 2.78-trillion-parameter model into production, built from the day-0 reports published by vLLM, SGLang and the people who tried it on hardware they already had. The hybrid KDA/MLA attention that makes a million-token context affordable and makes everything else harder, the memory budget, the two engines, the concurrency collapse, multi-node KV offloading and disaggregation, and the tool-calling surface. Every number is linked to where it came from.