Hardware/software partitioning and market microstructure

Hardware, in the datapath: feed arbitration and sequence-gap detection, market-data parse, book maintenance, feature computation, model evaluation, order encoding, pre-trade risk, TX.

Host: parameter and model fitting, position and P&L reconciliation, logging and analytics, recovery and retransmission requests, new-instrument onboarding, and a conventional trading path for anything not latency-critical.

The interface, which is where the design actually lives:

• Host → FPGA: a parameter region written by DMA, double-buffered with a single commit bit. The hardware reads bank A while the host writes bank B, then one atomic flip. Without this the fast path can observe a torn update — half the old parameters and half the new — and act on a combination that was never intended and never tested. This detail causes real incidents and it is invisible until it happens.
• FPGA → host: fills, plus a copy of every decision including the ones not acted on, so analytics can reconstruct behaviour offline. This is also your only real debugging channel.
• A kill register gating TX, writable by the host and by hardware watchdogs.

The statement that captures it: the host is never in the loop of a trade decision. It sets up the conditions under which hardware will trade, and hardware trades. If the host stops, the fast path must stop trading, not continue blind — fail-safe, not fail-open. A heartbeat from host to FPGA with a timeout that disables TX is the standard mechanism, and it is not optional.

Because its cost is variable and its worst case is unbounded, and in a deadline system the worst case is the specification. Collisions make probe counts data-dependent; a resize is a catastrophic pause at an arbitrary moment; and every lookup is a pointer chase whose latency depends on cache residency. An average of 20 ns with a p99.9 of 2 µs fails a 500 ns budget even though it “usually” fits. A tree is worse: $O(\log n)$ dependent pointer chases, plus rebalancing.

Use a price ladder: an array indexed by (price − reference) / tick size, with a bitmap marking occupied levels. Update is a direct index — one access, fixed cost. Finding the best is a priority encode over the bitmap, about four levels of leading-zero logic over 4096 bits, also fixed. No pointers, no collisions, no rebalancing, no variance.

Handling a price range too large for a full ladder is the interesting sub-problem. Use a windowed ladder around the current best — a few thousand ticks is ample for any liquid instrument — with a slow-path structure for distant prices. Two rules make this safe: the fast path must never wait on the slow path, and a price moving outside the window must be treated as a book-invalid event that stops trading that instrument rather than as something to handle inline.

The general principle transfers well beyond order books: choose data structures by their worst case and their variance, not their average. Most of the standard library is optimised for the wrong statistic when a deadline is involved.

Queueing at whatever stage has the least headroom — and by Module 4, a system sized for mean load is at high $\rho$ during the burst, where latency grows like $1/(1-\rho)$ and the queue takes far longer to drain than it took to build.

The failure modes, in increasing order of danger:

1. You fall behind. Decisions are made on stale book state. Bad, but detectable.
2. A buffer overflows and you drop a message. Now your book is wrong and you do not necessarily know it.
3. The dangerous one: you keep trading on a corrupted book. Silent, confident, and wrong — the only failure here that loses money at speed rather than merely missing opportunities.

The defences, and the third is the one that matters:

• Size for the burst, not the mean. Buffers, and $\rho \le 0.3$ at peak.
• A/B feed arbitration. Exchanges publish two identical feeds; take whichever packet arrives first and dedupe by sequence number. This is cheap in hardware and removes a whole class of tail events, since a microburst or loss on one feed does not stall you.
• Detect sequence gaps and stop immediately. Every message carries a sequence number; a gap means the book is unreliable. The book should publish a validity bit, and the strategy should be gated on it in hardware. Recovery — requesting retransmission, rebuilding from a snapshot — belongs on the host, and trading in that instrument resumes only when the book is provably correct again.

The principle worth stating: in a system that acts on state, the most important property is not speed but knowing when your state is untrustworthy. A fast wrong answer is worse than no answer, and it is the only outcome in this list that is unbounded.

Because a check must be independent of the failure it guards against, and the failure it guards against is your own strategy misbehaving — a bug, a bad parameter push, a model extrapolating somewhere it was never tested. A check that shares code, state, or a deployment unit with the strategy is disabled by exactly the events that would trigger it.

So the risk stage sits last, before TX, with its own registers, its own limits, and no dependence on anything upstream. The non-negotiable checks are all comparisons and cost about 10 ns in total: maximum order size, maximum notional, a price collar against a reference, maximum open position per instrument, a message rate limit, an instrument whitelist, and self-trade prevention. Behind them sits a kill switch — a single register gating all outbound orders, writable by the host and by hardware watchdogs including the host-heartbeat timeout.

It must be fail-closed. Sequence gap, heartbeat lost, watchdog expiry, parameter bank inconsistency: all disable TX. The default on any uncertainty is not to trade.

Module 11’s output bound is what makes this rigorous rather than merely defensive. Because a tree ensemble’s reachable output range is computable at compile time, you can set limits outside what correct operation can produce — so a firing limit is unambiguously a fault signal rather than a clipped aggressive signal, and it can safely trigger a shutdown instead of a clamp. That is a much stronger safety property than a limit you are constantly brushing against.

Knight Capital is the canonical illustration: an inconsistent deployment across servers activated dormant code, and 45 minutes without an effective kill switch cost about $440 million. The lesson is not “test more”. It is that the stopping mechanism must be independent of, and simpler than, the thing it stops.

Iteration speed, and it usually dominates the engineering outcome.

An FPGA change is a synthesis and place-and-route run measured in hours, plus verification, plus a deployment window. A parameter change is a register write. So the partition should be designed to maximise what can change without a rebuild: thresholds, sizes, enables and limits in registers; model constants loadable rather than compiled where the update cadence demands it (accepting Module 10’s 5–10× area penalty as the price of flexibility); feature computation parameterised rather than hard-coded. Deliberately spend area to keep things soft.

The deeper point, and the one this whole series has been building toward: latency is a constraint to satisfy, not a quantity to maximise. Past the point where you are faster than the opportunity requires — the quote you are racing for only updates every 10 µs, the counterparty only responds every millisecond — additional speed earns nothing while continuing to cost area, power, development time and flexibility. The teams that win are rarely the ones with the single fastest path. They are the ones fast enough who can change what the path does most quickly, because the edge decays and the ability to redeploy against a new one is worth more than another 50 ns.

Which reframes every optimisation in Parts B through E. They are worth doing until the constraint is met, and then they are worth stopping. Knowing where that line sits — for your venue, your strategy, your competitors — is the judgement that the technical material serves, and it is not itself a technical question.