All posts

28 July 2026 — Architecture

One model, every venue

Every exchange publishes its own dialect of the same handful of ideas. Normalizing them into a single binary model is what makes a dissector, a C struct, and a C# parser fall out of one specification.

Open Markets Initiative · 6 min read

Every exchange publishes its own dialect of the same handful of ideas. An order gets acknowledged, a trade prints, a book updates, a session heartbeats. The semantics barely move between venues. What moves is everything else: the field order, the width of a price, whether a length prefix counts itself, whether the timestamp is nanoseconds since epoch or since midnight, and which of the four competing definitions of "sequence number" is in play.

That variance is why protocol work has historically been done by hand, one venue at a time, and why it has historically been rewritten every time a firm adds a target language.

Normalize first, generate later

The pipeline has three layers, and the middle one is the interesting one.

1. Normalization turns a published specification — XML, FIX, or a PDF a human has to read — into five component kinds: types, groups, rules, values, and actions. Fifty-odd loaders exist because there are fifty-odd ways to publish the same information.
2. Compilation resolves those components into a binary model: a tree of elements with every dependency settled. No venue-specific knowledge survives this step.
3. Generation walks that tree and emits source code.

The discipline that makes it work is that layer three knows nothing about exchanges. A generator has never heard of ITCH or Pillar or SBE. It has heard of a fixed-width unsigned integer with an implied decimal place, and that is enough to emit a correct field accessor.

What falls out of it

Once the model is generic, targets are additive rather than multiplicative. A new venue costs one loader and reaches every language. A new language costs one generator and reaches every venue.

specification → normalize → compile → ┬→ Wireshark Lua dissector
                                      ├→ C struct + block handler
                                      ├→ C++ parser
                                      ├→ C# fixed-message struct
                                      └→ Java class

The alternative — the one most firms are living with — is a matrix. Thirty venues times five languages is a hundred and fifty artifacts to write, and a hundred and fifty artifacts to update when a venue revises a spec.

Determinism is the point

Generated output is a pure function of the compiled model. The same specification produces byte-identical source on every run, which means a spec revision shows up as a reviewable diff rather than an archaeology project. It also means the generated code is disposable: nothing is hand-patched downstream, so nothing is lost when it is regenerated.

That property is worth more than it sounds. Most of the cost of protocol maintenance is not writing the parser. It is knowing, six months later, whether the parser still matches the spec.