Tokens tell a model what exists, not what's allowed

Design system documentation is written for people. “Here is a button, it has six variants.” A human reads that, looks at the examples, and infers the rest from taste and context.

Hand the same documentation to a model and you get plausible slop. Not because it can’t read, because nothing in the source says when to reach for which. It picks the variant that sounds right for the sentence it was given.

So I spent the first three days of August building the layer that answers that, and the interesting part is not what I built. It is that my own eval found the first real bug in it, and the bug was mine.

Three layers, not one

The system has 1,040 design tokens: 250 primitives, 600 foundations, 190 semantic. That sounds like a lot of guidance and it is almost none, because tokens only enumerate.

• Tokens say which values exist. bg.intent.negative.strong is a real thing you may use.
• Rules say which combinations mean something. A generator can pull bg.intent.negative.strong out of the tokens on its own. Only the rules tell it that fg.base.primary on top of that is a contrast failure. There are 45 of these — 14 forbidden pairings, 7 surface rules, 12 layer pairings, 12 structural checks.
• Inventory says, per component: what it is for, what it is not for and what to use instead, what it composes with, and what accessibility obligations come attached. One entry for each of the 75 components.

The second layer is the one nobody ships. Token pipelines are common now. A machine-checkable statement that two legal values must not appear together is not, and it is the only layer that turns “here are the options” into “here is what would be wrong”.

Then you can score it

Rules that a machine can check are rules you can build a harness around. Six tasks, a model generates a UI spec for each, the spec gets validated against the system’s own rules.

Current baseline: six of six pass, 98% mean.

That number is a direction, not a benchmark. It is a mean over six tasks against one model baseline, which is thin enough that I would not defend any comparison built on it. What it is good for is catching regressions in the rules, which is exactly what it did.

The eval found a bug in my rules

Two of the six specs were rejected for putting a same-coloured border on a filled surface.

I went to check the model’s work and found the model was right. The source I derived the rules from does not forbid that. For strong fills it simply does not mention a border at all, and I had transcribed that silence as a prohibition.

Silence is not a prohibition. That is now written at the top of the rules file, because it is the failure mode of anyone converting prose guidance into machine-checkable constraints, and I walked straight into it on the first pass.

This is the same shape as the thing I wrote about last week, one level down. The rules encode my reading of the source. Nothing inside the system can question that reading, because the rules are the standard everything else is measured against. It took a generated spec disagreeing with me to surface it.

Which is a decent argument for having a harness at all, and I’d rather report it that way round than as a score.

What is derived and what is written

The working rule across all of it: derive, don’t write. Anything mechanically obtainable comes from a script.

• Tokens are generated, never hand-edited. A brand swap touches exactly one file, because components bind to semantic names and never to a palette colour, a hex value or an arbitrary utility.
• The inventory scaffold is generated. The judgement — what a component is not for — is written by hand, because that part does not automate.
• The variant list reads the components’ own cva definitions rather than a maintained list, which is the same drift I keep getting caught by elsewhere.

One thing worth being explicit about, since the numbers invite the wrong reading: 75 components and roughly 26,100 lines across 50 commits in three days is not me typing. It is the routing setup I already run, pointed at a domain where the output happens to be checkable. The rules layer is why that was possible rather than reckless — a generated component that violates a pairing rule fails before I ever look at it.

The CSS bug that would have cost all of it

One detail, because it nearly took the theming with it and it is not obvious.

A custom property containing var() resolves on the declaring element. Declaring --background: var(--color-bg-base-canvas) only in :root bakes in the light value and then inherits it into every .dark subtree. The toggle would have been dead while every individual token looked correct.

The fix is to declare in both scopes. The part I’d keep is the check that came after: a script resolves every var() chain in light and in dark and compares them. 334 resolved values identical, no drift. Which is the version of this I trust, given how many green checks this week have meant nothing.

Next

The harness needs more than six tasks and more than one model baseline before the number means anything, and I want it to fail on something I did not plant.

After that: the part I find most interesting is measuring how a system gets used rather than which parts of it get imported. Early result from one real application — the primary button variant appears five times, against 512 for the outline variant. That has its own post, with the caveat that one codebase makes it a self-diagnosis rather than a finding about anyone else.

If you maintain a design system: has anything in yours ever been checked by a machine beyond types and lint? I’m curious whether the rules layer is missing everywhere or just from the systems I’ve read.