A rule without a value is not a rule

A year of working with AI-generated UI against a real design system produced a finding I published and repeated: rules phrased as “never do X” erode, rules phrased as “do exactly this” hold. Write “never hardcode the badge height” and the rule erodes at generation time. Write badge height: 24px, always and it survives.

It sounded right. It matched published research showing models handle negated instructions worse as they scale. It matched what I had seen in production. It was also never tested against a control, because the two examples I kept citing differed in two ways at once — one was a prohibition and carried no value; the other was a directive and carried the value.

So I built a rig to separate them.

Below are two implementations of the same five components, generated from two versions of the same documentation. Both are live in this page — inspect them. One is exactly on-system. The other is off by 6.8 points.

Five components — button, text field, badge, card, tabs. A token file with 46 values. A rubric of 74 graded properties, written from the tokens before any documentation existed, so no condition could be written to suit it.

Three documents describe that same system. Each names every CSS selector. Each ships the same tokens.json. They differ in exactly one way at a time:

A → B isolates completeness. B → C isolates style. The prose conditions are 996 words; the directive one is 433. The wordier document is not the one carrying more information — B and C carry identical information.

Every run was generated by a separate agent that received only the token file and one condition, in a directory containing nothing else. No generator saw the rubric, the reference implementation, or another condition. Three runs per condition, same model, prompts identical but for the input path.

What the rubric does not grade

The contract covers 74 properties across 18 selectors. The generators emit around 96 more that it does not — display, cursor, box-sizing, font-family, shorthand border and padding — and 88 of those 96 appear in some runs and not others. That instability is real. It is not drift: nothing scored those properties, so nothing constrained them.

Left alone it would still be visible. A tab is a <button>, and a browser gives buttons a grey fill and a raised border unless told otherwise. A run that happens to write border: 0 looks clean; one that does not looks broken. Both score the same — the contract grades a tab’s height, padding, colour and weight, not its border.

So the harness pins them. structural.css sets the ungraded chrome identically in every panel — tab background and border, indicator positioning, box model, typeface — before any generated CSS loads. It cannot move a score, because none of those properties is graded. What it prevents is an undocumented difference between two runs being read as a measured one.

This was not designed in. The first build of the comparison did not normalise tab chrome, and two of the three runs of the strongest condition rendered as grey boxes while still scoring 100%. The fix is one rule in structural.css, committed forty-eight minutes after the rig itself.

My prediction was that B would sit somewhere between A and C — that prohibitions would erode even when the value was present. It did not happen. B and C are indistinguishable. Every prohibition in condition B survived intact, in all three runs, on all 74 properties.

What moved the number was naming the value. Nothing else did.

Across three runs of condition A, 222 graded properties: 15 wrong, and zero missing. That ratio is the finding. The model never left a gap and never flagged an ambiguity. It filled in every single value with something plausible.

Look at the card padding. The document said “one step up from the field padding scale”. Field padding is 12px. Every generator read “one step up” as the next token in the scale — 16px. The system means 20px.

That is what makes it expensive. A random error shows up in review. An error the model repeats every time agrees with itself and looks deliberate. Nobody catches it, because there is nothing that looks like a mistake — just a card that is 4px tighter than it should be, in a system nobody is measuring.

The last two rows are a different animal. The card padding was wrong in every run; the two gap values were wrong in two runs and in one. Those are not holes — the document said enough to get them right some of the time. A 3/3 failure is a gap in the documentation; a 1/3 failure is an ambiguity in it — and the ambiguity is the harder of the two to catch, because auditing a single run leaves you a two-in-three chance of seeing nothing wrong at all. A rule without a value is not a rule. A rule with an ambiguous value is a coin flip that lands your way often enough to pass review.

Every failure was a near-miss on a spacing step. Not one was a layout error, a missing state, or an invented colour.

Three faults in the first version, all of which would have pushed the result toward the answer I expected. They are recorded because an experiment you can only pass is not an experiment.

• The prose document named no CSS selectors. Two generators independently reported inventing their own — .button, .input__label. Audited as written, condition A would have scored near zero on naming, and the headline number would have had nothing to do with drift.
• The prose document was missing information, not just structured differently. It described spacing only in relative terms. “Prose loses because prose omits things” is nearly a tautology — it says nothing about whether phrasing matters. Fixing this is what produced condition B, and condition B is what killed my hypothesis.
• The isolation leaked. Input directories for different conditions were siblings under one parent. A generator that got stuck went looking, found another condition’s documentation, and said so in its report. Those runs were discarded and re-run against a filesystem where no other condition exists.

The prompt was deliberately not changed to forbid exploring, because one condition had already run without that instruction and prompt parity matters more than convenience.

What it is worth

There is no cost data in this lab. It is five components and a generic system, rebuilt in a clean room so the method could be checked without anything proprietary in it. The money was measured somewhere else.

On the production system this reproduces, making the documentation machine-readable moved on-system accuracy from about 25% to 65%, and testing that documentation by generating from it took it to 90%, and cut the correction step — pulling generated output back onto the system by hand — from roughly two hours a screen to fifteen minutes. Call it 1.75 hours returned on every screen a designer produces, before anything engineering-side. The cost case is there →

The lab runs once, as an experiment. In production the same question ran continuously — a loop that reads a component’s design source and its implementation in the code repository, reports the properties where the two have drifted, and opens the correction against the development branch. What you can check here is that loop with the proprietary parts removed.

What that number could not tell me was why it worked. More words? Firmer prohibitions? Better tone? This lab answers that, and the answer is narrow: naming the values did it, and nothing else did. The prose conditions run 996 words against the directive one’s 433 — the wordier document is not the one carrying more information.

So the return is measured on a real system and the instruction is one line. Finding out where you stand costs an afternoon. The fix is editing prose: no new components, no new tooling, no migration.

If you maintain a design system that AI writes against, the actionable finding is narrow and cheap:

Audit your documentation for properties that have no value. Not for tone, not for whether rules are phrased positively — for whether a machine reading the sentence can produce exactly one number. Anything relative is a guess waiting to happen: tighter than, one step up, generous, a standard gap.

That audit is now a tool. spec-resolvability → asks that one question of every rule in a set of docs, quotes back the sentences that fail it, and emits a contract.json stub — the input the auditor in this lab already takes. It is a skill file for an AI agent rather than a program: deciding whether “one step up from the field scale” resolves is a language judgement, not a pattern match. Read the docs, generate against them, measure what came out. It scores coverage, not correctness: documentation that is fully specified and wrong still scores 100%, and it over-flags, so the quoted sentences are a reading list rather than a verdict.

Pointed at GitHub’s Primer, the button page resolved 5 rules of 10 — under a section titled “Sizing and spacing” that names three sizes and measures none. Ours went from 93.2% to 100% by editing prose, not by changing a single word of tone.

Every value in this lab has a token. Real systems are not like that, and the gap turns out to matter more than anything the lab measured.

In a production design system I led, exactly two components carried a shadow. They were different shadows, each used once, in one component. Neither met the bar for a token, because the bar has always been reuse. Any designer reading the system would have made the same call.

Then I had a model author a component spec against that system — reading the design file, mapping each value to its token, writing the documentation the rest of the pipeline builds from. It handled the missing token the way you would want a careful colleague to. It noticed. It said so, unprompted:

Set that against section 05, where across 222 graded properties the model never once said it was unsure. The difference is the kind of gap. An under-specified value — “one step up from the field scale” — still resolves to something, so it resolves it and moves on. A value with nothing behind it at all cannot be resolved, and that is when it speaks up.

Which is the wrong way round for you. The silent one is the one that comes out wrong.

And then it wrote the raw values anyway.

That is the correct value, and it is worth being precise about what did and did not happen to it. It was not lost. It sat in the spec itself — 144 lines, 25 of its values pointing at a token, and this one just a number — a file I owned, that the reviewing agent read on every pass, with a ledger entry beside it saying what to do. Nothing was hidden and nothing was broken.

What happened is narrower: the value is in the system, but not in the token layer. Nothing can reference it by name. No token audit can see it. And the next component to need that shadow copies the number rather than pointing at it — because a number is all there is to copy.

And the flag was not wasted. That thread was the continuing reviewer for the kit — it held what the system knew and what to do next, and its entry is where the token would have started. The agent did the right thing: there was no token to point at, it would not invent one unilaterally, so it recorded the gap and moved on.

The cost is the gap between the two. The spec ships now. The token arrives later. In between, the system carries a value that nothing points to — and “in between” is measured in weeks, not minutes.

This is a different failure from the one the lab measures, and it is worth keeping them apart. In the lab, a value the documentation described but never named came out wrong. Here, a value with no token at all came out right — and landed as a literal rather than a name.

So should you tokenise everything?

No. But naming and tokenising are two different decisions, and I had them welded together.

The bar has always been reuse: will enough things use this to justify a token? Used once, in one component, does not clear it — and for a system that people read, that is the right answer. A designer changing a one-off shadow opens the one file it lives in.

The mistake is what follows from failing that bar. Because the value did not earn a token, it got no name at all — and a value with no name can only be written as a literal. Failing the reuse test should mean not a shared token yet. It should not mean a bare number.

Naming is unconditional and nearly free — a component-scoped name for one value used once is not a bad token, it is simply not a scale. Promotion to a shared token stays exactly where it was: when the value actually appears in a second place.

And that second part is the job to hand to the agent. Reuse used to be something you predicted up front, badly, because no one can hold a hundred components in their head and notice that the same shadow has quietly turned up three times. Something that reads the entire system on every pass can. Detecting the moment a value earns promotion is precisely the kind of work that has become cheap, and the threshold decision is better made from evidence, late, than from a guess, early.

So the two failures on this page take different fixes:

1. Value described but never named — “one step up from the field scale”. State the number. That is the whole fix, and it is what moved 93.2% to 100%.
2. Value with no token at all — stating the number is not the fix. That is what happened here: it was stated, correctly, and it was still a literal. Give it a name something can resolve, even a local one, and let promotion come later.

What I would not do is mint a system-level token for every value because a model might read it. That trades an invisible mess for a visible one — a namespace full of singletons nobody can navigate is its own failure. The goal is not more tokens. It is no bare numbers.

Five components, 74 properties, three runs per condition, one model. Enough to show a difference; not a benchmark. CSS only — nothing here tests layout, composition, accessibility or behaviour. The system is deliberately small and clean, so the absolute numbers will not transfer; the method is the part that does.

And the null result is a null result, not proof of a negative. It shows that prohibitions carrying values survived this rig. It does not show that negation is free in general — models do measurably worse on negated instructions as they scale, which remains a real reason to prefer positive phrasing. It just was not what was breaking my system.

The whole rig is public: the tokens, the rubric, all three documents, every one of the nine generated runs, the auditor, and the discarded runs from the version that was wrong.

If you re-run it and get a different answer, I would rather know.