Schema-constrained generation
Kansas City · Independent · since 2026-06 · updated 2026-08
Abstract
A filing with 369 fields that a language model has to help produce and must never be allowed to write.
Keywords: structured output, contracts, forms.
Two products here run on large, strict description files: one file per screen, saying what label goes where, what may be typed into which box, and which section appears when a box is ticked. There are more than eight thousand of them on one side alone, and a large share need updating every year when the forms change. Today people type them by hand.
The obvious idea is to let a model draft them. The obvious idea does not survive contact with the measurements.
| measure | value | conditions | source |
|---|---|---|---|
| Fields in the target file | 369 | — | wsz-schema-rag/forms |
| Rule the design turns on | the model never emits the file | it proposes values; a deterministic writer emits them | wsz-schema-rag/arch.md |
1Why you cannot just ask for the file
Current models are close to perfect at getting the shape of a strict file right and still get the contents wrong. When researchers asked the best available models to produce one complete form file in a single attempt, every model scored zero. Not poor. Zero.
Reading through the existing files, everything in them falls into three kinds, and only one of them is visible on the source document:
- What is printed on the form. The words, the boxes, where they sit. A model can read this, and so can plain software.
- What only an expert knows. Every input box has to be wired to the correct variable inside the calculation engine. The printed form does not say which one. Wire it to the wrong variable and the software computes the wrong answer while looking completely normal.
- House rules. Naming conventions, identifier schemes, fonts, spacing. Dull, and the file will not load if they are wrong.
The second kind is the whole difficulty. It is also the kind where a wrong answer is invisible until someone's numbers are wrong.
2The rule the design turns on
The model never emits the file. It proposes a value for one field at a time and says where it got it. A deterministic writer assembles the file, and refuses anything that fails the contract.
That inverts the usual arrangement. Instead of a fluent model producing a document that humans then audit, plain software produces the document and the model is a constrained source of suggestions for specific slots. The model works at three stations of a six-station line. Reading the source PDF is not one of them — modern form PDFs carry their text and box geometry inside them, so extracting that with code is both free and exact, and asking a model to do it would be strictly worse.
The other consequence is that every model output is small enough to check automatically. A proposed wiring either names a variable that exists in the engine or it does not.
3What is still open
This thread is paused. The design is written up and argued through; what does not exist yet is a measured accuracy number for the wiring step on held-out screens, which is the only number that would tell anyone whether the approach is worth building. Until that exists I would not claim more for it than a plan with a defensible shape.
4Evidence
This thread has no lab notebook yet. What exists is the working folder below and the measurements in Table 1; until the runs are written up as claims with verdicts, treat the numbers as this page states them and no further.
Where the work lives:
~/company/wsz-schema-rag— contracts, forms and the plan.~/company/wsz-schema-rag/plain_english.md— the argument without jargon.
§Related writing and lessons
- Thinking in Vectors · ColPali II: Inside the Model (lesson)
- Building ContentFlow CMS: A Journey into AI-Powered Development (writing)
- What is Faithfulness? (writing)
- There's an LLM Running in Your Browser Right Now — No Internet, No API, No Credits (writing)
§To remember
4 recall cards are drawn from this thread. They come back on a schedule in §7 Recall. The first: What happened when researchers asked the best models to write a complete form file in one attempt?
← §2.2 Inference economics§2.4 What agents should remember →