§3 Experiments · AI / LLM
active29 July 2026 → 6 August 2026 · 4 parts
Reading documents you can trust: the numbered findings
39 findings from eight days of measured runs on scanned documents — what sets accuracy, what is a dead end, and what makes an answer trustworthy.
In one breathBuy the best reader you can (it's cheap), feed it the highest-resolution image you can, and never let anything rewrite what it says. Spend all engineering on the OUTSIDE of the model: independent second readings, deterministic rules, structure skeletons injected only where fields are disputed, zoomed re-asks on conflicts, and honest abstention when evidence stays ambiguous. Put the whole policy in harness code, not prompts. The output isn't just answers — it's answers sorted by trustworthiness, with the untrustworthy 15% routed to humans, and every accepted field carrying a bounding-box citation you can click.
Summary
Every finding here states its evidence: runs on a 300-question seeded sample of a public document-question benchmark, or an 8-document, 113-field citation bench. The total spend for all of it was about three and a half dollars of API calls plus free local CPU.
They are numbered so that code and later write-ups can cite them. Findings 33 onward come from an arena that compared agents with a paired bootstrap and supersede any earlier claim based on a bare mean.
The protocol, in order
01What sets answer qualityReader quality is set by the reader — machinery can't add to it..insight02What does not workSystematic errors can't be outvoted..rejected03What works: the trust machineryDiversity beats repetition..adopted04Architecture and the harnessOur loop is a workflow, not an agent — and that's a feature..insight