Documents that cite themselves
Kansas City · Independent ·
Abstract
Reads a form or a scan, points at the exact words for every value, and says which values a person must check. A model that reads 95 values out of 100 correctly, and is equally confident about all 100, has saved nobody any work: the reviewer still has to check all 100 to find the 5. So the job is not reading. It is sorting: which answers can a person skip? Every value comes back with a rectangle drawn on the page showing where it was read, and one of three verdicts: accept, accepted after a second reading, or a person should look. On all 5,349 public validation questions the system reads at 97.53 ANLS, level with the best published systems, and not one point of it came from a larger model: the points were earned by the engineering around the model and confirmed on held-out questions. The measurements are public; the method is in a companion paper, on request.
1 · Article2 · PaperRequest access
Keywords: documents, evaluation, trust.
1Measurements
| measure | value | conditions | source |
|---|---|---|---|
| ANLS, all DocVQA validation questions | 97.53 | 5,349 questions, nothing removed | private research log: audits.tsv, REPORT.md:15-22 |
| Answers identical to gold | 93.56% | best published test entry 97.29 ANLS; people 98.11 | private research log: REPORT.md:56 |
| Fields filled without a person | 93.3% | at 100.00% precision, 0 wrong in 1,181 | grounding research repo (private) README.md:70 |
| Citations on the right ink | 99.5% | 6 wrong rectangles, 0 missing | grounding research repo (private) README.md:70 |
| Grounding score, 36 kept changes | 79.1 → 93.5 | development documents; held-out 79.7 → 90.9 | grounding research repo (private) results.tsv |
§Where the work lives
private: the extraction lab— the measured runs and the numbered findings.
§Related writing and lessons
- How does a semantic cache work? (writing)
- How do you actually measure a RAG system? (writing)