Documents that cite themselves
Read it once. Know which answers to check.
Reads a form or a scan into a schema, points at the exact words for every value, and says which values a person must check.
The idea
Two clerks read the same hundred forms. Both are right 95 times. The first swears by all 100. The second hands you 16 marked "look at these", with a finger on the exact line. Only the second has given you your afternoon back. This is the second clerk.
Read the full story →Why it matters
Anyone can rent a model that reads documents at about 94%. What a business cannot buy off the shelf is accountability: for every value, whether it may be used without a person, and if not, where on the page the person should look.
Forms and scansCitationsHuman review
How it works
- In
- A form or a scan, and the fields you want filled.
- Out
- Every value with the exact words it was read from, and a mark on the ones a person must check.
Closed source · results onlyHow it does this is not published. What it produces, and what that measured, is.
Explore deeper Each path is a different view of the same work.
Why it exists
Anyone can rent a model that reads documents at about 94%. That is a commodity, priced by the page. What a business cannot buy off the shelf is accountability: for every value the system filled, whether it may be used without a person, and if not, where on the page the person should look.
So this research measures three things, not one. Reading: is the answer right? Pointing: is the rectangle on the page drawn around the words the answer came from? Judging: did the system say accept, or accepted after a second reading, or a person should look, and was it right to say so?
What it is for
Tax forms, invoices, statements, filings: the paper that offices read all day and type into systems. The product fills the fields, cites the page for each one, and marks the few a reviewer must check. The metric that matters to the office is reviewer minutes per document.
What it measured
On the public DocVQA benchmark the system reads at 97.53 ANLS over all 5,349 validation questions with nothing removed; 93.56% of its answers are identical to the gold answer. The best published entry on the official leaderboard scores 97.29 and people score 98.11. On a grounding benchmark of generated business documents, 93.3% of fields are filled with no person at 100.00% precision, and 99.5% of citations land on the right ink.
Not one point came from a larger model: larger models, tried as readers, scored lower. The points were earned by the engineering around the model, and every one was confirmed on questions no change was designed on.
What it does not claim
The grounding benchmark is generated, not collected, and its development score sits 2.6 points above its held-out score. The reading results are on the validation set, as all public comparisons for this benchmark are, and not on the hidden test set.
The readers, the settings, the wording rules, the verdict thresholds and the 36 changes to the grounding engine are not published. They are in the companion paper, with every kept and discarded experiment, shared on request.
What was measured
Each number is read from the file named under it. Nothing here is rounded up.
private research log: audits.tsv:2 · REPORT.md:15-22private research log: REPORT.md:56private research log: REPORT.md:24-28grounding research repo (private) README.md:70grounding research repo (private) README.md:70grounding research repo (private) results.tsv:2,57Common questions
Is 97.53 the state of the art?
It is level with the best published systems on this benchmark and within half a point of people. The paper gives the comparison and its limits.
Is the code public?
No. The measurements and the public paper are. The engine and the method are proprietary.
How do I read the full paper?
Use the request door on this page and say who you are and what you are working on. It is shared on request.
Ask a question
Ask about what it does and what it measured. Answers come from what is written on this site.
Carry thisReading is the easy part. The product is knowing, for every value, whether a person needs to look, and showing them where.
proprietary · results public · the method on request