Accuracy
How often is it right? Here are our numbers.
Most AI document tools don't publish how accurate they are. We think you should be able to check that too. These are the results of our own tests on public contracts, how we ran them, and where Akraeus still gets things wrong.
Last updated October 2026. We'll update this page as we run more tests.
Grid · 20 distributor agreements · 10 questions each
178
Right
95% of 187 cells
5
Wrong, but flagged for review
the check caught them
2
Said “not stated” when the answer was there
a missed answer, not a wrong one
2
Wrong, without a flag
the outcome we work hardest to avoid
In other words: 98% of answers were either right or flagged for a person to check. The questions covered dates, the initial term, renewal terms, notice periods, governing law, exclusivity, non-compete clauses, termination for convenience and limits on liability.
Grid · 13 IT outsourcing, hosting and support agreements · 10 service-level questions each
7 of 130
Wrong, without a flag
About 5% of cells.
11
Flagged for review
6 of them were genuinely wrong, and the flag caught them.
Harder
On purpose
Service levels hide in schedules, tables and conditions. These contracts are messier than most.
In other words: 94% of answers were either right or flagged for a person to check. The questions covered where service levels are set out, availability targets, support hours, the highest severity level and its response and resolution times, reporting, service credits and their caps, and the right to terminate for missed service levels.
How we test
- Real contracts, publicly available. All 33 agreements come from CUAD, a public collection of commercial contracts filed with the US Securities and Exchange Commission, annotated by trained reviewers under the supervision of experienced lawyers.
- The product, unchanged. Each test runs the same code that answers your grid, on the same models. We also use these contracts to improve Grid, so we only make general changes — never a rule written for one contract or one industry — and we're adding fresh test sets to check that the results hold up.
- Graded against an answer key. The distributor set is graded against the expert labels. Where a label was itself debatable, we excluded that cell — 13 of 200 — rather than count it either way. CUAD doesn't label service levels, so we graded the IT services set by hand, and excluded the 8 of 130 cells where the contract itself is ambiguous or redacted.
- Every outcome counted. We separate answers that are wrong but flagged from answers that are wrong and look fine. A flagged answer costs you a minute; an unflagged wrong one can cost much more.
Where it still goes wrong
- A related clause instead of the right one. The most common mistake: answering a service-credit question with a general liability cap, for example. The independent check is there to catch this, and usually does — not always.
- Answers that depend on a condition. When a contract says different things in different circumstances, Grid tends to flag the cell rather than pick one. That is deliberate.
- Amendments filed as separate documents. Each document is answered on its own, so a later amendment isn't yet merged into the agreement it changes. Include the amendment as its own row and check both.
- Scanned pages. PDFs without selectable text can't be read yet.
What these numbers don't say
33 contracts is a meaningful test, not a guarantee. Your documents and questions are different, and results on them will vary. That's why every answer shows its source, and why Grid flags what it isn't sure of. Treat Akraeus as a fast, checkable first pass — and keep a person on the decisions that matter.
These tests measure Grid. Ask uses the same approach to citations, but we haven't published a separate benchmark for it yet.