How AI reads a legal document, stage by stage

Ask a vendor how their system reads a contract and you’ll almost always get an answer about the model: which one, how big, how recently trained. That’s the fifth stage of five, and it’s rarely the one that decides whether the answer is any good.

Here’s the whole sequence. Something parses the document into text, cuts that text into passages, indexes them, retrieves the handful it thinks are relevant, and only then hands those to a model to write an answer. The model never sees your document. It sees whatever retrieval chose. Which means that when retrieval chooses badly you get a confident, well-written answer built on the wrong evidence, and there’s nothing in the output to tell you that’s what happened.

Each stage fails in a different way.

Parsing

A PDF isn’t really a document. It’s a set of instructions about where to put marks on a page, and parsing is the work of reconstructing an actual document out of that: reading order, headings, tables, footnotes.

Legal documents are about as unfriendly to this as it gets. Executed agreements tend to arrive as scans, which means you’re relying on character recognition, and character recognition makes its mistakes in exactly the places where a digit matters. Obligations sit inside table cells, and a parser that flattens a table into a run of text has thrown away the relationship between the row label and the value sitting next to it. Schedules and footnotes get detached from the clauses that depend on them.

So when you’re evaluating something, feed it your worst scan and check whether the numbers survived. Ask it something whose answer lives in a table cell rather than a paragraph.

Splitting

Once you’ve got text, it gets cut into passages small enough to retrieve and hand to a model. Where those cuts land matters more than you’d think. A definition separated from the term it defines, or a proviso separated from the obligation it qualifies, produces a passage that reads perfectly complete and means something different from the clause it was taken out of.

You usually can’t inspect this stage at all, which is what makes it awkward. What you can do is ask questions whose answers turn on a qualification, the “does this apply if such-and-such?” kind rather than “what does this clause say?”. Those are the ones that go wrong when the cuts land in the wrong place.

Retrieval

Now the system has to choose. Passages get indexed so they can be found, usually by matching keywords and by matching meaning at the same time, then merging the two sets of results. Filters narrow things down to a folder, a date range, a matter.

This is the stage where answers are made or lost, and it goes wrong in two ways. Either it finds nothing relevant, in which case the honest thing to do is say so. Or it finds something plausible and wrong: the superseded version instead of the current one, or one of two documents that contradict each other.

The second one is the problem, because on the page it looks exactly like a good answer. Nothing in the response hints that a second document was sitting in the document set saying something different and simply wasn’t picked up.

The way to catch this is to build it into your test set deliberately. Put both the current and the superseded version in, then ask something the two answer differently. Then ask a question the document set genuinely can’t answer and see what comes back.

Generation

Finally the model gets the question and the retrieved passages and writes the answer. Constrained that way, it’s far less likely to invent a case or a clause than a general chatbot answering from memory would be. The failure everyone worries about most is largely handled by how the system is built.

What’s left is quieter. An answer can overstate what its sources actually support, or slip in a claim that none of the retrieved passages covers. That’s why a citation has to be checkable at the level of the individual claim rather than the answer as a whole. If clicking a reference opens the source, jumps to the page and marks the passage, a reviewer can check it in seconds. If it points at a 200-page PDF, nobody is going to check anything, and after a fortnight they’ll stop pretending they do.

What to take into an evaluation

Judge the pipeline rather than the demo, and four questions get you most of the way there.

Did it find the passage you’d have gone to yourself? That’s a sharper question than whether the answer sounds plausible. Can you check each claim on its own? Split the answer into the statements that matter and test each against its citation. Keep two scores: whether each citation is right, and how much of the answer carries one at all. What does it do when the document set has no answer, because whatever it does in your test it will also do in production. And what does it do when two documents disagree, which in any real estate is the ordinary case rather than the edge case.

Run all of that on your own documents, with the questions written down and the evidence you expect recorded before you see a single output. Vendor benchmarks are run on corpora chosen to make the vendor look good, and they almost never survive the move to yours.

The legal document work guide sets out the full approach, the evaluation kit has blank worksheets for running it, and legal AI tools compared covers which categories of system are worth putting through it in the first place.