What 70 Scientific Books Taught Me About Why AI Misreads a PDF

My name is printed in the front matter of more than seventy scientific volumes, and I have spent twenty-three years turning manuscripts into typeset pages. I now build AI systems that read documents for a living. Those two facts sound unrelated. They are the same problem seen from opposite ends, and the first one explains most of what goes wrong in the second.

Here is the short version. A document that has been typeset is not a bag of words. It is a structure, and almost all of that structure is thrown away the moment it becomes a PDF. When an AI reads that PDF, it is not reading the document. It is reading the shadow the document left on a page.

What a typesetter actually does

Setting a scientific book is not decoration. It is the act of encoding meaning into position.

A running head tells you which chapter you are in without reading a word. A float, a table or figure that moves to wherever it fits, carries a caption precisely because it has been separated from the paragraph that referred to it. Front matter is a defined sequence: half title, title, copyright, dedication, contents, preface. A displayed equation is numbered because something later will point back at it. A footnote sits at the bottom of a page and a citation sits in a bibliography, and those are different objects doing different jobs.

None of that is styling. Every one of those decisions is semantic. The compositor's craft is knowing that a two-line gap and a smaller point size mean this is an aside, not the argument, and that a reader will absorb that without ever being told.

In LaTeX, the source knows all of this explicitly. There is a \chapter, a \caption, a \label, a \cite. The structure is not implied by how it looks. The structure is written down, and the appearance is derived from it.

What the PDF keeps

The PDF keeps the appearance and discards the source.

A PDF is a set of instructions for placing glyphs at coordinates. Put this character at this x and y, in this font, at this size. That is essentially all it is. The fact that a particular run of glyphs was a chapter title, and another was a caption, and a third was a footnote, is not recorded anywhere. It is inferable by a human because humans read layout fluently. It is not stated.

So when a system ingests that PDF, it has to reconstruct, by guesswork, everything the compositor originally knew for certain. Is this larger text a heading or an emphasised sentence? Does this two-column page read left column then right, or across? Is this number a footnote marker or an exponent? Does this table have a header row, or is the first row just bold?

Every one of those guesses is a place where a document quietly becomes a different document.

The failures I see, and why they are structural

Three show up constantly, and none of them is a model problem.

Reading order. Two-column layouts are the classic. Extraction that follows the coordinate stream rather than the column flow will interleave the two columns line by line. The output is grammatical, fluent nonsense. Nothing errors. The text is all there. It is simply in an order that never existed.

Orphaned captions and floats. Because a float moves, a table can be printed pages away from the sentence that discusses it. Flatten that and the table arrives with no context, or worse, glued to whatever paragraph happened to be adjacent on the page. A figure that supports one claim gets silently attached to another.

Footnotes folded into the body. A footnote is a qualification, very often the exact qualification that limits the claim above it. Merge it into the running text and you get a sentence that asserts something the author deliberately hedged.

I want to be precise about the point. These are not hallucinations. The model is faithfully reporting what it was given. It was given a scrambled document.

Scans are a different problem again

Everything above assumes a born-digital PDF, where the glyphs are real characters. A scan is worse, because there are no characters at all, only pixels, and OCR has to invent the text before anything can read it.

OCR is very good now. It is not good at deciding that this block is a table and this one is a paragraph, and it has no idea that page 14 is a continuation of the table on page 13. In practice, tables are where scanned documents do the most damage, because a number lifted out of a misread table looks exactly as authoritative as a number lifted out of a correct one.

This is why "just OCR it" is not a plan. OCR converts pixels to characters. It does not convert a page back into a document.

What I do about it

The habit that came out of typesetting, and that I now build into software, is this: treat structure as evidence, and treat its absence as a fact worth recording.

Concretely, in the systems I build for reading commercial and financial documents:

Reconstruct reading order deliberately rather than trusting the coordinate stream. Column detection first, then flow.

Keep tables as tables through the whole pipeline. A table flattened into prose has lost the relationship between a row label and its value, and that relationship is usually the entire point of the table.

Carry the location with the content. Every extracted claim keeps the page it came from, so a reader can go and look. This is the same discipline as a citation in a bibliography, and it exists for the same reason.

Cut what cannot be traced. If a figure cannot be tied back to a specific place in a source document, it does not get repeated. It gets flagged. A missing number is an inconvenience. A confident wrong number is a decision made on fiction.

Say when the input was poor. If a document arrived as a low-quality scan, that belongs in the output, not buried in a log. The reader is entitled to know how much to trust what they are looking at.

The part that transfers

Typesetting taught me that most of what a document means is carried by how it is arranged, and that this arrangement is invisible until you try to reproduce it mechanically. Twenty-three years of putting structure into pages turns out to be unusually good preparation for the problem of getting structure back out of them.

The people who feel this most sharply are not publishers. They are anyone who has to make a decision from a hundred-page document that arrived as a PDF: an acquirer reading a data room, a lender reading management accounts, an analyst reading a filing. The document they were sent and the document their software read are not always the same document, and nothing in the process tells them so.

That gap is the whole reason I build the way I do. Cite the source or cut the claim, and when the source itself was damaged in transit, say that out loud.