On 20 September two of my own AI assistants were asked the same question and gave two different prices for the same service.
One answers visitors on my company's English site, the other on the Romanian one. Both had been given the same catalogue. Asked about one website package, the first said 11,900 lei. The second said 9,800 lei.
The catalogue said 11,900. The second assistant had been quoting a price 2,100 lei too low, to exactly the people who buy, since 27 August.
Nothing had failed. No log line, no error, no alert. Just an old copy of the catalogue inside a knowledge base, answering with full confidence. I saw it only because a second assistant happened to contradict the first. One assistant on its own cannot notice that it is wrong.
The claim was right once
The wrong price was not invented. It had been true. I had loaded the catalogue into that assistant weeks earlier, then the catalogue changed and the copy did not.
That is the normal way agents go wrong in production. Not hallucination in the dramatic sense. A copy that was correct on the day it was made, read back weeks later as if it were today.
So the fix is not a smarter model. The fix is to make every claim point at where it came from, so that a person or a check can follow the pointer and see whether the source still says it.
A table is not a measurement
Two weeks earlier I made the same mistake myself, with no AI model involved.
I compared my live catalogue with a file I had saved from another system and built a tidy table: 29 rows, a "verdict" column, percentages. It said none of 18 prices matched. It looked like a measurement.
The file was five days old. The two live pages I could read myself both showed the correct price. I checked them only after I was told I was wrong.
A comparison built from a file inherits the age of the file, however convincing it looks.
The shape of a cited claim
A claim is not a string. It is a string plus the place it came from, plus the exact words in that place that support it.
from dataclasses import dataclass
@dataclass
class Claim:
text: str # what the agent wants to say
source: str # where it came from: a URL, a file, a page number
quote: str # the exact words in the source that support it
def supported(claim, read_source):
"""True only if the source can be read NOW and still contains the quote."""
body = read_source(claim.source)
if body is None:
return None # could not check: never counts as supported
return claim.quote in body
The quote is checked when the claim is about to be used, not when it was written. That is what would have caught the old price: the catalogue no longer contained "9,800".
A source that cannot be read is not support. And a claim that fails is not rewritten by the model until it sounds right. It is dropped, and a person sees that it was dropped.
One source, and copies checked against it
After that day, every family of facts that changes got one named source. For my prices it is one file. Both websites, both assistants and every offer are copies, rebuilt from it, never edited by hand.
Each copy has a guard that compares it with the source every day. When the assistants' knowledge differs from the file, the guard rebuilds it, restarts the assistant and checks again. A guard that covers half the copies is worse than none: my first one checked one of two sites, said all was fine, and the other site served old prices for 70 minutes after a failed build.
What may be said without a source
"Hello" and "here is how to contact us" need no citation. Anything a customer could act on or hold me to does: a price, a date, a feature, a status, a number. If the agent cannot point at where it read it, it says it does not know and offers a person.
That makes the assistant look less clever on a slow day. It has not quoted a wrong price since.
Before an agent says something in your name, ask it what a careful colleague would ask: where did you read that, and does it still say so?
This is a shortened version of chapter 4 of my book in progress, Agents You Can Leave Alone, which I am writing in public on Leanpub: https://leanpub.com/agents-you-can-leave-alone