Deloitte gave the Australian government a report with academic references in it that do not exist. The firm refunded the final installment of the contract: AU$97,000.
Deloitte Canada put fabricated citations in a health workforce plan for Newfoundland and Labrador, on a contract worth one point six million dollars. A provincial accounting regulator opened an investigation, and the province rewrote its procurement rules.
EY published a cybersecurity report in which sixteen of twenty-seven citations were fabricated, misattributed, or pointed nowhere. The firm took it off its website.
KPMG published a report about how to do AI well. Five of its forty-five citations checked out.
Four firms whose entire product is being right about things. Each of them shipped a document with invented facts to a paying client and got caught by somebody outside the building.
Those are the ones we know about. They became public because a researcher checked, or a journalist checked, or a regulator checked.
Nobody checks the answer your AI gave your project manager on Tuesday.
A question nobody owns
One of the most common things people type about AI right now is a question about the 30% rule. The internet has five answers.
AI does seventy percent of the work, and humans keep thirty percent for judgment. AI does thirty and humans do seventy, which is the exact inverse, published with the same confidence. No more than thirty percent of your homework may be AI-written, which turns out to be a university plagiarism policy. Automate your first thirty percent of repetitive tasks. Put thirty percent of your AI budget into data quality.
Most of those pages admit, in their own text, that the rule is not real. No standard behind it. No regulator wrote it. The closest thing to a source is a Gartner prediction from July 2024 that thirty percent of generative AI projects would be abandoned after the proof of concept, which is about projects dying rather than about splitting work with a machine.
A question with no owner gets answered by people with something to sell, and they all sound certain. It is the whole failure, running in public, where you can watch it happen.
Your company has questions like that too. Yours are simply not on the internet where anyone can check them.
It was trained to guess
The usual explanation for wrong answers is that the model is broken or the technology is immature. It is neither.
OpenAI published a paper in September 2025 arguing that models make things up because we trained them to. The mechanism is worth understanding, because it explains why buying a better model does not fix this.
Models get scored on benchmarks. On most benchmarks, a wrong answer scores zero and “I do not know” also scores zero. Guessing is free. If you are unsure and you guess, you might be right. If you admit you are unsure, you definitely get nothing. Over millions of training examples, the version that survives is the one that always has an answer.
OpenAI demonstrated this on its own models. One of them declines to answer one percent of the time and gets seventy-five percent of its answers wrong. A newer one declines fifty-two percent of the time and gets twenty-six percent wrong. Accuracy between the two moved by two points. The difference is that one of them will tell you when it does not know.
Your AI is not lying to you. It was rewarded, for years, for never saying that it was unsure. So it does not.
## Your documents do not fix it
This is the part where the money goes, and it is the part almost everyone gets wrong.
The standard fix is to give the AI your documents. Point it at your drive, your wiki, your policies. Ground it in your own data. Every vendor in this market sells that, and I sell a version of it too, so weigh what follows accordingly.
It helps. It does not fix it.
RAGTruth, published at a computational linguistics conference in 2024, took about eighteen thousand AI answers where the model had been handed the source documents, and had people annotate them word by word. GPT-4, with the documents in front of it, made claims those documents did not support in roughly one answer out of six.
One in six, with the answer sitting in the file.
Then there is the worse finding. Stanford researchers ran twelve hundred questions where they planted a wrong answer in the retrieved document on purpose, and watched what the model did when that document disagreed with what it already knew correctly. It followed the document. More than sixty percent of the time, the model threw out its own correct knowledge and repeated the error it had been handed.
Grounding an AI in your documents does not make it right. It makes it obedient to whatever it pulled.
So if your drive holds two versions of the same policy and one of them is from 2021, you have not built a system that gives correct answers. You have built one that confidently hands you whichever version it reached first. Ask again next week and get the other one.
Researchers at Tsinghua and Cambridge have a name for this: Inter-context conflict is two retrieved sources contradicting each other, and their survey names outdated documents sitting beside current ones as a principal cause.
Last week I said your knowledge sits on islands. This is what happens when you hand a machine all the islands at once and do not tell it which one is current.
Nobody can tell you the number
People want to know what percentage of AI answers are wrong. The honest answer is that the number does not exist.
You will see three percent. That comes from a leaderboard where models are handed one document and told to summarise only what is in it. It measures faithfulness to a page, not truth about the world, and secondary coverage constantly relabels it.
You will see forty-five percent, from the European Broadcasting Union and the BBC. That is a real study across twenty-two broadcasters, with journalists grading twenty-seven hundred answers. Forty-five percent is the share of answers with at least one significant issue across four criteria, and a missing citation counts as an issue. The accuracy-specific figure in the same report is twenty percent. Guess which one gets quoted. Worth adding that the journalists doing the grading were employed by the same broadcasters whose content was being cited, which the report discloses openly.
You will see ninety-five percent, as in ninety-five percent of AI pilots fail. That traces to a report whose own text says something different, with the calculation never shown and the survey collected at four conferences.
Three different things, measured three different ways. Anyone quoting you one number for how often AI is wrong is selling you something, and that includes anyone quoting you a low one.
The number for your company is not in any study. It is in your building.
The hour that finds it
You need a room, four to eight people who answer questions for a living, and no preparation. Do not clean anything up first. Cleaning up first is how you get the pilot result instead of the real one.
Step one, fifteen minutes. Everyone writes privately. Read this out and then stop talking:
Write down the three questions you were asked this month that you could not answer in five minutes. Not the hard strategic ones. The ones where the answer definitely exists somewhere, and you could not get to it. Do not talk while you write.
The silence matters. The first person to speak sets the frame for the room.
Step two, ten minutes. Read them out and find the repeats. The question more than one person wrote down is your gap. It is almost never the one you would have picked.
Step three, twenty minutes. This is the step that does the work. Take the top repeat and trace the answer.
Where does it live?
How many places?
Are those copies the same?
When was each one last changed?
Who is allowed to change it?
Most rooms answer the first two confidently and then stall completely on whether the copies are the same. Somebody says there is a version in the shared drive. Somebody else says the real one is in a thread. Nobody knows which is current.
That stall is the whole argument. It is inter-context conflict, sitting in your company, found by hand in twenty minutes.
Step four, fifteen minutes. Price it.
How often does the question get asked?
Who answers it?
What is an hour of that person worth?
Then the one that hurts:
What happens when somebody acts on the wrong version?
Deloitte’s number was AU$97,000 and a corrected report on a government website. Yours will be different. Write it down anyway.
You leave with one sentence, and it reads something like this. The question we cannot answer quickly is this one; the answer lives in four places, two of them disagree, and it gets asked eleven times a month.
That sentence is worth more than any pilot, and here is why. You did, by hand, with people who work there, exactly what you expected the AI to do on its own. If your own team needs twenty minutes and an argument to work out which version of a policy is current, a model doing it in four hundred milliseconds is not being more careful. It is being faster, and it will not tell you it was unsure.
Fix the sentence, not the model
Pick the one question. Make one version of the answer current. Put it somewhere both your people and your systems can reach, and delete the rest. That is one afternoon. It is also the smallest possible piece of a company brain, and you will have built it before you have bought anything.
Run the hour this week, then tell me in the comments which question turned out to be the repeat. I am collecting them, and the pattern across companies is not what I expected.