New York City spent about $600,000 to build a chatbot that answers business owners’ questions about city rules. In March 2024, reporters at The Markup and the city found the chatbot
telling landlords they could refuse tenants holding Section 8 vouchers
telling employers they could take a cut of their workers’ tips
telling businesses they could refuse cash, three years after a city law required them to accept it.
The city kept it running at roughly half a million dollars a year until the incoming mayor canceled it in January 2026.
The easy read is that the technology failed. I do not think that is what happened.
New York City could not answer those questions quickly through any channel. The rules were real and public and scattered across agencies, notices, and pages written years apart by people who never met, with nothing to say which one won.
The chatbot did not invent a knowledge problem. It published one.
Sit with that, because it is the whole argument.
An AI system does not create the gap between what your organization knows and what it can retrieve. It makes the gap
legible
at scale
to strangers
in writing
at the speed of a search box.
So before you evaluate another vendor, run the seven questions below on your own company. None of them are hard. That’s the point. Every one has an answer, and someone in your building knows it. The only variable is how long it takes to reach.
1. What did we tell the last customer who asked this?
Your team answers the same forty questions all year. Pricing edge cases. Whether you support the thing you technically support but do not advertise. What happens when a project goes over scope?
Almost no organization can retrieve its own prior answer to a repeated question. So each person reconstructs one. They are all reasonable, they are all slightly different, and they drift further apart each quarter as the people who gave the early answers move on.
Nobody lied. Nobody was careless. The answer simply has no home, so it gets rebuilt from memory every time, and memory has a slope.
Your customers compare notes. You do not.
2. Which version of this is current?
In February 2024, the British Columbia Civil Resolution Tribunal ordered Air Canada to pay a passenger about $650 after its website chatbot told him he could apply for a bereavement fare retroactively. The airline’s own bereavement travel page said the opposite.
Both statements were live on aircanada.com at the same time.The money is trivial. The defense is the story. Air Canada argued that the chatbot was, in the tribunal’s summary, a separate legal entity responsible for its own actions. The tribunal’s answer runs eleven words longer than it needed to and is better for it:
“It should be obvious to Air Canada that it is responsible for all the information on its website. It makes no difference whether the information comes from a static page or a chatbot.”
A company stood up in front of a tribunal and argued that it was not responsible for part of its own website. That is what happens when an organization loses track of which version of a policy is the real one. It can no longer recognize its own voice.
Ask what your current refund policy is. Then count how many places a customer could find a different answer.
3. Why did we decide it this way?
Organizations are good at preserving decisions and terrible at preserving reasons.
The rule survives. The constraint that produced it does not. Two years later, nobody can tell you whether the rule still holds, because nobody can tell you what it was for. So it stays, because removing a rule you do not understand feels riskier than keeping it.
This is how companies accumulate policies that everyone follows, and nobody will defend. Ask why the approval threshold is set where it is. Watch what happens when the honest answer is that someone set it in 2019 for a reason that stopped applying in 2021.
Of the seven, this one costs the most and shows up on no report anywhere.
4. Who owns this?
A question with no owner has no maintainer. A page with no maintainer isnt stable; its just unedited.
Most companies have a wiki full of documents in exactly this state. They look maintained. They are abandoned in place.
5. Has anyone here already done this?
Someone on your team is currently rebuilding an analysis, a template, a deck, or a process map that already exists somewhere in the company.
They are not lazy. They looked. The search returned nothing useful, so they concluded nothing existed, which is a rational inference from the evidence available to them and completely wrong. The work was there. It was named something else, filed by someone who has since left, in a place they had no reason to look.
You pay for that work twice and count it once.
6. What do we actually promise, in writing?
Not what you intend to promise. What your published surfaces currently say, added together, to a customer who reads them all.
New York City believed it was publishing help for small businesses. What it published was permission to break the law. Air Canada believed it had a bereavement policy. What it had was two.
Every organization has a gap here, because promises accumulate across contracts, proposals, marketing pages, support macros, and whatever a salesperson said in a meeting last March to close a deal. Nobody holds the sum. The sum is what you are accountable for.
7. What did we learn the last time this went wrong?
You have had this outage before. This client relationship has ended this way before. This hiring mistake has a precedent inside your own company, with a post-mortem written by someone thoughtful, saved to a folder that has not been opened since.
An organization that cannot reach its own post-mortems does not have institutional memory. It has an archive, which is a different thing. An archive is where knowledge goes when nobody built a path back to it.
What the seven have in common
None of these are intelligence questions. Every one of them is a retrieval question.
The answers exist.
They’re already inside the company, held by a person, a file, a thread, or a system you are paying for. Nothing on this list requires new insight, new research, or a smarter team. Each one requires a path from question to answer that is shorter than the effort someone is willing to spend before giving up and rebuilding it from scratch.
My own rough number for that threshold is five minutes. I have not measured it, and I am not aware of anyone who has. It is what I have watched happen, repeatedly, and I would rather give you the figure I actually use than a sourced one I do not believe. Under five minutes, people retrieve. Over five minutes, people improvise and move on, and their improvisation becomes the new answer for everyone who asks them next.
Your company is not running on what it knows. It is running on what it can reach inside five minutes. The rest is inventory you are storing and paying for and cannot sell.
What to do about it, in order
Do not buy anything yet.
Take the seven questions into a room with the people who would actually be asked them. Do not discuss them. Time them. Pick one real instance of each, start a clock, and find the answer the way a normal person would.
You are measuring time to answer, not whether an answer exists. Everyone already believes the answers exist. The number that matters is how long the path is, and whether two people walking it separately arrive at the same place.
Rank the seven by how long they took. That ranking is your build order, and my bet is it will not match the order you would have guessed before you started. I want to be clear that this is a prediction based on watching it happen, not a finding from a study. The reason I expect it is structural: leadership gets fast answers by asking a person directly, and that path does not exist for anyone else in the company. The gap you cannot see is the one nobody has ever had to walk around you to reach.
Then fix them in that order. Fixing means giving each answer one home, one owner, one date, and one path that anyone can walk without knowing whom to ask. Do that seven times, and you have started building what I call a company brain: the place where what your company knows is organized, current, and reachable by your people and by any system you point at it.
That work is unglamorous, mostly not technical, and it decides whether every AI dollar you spend after it returns anything.
The bottom line
New York City did not have an AI problem. It had a knowledge problem that a $600,000 AI system read aloud to the public for two years.
Your AI will answer all seven of these questions tomorrow, instantly, in complete sentences, to anyone who asks. It will sound certain either way. The only thing you get to decide is whether it is right.
By the way, I write a weekly piece on the knowledge your company already has but cannot reach. halfofit.trickywombat.ai