How to Stop an AI Support Agent Inventing Answers About Your Own Product
September 15, 2026 · 10 min read · by the hiy team
The short answer
An AI support agent invents answers in five predictable places, and each has a control that is specific rather than general. Thin retrieval is closed by a strict scope that answers only what the material supports. Numbers reconstructed from memory are closed by keeping limits, prices and known issues as rows the agent quotes verbatim. Stale documentation is closed by corrections that override the old text. A false premise in the question is closed by an agent allowed to refuse the premise, not just the answer. A helpful tone that rounds up is closed by a neutral voice with no incentive to please. Then you prove it with a test set of questions you know are outside the material — and the test has to be run before customers are, because the first invented answer is public and in your product's name.
None of these is "write a better prompt". A prompt is a request. The controls below are facts about what reaches the model and what leaves it.
Why does a support agent make things up?
Because a language model is a completion engine, and a confident paragraph is its most natural completion. It does not know that it does not know; it knows that the sentence after "Your Team plan includes" usually contains a feature list.
Support makes this worse than general chat, for a reason that is easy to miss: the shape of the answer is obvious even when the content is not. Every product has a refund window, a seat limit, an export format, a rate limit. The model has seen ten thousand of each. Asked about yours, it does not need your documentation to produce something that looks exactly like the right answer. It only needs your documentation to produce the right one.
So the invented answer is not an error the model can feel. It is fluent, formatted like your docs, and wrong in a number.
Growth: up to 25 members included. Additional members are billed per seat at the rate shown on your invoice.
The five leaks, and the control for each
| Where it leaks | What it looks like | The control |
|---|---|---|
| Thin retrieval | The docs mention exports, seats and pricing; the customer asked about export seats; the agent stitches a policy out of three near-misses | A strict scope: if nothing clears the retrieval bar, nothing reaches the model, and the only available output is a refusal |
| Numbers from memory | "Up to 10 API keys" when the page says 5 — the model remembered the shape and filled in a typical value | Keep limits, prices, error codes and known issues as list rows the agent looks up and quotes exactly, never paraphrases |
| Stale documentation | The old pricing page is still indexed beside the new one; the agent picks whichever scores higher today | A correction that overrides every other passage on that question, and a re-import when the page changes |
| A false premise | "Why was the free plan's 1,000-message limit reduced?" when there never was one; the agent explains the reduction | An agent permitted to refuse the premise, not just the answer — and a test that checks it does |
| A helpful tone | "Yes, that should work" about an integration that does not exist, because agreeing is friendlier than declining | A neutral product voice with no persona to please and nothing to sell |
Leak one: thin retrieval, and why a prompt cannot fix it
The common case is not a question about something you have never touched. It is a question adjacent to three things you have written. Retrieval returns your onboarding page, your terms summary and a post about team plans — all somewhat about seats and money, none of them your seat policy. The model now has plausible-looking material and an instruction to use it. It writes a seat policy.
No wording of "only answer from the context" removes this, because the prompt cannot see that the passages are off-target. The fix is upstream: a threshold on what is allowed to reach the model at all, so that a weak match produces an empty context and an empty context produces a refusal by arithmetic rather than by obedience. The layered version of this argument, with how to set the threshold empirically, is in a chatbot that only answers from your documents.
In hiy a Support Agent starts on Strict — the firmest of the three scope settings — which answers only what your own material supports and declines everything else plainly. Your own personal twin starts one notch looser, because a coach answering a general question about their field is helpful; a support agent answering a general question about your product is inventing it.
Leak two: numbers from memory
A passage that says "the Growth plan includes 25 members" is a fact the model has to read out of a paragraph. It usually does. Sometimes, when the paragraph also mentions 2,500 messages and a 25% discount, it does not.
The control is to stop asking it to read numbers out of prose. Keep the things a customer needs exactly — limits, prices, error codes, what's broken this week and the workaround — as rows in a list, and have the agent look the row up and quote it. hiy's lists do this: the agent quotes only what a lookup actually returned, shows a price exactly as you typed it ("from £2,000", "DOE") and never computes or converts one, and if no row matches it says so rather than describing what it expects you'd have.
The single highest-value list for a support agent is Known issues. It answers the question customers ask most when something is wrong — is it me, or is it you? — from a row you wrote this morning, with the workaround attached, instead of from a help page written before the bug existed.
Leak three: stale documentation
Docs go stale in two ways. The page changed and the index did not; or the page changed and the old page is still there, one folder over, scoring nicely.
Re-importing closes the first. For the second, you need a mechanism that lets a newer statement win over an older one without deleting the older one — because deleting is slow, and because the old text may still be the right citation for a different question. In hiy that is a correction: when a corrected passage and the correction would come up together, the agent answers from the correction, in its wording, and the corrected material stays searchable and citable for everything else. A correction is also the cheapest way to close a gap the queue surfaced: answer once, in the words a customer would use to ask, and the agent knows it from then on.
Leak four: the false premise
"What did the release notes say about the 1,000-message limit on Free?" There was never such a limit. A weak system narrates the release notes. A good one says there is no such limit in the material.
This one is rarely tested and often leaked, because the model's helpfulness is pointed at the question, and the question has already asserted the fact. The control is partly the strict scope above — the premise has no passage to stand on, so the retrieval is thin — and partly a check you have to run yourself, which is why it is one of the five kinds in the 20-question test.
Leak five: the helpful tone
The most human leak. Asked "will this work with Zapier?", an assistant tuned to be agreeable says "yes, that should work" — because agreement is warm and refusal is awkward, and because a persona built to sound like a friendly person carries that person's instinct to reassure.
A support agent should not have that instinct. It should answer in a neutral product voice, say it is the product's support, and have nothing to sell — because an agent that steers toward a sale is also an agent that rounds up. In hiy a Support Agent has no persona layer at all (the part of the system that carries somebody's phrasing is never built for it), the services catalogue is dropped before the prompt is assembled, and the booking link is withheld from every answer. Those are bounds in code, not preferences in a prompt.
The decline has to be matched, not assumed
One subtlety that decides whether any of this pays off. When the agent declines, that miss should go to a queue you answer. But detecting a decline is harder than it looks: a strict scope tells the model to say "that isn't something I have from Acme", not the literal "I don't know", and a detector that only looks for the literal phrase will miss nearly every real refusal. Infer misses from weak retrieval scores instead and you get the opposite failure — a queue full of questions the agent did answer, which is a queue nobody reads.
hiy's gap queue is written when the reply reads as a decline in the wordings the model actually produces, checked against a corpus of real stored answers. It is a small thing that determines whether the loop closes. Ask any vendor how their "unanswered questions" list is populated; the answer tells you whether it is worth opening.
Prove it before anyone sees it
Write down three questions you are certain are outside your material, plus three you know it covers phrased in words you did not use, and ask all six in a sandbox that does not count. Three refusals and three cited answers means the controls hold. Three fluent paragraphs means you have a prompt and a hope. The full protocol — twenty questions across five kinds, and what each failure tells you to fix — is in how to test a support agent before launch; in hiy the Sandbox is where you run it, and nothing asked there reaches your allowance, your Insights or your queue.
The honest version
These controls stop invention from thin air. They do not stop a confident misreading of a passage that genuinely was retrieved — something scored well, a citation was attached, and the paraphrase is subtly wrong. That failure passes every check above, and the only defence is the citation itself: a customer who can open the passage can catch it. It is why the passage has to be readable, not just named.
And strictness has a price. An agent tuned never to leak will sometimes refuse a question the docs did in fact answer, phrased in a way the retrieval missed. You are choosing which error to explain to a customer: an unhelpful refusal, or a helpful invention. For support, the refusal is the cheaper apology.
Where to start
Turn your limits, prices and current bugs into rows before you tune anything else — it closes the most damaging leak in an afternoon. Then run the six questions. If you are starting from nothing, a hiy Support Agent begins on strict scope with the catalogue withheld and the queue matched to real declines, so the six questions are the first thing you do rather than the last thing you build.
Questions people ask
Can a system prompt stop an AI support agent from hallucinating?
Not on its own. A prompt is a request the model treats as a preference; given three loosely related passages it will still stitch a confident answer. The controls that hold are upstream of the prompt: a retrieval threshold that leaves the context empty when nothing matches, lists the agent quotes verbatim, and corrections that override stale text.
What is the single most effective way to stop invented numbers?
Stop asking the model to read numbers out of prose. Keep limits, prices, error codes and known issues as rows in a list the agent looks up and quotes exactly. A row cannot be paraphrased into a different number.
How do I know if my support agent is inventing answers right now?
Ask it three questions you are certain the documentation does not cover — one adjacent to your product, one asking for a specific number you never published, and one with a false premise — in a sandbox. Three refusals means the controls hold; three plausible paragraphs means it is inventing.