If you are putting AI agents to work on emails, summaries, customer answers or posts, this is for you. The model is rarely the hard part. The hard part is stopping it from inventing a quote, a number or a step that never happened, and then saying it with total confidence.
Short answer: Treat invented facts as a process problem, not a model quirk. Make the agent work only from a source it actually read, turn anything it does not know into an open question for a person, drop every claim nobody can verify, and keep a human as the last gate before anything reaches a customer. Add an automated grounding check when you have a reference source to score against.
Why a confident wrong answer costs more than a blank one
A confident wrong answer costs more than "I don't know" because people act on it. A blank answer sends someone to look things up. A smooth, specific, wrong answer gets forwarded to a colleague, pasted into a quote or sent to a customer, and you find out later.
The AI vendors say this plainly. OpenAI's safety best practices list "hallucinating inaccurate information" first among the limitations you should understand and communicate before you put a model in front of users. That is a model vendor telling you it will sometimes make things up.
So the useful question for a business owner is not "which model never invents things?" It is "what rule stops an invented claim from going out?" That rule is where we put our effort.
1. Make the agent work from a source it actually read
The first rule is simple: the agent writes only from material it fetched for this task, never from memory or a guess. If it is answering a customer, it works from your actual policy or price list. If it is summarizing a call, it works from the transcript. If it is writing about someone's post, it works from the text of that post.
This lines up with what OpenAI recommends. Its safety guidance says that returning outputs from a validated set of materials, where possible, can be safer than generating something new. Its example is routing a customer question to the best-matching existing support article instead of answering from scratch.
OpenAI's guide to optimizing LLM accuracy explains why the source matters so much. When a model lacks your company's knowledge, or its knowledge is out of date, you have to put the right information in front of it. Retrieval-augmented generation (RAG) is the industrial version of that: the system looks up relevant content and adds it to the prompt before the model answers. The same guide warns that RAG can fail in two places. The lookup can supply the wrong context, or so much irrelevant context that it causes hallucinations, and the model can get the right context and still do the wrong thing with it.
What this looks like in a small business
You do not need a RAG system to start. You need a habit: no source, no claim. If your team asks an agent to draft a reply about delivery times, the agent should be handed the current delivery policy. If nobody can hand it a source, the claim does not go in the reply.
2. Turn unknown details into open questions
When the agent does not know something, it should ask, not fill the gap. A missing number becomes a question for the person who knows. A missing example becomes a note that says "we need a real one here." A detail nobody can confirm is left out.
We use this rule on our own content work. Unknown is an allowed answer. A draft that says "we don't have this number yet" is a draft a person can finish. A draft with a plausible invented number is a draft that looks finished and is wrong.
It also gives you a clear audit trail. When the gaps are written down as questions, a reviewer can see exactly what the agent did not know, instead of hunting for the sentence it quietly made up.
3. Keep a person as the last gate
A person should approve anything customer-facing before it goes out. OpenAI's safety guidance recommends having a human review outputs before they are used wherever possible, and calls this especially critical in high-stakes domains.
The same guidance adds a detail that is easy to miss: the reviewer needs access to whatever is required to verify the output. Its example is a summary of notes, where the reviewer should be able to open the original notes. A reviewer who only sees a polished paragraph is checking tone, not truth.
Make the gate a rule, not a habit
The gate works best when the software enforces it. If an agent can approve its own work, sooner or later a confident sentence will slip through on a busy day. If the system simply does not let the agent approve, the only way out is past a person.
4. Add a machine check when you have a reference source
An automated grounding check scores whether an answer is supported by a source you provide, and it can block answers that score too low. It is a useful second layer when you already have the source the answer should come from.
What Amazon Bedrock Guardrails checks
The contextual grounding check in Amazon Bedrock Guardrails looks at two things. Grounding asks whether the response is accurate based on the source, and any new information the response introduces counts as ungrounded. Relevance asks whether the response actually answers the user's question.
AWS's own example makes the difference clear. With a source that says London is the capital of the UK and Tokyo is the capital of Japan, the answer "The capital of Japan is London" is ungrounded. The answer "The capital of UK is London" is grounded but irrelevant, because nobody asked about the UK.
You set a threshold for each score between 0 and 0.99. AWS gives the example of setting both to 0.7, so any response scoring below 0.7 on either is treated as a hallucination and blocked. A higher threshold blocks more ungrounded content, and a threshold of 1 is not allowed because it would block everything.
Know the limits of the check
The check needs three things to work: the source, the question and the response. The documentation lists summarization, paraphrasing and question answering as supported uses and says conversational chatbot use is not supported. The Guardrails overview describes it as one filter among several, alongside content filters, denied topics, word filters, sensitive information filters and Automated Reasoning checks.
A grounding score is a good alarm. It is not a replacement for the person at the gate, because it can only judge an answer against the source you gave it. If the source is wrong, a perfectly grounded answer is still wrong.
Where grounding still fails
Grounding often fails when the source itself is the problem. Three gaps are worth watching.
The source is out of date
An agent that faithfully repeats last year's price list is grounded and wrong. Sources need an owner and a date, and someone has to retire the old ones.
The agent read the wrong thing
We learned this one directly. A comment we drafted for someone else's post reached posting time, and when the post itself was opened, it was about something different from what the comment answered. It went back to review unposted. The rule we wrote afterwards is that a comment is written only from the target post's actual text, never from a guessed or summarized excerpt.
Nobody can see what the agent saw
If a reviewer cannot open the material the agent worked from, they cannot check the claim. Keep the source next to the draft, which is the same point OpenAI makes about giving reviewers the original notes.
Questions to ask your team this week
You can test your own setup with a few plain questions. Ask what source each AI-written answer comes from, and whether a reviewer can open that source in one click. Ask what the agent does when it does not know something: does it say so, or does it guess?
Then ask who approves anything that reaches a customer, and whether the software forces that approval or just hopes for it. Finally, ask when your sources were last updated and who owns them. If any answer is "nobody" or "we're not sure," that is your first fix.
How we approach this at Dasnuve
What worked for us is treating truthfulness as a process rule, not a feature of the model. We run AI agents on our own content work, and the rules are boring on purpose.
Writers verify every external claim against the sources in the brief, and anything that cannot be verified is dropped, not softened. Unknown details go into a note as open questions for a person instead of being filled in. The body of each draft holds only the exact text that will go out, and it is read back after saving to confirm it matches. Every agent uses its own scoped key and role, and there is a human approval column that agents cannot get past: a bot can move a draft to review, but only a person can approve it. When we make a mistake, we write the lesson down, and our writing and research bots read those lessons before each batch.
For more on the access side, see how we let AI bots do real work without losing control and how to safely give an AI agent access to your CRM. If you want help building AI into the systems you already run, see our work on intelligent applications.
FAQ
Why do AI agents make things up?
Language models generate likely text, and likely text is not always true text. When the model lacks the facts, it can fill the gap with something that sounds right. OpenAI lists hallucinating inaccurate information among the limitations businesses should understand before relying on a model.
Will a better model stop hallucinations?
A better model may make fewer mistakes, but it will not make them zero. The safer approach is a rule around the model: work from a real source, admit unknowns, and have a person approve customer-facing output.
What is a grounding check?
A grounding check compares an AI answer to a reference source and scores whether the answer is supported by it. Amazon Bedrock Guardrails scores grounding and relevance and can block answers below a threshold you set between 0 and 0.99.
Do I need RAG to stop invented facts?
No. RAG is one way to put the right information in front of a model, and OpenAI notes it can still fail if retrieval returns the wrong or noisy context. Many teams can start with a simpler rule: hand the agent the source for each task and drop any claim without one.
Should an AI agent ever publish without human review?
For anything a customer will see, we would not allow it. OpenAI recommends human review wherever possible, especially in high-stakes domains. Let agents draft and flag problems, and keep the final yes with a person.
Where to start
Pick one place where an AI agent already writes something your customers or staff act on, such as support replies or meeting summaries. For that one workflow, write down the source the agent must use, what it should say when it does not know, and who approves the output before it goes out.
Run it that way for a week and read the open questions the agent raises. They will show you exactly where your sources are missing, which is a far cheaper discovery than a confident wrong answer in a customer's inbox.