Back to blog

Hallucinations in customer service: how to make AI replies safe

Published on November 8, 2026

A Humane figurine with a magnifying glass checks a draft reply while another marks a wrong line with a red pencil.

A customer asks: “Can I still return the jacket after 30 days?” The AI replies: “Yes, of course. You have 60 days with us, free of charge and no questions asked.” The sentence is friendly, grammatically clean and in your brand's tone. It just isn't true. Your return period is 14 days, and return shipping costs the customer postage. The AI didn't look it up. It formed a sentence that fits a thousand other online shops and slipped it under your name.

That's a hallucination, and it's the biggest risk in using AI for support. Not because it's frequent, but because it's invisible. A bad reply is easy to spot. A wrong, well-written reply is only spotted when the customer is standing at the door with the parcel, insisting on what was promised.

This article explains where hallucinations come from, what they cost, where in support they typically occur, and which five safeguards a system needs so that only verified information goes out.

Why language models say something when they know nothing

A language model has no concept of “I don't know”. It generates the most probable next sentence, given everything it has seen so far. If your actual return period is in that context, the most probable sentence is correct. If it's missing, the most probable sentence is what most online shops write. And it sounds just as confident.

That isn't a bug a better model fixes. It's a property of the technology. So the question isn't whether a model can hallucinate, but whether the system around it makes sure the model has the right information, only answers within that information, and that someone looks when it's unsure.

What a wrong reply costs

The best-known case is Air Canada's. A customer had asked the airline's chatbot about bereavement fares and was told he could apply for the discount retroactively. The actual policy didn't allow that. Before the tribunal, Air Canada argued the chatbot was a separate entity whose statements the airline wasn't responsible for. The tribunal rejected that in February 2024: a company is responsible for all information on its website, whether it comes from a static page or a chatbot, and Air Canada had to refund the difference, as McCarthy Tétrault summarises the case.

The amount was small. The principle isn't: what your AI promises the customer, you have promised. A hallucinated 60-day period is a 60-day period, at least for that one customer, and very probably for the next one who sees the screenshot in a review.

The second case is less legal, but just as instructive. In January 2024 a customer got the parcel carrier DPD's chatbot to swear and write a poem about its own uselessness. DPD switched off the AI component; according to TIME, an error after a system update was the cause. The damage wasn't a sum of money but a screenshot that spread by the million.

And then there are the cases that never make the news: the customer who got a wrong delivery promise and never uses the shop again. According to a Gartner survey from 2026, only 27 percent of customers would try a chatbot a second time after a negative experience. A hallucination is almost always a negative experience, just delayed.

Where hallucinations typically occur in support

Not every request is equally at risk. Five spots are particularly vulnerable:

  • Deadlines and conditions. Return period, shipping costs, delivery time, warranty. Every shop has its own rule here, and the model knows the rules of a thousand other shops.
  • Shipment status without data. If there's no scan, “your parcel is on its way” is the most obvious wording, and it's a claim without basis.
  • Product properties. Material, size, compatibility, ingredients. If the knowledge base doesn't contain the answer, the model fills the gap with what's usual.
  • Commitments. “We'll send a replacement right away”, “Your refund is already on its way”. Sentences that reassure the customer and promise an action nobody has triggered.
  • Legal statements. Withdrawal, warranty, liability for transport damage. The model likes to sound binding where it should stay general.

Look at your own support and you'll notice: these are almost exactly the spots where a new employee makes mistakes in their first week, too. So the solution is similar: clear rules, looking up instead of guessing, and someone reading along at the start.

Five safeguards a system must have

1. Replies from data only, not from memory

The most important safeguard is that the AI looks things up before it replies. The order from Shopify, the shipment status from the carrier, the rules from your knowledge base. What isn't in those sources must not be in the reply. Here's how it works at Humane: the agent pulls order and customer data from Shopify and the live shipment status from the carrier before it forms a sentence. If the carrier has no scan, the agent writes that the shipment is registered and there's no status yet. It doesn't write “on its way”.

2. Rules that are checked on every reply

Your return period, your goodwill limits, your handling of transport damage belong in a rule set that's checked against every reply, not just during training. At Humane that's the Business Rules Engine: stored rules are applied on every reply. A reply that names a 60-day period when the rule says 14 days doesn't get through.

3. Draft instead of send when the AI is unsure

A system must be able to distinguish between “I have all the data and the rule applies” and “something is missing here”. In the first case the reply may go out automatically, if you've approved the category. In the second case it becomes a draft you see before the customer does. That's the product core many chatbots lack: the ability not to answer. Which tickets go out at Humane without approval and which never do is in Auto-send: which tickets Humane answers without approval.

4. Review before sending, for everything at the start

The first week should be a training week in which every reply is a draft. You read what the agent would have written and correct it. That's not just safety; it's also the fastest way to find hallucinations: they show up in the draft before they reach the customer. Only once you've waved a category through without corrections for a while do you enable automation there.

5. Corrections that become rules

When you correct a draft, the system has to learn from it. Otherwise you correct the same hallucination anew every week. At Humane a correction becomes a rule: correct once, right every time after. How that works in detail is in Correct once, right every time.

How to test your own system

You don't need to be an engineer to test an AI tool for hallucinations. Five test emails are enough, and you can send them during any trial period:

  1. A question about a deadline that's in your rules. Does the right number come back?
  2. A question about a deadline that isn't in your rules. Does the AI say it doesn't know, or does it invent something?
  3. An order number that doesn't exist. The right reaction is a follow-up question or a draft, not a reply about “the order”.
  4. A delivery question about a shipment with no scan. Does the reply say “on its way” or “no status yet”?
  5. A question about a product you don't sell. Does the AI recommend it anyway?

If a system passes all five, you have no guarantee, but you know it has the basic mechanics. If it fails two of them, it's a chatbot that writes well, not an agent that looks things up. The difference is described in AI chatbot vs. AI agent.

What even the best safeguards can't do

Honestly: none of the five safeguards makes hallucinations impossible. They make them rare, visible and correctable. What remains are two responsibilities on your side. First: the knowledge base has to be right. If an outdated deadline sits there, the AI answers consistently wrong without hallucinating at all. Second: automation belongs only in categories where you've seen it work. A system that asks you before it commits isn't slower. It's what separates an agent from a liability.

In short

Hallucinations happen when a language model replies without having the data. The solution isn't a better model but a system that looks up instead of guessing, checks your rules on every reply, produces a draft instead of an email when unsure, presents everything for approval at the start, and learns from every correction. Test every tool with a question it can't know. The answer to that one question tells you more than any demo. The full overview is in AI in customer service: what actually works in 2026.

If you want to see what the agent presents as a draft for your requests, and what it doesn't dare to answer: try 100 customer requests for free, no credit card.

Sources