In brief
Conversational AI systems like ChatGPT, Claude, or Gemini sometimes fabricate facts entirely from scratch — we call this hallucination. This is not a rare bug: it is a direct consequence of how they work. They don’t query a reliable database; they generate text one word at a time, picking the most probable next word. Understanding this limitation is the first step toward using these tools correctly.
A text generator, not a truth engine
To understand why an AI makes things up, you first need to understand what it actually does.
An LLM (large language model — the engine behind ChatGPT or Claude) doesn’t “know” things in the way a human knows something. It was trained on billions of texts — articles, books, forums, web pages — and it learned to predict the next word in a sentence. That’s it. When you ask it a question, it doesn’t look up the answer in a database. It constructs, word by word, the sequence of text that seems most plausible.
Imagine someone who has read thousands of medical textbooks but isn’t a doctor. If you ask them a question, they’ll tell you something that sounds like what a doctor would say. Most of the time, it’ll be correct. But sometimes they’ll mix up information, fill a memory gap with an invented detail, or stitch together pieces of knowledge that don’t fit together. And they’ll do it with the same confidence as when giving a correct answer.
In short: an LLM is an imitator trained to produce plausible-looking text. Plausibility and truthfulness are two different criteria — and the AI only controls the first one.
Confidence means nothing
This is the main trap. The AI states false things with exactly the same tone as true ones.
There is no built-in warning signal. No hesitation, no “I’m not sure,” no change in tone when the answer is fabricated. The model generates fluent, assured text in every case, because that’s what it learned to do: produce text that looks correct.
It’s like talking to someone who never says “I don’t know.” Rather than admit a gap, they’d rather invent a plausible answer — and deliver it with complete confidence. The problem is that you, on the other end, have no way to distinguish the moments when they truly know from the moments when they’re improvising.
In short: the fluency of a response is not a reliability indicator. On well-documented topics (Wikipedia, classic textbooks), the AI is often correct. On precise details (numbers, dates, citations), it errs regularly — always with the same confident tone.
Concrete examples
Hallucinations take various forms. Here are the most common:
Sources that don’t exist
You ask for references on a subject. The AI gives you article titles, author names, scientific journals — all perfectly formatted. Except that when you check, some of those sources were never published. The AI fabricated a reference that looks like a real one.
Wrong dates and figures
“This law was passed in 2019.” In reality, it was 2021. The AI doesn’t consult a calendar: it generates the date that seems most probable in that context. The less documented the subject is in its training data, the higher the risk of error.
Incorrect attributions
“This quote is from Einstein.” Except it isn’t. The AI associates famous phrases with well-known figures because that pattern appears frequently in the texts it read — even when the original attribution was already wrong.
Plausible but invented facts
“The city of X has a population of 45,000 inhabitants.” The number seems reasonable, the format is credible, but it came from nowhere. The AI doesn’t “verify”: it produces what seems consistent.
How to protect yourself
The good news: now that you know this happens, you can guard against it. A few simple habits are enough.
| Use case | Vigilance level | Reflex to adopt |
|---|---|---|
| Drafts, brainstorming, rephrasing | Low | Use as-is, the error has no consequence. |
| Summary for a colleague | Medium | Reread, verify 1–2 numerical claims before sending. |
| Professional or public document | High | Verify each fact, figure, citation against an independent source. |
| High-stakes decision (health, legal, financial) | Maximum | The AI is only a starting point — confirm with a professional or an official source. |
Verify important facts
If information will serve as the basis for a decision (a contract, an article, a medical choice), verify it with an independent source. A classic search engine, an official website, a reference work. AI is a starting point, not a final source.
Ask for sources
You can explicitly ask: “Give me the sources for this information.” The AI doesn’t always provide them spontaneously, and the ones it provides aren’t always real — but it gives you something to check.
Rephrase the question
If you doubt a response, ask the same question differently. Or approach it from a different angle. If the answers contradict each other, that’s a signal that the AI doesn’t have a solid foundation on that point.
Cross-reference with a classic search
AI complements a web search; it doesn’t replace one. For factual subjects (dates, figures, laws, people), classic search remains more reliable.
Use AI for what it does well
AI excels at rephrasing, summarizing, structuring, exploring ideas, and drafting content. It is less reliable for precise facts, specific figures, and bibliographic references. Play to its strengths.
Key takeaways
- An LLM (large language model) predicts the next word — it doesn’t query a reliable database. It is a text generator, not a truth engine.
- Hallucinations (fabricating facts) are a structural limitation, not an occasional bug.
- The AI’s confident tone guarantees nothing: it states false things with the same assurance as true ones.
- Systematically verify important facts with independent sources.
- AI is a powerful tool with a known limitation. Being aware of it is enough to protect yourself in everyday use.