You ask an AI about a clause in your contract, it answers confidently, citing an article… that doesn’t exist. This isn’t a rare bug: it’s a feature of how these models work.
Why it happens
A language model is trained to produce the most plausible text, not to check that it is true. When it doesn’t know, it can still write something that looks right. That’s what’s called a “hallucination”.
How to reduce the risk
- Give it your documents: by plugging it into your files (the RAG principle), it answers from your sources rather than from memory.
- Ask for sources: an answer with no supporting passage should raise a flag.
- Allow “I don’t know”: tell it explicitly to hold back when information is missing.
- Test with questions whose answer you know before trusting it.
- Keep a human in the loop for anything that matters.
The limits
No safeguard brings the risk down to zero. For sensitive decisions (legal, finance, health), the AI prepares, a professional checks.
