What are AI Hallucinations?

Summary

AI hallucinations are coherent, confident outputs that are factually wrong, fabricated, or unsupported, appearing in chatbots, image generators, and multimodal systems. Generative models predict likely continuations rather than retrieve guaranteed facts, while training incentives, limited or flawed data, generation settings, and error compounding in reasoning models can increase the risk. Examples include Google Bard’s incorrect JWST claim, Air Canada’s nonexistent bereavement policy, Microsoft Sydney’s erratic responses, and fabricated legal citations that led to sanctions. Hallucinations create enterprise risks involving safety, liability, compliance, security, reputation, and trust; recommended controls include guardrails, source verification, adversarial testing, monitoring, human review, retrieval-augmented generation, governed data, and evaluation throughout development and deployment.

Context

Generative AI models can produce plausible-sounding falsehoods because they predict likely outputs rather than reliably retrieve facts. The resulting hallucinations create risks involving health and safety, misinformation, security, legal liability, regulatory compliance, financial exposure, reputation, and user trust.

Approach / What changed

The recommended response treats hallucination prevention as an engineering discipline: use guardrails, source verification, adversarial testing, continuous monitoring, human review for high-stakes outputs, retrieval-augmented generation grounded in governed data, and systematic evaluation across the development lifecycle.

Takeaways

• Hallucinations can arise from training incentives that favor answering over expressing uncertainty, incomplete or outdated training data, and generation settings such as temperature that trade reliability for creativity.
• Examples include Bard’s false claim about the first exoplanet images, Air Canada’s nonexistent bereavement policy, and fabricated legal citations that resulted in court sanctions.
• Detection and mitigation can combine factual consistency checks, semantic entailment, retrieval-based verification, CI/CD evaluation, and human review for high-risk or low-confidence cases.

Related reading