Hallucination
- Definition
- When an AI model generates confident-sounding but factually incorrect or fabricated information. Hallucination is the number-one barrier to enterprise AI adoption and a major focus of current research.
- Why it matters
- Hallucinations are why enterprises remain cautious about AI deployment. A model that makes things up with perfect confidence is worse than one that says 'I don't know,' because users trust the confident-sounding output. In high-stakes domains, hallucinated medical advice, legal citations, or financial data can cause real harm. The industry has made significant progress, with hallucination rates dropping from 15-20% to 2-5% on standard benchmarks, but even 2% is unacceptable for many applications. The strategic response is not to wait for hallucination-free models (which may never exist) but to build systems that detect, mitigate, and recover from hallucinations through grounding, citations, and human oversight.
- In practice
- In 2023, a New York lawyer was sanctioned after submitting a brief with AI-hallucinated case citations, becoming the poster child for hallucination risk. Vectara's Hughes Hallucination Evaluation Model tracks hallucination rates across models, showing a steady decline from 2023 to 2025. RAG (retrieval-augmented generation) reduced factual hallucination by 50-80% in enterprise deployments. Anthropic and Google added citation capabilities so models can reference specific source documents. Production systems now commonly use hallucination detection as a post-processing step, flagging outputs that contain claims not grounded in provided context.
We cover safety & governance every week.
Get the 5 AI stories that matter — free, every Friday.
Related terms
Grounding
Techniques that anchor AI outputs in verifiable facts by connecting models to external knowledge sources. Grounding reduces hallucination and is essential for enterprise use cases where accuracy is non-negotiable.
RAG (Retrieval-Augmented Generation)
A technique that retrieves relevant documents from an external knowledge base and feeds them to a model alongside the user's query. RAG reduces hallucination and keeps responses grounded in current, factual data.
Guardrails
Programmatic rules and safety layers that constrain AI model behavior in production. Guardrails can block prompt injection, enforce output formats, prevent policy violations, and ensure brand-safe responses.
Evals
Systematic evaluation frameworks that measure AI model performance on specific tasks relevant to your use case, going beyond generic benchmarks to test the behaviors that actually matter for your application.
Know the terms. Know the moves.
Get the 5 AI stories that matter every Friday — free.
Free forever. No spam.