👻 Hallucinations

The Confident Storyteller Analogy

You ask your friend about a movie they haven't seen. Instead of saying "I don't know," they confidently make up a plot summary that sounds completely plausible.

AI hallucinations are a lot like this.

When LLMs don't know something, they may not say "I don't know." Instead, they can generate a confident, fluent, plausible-sounding response that turns out to be false.

The tricky part: hallucinations can sound just as confident as true statements. You often can't tell them apart by tone alone.

Why This Is a Big Problem

Hallucinations Are Invisible

Real fact: "The Eiffel Tower is a well-known landmark in Paris."
Hallucination: "The Eiffel Tower was built in <made-up-year>."

Both sound equally confident. There's no "I'm making this up" signal.

Real Consequences

Examples of Hallucinations

Fake Citations

You: "Find me research papers on climate change"
AI: "According to Smith et al. (YEAR) published in a top journal..."

Reality: This paper doesn't exist. The author, journal, and year were invented.

Made-Up Facts

You: "Tell me about the founder of Google"
AI: "Larry Page founded Google in <made-up-year> in his Stanford dorm room..."

Reality: The details are wrong (and the year is made up).

Imaginary Code Libraries

You: "How do I use the python-magic-solver package?"
AI: [Provides detailed installation and usage instructions]

Reality: This package doesn't exist. The AI invented it and documentation for it.

False Historical Events

You: "What happened in the Battle of Springfield?"
AI: [Provides detailed account of battle with dates, commanders, casualties]

Reality: No such battle occurred. Every detail was fabricated.

Why LLMs Hallucinate

Fundamental Design

LLMs are trained to predict the next word that sounds most plausible:

Training objective: "What word would typically follow?"
NOT: "What's actually true?"

They're pattern matchers, not fact databases.

No Knowledge of Truth

LLM doesn't have:
- Access to verify facts
- A database to look things up
- Ability to say "I have no information on this"

LLM has:
- Patterns learned from training text
- Strong drive to produce fluent responses
- No mechanism to distinguish known from unknown

Training to Avoid Blank Outputs

LLMs are often trained to provide a response. Being unhelpful can be punished, so they may produce something plausible-sounding instead of saying "I don't know."

Knowledge Cutoff

Information after the training date may be missing:

You: "Who won a very recent championship?"
AI (trained on older data): [Makes educated guess that may be wrong]

The Spectrum of Hallucination

Not all hallucinations are equal:

How to Reduce Hallucinations

1. RAG (Retrieval-Augmented Generation)

Give the model actual documents to reference:

Without RAG:
"Tell me about Policy X" → [May hallucinate details]

With RAG:
"Here's the actual Policy X document. Summarize it."
→ [Answers based on real content]

2. Lower Temperature

Higher temperature can be more creative and may increase hallucination risk

Lower temperature: more consistent, often lower risk
Higher temperature: more varied, sometimes higher risk

For factual tasks, use low temperature.

3. Ask for Sources

Prompt: "Answer based on verifiable facts and cite your sources."

Then: Actually check those sources! AI can cite fake sources too.

4. Grounding in Context

Provide facts in your prompt:

"Given that Google was founded by Larry Page and Sergey Brin,
summarize the early history of the company."

The AI now has correct facts to work with.

5. Structured Outputs

"If you don't know, explicitly say 'I don't have information about this.'"

Forces the model to acknowledge uncertainty.

6. Use Newer Models

Newer model generations often hallucinate less than older ones, but none eliminate the issue.

How to Detect Hallucinations

Red Flags

• Very specific numbers or dates when vague would make sense
• Detailed information about obscure topics
• Very polished-sounding citations
• Information that seems "too good"

Verification Steps

1. Search for cited sources - Do they exist?
2. Cross-reference facts - Can you find them elsewhere?
3. Check dates and numbers - Are they plausible?
4. Ask follow-up questions - Hallucinations often fall apart under probing

FAQ

Q: Will hallucinations ever be completely fixed?

Unlikely. LLMs generate based on patterns, not verified knowledge. We can reduce hallucinations but probably not eliminate them.

Q: Do all LLMs hallucinate?

Most do. Some more than others. Larger, newer models may hallucinate less. RAG-enabled systems often hallucinate less on grounded topics.

Q: Why doesn't the AI just say "I don't know"?

Training incentivizes helpfulness. Saying "I don't know" was historically punished. Newer models are better trained to express uncertainty.

Q: How do I know if I can trust AI output?

For important decisions: generally verify independently. Treat AI as a starting point, not a source of truth.

Q: Is hallucination the same as bias?

No. Bias = systematically wrong in one direction. Hallucination = confidently wrong about random facts.

Q: Can I sue if AI gives me wrong information?

Developing area of law. Generally, you're responsible for verifying important information. But this is evolving.

Summary

AI Hallucinations occur when LLMs confidently generate false information. They happen because models generate plausible text, not verified facts. It's a good idea to verify important AI outputs.

Key Takeaways:

• LLMs make up facts confidently and fluently
• Common types: fake citations, wrong facts, imaginary code
• Root cause: predicting plausible words, not truth
• Reduce with: RAG, low temperature, verification, grounding
• Newer/larger models may hallucinate less
• Fact-check important AI outputs

Rule of thumb: don't treat AI as a fact source without verification - especially for anything important.