Hallucination

Notes on AI

Is AI a Student or an Actor

AI outputs are completions — continuations of patterns, not fact-checked answers retrieved from a database. Like an improv actor following the "yes, and" rule, the model streams tokens to maintain the flow and never stops the scene. Modern models are trained to play the role of a truthful, helpful assistant, but the underlying mechanism is the same: predicting what comes next — and the model is performing truthfulness, not accessing it.

Notes on AI

What AI Is Good At vs Bad At

This episode introduces a practical framework for understanding when to use AI by framing it as a "special employee" with three distinct characteristics.

Notes on AI

Why AI Sounds Confident

LLMs are optimised to produce fluent, grammatically correct language — not to verify whether what they say is true. The model generates the most likely continuation token by token, and when those tokens form smooth professional prose, the result sounds confident even if the underlying content is wrong. This is fluency bias: our brains mistake the quality of language for the quality of information. When evaluating an AI answer, ignore tone — look at evidence, sources, and consistency.

Notes on AI

Grounding

Pasting a document into AI gives the model access to it — but the model still draws on everything it was trained on and fills gaps silently. Grounding adds a boundary with two instructions: tell the model to answer only from the document, and tell it to say "I don't know" when something is missing. Without the second sentence, the model transitions from your source to its training without any signal. Two practical limits: documents larger than the context window can't be fully read, and scanned PDFs are images — the model can't read the words inside.

Notes on AI

What Hallucinations Are

A hallucination is what happens when an AI model produces plausible-sounding text that isn't actually true. The word suggests something unusual — a drift, a malfunction. But the model isn't doing a...

Notes on AI

When You Should Trust AI

Whether to trust AI output is not about the model — it's about the task. Some tasks have a natural safety net where errors surface before they cause harm; others don't, and a wrong answer looks exactly like a right one. The real skill is judging the risk of the mistake, not the quality of the model.

Notes on AI

Red Flags

AI errors don't look wrong — they look precise. Specific numbers, named reports, and exact citations are the parts of an AI answer most likely to be fabricated and hardest to question, because they look exactly like the signals we associate with credibility. The actual red flag is not vagueness. It is unverifiable specificity.