The answer to check
“The shop opens on Monday at 8 am. Its owner is Neri.”
Authored example, not a model transcript.
Machine learning · extension
Hallucination and uncertainty
An answer says exactly when a shop opens and names its owner. It sounds convincing. But where did those details come from? Compare the answer with a short notice before trusting it.
Standalone lesson. No coding required. Results are illustrative and stay on this page; checks do not save to the core course notebook.
01 · check before trusting
Which details does the notice actually support?
Read the answer and the notice side by side. Predict which detail is supported, which conflicts with the notice, and which the notice never mentions. Then check each claim.
Illustrative · fictional source exercise
The shop notice
- S1. We open on Monday.
- S2. Doors open at 9 am.
- S3. Find us beside the library.
Classify each claim using this fictional notice only. “Supported” means the notice backs it up. “Contradicted” means the notice says something incompatible. “Not enough evidence” means the notice does not settle it.
A citation helps only when its source supports the claim. This exercise tests support within a fictional notice; real sources also need checks for reliability and recency.
02 · explain what happened
Sounding certain does not make a claim true
The answer joined three details into one smooth sentence. Only one was supported by the notice. Clear wording, a precise time and a person's name can make text feel trustworthy without adding evidence.
A language model learns patterns that help it continue text. Producing a plausible continuation and checking a claim against a reliable source are different jobs. Here, hallucination means generated factual content that is false or unsupported by the relevant evidence.
A false statement alone does not show that a model deliberately tried to mislead you. Calling every error a “lie” assumes an intention that the answer itself does not establish. Agreeing with a user's mistaken belief is another behaviour to examine: sycophancy.
These examples are written for the lesson. They are not outputs from a live model or measurements of its confidence.
Try a different example
A library notice says: “Members may borrow six books.” An answer says: “Members may borrow six books for three weeks.”
Explain it back: what would you tell a friend who says, “It gave a precise answer, so it must know”?
Compare with one explanation
“Precise wording can still contain a mistake. I would check each important detail against a reliable, current source. If the source does not mention a detail, I need more evidence or a narrower answer.”
You can stop here: match each claim to evidence, separate a contradiction from a missing detail, and avoid treating confident wording as proof. A citation is useful only if its source supports the actual claim.
Optional: why might a scoring rule encourage guessing?
Imagine a quiz gives one point for a correct answer, and zero for both a wrong answer and “I don't know”. Guessing has a chance to earn a point; staying silent cannot. That scoring rule can favour guessing. It is one contributing mechanism, not an explanation for every hallucination.
In this toy exercise, “abstain” means decline to answer. Start at zero penalties, then raise the wrong-answer penalty to 2. Finally raise abstention credit to 1. Watch how many questions get answered as well as how many answers are correct.
Illustrative · computed scoring experiment
A correct answer earns 1 point. These six fictional questions have stipulated probabilities of a correct answer, not token probabilities or measured model confidence. Sample outcomes are fixed; moving a slider never reruns them.
Answer expected score = p × 1 − (1 − p) × penalty. Abstention score = credit. The rule answers only when its expected score is higher; ties abstain.
Which bay holds capsule A?
Stipulated p: 95%
Answer score 0.95 · abstain 0.00
Rule: answer. Fixed answer outcome: correct.
How many samples arrived?
Stipulated p: 80%
Answer score 0.80 · abstain 0.00
Rule: answer. Fixed answer outcome: correct.
Which dock received capsule B?
Stipulated p: 65%
Answer score 0.65 · abstain 0.00
Rule: answer. Fixed answer outcome: incorrect.
Which courier carried it?
Stipulated p: 50%
Answer score 0.50 · abstain 0.00
Rule: answer. Fixed answer outcome: correct.
At what minute did it arrive?
Stipulated p: 30%
Answer score 0.30 · abstain 0.00
Rule: answer. Fixed answer outcome: incorrect.
What was its launch temperature?
Stipulated p: 10%
Answer score 0.10 · abstain 0.00
Rule: answer. Fixed answer outcome: incorrect.
Expected total 3.30
Realised total, fixed sample 3.00
Coverage: answered / all 6/6 · 100%
Accuracy among answers 3/6 · 50%
Expected score averages hypothetical repeated trials under the stipulated probabilities. Realised score uses this one fixed sample. High conditional accuracy with low coverage does not mean every question was answered correctly.
Research connection: Kalai et al., Why Language Models Hallucinate (2025) analyse statistical errors and scoring incentives that favour guessing over abstention. This exercise isolates one incentive; it does not simulate real training.
Language models assign probabilities to possible next pieces of text, called tokens. A high token probability is not a measured probability that a factual claim is true. Measuring confidence needs checks against correct outcomes. Retrieval can provide sources, but irrelevant or misread sources can still lead to unsupported claims.