Hallucination

The scene: wrong, and completely sure of itself. Picture a common scene. Ask it to list a product’s specifications: the answer comes back in a second — a dozen rows, figures with decimal points, everything lined up neatly. Then you check the list against the official page, and typically a few entries don’t match. Two of them don’t exist on that product at all. Ask it for book recommendations and you get titles, authors, the works — then you go looking, and find none of them. Through all of this it never stalls. Not once. Not for a second.

What’s really happening: it continues; it doesn’t check. The job it is doing is not “looking things up.” It is continuing. You give it the first half of a sentence; it places the word that sounds most natural next, then the next one — the same trick your phone’s keyboard uses when it guesses what you’ll type. The difference is how much it has read. Enough to sound convincing. But there is nothing at its side to check against, and no exit marked “I don’t know”: where the thread runs out, it invents a piece and sews it on. That’s the scene people call a hallucination. The words are smooth. The facts aren’t there.

Why: not malice — the scorecard never asked. Hold off on calling it a liar. Lying takes intent, and there is none here. While it was being trained, the points went to answers that sounded smooth and finished the thought; saying “I can’t find that” earned nothing. So, rather than stop halfway, it rounds the story off and keeps going. This isn’t a character flaw. It was simply never taught that stopping is allowed.

The fix: three steps are enough. This flaw can be fenced in, and it doesn’t take anything fancy — three steps: feed it first, check the numbers that matter yourself, and make “not found” an acceptable answer. The steps are written out below. Why feed it first? Because invention grows in the gaps. Hand it the manual, the file, the page before it answers, and the gaps are already occupied by something real; the room left for making things up shrinks on its own. Numbers deserve the most suspicion. A price, a date, a certificate number — the smoother they look, the more likely they were never checked. Anything you plan to use, look it up once at the original source. The last step costs one sentence. Say it up front when you ask: “If you can’t find it, write that you can’t find it.” It was never refusing to admit a gap. The option simply wasn’t on the table. Put it there, and it usually takes it.

The fix, in three steps

1. Feed it first, then ask. Show it the manual, the file, the web page before it answers. With real material underneath, the gaps it can fill with invention get much smaller.
2. Check the numbers that matter yourself. Prices, dates, specs, book titles, certificate numbers — anything you plan to use, go back to the original source and look once before you rely on it.
3. Make “not found” an acceptable answer. Add one line when you ask: “If you can’t find it, write that you can’t find it — no guessing.” Not knowing is fine. An invented answer is the real problem.

Sources for this episode

The topic comes from the AI-hallucination installment of Liu Yi’s “AI小知识” (AI Notes) column: the subject and its main points are the column’s; the lyrics and these notes were written fresh for this site. This episode cites no outside statistics. The three cases in the song — invented specifications, a certification that doesn’t exist, recommended books that can’t be found — are the three most common shapes this flaw takes. Whether the answer in front of you right now is real or invented, only the three steps above can tell you.

Process note

Once the words were settled, this song was recorded in two versions. We listened to all 32 lines one by one and checked each against the lyric sheet. In the first version, the chorus hook — “顺，不等于真” (“smooth isn’t true”) — came out blurred all three times, impossible to make out. That version had to go. In the second version there is one pass where those words stop on their own, each syllable landing clean; that’s the one you’re hearing. Two phrases fought us in both versions: “参数规格” (specs and parameters) and “认证头衔” (certification titles) — hard words packed together trip the mouth at speed. Noted for next time: keep dense, hard phrases out of the fast lines. The visuals had been built for an 80-second cut; when the song came in longer, all eight scenes were re-timed and stretched, one at a time, to fit.