Where it slips up

AI's big catch: it can be confidently wrong. It states made-up things in the exact same calm, certain voice it uses for true ones — so how do you tell which is which?

Beginner · 3 min read

A question worth testing: AI predicts plausible words, not verified facts. It sounds sure either way — so can its confidence tell you whether it's right?

See for yourself. You asked AI for three quick facts — pick the one you think it made up:

Spot the made-up one

Several rounds of "quick facts." Each one the AI states with the same calm certainty — but one per round is invented. Catch it, see why it was made up, and learn the one-line way you'd check.

Honey never really spoils — sealed pots thousands of years old have been found still edible.

Bees make honey from the nectar they gather from flowers.

A single worker bee makes about a full jar of honey in its lifetime. ✗ this is the invented one

The third is invented — a single bee makes only about a twelfth of a teaspoon of honey in its whole life. Notice the AI sounded just as sure about the false one as the true ones — so its confidence tracked nothing here. What would that mean for the next answer you trust? (Turn on JavaScript for the full game — several rounds, each trickier.)

Built so you get it by trying it — Bret Victor's idea of an explorable explanation.

So where does that leave you? AI is fluent, not accountable — it has no internal sense of true versus false, only of what words usually come next. One answer that holds up: keep a human check on anything that matters. That's the bet behind augmented intelligence — AI does the work, a person verifies and decides — and you can weigh whether it's the right one.

Why can't you trust an AI answer just because it sounds confident?

It predicts plausible words, not verified facts — so it states made-up things in the same confident voice as true ones. Confidence tells you nothing about truth; verify anything that matters.

Try it on your own work

Next time you lean on an AI answer for something that matters — a fact, a number, a recommendation:

Take its most confident-sounding claim.
Check it against a real source — and notice how convincing the wrong version can look.

If you can't think of a way to check a task, that's the task to keep a closer eye on.