Did you know? · Science & Medicine

Daniel Kahneman named the illusion of validity

Watching army officer candidates, Daniel Kahneman noticed that assessors stayed confident despite proof their predictions failed. He called it the illusion of validity, his first cognitive illusion.

Daniel Kahneman
Daniel Kahneman · 1934–2024
Person
Daniel Kahneman
Lived
1934–2024
Field
Psychology, behavioural economics
From
Israeli-American

The story

Daniel Kahneman graduated from the Hebrew University of Jerusalem in 1954 with a major in psychology and a minor in mathematics. Drafted that year as a second lieutenant, he spent a year as an infantry platoon leader and was then transferred to the psychology branch of the Israel Defense Forces.

One of his duties there was to help assess candidates for officer training, using methods developed by the British Army in the Second World War. In one exercise, a leaderless group of eight soldiers, stripped of rank insignia and identified only by numbers, had to carry a telephone pole over a wall about 2.5 metres high without letting the pole or themselves touch it. Two assessors watched, often for half an hour or more, and felt certain they could see each candidate's true character: leaders, loyal followers, empty boasters.

Feedback from the officer-training school, delivered every month or so on what the unit called statistics day, showed that the assessors' ratings predicted almost nothing. Yet when the next group arrived, Kahneman and his colleagues felt just as confident as before. He was struck by the complete lack of connection between the statistical evidence and the compelling feeling of insight, and he coined a term for it: the illusion of validity.

Almost twenty years later the term entered the technical literature in On the Psychology of Prediction, a 1973 paper with Amos Tversky. Kahneman described it as the first cognitive illusion he discovered, and the theme of overconfident intuitive prediction ran through his whole career. Much later he explained that the assessors had answered an easier question than the one they were asked: they judged an hour of behaviour in an artificial setting and told themselves a story about the future, a pattern he summed up as what you see is all there is.

He found the same illusion elsewhere. Studying the results of 25 wealth advisers over eight years, he found that none stood out consistently, and told the firm's directors that they were rewarding luck as if it were skill. His advice was to trust confident judgments only when there is independent reason to believe the person knows what they are talking about.

Why it matters

The illusion of validity helps explain why experts in fields from finance to hiring overrate their own judgment, and why checking predictions against data matters.

About Daniel Kahneman

Child survivor of Nazi-occupied France who, with Amos Tversky, mapped the systematic errors of human judgment, founded behavioural economics and won the 2002 Nobel Prize in economics.

Read the full profile
Daniel Kahneman
1934–2024 · Psychology, behavioural economics
View profile →

More about Daniel Kahneman

Sources

Another act of tikkun olam. This is part of tikkun olam, the Jewish idea of repairing the world, and every one of us can add to it.

More in Science & Medicine

On kindness & the treatment of others
“A friend is devoted at all times; a brother is born to share adversity.”
Proverbs 17:17