Predictions That Make Themselves True

The idea in plain words

A rumour spreads that a sound bank is failing. Depositors rush to withdraw their money, and the bank, which could have met normal withdrawals, collapses. The prediction caused its own truth. A recommendation system predicts that a video will be popular, shows it to millions, and it becomes popular. A credit score labels someone risky, they are charged more, and they struggle to pay.

Predictions about people and markets differ from predictions about planets, because the people predicted can respond. When enough of them act on a forecast, the forecast can come true, or fail, because it was made.

Where it comes from

The sociologist Robert K. Merton named the self-fulfilling prophecy in 1948, defining it as a false definition of a situation that evokes behaviour which makes the originally false conception come true. His examples included bank runs and racial discrimination, in which prejudice created the conditions that seemed to confirm it.

The philosopher Ian Hacking described in the 1990s the looping effects of human kinds: when scientists classify people, as with medical or psychological categories, the people classified learn of the classification and change, so the category changes too. The sociologist Donald MacKenzie argued in An Engine, Not a Camera (2006) that financial models, such as the Black–Scholes formula for pricing options, did not merely describe markets but changed them, as traders adopted them, until the models’ crash in 1987 revealed the limits. The investor George Soros called the same phenomenon reflexivity.

What it does to the mind

People treat forecasts as information about the world, not as interventions in it, and so they do not see the loop. When a prediction comes true, it confirms belief in the predictor, even if the predictor caused the outcome. Labels also shape self-understanding: people told they are a certain kind of person often come to act like one, a finding studied in education as the Pygmalion effect, though its size is debated.

Where it shows up

The essay on recommendation feeds describes systems that show people what they predict they will like and then learn from what was shown, and creators who make what the system rewards. The essay on web search describes rankings that create the links and clicks they measure. The essay on subscription software describes valuations that rested on a convention about the future, and fell when the convention changed.

What it asks of someone deciding

When a forecast will be acted on widely, ask how the action will change the outcome. Treat influential forecasts as interventions.

Break feedback loops in data. If your system learns from behaviour it shaped, run experiments and deliberate exploration to see what people would do otherwise.

Be careful with labels for people. Classifications that determine treatment, such as high potential or low performer, can create what they claim to find.