How to Judge a Decision
How can anyone tell a good decision from a lucky one, and become better at making them?
Most organisations judge decisions by their results. The books on this path, read in order, make that habit hard to keep.
The path begins with the mind: the systematic errors of intuition that Daniel Kahneman and Amos Tversky documented. It then turns to luck, the large part of any outcome that nobody controls, and to the habit of mistaking survivors’ stories for skill. From there it becomes practical: how forecasters improve, how organisations can reduce inconsistency in judgement, and why the hardest part of judging well is wanting to.
The thread running through it is the distinction this site returns to often, between the quality of a decision and the quality of its outcome. The Ideas on moral luck, hindsight and calibration are companions to these books, and the Decide cases are a place to practise what they teach.
-
The foundation. Kahneman's account of two modes of thinking, fast and intuitive against slow and deliberate, gives names to most of the errors the rest of this path is about.
- Read it for
- The catalogue of biases, and especially the chapters on overconfidence, the planning fallacy and the inside and outside views.
- Carry forward
- Which of my confident judgements came from the fast system without my noticing?
-
The story of the friendship between Kahneman and Amos Tversky shows how these ideas were discovered, argued over and tested, which makes them feel less like rules and more like hard-won insights.
- Read it for
- How two very different minds worked together, and how ideas that now seem obvious were once controversial.
- Carry forward
- What kind of colleague makes my thinking sharper?
-
Moves from the psychology of error to the practice of deciding. Treating decisions as bets separates the quality of a choice from the luck of its outcome.
- Read it for
- The idea of "resulting", judging decisions by outcomes, and the habit of asking how sure you really are.
- Carry forward
- If I had to bet on this belief, how much would I stake?
-
Deepens the role of luck. Taleb shows how often success is mistaken for skill, especially in fields where outcomes are noisy and survivors write the history.
- Read it for
- Survivorship, the asymmetry between rare events and our intuitions, and alternative histories, the paths that did not happen.
- Carry forward
- How much of this success would survive if I replayed history a hundred times?
-
After the warnings, a constructive answer. Tetlock and Gardner show that some people forecast far better than others, and that the habits behind it can be learned.
- Read it for
- Breaking questions into parts, starting from base rates, updating in small steps and keeping score.
- Carry forward
- What is my base rate, and how far should this case move me from it?
-
Noise: A Flaw in Human Judgment
Adds the second kind of error. Bias pushes judgements in one direction; noise scatters them. Organisations rarely measure it.
- Read it for
- Noise audits, decision hygiene and the case for structured judgement.
- Carry forward
- If two of our best people judged this independently, how far apart would they be?
-
Turns from technique to motivation. Good judgement requires wanting to see clearly, even when the truth is unwelcome.
- Read it for
- The contrast between defending beliefs and testing them, and ways to notice motivated reasoning in oneself.
- Carry forward
- What would I prefer to be true here, and is that shaping what I see?
-
Ends with the social side of changing minds: how to rethink one's own views and help others rethink theirs.
- Read it for
- Confident humility, and the difference between preaching, prosecuting and thinking like a scientist.
- Carry forward
- Which of my strongly held views have I not reconsidered in years?
Follow the thread