When a Measure Becomes a Target
Any number used to judge and reward people starts to change their behaviour in ways that make it a worse measure of what it was meant to track.
The idea in plain words
A team is told to increase daily active users. Notifications multiply, emails become more urgent, features are designed to bring people back rather than to help them. The number rises. Whether the product has become better for its users is another matter, and the number can no longer tell you, because the number is what everyone is now working on.
The general principle is usually given in the anthropologist Marilyn Strathern’s wording: when a measure becomes a target, it ceases to be a good measure. It is known as Goodhart’s law, after the economist who first stated a version of it.
Where it comes from
Charles Goodhart was an adviser at the Bank of England in the 1970s, when governments tried to control inflation by targeting measures of the money supply. He observed that whenever a particular measure was chosen as a policy target, its relationship with the rest of the economy changed: banks and borrowers adapted, and the measure stopped tracking what it had tracked before. In a paper of 1975 he wrote, in effect, that any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.
The social psychologist Donald Campbell had reached a similar conclusion about public policy. His version, published in 1979, held that the more any quantitative social indicator is used for social decision-making, the more subject it becomes to corruption pressures, and the more apt it is to distort the processes it is intended to monitor. Police clearance rates, school test scores and hospital waiting times all show the pattern.
In 1976 the economist Robert Lucas made a related point about economic models: relationships estimated from past data break down when policy changes, because people change their behaviour in response to policy. Strathern’s 1997 phrasing, written about audits in British universities, generalised it into the form now widely quoted.
The deepest philosophical background is in Alasdair MacIntyre’s distinction, from After Virtue (1981), between internal and external goods. Internal goods are the excellences of a practice itself, such as good medicine, good teaching or good software. External goods are things like money, status and scores, which can be obtained in many ways. When external rewards are tied to a measure, people pursue the measure, and the internal goods of the practice can be crowded out.
What it does to the mind
Metrics change what people pay attention to. Whatever is measured becomes vivid; whatever is not becomes invisible. People are rarely cynical about this. They come to believe that the metric is the goal, because the metric is what is discussed in every review and rewarded in every bonus. The substitution of measure for purpose happens quietly, through attention, and is felt as diligence.
There is also a moral dimension. Gaming a metric can feel like cheating, and good people resist it. But when the system rewards gaming and penalises restraint, the people who stay are those most willing to optimise the number. Over time the culture shifts.
Where it shows up
The social media essay traces how engagement metrics shaped feeds, and what that did to the people using them. The email essay shows senders optimising open rates and receivers building defences. Amazon’s 1997 letter is an example of deliberately choosing a long-term measure, free cash flow and market leadership, to avoid the pressures of short-term ones. The metric becomes target pattern collects the cases across essays.
What it asks of someone deciding
Use several measures that pull in different directions, so that gaming one shows up in another. Engagement paired with satisfaction, speed paired with quality.
Rotate and review metrics. A measure that worked for a year may have been absorbed by the system; check whether it still tracks the outcome you care about.
Keep judgement in the loop. Metrics should inform decisions, not make them. When a number and the considered view of people close to the work disagree, investigate before trusting the number.
Argue with it
The strongest objection
Without measures, organisations fall back on opinion, politics and favouritism. The answer to bad metrics is usually better metrics, several of them, revised often, not the abandonment of measurement.
Take it to your team
Questions to ask
- What is the easiest way to improve this metric without improving the product?
- Which important outcomes have no metric at all, and are they being neglected as a result?
- When did we last check that this number still means what it meant when we chose it?
Follow the thread
Connections
- Everyone You Know, and Everyone You Don’tSocial Media
- An Address Anyone Can Write ToEmail
- The Machine That AnswersLarge Language Models
- The Machine for Asking What IfSpreadsheets
- The Auction Behind the AnswerWeb Search
- Software You Never OwnSubscription Software (SaaS)
- What You Might Also LikeRecommendation Feeds
- The Company Inside the MachineEnterprise Software (ERP)
Read further
Bibliography
- Charles A. E. Goodhart, “Problems of Monetary Management: The U.K. Experience,” Papers in Monetary Economics (Reserve Bank of Australia, 1975); reprinted in Monetary Theory and Practice (Macmillan, 1984).
- Donald T. Campbell, “Assessing the Impact of Planned Social Change,” Evaluation and Program Planning 2, no. 1 (1979), 67–90.
- Marilyn Strathern, “‘Improving Ratings’: Audit in the British University System,” European Review 5, no. 3 (1997), 305–321.
- Robert E. Lucas Jr., “Econometric Policy Evaluation: A Critique,” Carnegie-Rochester Conference Series on Public Policy 1 (1976), 19–46.
- Jerry Z. Muller, The Tyranny of Metrics (Princeton University Press, 2018).
- Alasdair MacIntyre, After Virtue (University of Notre Dame Press, 1981), chapter 14, on internal and external goods.
Shrikant Kumar. “When a Measure Becomes a Target.” Idea, Telos. https://shrikantk.com/ideas/goodhart/