Exploring and Exploiting

The idea in plain words

Do you go back to your favourite restaurant or try a new one? The favourite is a known pleasure; the new place might be better or worse. If you always return to the favourite, you never discover anything better. If you always try new places, you rarely enjoy the best one you have found.

That is the trade-off between exploitation, using what you know, and exploration, gathering information that might pay off later. It appears in every domain where choices repeat: product portfolios, hiring, research, careers and the design of recommendation systems.

Where it comes from

Mathematicians formalised the problem as the multi-armed bandit, after a gambler facing a row of slot machines with unknown payouts. Herbert Robbins framed it in 1952, and in 1979 John Gittins found an optimal strategy for an important class of cases: give each option an index that reflects both its known value and the value of learning more about it. The general insight is that the right amount of exploration depends on how much future remains in which to use what you learn.

In 1991 the organisational theorist James March argued that organisations face the same trade-off and are systematically biased towards exploitation, because its returns are more certain, faster and closer. Exploitation drives out exploration, he warned, leaving organisations efficient at what they do and unable to adapt when the world changes.

What it does to the mind

People prefer the known. Exploring involves uncertainty, possible failure and short-term loss, and loss aversion makes each feel worse than the equivalent gain. Successful people and organisations are especially prone to over-exploitation, because their success confirms that their current approach works. Exploration also tends to decline with age, partly rationally, since there is less time left to use what is learned.

Where it shows up

The essay on recommendation feeds describes systems that must balance showing people what they already like with learning what else they might, and the pressure to favour immediate engagement. The essay on web search describes a leader whose core business left little reason to explore the answer engines that later threatened it. The idea of the innovator’s dilemma describes how exploiting existing customers blinds firms to new markets.

What it asks of someone deciding

Set aside deliberate capacity for exploration, protected from the metrics that judge exploitation, because otherwise it will be crowded out.

Explore more when the horizon is long and the environment is changing; exploit more when time is short or the world is stable.

Count what experiments teach, not only whether they succeed. An exploration that rules out an option has done its job.