Trustworthy Computing

The situation

At the turn of the millennium Microsoft’s software was everywhere, and so were attacks on it. In 2001 the Code Red and Nimda worms spread across hundreds of thousands of computers, exploiting flaws in Microsoft’s products. Corporate customers were losing patience, and governments were beginning to ask whether software this important could be trusted. Microsoft was also asking customers to rely on it for a new generation of internet services, under the name .NET, which made trust more important than ever.

Gates had a habit of using memos to reset the company’s priorities; his 1995 memo on the internet is the most famous example. In January 2002 he wrote another.

What the memo argues

It opens by placing itself in that tradition: “Every few years I have sent out a memo talking about the highest priority for Microsoft.” The new priority was defined by analogy: “Trustworthy Computing is computing that is as available, reliable and secure as electricity, water services and telephony.” People do not think about whether the lights will come on; Gates wanted software to earn the same unthinking trust.

He stated the rank without qualification: “Trustworthy Computing is the highest priority for all the work we are doing.” And then came the sentence that gave the memo its force: “So now, when we face a choice between adding features and resolving security issues, we need to choose security.”

The memo divides trustworthiness into parts: availability, meaning products that work whenever customers need them; security, meaning customers’ data protected from harm; and privacy, meaning that users, in Gates’s words, “should be in control of how their data is used.” It also calls for secure defaults: “Our products should emphasize security right out of the box.” Gates connects the push to .NET, Microsoft’s strategy for internet services, and to the events of the previous autumn, referring to “September’s terrorist attacks” and to “a number of malicious and highly publicized computer viruses.”

He ends with confidence rather than contrition: “The challenge here is one that Microsoft is uniquely suited to solve.”

The decision it carried

The memo’s decision was a change in the order of priorities across the company: when security and features conflicted, security would win. It was followed immediately by a practical one. In February 2002 Microsoft paused new development of Windows so that its engineers could be trained in secure programming and could review code for flaws; contemporary reports put the number of engineers at about 7,000 to more than 8,000, and Microsoft later described the effort as lasting more than two months across several products.

What happened

Results were not immediate. Some of the worst attacks in Windows history came after the memo, including SQL Slammer in January 2003 and Blaster in August 2003, because they exploited flaws in software already in customers’ hands. But the way Microsoft built software changed. In October 2003 it moved to a predictable monthly schedule for security updates, which made patching manageable for companies. In 2004 Windows XP Service Pack 2 turned on a firewall by default and tightened many settings, and Microsoft made its Security Development Lifecycle, a set of required security practices for every product, mandatory. Over the following decade the company’s software became significantly harder to attack, and its security practices were widely copied.

How to read it as a product manager

The memo is a model for changing a priority that conflicts with what the organisation is used to rewarding.

It defines the goal in terms anyone understands. Comparing software to electricity and the telephone gave engineers and customers alike a clear picture of the target.

It settles the trade-off in advance. Product teams constantly face choices between new capabilities and invisible qualities such as security or reliability. Most organisations say the invisible quality matters and then reward the visible features. A sentence like “choose security” changes the default answer in thousands of small decisions.

It insists on defaults. “Right out of the box” recognises that most customers never change settings. A product is as safe as its defaults, not as safe as its options.

It shows the cost of catching up. Fixing a quality problem that has accumulated for years requires stopping work, which is why it is cheaper to treat such qualities as priorities from the beginning.

The decision, examined closely

What triggered it

The trigger was a combination of public failures and a strategic need. The 2001 worms were front-page news and affected large customers directly. At the same time, Microsoft’s .NET strategy asked customers to trust it with services running over the internet. A company widely seen as insecure could not sell that future.

The constraints

Microsoft’s products contained millions of lines of code written over many years by engineers trained to optimise for features and compatibility. Customers expected new versions on schedule. And security improvements often broke compatibility with older software, which customers valued highly. Any serious security push would cost time, money and some compatibility.

What the author was optimising for

The memo optimises for long-term trust in Microsoft’s platform, especially among businesses and governments. That trust was the foundation of both the existing Windows business and the new internet services. Gates was willing to pay in short-term feature delivery to protect it.

Second-order effects

The security push professionalised software security across the industry. Practices that Microsoft developed, such as threat modelling and a mandated security lifecycle, were published and adopted elsewhere. The monthly update schedule became an industry norm. Inside Microsoft, the memo gave engineers permission to delay features for security reasons, which changed the culture of product teams.

The counterfactual

Without the memo, Microsoft might have continued patching vulnerabilities one by one while competitors and regulators used security as an argument against its products. Security would probably have improved eventually under external pressure, but more slowly and at greater cost to Microsoft’s reputation. The memo turned an external crisis into an internal priority.

What is contested

Security experts at the time were sceptical, noting that memos do not fix software and that progress would be hard to measure. Some later accounts point out that engineers inside Microsoft had been raising security concerns before the memo, so it formalised a shift already under way. And the 2003 worms showed that years of legacy code would take years to fix. The fair conclusion is that the memo started a long effort rather than solving the problem.

Sources and limitations

Quotations were verified against the full text of the memo as published by The Register shortly after it was sent. The size and duration of the development pause vary between sources and are given as ranges. Details of later security practices come from news reporting and widely documented industry history. The judgement of the memo’s long-term effect is the author’s interpretation.