Roads not taken 1974 → 1993 Beaten by good enough

Lisp Machines

Computers Built for Thinking

MIT AI Lab, Symbolics, Lisp Machines Inc., Xerox and Texas Instruments

Why did the most advanced programming environments of the 1980s lose to cheaper, general-purpose machines that did less?

The world it was born into

Lisp, designed by John McCarthy at MIT in 1958, was the language of artificial intelligence research for decades. It treated programs as data, made it easy to manipulate symbols and lists, managed memory automatically and encouraged interactive, exploratory programming. These qualities suited AI research, where programmers often did not know in advance what their programs should do.

They also made Lisp slow on ordinary computers. Automatic memory management, checking the types of values at run time and the heavy use of lists all cost time and memory on hardware designed for numerical calculation. By the mid-1970s AI researchers at MIT were running large Lisp programs on shared time-sharing machines and finding them painfully slow.

The bet

Richard Greenblatt and Tom Knight at the MIT AI Lab proposed a radical solution: build a personal computer designed specifically to run Lisp. Work on the first machine, CONS, began around 1974; a second generation, the CADR, was built in the late 1970s and used in the lab. The hardware supported Lisp’s needs directly, with tags identifying the type of each value and support for memory management. Each researcher would have a whole machine, an unusual luxury at the time.

Around 1979–80, two companies were founded to sell commercial versions: Lisp Machines Inc. (LMI) and Symbolics. Xerox, whose research centre had developed its own Lisp environment, and later Texas Instruments joined the market. The bet was that AI and advanced software development would grow into a large market that needed the best possible Lisp environment, and that custom hardware would provide it.

Why it made sense

Lisp machines were extraordinary. Their environments offered high-resolution graphical displays, windows, a mouse, networking and powerful development tools: programmers could inspect any object, change code while it ran and debug in ways most developers would not see for another decade or more. The whole operating system was written in Lisp and could be examined and modified by the user.

The market looked promising. The early 1980s brought an AI boom. Expert systems, programs encoding the rules of human specialists, attracted investment. Japan’s Fifth Generation Computer project, launched in 1982, alarmed American and European governments, which responded with large programmes of their own, including the American Strategic Computing Initiative of 1983. Lisp machine makers sold to universities, defence contractors, oil companies and AI start-ups. Symbolics, the largest, became a significant company. In March 1985 it registered symbolics.com, the first .com domain name.

How it lost

Two things happened at once. First, general-purpose workstations from Sun, Apollo and others, built on fast commodity microprocessors, improved rapidly, riding the industry-wide gains described in the compute essay. Companies such as Lucid and Franz built high-quality Lisp implementations for these machines. By the mid-1980s a standard workstation could run Lisp acceptably for many purposes at a fraction of the cost of a Lisp machine, and it could also run everything else a company needed.

Second, the AI boom ended. Expert systems proved more limited and harder to maintain than promised. Funding and corporate enthusiasm cooled in what became known as the second AI winter. Around 1987 the market for specialised AI hardware collapsed, by many accounts almost overnight. Symbolics struggled on and filed for bankruptcy protection in 1993.

Explaining both the same way

In 1990–91 Richard Gabriel, a Lisp pioneer, circulated an essay containing a section called “The Rise of Worse Is Better”. He contrasted two design philosophies. The “MIT approach”, which he associated with Lisp, held that a design must be correct, consistent and complete, even if that made it complex to implement. The “New Jersey approach”, associated with Unix and C at Bell Labs, favoured simplicity of implementation, even at some cost to correctness and completeness. Gabriel argued, with evident ambivalence, that the second approach had better survival characteristics: simpler systems spread faster, run on more machines and improve over time.

That is a symmetrical explanation, and its strength is that it calls neither side foolish. Lisp machines were better at what they did. Workstations running Unix were good enough at it and better at fitting into the rest of computing. Where general-purpose hardware improved relentlessly and cheaply, the general system’s advantages compounded and the specialised system’s advantages eroded. Christensen’s innovator’s dilemma describes the same shape from the other side: the cheaper, simpler product, at first inadequate, improves until it is enough.

What the loss teaches

The central lesson is about where to put value when the platform beneath you is being commoditised. The Lisp machine companies’ deepest asset was their software environment, not their hardware. Their fate might have been different had they moved that environment onto general-purpose machines quickly, though with lower margins and fierce competition. Holding on to the hardware meant holding on to the part of the business the market was making obsolete.

The second lesson concerns booms. Lisp machines rose and fell with expectations for AI. A company tied to one market’s mood is exposed to its swings, a warning with an obvious contemporary resonance.

Finally, little of value was lost. Integrated development environments, interactive debugging, garbage collection and dynamic languages, all developed or refined in the Lisp machine world, became standard. The machines died. Their way of programming won.

Sources and limitations

The market collapse of 1987 is widely described but poorly quantified in public sources; revenue figures for Symbolics and others are not given here because reliable figures were not found. Technical descriptions draw on the 1980 paper by Greenblatt and colleagues and on Steele and Gabriel’s history of Lisp. Gabriel’s essay is available on his own website. The interpretation that a faster move to software might have helped is the author’s, and it is uncertain: software companies selling Lisp on workstations also struggled once the AI winter set in.

The minds involved

The forces at work

  1. Specialised excellence versus general improvement

    Lisp machines were superb at one thing. General-purpose processors improved for every customer at once, funded by a far larger market. Each year the gap in Lisp performance narrowed while the gap in price stayed wide, until good enough at a fraction of the price won.

  2. Riding a boom

    The early 1980s brought great optimism about expert systems and AI, with large government programmes in Japan, the United States and Britain. Lisp machine demand rose with that optimism. When expectations proved too high, demand fell with them, and the machines were tied to a single market's mood.

  3. The good-enough rule

    Most buyers did not need the best Lisp environment. They needed one that was adequate and ran on the same computers as their other software. Buyers who satisfice, choosing the first acceptable option, favoured general-purpose workstations.

  4. Explaining "worse is better"

    Richard Gabriel's famous essay contrasted the MIT approach, which favoured correctness and completeness, with the approach of Unix and C, which favoured simplicity of implementation. The symmetrical point is that both approaches were coherent; the environment of cheap, spreading hardware rewarded the second.

For people who build products

Decisions worth studying

3 decisions from this history, each with the situation, the alternatives, what happened and a verdict. Open one for the full case. The verdict says how a decision turned out, which is not the same as whether it was wise; see moral luck.

  1. 1979–1980 · MIT AI Lab researchers who founded LMI and Symbolics Sound process Mixed Commercialise the MIT Lisp machine as dedicated hardware rather than as software for general-purpose computers. A hardware advantage built on today's gap is a bet that the gap will last.
    Situation
    General-purpose computers of the late 1970s ran Lisp slowly and had small memories. Custom hardware gave dramatic gains in speed and capability.
    Alternatives
    Port the Lisp environment to general-purpose minicomputers and workstations; sell software only.
    What happened
    The companies grew quickly in the early 1980s, selling to research labs, defence contractors and AI start-ups.
    Verdict
    Correct for the hardware of 1979. It tied the companies' fortunes to the assumption that specialised hardware would stay ahead of general-purpose processors.

    Pattern Hiding the machine wins once the machine is cheapMaking something cheap moves the cost to someone else

  2. Mid-1980s · Symbolics and other Lisp machine makers Weak process Backfired Continue investing primarily in proprietary Lisp hardware as general-purpose workstations improved rapidly. When a cheaper general platform is catching up, move your value into what runs on it before it arrives.
    Situation
    Sun, Apollo and others were selling workstations based on fast commodity microprocessors. Commercial Lisp implementations for these machines were improving.
    Alternatives
    Shift to software, porting the environment to standard workstations; partner with workstation makers.
    What happened
    Around 1987 demand for Lisp machines collapsed as customers moved to cheaper general-purpose machines. Symbolics filed for bankruptcy protection in 1993.
    Verdict
    The trend in general-purpose processor performance was visible and widely discussed. A faster move to software might have preserved the companies' most valuable asset, their environments.

    Pattern A defining strength becomes the constraintIncumbents absorb the challenger’s best idea

  3. 1984–1994 · The Lisp community and standards bodies Sound process Mixed Standardise Common Lisp so that Lisp programs could run on many machines, including general-purpose ones. Standards that help your users can also dissolve your hardware advantage; plan for that.
    Situation
    Many incompatible Lisp dialects existed. Users wanted portability.
    Alternatives
    Remain with machine-specific dialects.
    What happened
    Common Lisp became an ANSI standard in 1994. Portability helped Lisp survive the hardware collapse, though in a much smaller market.
    Verdict
    Portability was the right move for users. It also removed one of the main reasons to buy dedicated hardware.

    Pattern Compatibility becomes gravity

Nothing quite dies

What survived

  • Integrated development environments with interactive debugging, inspection and live code changes, now standard in modern tools.
  • Automatic memory management (garbage collection), which Lisp machines supported in hardware and which is now standard in most popular languages.
  • Dynamic languages such as Python, JavaScript and Ruby, which carry many ideas first developed in the Lisp tradition.
  • Common Lisp, standardised in 1994, and later Lisp dialects such as Clojure.
  • Symbolics registered symbolics.com in March 1985, the first domain name registered under .com.

Rise and fall

Timeline

  1. 1958 John McCarthy designs Lisp at MIT.
  2. 1974 Work begins at the MIT AI Lab on the CONS Lisp machine.
  3. Late 1970s The CADR, a second-generation MIT Lisp machine, is built and used within the lab.
  4. 1979 Lisp Machines Inc. (LMI) is founded.
  5. 1980 Symbolics is founded.
  6. 1982 Japan launches its Fifth Generation Computer project, intensifying interest in AI.
  7. 1985 Symbolics registers symbolics.com, the first .com domain.
  8. 1987 The market for specialised AI hardware collapses.
  9. 1990–1991 Richard Gabriel's "worse is better" argument circulates.
  10. 1993 Symbolics files for bankruptcy protection.
  11. 1994 Common Lisp becomes an ANSI standard.

Take it to your team

Questions to ask

  • Is our advantage built on a gap that a cheaper, general technology is closing?
  • If our best feature ran on everyone's standard platform, would customers still need our product?
  • Are we tied to a single market's mood, and what happens to us if that market cools?

Follow the thread

Connections

Where this comes from

Sources

  1. Richard P. Gabriel, “Lisp. Good News, Bad News, How to Win Big” (1990–1991), including the section “The Rise of Worse Is Better.”
  2. Richard D. Greenblatt, Thomas F. Knight Jr., John Holloway and David A. Moon, “A LISP Machine,” Proceedings of the Fifth Workshop on Computer Architecture for Non-Numeric Processing (1980).
  3. Guy L. Steele Jr. and Richard P. Gabriel, “The Evolution of Lisp,” ACM SIGPLAN Notices 28, no. 3 (1993), 231–270.
  4. Steven Levy, Hackers. Heroes of the Computer Revolution (Doubleday, 1984).
  5. Daniel Crevier, AI. The Tumultuous History of the Search for Artificial Intelligence (Basic Books, 1993).

How to cite

Shrikant Kumar. “Lisp Machines: Computers Built for Thinking.” Road not taken, Telos. https://shrikantk.com/roads-not-taken/lisp-machines/

↑ ↓ to move↵ to open⌘K to toggle