Lisp Machines
Computers Built for Thinking
MIT AI Lab, Symbolics, Lisp Machines Inc., Xerox and Texas Instruments
Why did the most advanced programming environments of the 1980s lose to cheaper, general-purpose machines that did less?
The world it was born into
Lisp, designed by John McCarthy at MIT in 1958, was the language of artificial intelligence research for decades. It treated programs as data, made it easy to manipulate symbols and lists, managed memory automatically and encouraged interactive, exploratory programming. These qualities suited AI research, where programmers often did not know in advance what their programs should do.
They also made Lisp slow on ordinary computers. Automatic memory management, checking the types of values at run time and the heavy use of lists all cost time and memory on hardware designed for numerical calculation. By the mid-1970s AI researchers at MIT were running large Lisp programs on shared time-sharing machines and finding them painfully slow.
The bet
Richard Greenblatt and Tom Knight at the MIT AI Lab proposed a radical solution: build a personal computer designed specifically to run Lisp. Work on the first machine, CONS, began around 1974; a second generation, the CADR, was built in the late 1970s and used in the lab. The hardware supported Lisp’s needs directly, with tags identifying the type of each value and support for memory management. Each researcher would have a whole machine, an unusual luxury at the time.
Around 1979–80, two companies were founded to sell commercial versions: Lisp Machines Inc. (LMI) and Symbolics. Xerox, whose research centre had developed its own Lisp environment, and later Texas Instruments joined the market. The bet was that AI and advanced software development would grow into a large market that needed the best possible Lisp environment, and that custom hardware would provide it.
Why it made sense
Lisp machines were extraordinary. Their environments offered high-resolution graphical displays, windows, a mouse, networking and powerful development tools: programmers could inspect any object, change code while it ran and debug in ways most developers would not see for another decade or more. The whole operating system was written in Lisp and could be examined and modified by the user.
The market looked promising. The early 1980s brought an AI boom. Expert systems, programs encoding the rules of human specialists, attracted investment. Japan’s Fifth Generation Computer project, launched in 1982, alarmed American and European governments, which responded with large programmes of their own, including the American Strategic Computing Initiative of 1983. Lisp machine makers sold to universities, defence contractors, oil companies and AI start-ups. Symbolics, the largest, became a significant company. In March 1985 it registered symbolics.com, the first .com domain name.
How it lost
Two things happened at once. First, general-purpose workstations from Sun, Apollo and others, built on fast commodity microprocessors, improved rapidly, riding the industry-wide gains described in the compute essay. Companies such as Lucid and Franz built high-quality Lisp implementations for these machines. By the mid-1980s a standard workstation could run Lisp acceptably for many purposes at a fraction of the cost of a Lisp machine, and it could also run everything else a company needed.
Second, the AI boom ended. Expert systems proved more limited and harder to maintain than promised. Funding and corporate enthusiasm cooled in what became known as the second AI winter. Around 1987 the market for specialised AI hardware collapsed, by many accounts almost overnight. Symbolics struggled on and filed for bankruptcy protection in 1993.
Explaining both the same way
In 1990–91 Richard Gabriel, a Lisp pioneer, circulated an essay containing a section called “The Rise of Worse Is Better”. He contrasted two design philosophies. The “MIT approach”, which he associated with Lisp, held that a design must be correct, consistent and complete, even if that made it complex to implement. The “New Jersey approach”, associated with Unix and C at Bell Labs, favoured simplicity of implementation, even at some cost to correctness and completeness. Gabriel argued, with evident ambivalence, that the second approach had better survival characteristics: simpler systems spread faster, run on more machines and improve over time.
That is a symmetrical explanation, and its strength is that it calls neither side foolish. Lisp machines were better at what they did. Workstations running Unix were good enough at it and better at fitting into the rest of computing. Where general-purpose hardware improved relentlessly and cheaply, the general system’s advantages compounded and the specialised system’s advantages eroded. Christensen’s innovator’s dilemma describes the same shape from the other side: the cheaper, simpler product, at first inadequate, improves until it is enough.
What the loss teaches
The central lesson is about where to put value when the platform beneath you is being commoditised. The Lisp machine companies’ deepest asset was their software environment, not their hardware. Their fate might have been different had they moved that environment onto general-purpose machines quickly, though with lower margins and fierce competition. Holding on to the hardware meant holding on to the part of the business the market was making obsolete.
The second lesson concerns booms. Lisp machines rose and fell with expectations for AI. A company tied to one market’s mood is exposed to its swings, a warning with an obvious contemporary resonance.
Finally, little of value was lost. Integrated development environments, interactive debugging, garbage collection and dynamic languages, all developed or refined in the Lisp machine world, became standard. The machines died. Their way of programming won.
Sources and limitations
The market collapse of 1987 is widely described but poorly quantified in public sources; revenue figures for Symbolics and others are not given here because reliable figures were not found. Technical descriptions draw on the 1980 paper by Greenblatt and colleagues and on Steele and Gabriel’s history of Lisp. Gabriel’s essay is available on his own website. The interpretation that a faster move to software might have helped is the author’s, and it is uncertain: software companies selling Lisp on workstations also struggled once the AI winter set in.
The minds involved
The forces at work
-
Specialised excellence versus general improvement
Lisp machines were superb at one thing. General-purpose processors improved for every customer at once, funded by a far larger market. Each year the gap in Lisp performance narrowed while the gap in price stayed wide, until good enough at a fraction of the price won.
-
The early 1980s brought great optimism about expert systems and AI, with large government programmes in Japan, the United States and Britain. Lisp machine demand rose with that optimism. When expectations proved too high, demand fell with them, and the machines were tied to a single market's mood.
-
Most buyers did not need the best Lisp environment. They needed one that was adequate and ran on the same computers as their other software. Buyers who satisfice, choosing the first acceptable option, favoured general-purpose workstations.
-
Richard Gabriel's famous essay contrasted the MIT approach, which favoured correctness and completeness, with the approach of Unix and C, which favoured simplicity of implementation. The symmetrical point is that both approaches were coherent; the environment of cheap, spreading hardware rewarded the second.
For people who build products
Decisions worth studying
3 decisions from this history, each with the situation, the alternatives, what happened and a verdict. Open one for the full case. The verdict says how a decision turned out, which is not the same as whether it was wise; see moral luck.
-
Commercialise the MIT Lisp machine as dedicated hardware rather than as software for general-purpose computers. A hardware advantage built on today's gap is a bet that the gap will last.
- Situation
- General-purpose computers of the late 1970s ran Lisp slowly and had small memories. Custom hardware gave dramatic gains in speed and capability.
- Alternatives
- Port the Lisp environment to general-purpose minicomputers and workstations; sell software only.
- What happened
- The companies grew quickly in the early 1980s, selling to research labs, defence contractors and AI start-ups.
- Verdict
- Correct for the hardware of 1979. It tied the companies' fortunes to the assumption that specialised hardware would stay ahead of general-purpose processors.
Hiding the machine wins once the machine is cheapMaking something cheap moves the cost to someone else
-
Continue investing primarily in proprietary Lisp hardware as general-purpose workstations improved rapidly. When a cheaper general platform is catching up, move your value into what runs on it before it arrives.
- Situation
- Sun, Apollo and others were selling workstations based on fast commodity microprocessors. Commercial Lisp implementations for these machines were improving.
- Alternatives
- Shift to software, porting the environment to standard workstations; partner with workstation makers.
- What happened
- Around 1987 demand for Lisp machines collapsed as customers moved to cheaper general-purpose machines. Symbolics filed for bankruptcy protection in 1993.
- Verdict
- The trend in general-purpose processor performance was visible and widely discussed. A faster move to software might have preserved the companies' most valuable asset, their environments.
A defining strength becomes the constraintIncumbents absorb the challenger’s best idea
-
Standardise Common Lisp so that Lisp programs could run on many machines, including general-purpose ones. Standards that help your users can also dissolve your hardware advantage; plan for that.
- Situation
- Many incompatible Lisp dialects existed. Users wanted portability.
- Alternatives
- Remain with machine-specific dialects.
- What happened
- Common Lisp became an ANSI standard in 1994. Portability helped Lisp survive the hardware collapse, though in a much smaller market.
- Verdict
- Portability was the right move for users. It also removed one of the main reasons to buy dedicated hardware.
Nothing quite dies
What survived
- Integrated development environments with interactive debugging, inspection and live code changes, now standard in modern tools.
- Automatic memory management (garbage collection), which Lisp machines supported in hardware and which is now standard in most popular languages.
- Dynamic languages such as Python, JavaScript and Ruby, which carry many ideas first developed in the Lisp tradition.
- Common Lisp, standardised in 1994, and later Lisp dialects such as Clojure.
- Symbolics registered symbolics.com in March 1985, the first domain name registered under .com.
Rise and fall
Timeline
- John McCarthy designs Lisp at MIT.
- Work begins at the MIT AI Lab on the CONS Lisp machine.
- The CADR, a second-generation MIT Lisp machine, is built and used within the lab.
- Lisp Machines Inc. (LMI) is founded.
- Symbolics is founded.
- Japan launches its Fifth Generation Computer project, intensifying interest in AI.
- Symbolics registers symbolics.com, the first .com domain.
- The market for specialised AI hardware collapses.
- Richard Gabriel's "worse is better" argument circulates.
- Symbolics files for bankruptcy protection.
- Common Lisp becomes an ANSI standard.
Take it to your team
Questions to ask
- Is our advantage built on a gap that a cheaper, general technology is closing?
- If our best feature ran on everyone's standard platform, would customers still need our product?
- Are we tied to a single market's mood, and what happens to us if that market cools?
Follow the thread
Connections
Where this comes from
Sources
- Richard P. Gabriel, “Lisp. Good News, Bad News, How to Win Big” (1990–1991), including the section “The Rise of Worse Is Better.”
- Richard D. Greenblatt, Thomas F. Knight Jr., John Holloway and David A. Moon, “A LISP Machine,” Proceedings of the Fifth Workshop on Computer Architecture for Non-Numeric Processing (1980).
- Guy L. Steele Jr. and Richard P. Gabriel, “The Evolution of Lisp,” ACM SIGPLAN Notices 28, no. 3 (1993), 231–270.
- Steven Levy, Hackers. Heroes of the Computer Revolution (Doubleday, 1984).
- Daniel Crevier, AI. The Tumultuous History of the Search for Artificial Intelligence (Basic Books, 1993).
Shrikant Kumar. “Lisp Machines: Computers Built for Thinking.” Road not taken, Telos. https://shrikantk.com/roads-not-taken/lisp-machines/