A Relational Model of Data for Large Shared Data Banks

The situation

By the late 1960s large organisations stored their records in database systems such as IBM’s IMS, built for the Apollo programme and widely sold, and systems following the CODASYL network model. These systems were efficient, but they asked programmers to navigate the data: to know how records were ordered, which indexes existed and which pointers linked one record to another. Programs were written around those details. When the storage changed, because of growth or a new kind of query, programs broke.

Edgar F. Codd was a British-born mathematician working at IBM’s research laboratory in San Jose. He saw this dependence as the central problem.

What it argues

The paper’s first sentence states the goal: “Future users of large data banks must be protected from having to know how the data is organized in the machine (the internal representation).” He immediately rules out a half-measure: “A prompting service which supplies such information is not a satisfactory solution.” Applications should remain unaffected when the internal representation changes.

Codd names three kinds of dependence in existing systems that needed to be removed: “ordering dependence, indexing dependence, and access path dependence.” Programs relied on records being stored in a particular order, on particular indexes existing and on particular paths through the data. Each made programs fragile.

His solution was to describe data “with its natural structure only—that is, without superimposing any additional structure for machine representation purposes.” He proposed the relation, from mathematics, a set of rows (tuples) with named columns, as the only structure users would see. Data would be manipulated with operations such as selection, projection and join, grounded in set theory and logic. He introduced a normal form to remove redundancy and showed how the model supported reasoning about consistency. He argued that the network model then in vogue had “spawned a number of confusions”, among them mistaking connections between records for genuine relations.

Notably, the paper states that implementations of systems to support the relational model are not discussed. It was a theory of what users should see, not a design for making it fast.

The idea underneath

Codd’s paper is a profound piece of abstraction: a decision about who has to know what. The databases essay takes its title from exactly this question. Codd moved knowledge of storage from the user to the system. The cost was efficiency, at first. The benefit was that people could ask questions of data without knowing how it was kept, and the system could reorganise itself without breaking them.

It is also an exercise in legibility. Relations make data legible: uniform, rigorous, inspectable. Like all legibility, it simplifies. Later movements, from object databases to NoSQL systems, argued that some data, such as documents, graphs and rapidly changing records, fits tables badly. The relational model absorbed many of these challenges, but the trade-off it made is real.

How it was received and what followed

The paper was resisted inside IBM, where IMS was a very successful product. Researchers at IBM built a prototype, System R, and developed a query language called SEQUEL, later SQL, but IBM was slow to commercialise it. Others moved faster. Relational Software, which became Oracle, shipped a commercial relational database in 1979; a project at Berkeley, Ingres, led to several companies. IBM released its own relational products in the early 1980s. By the 1990s relational databases and SQL dominated, and they remain central to computing today. Codd received the Turing Award in 1981.

The resistance inside IBM is a classic case of the innovator’s dilemma and of loss aversion: a superior idea owned by a company whose existing product it threatened.

Reading it today as a decision-maker

The paper’s most transferable idea is data independence, generalised: separate what users need to know from what the system needs to know, so each can change without breaking the other. Interfaces, APIs and platform boundaries all apply the same principle. Ask of any product what its users are forced to understand about its internals.

Its history adds a strategic lesson. Owning the better idea is not enough if a successful product inside the same company has reasons to resist it. Codd’s model reached the market first through competitors.

Sources and limitations

Quotations are from the published paper, checked against the extracted text of the copy hosted by the University of Pennsylvania. The account of IBM’s slowness to commercialise the model and Oracle’s early product draws on histories of the software industry; participants describe the internal debates differently. The interpretation in terms of legibility and the innovator’s dilemma is the author’s.