Who Has to Know Where the Data Lives?

The most consequential sentence in the history of data management is also one of the plainest. It opens a paper published in the June 1970 issue of Communications of the ACM by Edgar F. Codd, a mathematician working at IBM’s research laboratory in San Jose:

Future users of large data banks must be protected from having to know how the data is organized in the machine (the internal representation).¹

Read quickly, this sounds like a technical preference. Read slowly, it is a claim about responsibility. Someone has to know where data lives on a disk, which records point to which, and which path through them is fast. Codd’s sentence says that this someone should not be the person asking the question. The word he chose was protected, as if knowledge of storage were a burden or a hazard.

Every database built in the half century since has taken a side on that sentence, sometimes without noticing. The systems before Codd answered it one way. The systems that grew out of his paper answered it the other. The web-scale systems of the 2000s partly reversed the answer for good reasons, and the systems of the last decade have been quietly reversing it back. To understand why the database looks the way it does today, and why it has become the one part of software that is expected to outlast every program that touches it, it helps to follow that single question from the beginning.

Before the database: the file belonged to the program

For most of recorded history, keeping data meant keeping books. Merchants recorded transactions in ledgers, and over centuries accountants developed rules that look strikingly modern. Jim Gray, one of the central figures in database research, described them in a 1981 paper. Double-entry bookkeeping made calculations self-checking. And “one never alters the books; if an error is made, it is annotated and a new compensating entry is made in the books. The books are thus a complete history of the transactions of the business.”² Keep that rule in mind. It disappears for a few decades and then returns in a different form.

Machines entered the story through counting. After the 1880 United States census proved slow to tabulate by hand, the Census Bureau held a competition in 1888 to find faster methods for 1890. Herman Hollerith won with a system that recorded each person as holes in a card and counted them electrically, as pins passed through the holes and closed circuits.³ The punched card became the standard unit of business data for more than half a century, and the company that grew from Hollerith’s work eventually became IBM.

Cards, and later magnetic tape, imposed a particular shape on data. Both were sequential. To find a record you read from the beginning. To change a record you produced a new copy of the whole file. Gray describes how early computer systems simply carried the bookkeeper’s discipline forward: a nightly run took the old master file and the day’s transactions and produced a new master, and the old master was never updated. This was good accounting practice, he notes, but it was also forced by the medium, since “writing a new tape was easier than re-writing the old tape.”²

The deeper consequence of this period was organisational. Each application owned its files. The payroll program had a payroll tape whose layout lived inside the payroll program’s code. Inventory had its own tape, laid out differently, with its own copy of part numbers and descriptions. If the same fact appeared in two places, nothing kept the two copies in agreement except human care. And if someone changed a file’s layout, every program that read it had to be rewritten. Data was not a shared resource. It was an extension of a particular program.

The disk changes the question

Direct access storage, first on drums and then on disk, broke this arrangement. A disk could jump to any location. For the first time it was practical to keep data permanently online, to update a single record in place, and to let several applications reach the same records. As Gray put it, it “became attractive to update just the parts that changed.”² That attraction carried a cost the ledger keepers would have recognised: once you overwrite in place, the history is gone unless you deliberately keep it somewhere else.

The disk also raised a question that tape never had to answer. If records can be reached in any order, how do you find the right one, and how do you represent the relationships between them? A customer has many orders. An order has many lines. A finished product is assembled from parts, which are assembled from other parts. On tape these relationships were handled by sorting files into matching order and reading them side by side. On disk they could be stored directly, as links.

Bachman and the integrated store

The first system to take this seriously was built at General Electric. Charles Bachman, a mechanical engineer who had come to computing through industrial work, joined GE in 1961 and was assigned to an “integrated systems project” to produce a generic manufacturing control system for GE’s departments, running on the new GE 225 computer.⁴ The manufacturing system, called MIACS, had to plan and schedule work across a factory. Underneath it Bachman built the Integrated Data Store, or IDS, which shipped in 1964 and is widely regarded as the first database management system used in everyday production.⁵

Two details from Bachman’s own account explain a great deal about its design. First, it was built under severe constraints. GE barely had disk drives when the work started; during the first programming of IDS, the team had no disk to debug on and used two magnetic tape drives to simulate one.⁶ Second, Bachman thought of the database as a large virtual memory made of pages, with records as lines on those pages, and relationships as chains of pointers linking records together.⁶ An item record could own a chain of backorder records. A problem record could own a chain of data records. To answer a question, a program started at a known record and followed the chains.

This was an elegant answer to the constraints of the time. Following a pointer to a record already in a memory page was cheap; scanning a whole file was expensive. If you knew in advance how the data would be used, you could lay out the chains so that common tasks touched as few pages as possible. Bachman’s oral history describes tuning exactly this kind of thing: a chain of backorders that was slow because the program had to walk to the end of it every time, fixed by adding a pointer to the last member.⁷ Performance came from the programmer’s knowledge of the structure.

Bachman also invented a way to draw these structures, the data structure diagram, and his design became the basis for the industry standard proposed by the CODASYL Data Base Task Group in the late 1960s and early 1970s.⁴ The model it embodied, records joined by owner and member sets into a network, is known as the network model.

IMS and the hierarchy

At almost the same time, IBM built a different system for a different problem. North American Aviation, later North American Rockwell, was the prime contractor for the Apollo command and service module and needed to track a bill of materials with an enormous number of parts. IBM worked with Rockwell (and with Caterpillar Tractor) on software that became the Information Management System, IMS.⁸ IBM’s own history dates the request to 1963 and gives 1968 as the year the system went into use; other accounts place the formal partnership in 1965 and the commercial rename in 1969.⁸ The differences matter less than the shape of the solution. A bill of materials is naturally a tree: a spacecraft contains assemblies, which contain subassemblies, which contain parts. IMS organised data as hierarchies of segments, and programs navigated down and across those trees.

Both systems solved real problems, and both were successful for decades. IMS is still running in large institutions today. Michael Stonebraker and Andrew Pavlo note drily that this “persistence is more of a testament to the ‘stickiness’ of data rather than the lasting power of these systems.”⁹ That observation will matter at the end of this essay. Data, once accumulated, is very hard to move.

The programmer as navigator

What the network and hierarchical systems had in common was a division of labour. The database stored records and maintained links. The application program decided which links to follow. To retrieve anything, a programmer wrote code that moved through the structure one record at a time.

In 1973 Bachman received the ACM Turing Award, the field’s highest honour, and his lecture was titled “The Programmer as Navigator.” He framed it around Copernicus, whose heliocentric model had been published 430 years earlier. Bachman’s argument was that computing needed its own shift of perspective: away from the computer at the centre, with data flowing through it in sequence, toward the database at the centre, with the programmer moving through it, choosing a route from record to record.¹⁰ It was a confident and influential vision, and within its frame it was correct. Disk-based databases had made the programmer free to navigate.

Two young researchers at IBM experienced that freedom from the other side. Donald Chamberlin, recalling his introduction to the CODASYL proposal around 1972, wrote that he and Raymond Boyce spent time “learning concepts such as ‘currency indicators’ and ‘set occurrence selection.’ With a little practice, we learned how to represent a database query in the form of a program that navigated through a network of pointers to find the desired information.”¹¹ A currency indicator was the system’s record of where in the network you currently stood. Asking a question meant writing a small journey.

Codd’s objection

Codd’s 1970 paper was an argument that this division of labour was wrong, and the paper is precise about why. He named three kinds of dependence that tied programs to storage. Ordering dependence: programs assumed records came back in the order they were stored, so changing the order broke them. Indexing dependence: programs referred to particular indexes, so adding or dropping an index for performance broke them. Access path dependence: programs were written to follow particular chains or hierarchies, so reorganising the structure broke them.¹

The goal he set against these was data independence, which he defined as “the independence of application programs and terminal activities from growth in data types and changes in data representation.”¹ The opening of the paper makes the practical stakes plain. Changes in how data is stored “will often be needed as a result of changes in query, update, and report traffic and natural growth in the types of stored information.”¹ In other words, a database that is used will change, and if every change ripples into every program, the cost of using data grows with the amount of it.

His proposal was to describe data in terms of mathematical relations: sets of tuples, which in practice look like tables of rows and columns. A relation, Codd wrote, provides “a means of describing data with its natural structure only, that is, without superimposing any additional structure for machine representation purposes.”¹ Relationships between records would no longer be stored as hidden pointers. They would be expressed through values: an order row contains the customer’s identifier, and the connection exists because the values match. Questions would be asked in a high-level language based on logic, stating what was wanted, and the system would work out how to get it.

Codd also pointed to a subtler problem with the network approach, which he called the “connection trap”: mistaking the existence of a stored link for the existence of a logical relationship, and so deriving conclusions the data does not support.¹ The point was philosophical as much as technical. In a navigational system the meaning of the data was partly carried by its physical wiring. In the relational view, meaning was carried by values and by logic alone.

Why this idea, why then

It is worth asking why the argument appeared in 1970 rather than earlier or later. Several conditions had come together.

The first was scale and sharing. Codd’s title referred to “large shared data banks.” By the late 1960s organisations had begun to consolidate data that many programs and many terminal users needed at once. A file owned by one program could be changed by that program’s authors. A bank shared by hundreds of programs could not be changed safely if each depended on its layout.

The second was that the costs of navigation were visible. The CODASYL proposals were being standardised, and people who studied them closely, as Chamberlin did, could see how much program code a simple question required.

The third was Codd’s own background. He was a mathematician, and relation theory and predicate logic were familiar tools to him. The relational model drew on ideas, sets and first-order logic, that were not new at all. What was new was the claim that they were a practical foundation for commercial data processing.

What the paper did not provide was proof that this could be fast. Its last line of the introduction is candid: “Implementations of systems to support the relational model are not discussed.”¹ That gap was the reasonable heart of the resistance. A navigational programmer who knew the data could choose an efficient path. Could a general-purpose system, given only a logical question, choose as well? In 1970 nobody knew.

The disagreement came to a head at the 1974 ACM SIGFIDET workshop in Ann Arbor, where Codd and Bachman appeared in a much-remembered debate between the relational and network positions; the proceedings volume carried the subtitle “Data-Structure-Set versus Relational.”¹² Accounts of who “won” mostly come from people who had already chosen sides, and they are best read that way. The more useful lesson is that both positions were coherent. Bachman’s was about control and performance under real constraints. Codd’s was about the long-term cost of coupling meaning to storage. The question was empirical: could the machine take over the navigator’s job?

Making the machine navigate

A language for people who are not programmers

Chamberlin and Boyce first met Codd at a symposium he organised at IBM’s Yorktown Heights laboratory in 1972. Chamberlin later wrote that the exposure to the relational model “was a revelation. For the first time, we could see how a query that would require a complex program in the DBTG language could be reduced to a few simple lines.”¹¹ They made a game of inventing queries and challenging each other to express them, and one became a classic: find the names of employees who earn more than their managers.

Codd’s own languages, relational algebra and relational calculus, used mathematical notation. Chamberlin identified this as a barrier, partly a superficial one: the symbols were hard to type on an ordinary keyboard.¹¹ Their first attempt, called SQUARE, used a subscript notation that was still difficult to type. After moving to IBM’s San Jose laboratory in 1973 to join a project called System R, they designed SEQUEL, intended to be read “almost as though they were English prose.”¹³

Their ambitions for it are recorded plainly. Chamberlin wrote that they expected the language to be used mainly for “ad-hoc queries by planners and other professionals whose domain of expertise was not primarily database management,” and that they wanted it “simple enough that ordinary people could ‘walk up and use it’ with a minimum of training.”¹⁴ That aspiration explains the language’s most distinctive design choices: keywords instead of symbols, a structure that reads from left to right as a request, and no notion of position or pointer anywhere in a query.

Boyce died suddenly in 1974, at 26, about a month after presenting SEQUEL at the Ann Arbor conference.¹³ The language continued to evolve inside System R, was tested at three IBM customer sites, and in the late 1970s was renamed SQL because of a trademark issue.¹⁵

The optimizer: the navigator moves inside

The language was half the answer. The other half was the component that decided how to execute a query. In 1979 Patricia Selinger and colleagues at IBM published “Access Path Selection in a Relational Database Management System,” describing how System R chose among ways to run a query: which indexes to use, in what order to join tables, by which method. It estimated the cost of each plan using statistics about the data, and picked the cheapest.¹⁶

This is the moment the old navigator’s job moved inside the database. The programmer in the CODASYL world had chosen access paths using knowledge of the data. The optimizer chose access paths using statistics about the data, recomputed as the data changed. Because the choice was made at run time rather than written into the program, the database administrator could add an index, reorganise storage, or let tables grow by orders of magnitude, and the same query would keep working, often faster. Data independence, which had been a principle, became an engineering fact. The cost-based approach in that paper remains the ancestor of the optimizers in most relational systems in use today.

The same years produced a parallel effort at the University of California, Berkeley, where Stonebraker and Eugene Wong built Ingres, with its own query language, QUEL. Commercial systems followed quickly. A small company that became Oracle shipped a SQL database in 1979, and IBM released SQL/DS in 1981 and DB2 in 1983. Chamberlin notes that commercial implementations, “such as Oracle and DB2, began to appear in the late 1970s and early 1980s,” and by 1986 SQL had become an ANSI standard.¹⁵ Codd received the Turing Award in 1981.

What the relational win actually decided

It is easy to tell this story as the triumph of a better idea, and the idea was good. But the reasons it prevailed are worth stating carefully, because they explain what happened later.

The relational model won partly because its central promise said nothing about hardware. A table does not specify whether it lives on a disk, in memory, in rows or in columns, on one machine or a thousand. That silence was deliberate. It meant that every improvement in storage technology could be absorbed underneath the same interface. Programs written against SQL in the 1980s could benefit from storage engines built decades later.

It won partly because the optimizer made the bet pay off in performance often enough. And it won partly because a standard language let skills, tools and applications move between products. None of these was guaranteed in 1970. They had to be built.

There was also a cost. A relational system asks you to decompose information into normalised tables and reassemble it with joins. Programs written in languages that think in terms of objects and nested structures must translate back and forth, a friction that came to be called the “impedance mismatch.” Chamberlin himself noted that this mismatch “tends to increase application complexity and interfere with global optimization.”¹⁴ Much of the later history of databases is a series of attempts to remove that friction, and most of them were eventually folded back into relational systems.

The database as a keeper of promises

So far the story has been about finding data. An equally important strand is about changing it safely, especially when many people change it at once.

Gray’s 1981 paper, “The Transaction Concept: Virtues and Limitations,” begins not with computers but with law. “The transaction concept derives from contract law,” he wrote. Parties negotiate and then make a deal binding by a joint act, and if they are cautious they appoint an intermediary to coordinate the commitment.¹⁷ From this he drew the properties a transaction must have: atomicity, it happens entirely or not at all; consistency, it obeys the rules; and durability, once committed it cannot be undone except by a further, compensating transaction.¹⁷ Theo Härder and Andreas Reuter added isolation, that concurrent transactions should not see each other’s partial work, and gave the set the acronym ACID in 1983.¹⁸

This is where the ledger’s old rule quietly returns. Disks had made update in place attractive, and Gray devoted a section of his paper to update in place under the heading “A poison apple?”² The practical answer that databases converged on was the log: before changing anything in place, write a record of the intended change to an append-only log. If the system crashes, the log is replayed or rolled back. The tables you query are, in effect, a convenient summary of a history that is kept elsewhere, in order, never altered. The bookkeepers’ principle survived the disk by moving underneath it.

The transaction changed what a database was. It was no longer just a store. It became the place where an organisation’s commitments were recorded and enforced: a seat sold once, money moved without being lost or created. Chamberlin, looking back in 2012, admitted he had not expected this. He had imagined SQL for occasional questions by non-specialists, but found it “more frequently used by trained database specialists to implement repetitive transactions such as bank deposits, credit card purchases, and online auctions.”¹⁴ The language designed for curiosity became the language of commitment.

This is an instructive gap between intention and consequence. The intended user, the planner walking up to a terminal, never fully arrived; Chamberlin wrote that SQL “has not proved to be as accessible to untrained users as Ray and I originally hoped.”¹⁴ But the decision to make queries declarative turned out to matter most for an audience nobody had designed for: programmers who needed the system, not themselves, to guarantee correctness under concurrency and failure.

Two kinds of questions

As relational systems spread through the 1980s, a tension appeared between two kinds of work. Transaction processing, the bank deposits and seat reservations, needed many small reads and writes, each fast and safe. Analysis needed a few enormous questions over years of history: what sold, where, and why. The same database struggled to serve both; long analytical queries slowed the transactions that kept the business running.

The response was to separate them. Data was copied, usually overnight, from operational systems into a separate store organised for analysis, which came to be called a data warehouse. This is, again, a story about the ledger: the operational system holds the current state, the warehouse holds the history. The warehouse was relational too, but it was tuned for reading. Over time, engineers found that analytical queries ran far faster if data was stored by column rather than by row, since a question about sales totals reads one column across many rows. Stonebraker and Pavlo describe column stores as a new implementation of the same model, and argue that such implementation changes, rather than new data models, are where most progress has come from.⁹

The same period produced a series of challenges to the relational model itself. Object-oriented databases in the late 1980s and early 1990s promised to store programming objects directly and so remove the impedance mismatch. XML databases in the late 1990s promised the same for documents. Stonebraker and Hellerstein’s 2005 review of forty years of data models concluded that “the relational model with an extendable type system (i.e., object-relational) has dominated all comers.”⁹ The useful parts of each challenger, such as richer types and nested data, were added to relational systems, and the challengers themselves faded.

This pattern deserves attention because it repeats. A real limitation is noticed. A new system is built around fixing it, often by moving some responsibility back to the application. It succeeds in its niche. Then relational systems absorb the useful idea, and the separate system loses its reason to exist. It is tempting to see this as relational imperialism. A more careful reading is that the relational contract, say what you want and let the system decide how, was flexible enough to swallow new features without breaking old programs, which is exactly what Codd designed it to do.

The web breaks the single machine

The most serious challenge came not from a new data model but from a new scale. In the early 2000s a handful of internet companies were storing and serving data in volumes, and at request rates, that no single machine could handle. Relational databases had grown up on single, expensive, reliable servers. Spreading one across hundreds of cheap machines, any of which might fail at any moment, while keeping every transaction perfectly consistent, was not something existing products did well.

A different set of priorities

Google’s response was a set of systems built for its own needs: a distributed file system (2003), MapReduce for batch processing over it (2004), and Bigtable for structured storage (2006). Stonebraker and Pavlo describe MapReduce as having been built as a “point solution” for processing Google’s periodic crawl of the web, by a company that “had little expertise in DBMS technology” at the time.¹⁹ That is not a criticism; it is a description of priorities. These were systems designed by people whose overriding constraint was scale and machine failure, not by people trying to implement Codd.

Amazon’s Dynamo paper, published in 2007, states its priorities more explicitly than almost any document in the field. Reliability at massive scale was the first concern, since “even the slightest outage has significant financial consequences and impacts customer trust.” Dynamo targeted “an ‘always writeable’ data store,” because for many services “rejecting customer updates could result in a poor customer experience.” The example was the shopping cart, which had to let customers add and remove items even while parts of the system failed. And the paper observed that most of the services in question “only store and retrieve data by primary key and do not require the complex querying and management functionality offered by an RDBMS.”²⁰

Then came the decisive trade: “To achieve this level of availability, Dynamo sacrifices consistency under certain failure scenarios.” Dynamo was “an eventually consistent data store; that is all updates reach all replicas eventually.”²⁰ If two copies of a shopping cart diverged during a failure, the system would keep both and let the application reconcile them later. Adding an item to a cart twice was a better outcome than refusing to add it at all.

CAP and the language of trade-offs

The intellectual frame for these decisions had been offered a few years earlier by Eric Brewer, who presented what became known as the CAP theorem in the late 1990s, most famously in a keynote at the 2000 Symposium on Principles of Distributed Computing.²¹ In its popular form it said a distributed system can provide only two of three properties: consistency, availability, and tolerance of network partitions. Since networks do partition, the real choice during a partition is between staying consistent and staying available.

Brewer and his colleagues had also coined BASE, a deliberately chemical counterpart to ACID, for designs that favoured availability.²¹ Twelve years later he wrote a careful revision, warning that “the ‘2 of 3’ formulation was always misleading because it tended to oversimplify the tensions among properties,” and describing ACID and BASE as “two design philosophies at opposite ends of the consistency-availability spectrum.”²¹ The revision matters because many systems had treated CAP as permission to abandon consistency altogether, when the actual trade-off applied only during failures and could be managed with much more nuance.

NoSQL and the return of the navigator

In June 2009 a meetup in San Francisco, organised by Johan Oskarsson, gathered developers working on new distributed stores; the hashtag chosen for it, NoSQL, became the name of a movement.²² The systems under that banner varied widely: key/value stores, document stores holding JSON, wide-column stores modelled on Bigtable, graph databases. What they mostly shared was a willingness to give up some combination of SQL, joins, schemas and multi-record transactions in exchange for scale, flexibility and simplicity of operation.

There were good reasons for this. Web applications were increasingly written in JavaScript, and JSON was their natural data format; storing it directly removed a translation step. Startups changed their data models weekly, and a “schema later” approach let them move fast. For workloads that really were simple lookups by key, a relational engine was more machinery than necessary.

But look at what moved where. In a key/value store, Stonebraker and Pavlo point out, the value “is typically an untyped array of bytes,” the database “is unaware of its contents,” and “it is up to the application to maintain the schema and parse the value into its corresponding parts.”¹⁹ Joins, if needed, are implemented in the application. In a document store, related data is nested inside documents for speed, which brings back the old problems of denormalisation, including duplicated data and, in their words, “no data independence.”²³

In other words, a significant part of the NoSQL movement was a deliberate return to the navigator. The application again had to know how data was laid out and how to traverse it. This was a reasonable trade in many cases, the same one Bachman made in 1964: when constraints are severe and access patterns are known in advance, putting knowledge of the structure in the program buys performance. The cost arrives later, when access patterns change, which they always do.

The guarantees come back

What happened next is the part of the story that is easiest to miss, because it unfolded gradually.

In 2012 Google published Spanner, a database distributed across data centres around the world that nevertheless supported “externally-consistent distributed transactions,” transactions whose order matched the real-time order in which they happened, everywhere.²⁴ It did this partly with an unusual piece of engineering: an interface called TrueTime that exposed the uncertainty in each server’s clock, backed by GPS receivers and atomic clocks, so the system could wait out the uncertainty instead of ignoring it. Spanner showed that with enough engineering, the trade-off CAP described could be pushed much further toward consistency than many had assumed. It was, notably, built by the same company whose earlier systems had relaxed those guarantees.

Meanwhile the NoSQL systems were adding back what they had removed. By the end of the 2010s, almost every major one had added a SQL interface: DynamoDB’s PartiQL, Cassandra’s CQL, Couchbase’s SQL++, and MongoDB’s SQL support for its Atlas service in 2021.²⁵ Many added ACID transactions. Stonebraker and Pavlo summarise the drift memorably: the message “has morphed from ‘Do not use SQL – it is too slow!’ to ‘Not only SQL’.”²⁵ In the opposite direction, the SQL standard added a JSON data type and functions for semi-structured data, narrowing the remaining difference.

It would be wrong to read this as the NoSQL experiment having failed. It changed relational systems profoundly. It proved that cheap, failure-prone machines could hold the world’s data, forced serious engineering into distributed consensus and replication, and established flexible schemas and JSON as normal expectations. What it did not do was displace the relational contract. As with object and XML databases before, the useful ideas were absorbed.

Hardware rewrites the “how”, again

If the relational model kept winning at the level of what, nearly everything changed at the level of how. Stonebraker and Pavlo’s 2024 review argues that while relational fundamentals barely moved, “there were dramatic changes in RM system implementations,” and that the major advancements were “primarily driven by changing hardware characteristics.”⁹

Three of these changes are worth tracing because they show data independence doing exactly the work Codd intended.

The first is the cloud. In a 2016 paper, Snowflake’s engineers described a data warehouse built as a “multi-cluster, shared-data architecture,” in which data sits in cheap cloud object storage and separate, elastic clusters of compute are attached to it as needed.²⁶ Storage and processing, once welded together in a single server, became independent resources priced separately. Users kept writing SQL. Underneath, the physical arrangement was unrecognisable compared with a 1990s warehouse.

The second is open file formats. Data lakes stored raw files, often in column-oriented formats such as Parquet, cheaply and in quantity, but without the transactional guarantees of a warehouse. In 2021 researchers at Databricks proposed the “lakehouse”: warehouse features such as transactions, versioning and performance, layered over open files that many different engines can read directly.²⁷ What is striking philosophically is that this moves data independence down a level. The promise is no longer only that programs need not know how one database stores data. It is that the data can outlive any particular database engine, in a format no single vendor controls.

The third is scale in the other direction: very small. In 2000, D. Richard Hipp was writing damage-control software for a US Navy destroyer that depended on a separate database server; when the server was unavailable, the software could not work. His response was SQLite, a complete SQL database that runs inside the application and stores everything in a single ordinary file, with no server at all.²⁸ Its own documentation estimates that with billions of smartphones each holding hundreds of SQLite files, “there are over one trillion (1e12) SQLite databases in active use,” in every Android and iOS device, every major web browser and many other places.²⁹ More recently, embedded analytical engines such as DuckDB apply the same idea to analysis. The relational contract that began on mainframes now runs, invisibly, in nearly every pocket.

What a database is today

A survey of the present landscape shows how much has converged.

Among developers, the relational systems dominate use. In Stack Overflow’s 2025 developer survey, 55.6 percent of respondents who answered the question reported using PostgreSQL in the past year, followed by MySQL at 40.5 percent and SQLite at 37.5 percent.³⁰ PostgreSQL itself descends from Postgres, a Berkeley research project of the mid-1980s whose central idea was an extensible type system, the very feature that let relational systems absorb challengers.

New data models keep arriving and keep being absorbed. Graph queries, long the territory of specialised graph databases, were added to the SQL standard in 2023 as property graph queries, and a dedicated Graph Query Language, GQL, was published as an ISO standard in April 2024 by the same committee that maintains SQL.³¹ The newest arrival is the vector database, built to store the numerical embeddings produced by AI models and to find records whose vectors are nearest to a query vector. Stonebraker and Pavlo note that vector systems are attracting attention “similar to the NoSQL fad,” and observe how quickly the familiar pattern started: after ChatGPT made large language models mainstream in late 2022, it took less than a year for several relational systems to add vector indexes of their own, often by integrating open-source libraries such as pgvector. They describe such indexes as “a feature, not the foundation of a new system architecture,” and anticipate that vector databases will become more relational over time while the incumbents add vectors to their feature lists and move on.³² Whether that prediction holds is a question for the next decade.

Meanwhile, the physical architecture of serious data systems is now usually distributed, replicated, logged and cloud-hosted, with storage separated from compute. The query a user types looks much like one Chamberlin might have written in 1974.

Understanding the philosophy

The history above can be read as engineering, and it is. But the arguments engineers had about databases were often arguments about very old philosophical questions: what a fact is, what it means to know something, whether the past can change, what counts as true when two records disagree. This section takes the main ideas embedded in database design one at a time. For each, it explains the idea in plain terms, introduces the philosophical tradition it connects to, shows where it appears in the history, sets out the strongest objection, and says what it means for anyone who designs or depends on data today. The connections to named philosophers are interpretations; with the exceptions noted, the engineers involved did not cite them.

1. A database is a collection of facts, not a pile of things

The idea in plain terms

Look at one row in a relational table: Baker, 50,000, Smith. On its own it is three values. Read with its table’s heading, it becomes a sentence: “The employee named Baker earns 50,000 and reports to Smith.” That sentence is either true or false. A relational database is, in this sense, a large collection of statements its owners assert to be true. Codd built the model on the branch of logic that deals with such statements, the predicate calculus, which is why a query in SQL is really a logical question: “for which values does this statement come out true?”

The tradition

In 1921 Ludwig Wittgenstein opened his Tractatus Logico-Philosophicus with two sentences that read almost like a design document for the relational model: “The world is everything that is the case. The world is the totality of facts, not of things.”³³ Wittgenstein’s point was that a list of objects tells you nothing. Baker, Smith and the number 50,000 are inert until they stand in some relation, and that arrangement, a state of affairs, is what can be true or false. Knowledge, on this view, is knowledge of facts, and facts are things standing in relations.

Where it shows up

The network databases that came before Codd stored things, records, and connections between them, pointers. Meaning was carried partly by the wiring. Codd’s objection, the “connection trap,” was that people confused a stored link with a logical relationship and drew conclusions the data did not support.¹ The relational model stored facts, and let connections be derived from them: Baker’s manager is found because the values match, not because a pointer was laid. In Wittgenstein’s terms, the network model stored things and the roads between them; the relational model stored the world as the totality of facts.

The strongest objection

Some knowledge seems naturally to be about connection rather than about isolated facts: social networks, supply chains, the web of citations. Graph databases were built on that intuition. And real data is rarely a tidy set of true sentences; it is incomplete, provisional and contested. The relational answer has been to absorb graphs rather than reject them, adding property graph queries to SQL in 2023. A defender of the fact view would say that a connection is itself a fact (“A follows B”), so nothing is lost. A critic would say that representing a network as facts is possible but does not always fit how people think about it.

What it means today

Every time someone designs a table, they decide what kind of sentence their organisation will be able to assert. A table of “customers” with a column for “churn risk” asserts something about people that may be a guess dressed as a fact. Understanding a database as a body of assertions is a useful discipline: it prompts the question of who is making each claim, and on what evidence.

2. What is not recorded is treated as false

The idea in plain terms

Ask an airline’s database for flights from Delhi to Kotdwar on Tuesday. If none are recorded, the system answers “none,” and everyone treats that as meaning there are none. A database typically assumes that it is complete: whatever it does not contain is not true.

The tradition

In 1978 the computer scientist Raymond Reiter gave this assumption a name in a paper titled “On Closed World Data Bases.”³⁴ Under the closed-world assumption, the absence of a fact means the fact is false. Under the open-world assumption, absence means only that the fact is unknown. The distinction is an old one in the theory of knowledge, captured in the maxim that absence of evidence is not evidence of absence. Human reasoning moves between the two depending on context. If your name is not on a guest list, you are not invited; if a species has not been found on an island, it may simply not have been looked for.

Where it shows up

Relational databases and SQL are built for the closed world, which is what lets them give definite answers to questions such as “which accounts have no transactions this month?” Codd himself recognised that some facts are genuinely unknown, and introduced the NULL marker and a three-valued logic (true, false, unknown) to handle them, a design that has been debated ever since because it complicates the clean two-valued logic the model began with. Systems built for the open web, such as knowledge graphs, generally take the open-world view instead.

The strongest objection

The closed world is powerful precisely because it is presumptuous. When it is wrong, the error is invisible: a missing record looks exactly like a true negative. Open-world systems are more honest about ignorance, but they cannot answer simple questions decisively. There is no neutral choice, only a decision about which kind of error matters more.

What it means today

A database’s silence is a claim. If a benefit is denied because a record is missing, or a person is flagged because a payment “never arrived,” the closed-world assumption has acted on someone’s life. Anyone building systems that affect people should know where their system treats absence as truth, and what happens to those it gets wrong.

3. Not having to know: abstraction as a division of knowledge

The idea in plain terms

Codd’s central principle was that users should be “protected from having to know how the data is organized in the machine.”¹ Most people drive without understanding the engine, use money without understanding central banking, and take medicine without understanding chemistry. A good abstraction lets people act effectively on knowledge held somewhere else.

The tradition

Two years after Codd, the software engineer David Parnas published a paper on how to divide programs into modules, arguing that each module should hide a design decision likely to change, so that other parts of the system do not depend on it.³⁵ This information hiding became a founding principle of software engineering, and Codd’s data independence is its application to stored data. A broader philosophical version comes from Hilary Putnam, who in 1975 described a “division of linguistic labour”: most people can use the word gold correctly without being able to test whether something is gold, because they rely on experts who can.³⁶ Knowledge in a society is distributed, and ordinary competence depends on trusting the parts one does not hold.

Where it shows up

The query optimizer is the expert in this arrangement. It holds the knowledge of storage, indexes and access paths, and the person writing a query relies on it, as Putnam’s speaker relies on the metallurgist. When web-scale systems in the 2000s pushed knowledge of data layout back into application code, they reversed the division of labour, and the programmer had to become the expert again.

The strongest objection

In 2002 the programmer Joel Spolsky stated what he called the Law of Leaky Abstractions: “All non-trivial abstractions, to some degree, are leaky.”³⁷ The optimizer sometimes chooses badly, and then the person who was “protected from having to know” must suddenly know a great deal. Object-relational mappers, which hide SQL behind programming-language objects, are a well-known source of such leaks. Abstraction does not abolish the hidden knowledge; it postpones the moment when someone needs it.

What it means today

Every abstraction is an act of trust. The practical wisdom the history teaches is to rely on the abstraction by default, to understand roughly what lies beneath it, and to recognise the symptoms that tell you it has started to leak.

4. The commit as a promise that makes itself true

The idea in plain terms

Before a transaction commits, nothing has officially happened: the money has not moved, the seat is not sold. At the moment of commit, it has. The word commit does not describe a change that already occurred; it is the act that makes the change real.

The tradition

In lectures published in 1962 as How to Do Things with Words, the philosopher J. L. Austin drew attention to sentences that do things rather than describe them, which he called performatives.³⁸ Saying “I promise” is promising; saying “I do” at a wedding is marrying; saying “I bet you sixpence” is betting. Such utterances are not true or false. They succeed or fail, depending on whether the right conditions hold: the right people, the right procedure, the right moment.

Where it shows up

Jim Gray reached for the same examples. His 1981 paper derives the transaction from contract law and uses a wedding to explain how a transaction is committed: the minister asks whether anyone objects, asks each party whether they agree, and only if both say “I do” pronounces them married.¹⁷ This is, almost exactly, the two-phase commit protocol that distributed databases use: a coordinator asks every participant whether it is ready, and only if all agree does it declare the transaction committed. Gray’s analogy and Austin’s philosophy describe the same thing: a binding act that succeeds only when a procedure is followed exactly.

The strongest objection

Promises made across unreliable channels have limits. Gray himself described what became known as the “two generals” problem in 1978: two parties communicating only by messengers who may be lost can never be absolutely certain they have agreed.³⁹ In real distributed systems, a commit can be left in doubt when the network fails at the wrong moment. Engineering can make such failures rare and recoverable, but not impossible. Some performatives, it turns out, cannot be guaranteed to land.

What it means today

Every bank transfer, ticket purchase and stock trade rests on this philosophy of the binding act. When a system says “confirmed,” it is not reporting a fact about the world. It is making a promise, and the trustworthiness of the system is the trustworthiness of its promises.

5. The past is fixed; the present is a summary

The idea in plain terms

Inside a modern database, the tables you query show the current state: today’s balances, today’s addresses. Underneath, the log records every change in order and is never altered. The present is, in a real sense, a summary computed from an unchangeable history.

The tradition

Accountants knew this before computers, which is why Gray quotes their rule that “one never alters the books.”² Philosophers have long treated the past as the one thing beyond change. In the Nicomachean Ethics, Aristotle quotes the poet Agathon to the effect that even a god cannot make what has been done undone.⁴⁰ The past is fixed in a way the present and future are not, which is why, for Aristotle, deliberation concerns only what is still to come.

Where it shows up

Write-ahead logs, replication streams, “event sourcing” architectures that store every event and derive state from it, and lakehouse tables that let users query the data “as of” a past moment all express this idea. They treat history as primary and the present as derived.

The strongest objection

Some things should be forgotten. The European Union’s General Data Protection Regulation, in force since 2018, gives people a right to have personal data erased in many circumstances.⁴¹ An architecture built on an immutable record of everything sits uneasily with a legal and moral duty to forget. Engineers have developed techniques such as deleting the encryption keys for a person’s data, but the philosophical tension is real: the bookkeeper’s virtue of never altering the record can become a vice when the record is about a person who wants to move on.

What it means today

Designers increasingly have to decide, deliberately, what their systems should remember forever and what they should be able to forget. That is not a technical question. It is a question about what kind of memory an institution should have.

6. Truth that arrives later

The idea in plain terms

In an eventually consistent system such as Amazon’s Dynamo, two copies of the same data may briefly disagree. The system promises that, if updates stop, all copies will eventually agree. Truth, in such a system, is not a property of any one copy at a given moment. It is what the copies converge on.

The tradition

The American philosopher Charles Sanders Peirce proposed in 1878 a theory of truth that fits this design strikingly well: “The opinion which is fated to be ultimately agreed to by all who investigate, is what we mean by the truth.”⁴² For Peirce, truth was the end point of inquiry, the belief that investigation would converge on if pursued long enough. A related idea comes from physics. Einstein’s special relativity showed that observers in different places cannot always agree on whether two events were simultaneous. The computer scientist Leslie Lamport has said that his foundational 1978 paper on ordering events in distributed systems, “Time, Clocks, and the Ordering of Events in a Distributed System,” grew out of thinking about relativity.⁴³ In a distributed system, as in the universe, there is no single “now” that everyone can see.

Where it shows up

Brewer’s CAP theorem framed the choice: during a network partition, a system must choose between staying available and staying consistent. Dynamo chose availability and accepted temporary disagreement, letting the application reconcile conflicting shopping carts later.²⁰ Google’s Spanner chose the other side, and its TrueTime interface is, philosophically, an admission of uncertainty: rather than pretending clocks agree, it measures how much they might disagree and waits out the difference.²⁴

The strongest objection

For some data, “eventually” is not good enough. A bank balance that is briefly wrong can allow an overdraft; a medical record that is briefly stale can mislead a doctor. The choice between consistency and availability is ultimately a moral judgement about which harm is worse: refusing to answer, or answering wrongly.

What it means today

Systems that span the globe cannot have a single instantaneous truth. The engineering question “how consistent must this be?” is really the question “who is harmed if two people see different versions of the world for a moment, and is that acceptable?”

7. The schema as a theory of what exists

The idea in plain terms

Before storing anything in a relational database, someone must decide what kinds of things exist (customers, orders, products) and what can be said about them. That decision shapes every question the system can later answer.

The tradition

In philosophy, ontology is the study of what exists and how it should be categorised. Aristotle’s Categories tried to list the basic kinds of things that can be said about anything: substance, quantity, quality, relation, place, time and others. In 1999 the scholars Geoffrey Bowker and Susan Leigh Star showed in Sorting Things Out how classification systems, from disease codes to racial categories, shape what institutions can see and do, and whom they leave out.⁴⁴

Where it shows up

The relational model asks for the ontology first: define the schema, then store the data. The document databases of the NoSQL era offered “schema later,” letting applications store data first and decide its structure afterwards. Each approach reflects a view about whether the categories of the world should be decided in advance.

The strongest objection

Fixed schemas can encode outdated or narrow assumptions, a form field with only two options, a name field that cannot hold names from other cultures. Flexible schemas avoid imposing categories early, but they do not escape the need for them; the decision moves into application code, where it is less visible and harder to examine.

What it means today

Designing a schema is designing a small world. It is worth asking, for every table and every field, whose view of reality it encodes and who might not fit.

Bringing the threads together

Seen this way, the relational database is a coherent philosophy built into software: the world as a set of facts; absence as falsehood; the hidden machinery of knowledge entrusted to an expert; commitments as binding acts; the past as fixed and the present as a summary; truth as something every observer must see the same way; and the categories of existence decided in advance. Each later movement relaxed one of these commitments for good reasons. NoSQL relaxed the agreement on truth and the prior ontology; data lakes relaxed the schema; open-world knowledge graphs relaxed the closed world. And each time, as the history shows, much of the original philosophy came back, because the institutions that depend on data turned out to need it. That is why arguments about databases so often feel like arguments about truth. In a quiet way, they are.

Why the database outlives its programs

Return to Codd’s sentence. Future users of large data banks must be protected from having to know how the data is organized in the machine. At first it reads as a statement about convenience. After following the history, it reads more like a statement about time.

Programs are short-lived. They are rewritten, replaced, ported to new languages. Data, by contrast, accumulates and persists; it is the residue of everything an organisation has done. The IMS databases still running decades after their programs were written, and the observation that their persistence reflects the stickiness of data more than the merit of the software, make the point bluntly.⁹ If the meaning of data is tied to the structure of the programs that wrote it, then the data is trapped inside those programs and ages with them. If the meaning is expressed independently, in values and logic that any future program can read, the data can outlive every program that ever touched it.

That is, I think, the deepest logic of the database. Bachman’s navigators were not wrong; they built the first systems that made shared data possible at all, and their approach returns whenever constraints demand it. But the long arc bends toward a system that takes responsibility for the how, keeps an unaltered record of what happened, and lets each new generation of programs ask its own questions of data it did not create. The ledger keepers who never altered the books were solving the same problem. They knew that the records would have to make sense to people who were not there when they were written.

Selected bibliography

• Codd, E. F. “A Relational Model of Data for Large Shared Data Banks.” Communications of the ACM 13, no. 6 (June 1970): 377–387. doi:10.1145/362384.362685
• Bachman, Charles W. “The Programmer as Navigator.” Communications of the ACM 16, no. 11 (November 1973).
• Bachman, Charles W. Oral history interview by Thomas Haigh, 25–26 September 2004. Charles Babbage Institute, University of Minnesota. PDF
• Chamberlin, Donald D. “Early History of SQL.” IEEE Annals of the History of Computing 34, no. 4 (2012): 78–82. doi:10.1109/MAHC.2012.61
• Selinger, P. Griffiths, M. M. Astrahan, D. D. Chamberlin, R. A. Lorie, and T. G. Price. “Access Path Selection in a Relational Database Management System.” Proceedings of ACM SIGMOD (1979). doi:10.1145/582095.582099
• Gray, Jim. “The Transaction Concept: Virtues and Limitations.” Proceedings of VLDB (1981); Tandem TR 81.3. PDF
• Härder, Theo, and Andreas Reuter. “Principles of Transaction-Oriented Database Recovery.” ACM Computing Surveys 15, no. 4 (1983): 287–317.
• DeCandia, Giuseppe, et al. “Dynamo: Amazon’s Highly Available Key-Value Store.” Proceedings of SOSP (2007). Summary and paper
• Brewer, Eric. “CAP Twelve Years Later: How the ‘Rules’ Have Changed.” Computer (IEEE), 2012. InfoQ republication
• Corbett, James C., et al. “Spanner: Google’s Globally-Distributed Database.” Proceedings of OSDI (2012). Google Research
• Dageville, Benoit, et al. “The Snowflake Elastic Data Warehouse.” Proceedings of SIGMOD (2016).
• Armbrust, Michael, Ali Ghodsi, Reynold Xin, and Matei Zaharia. “Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics.” CIDR (2021). CIDR
• Stonebraker, Michael, and Andrew Pavlo. “What Goes Around Comes Around… And Around…” SIGMOD Record 53, no. 2 (June 2024): 21–37. PDF
• Haigh, Thomas. “How Data Got Its Base: Information Storage Software in the 1950s and 1960s.” IEEE Annals of the History of Computing 31, no. 4 (2009): 6–25.
• Wittgenstein, Ludwig. Tractatus Logico-Philosophicus. 1921; trans. C. K. Ogden, 1922.
• Reiter, Raymond. “On Closed World Data Bases.” In Logic and Data Bases, ed. H. Gallaire and J. Minker. Plenum, 1978.
• Parnas, David L. “On the Criteria to Be Used in Decomposing Systems into Modules.” Communications of the ACM 15, no. 12 (1972).
• Putnam, Hilary. “The Meaning of ‘Meaning’.” Minnesota Studies in the Philosophy of Science 7 (1975).
• Spolsky, Joel. “The Law of Leaky Abstractions.” Joel on Software, 11 November 2002. joelonsoftware.com
• Austin, J. L. How to Do Things with Words. Oxford University Press, 1962.
• Gray, Jim. “Notes on Data Base Operating Systems.” In Operating Systems: An Advanced Course. Springer, 1978.
• Aristotle. Nicomachean Ethics, Book VI.
• Peirce, Charles S. “How to Make Our Ideas Clear.” Popular Science Monthly, January 1878.
• Lamport, Leslie. “Time, Clocks, and the Ordering of Events in a Distributed System.” Communications of the ACM 21, no. 7 (1978).
• Bowker, Geoffrey C., and Susan Leigh Star. Sorting Things Out: Classification and Its Consequences. MIT Press, 1999.
• Regulation (EU) 2016/679 (General Data Protection Regulation), Article 17.

Research limitations

Some sources could not be consulted in full. Bachman’s 1973 Turing lecture sits behind a paywall, so its argument is paraphrased from its published abstract and secondary descriptions rather than quoted. Dates for the early IMS work differ between IBM’s own history pages and other accounts, and both are given. Accounts of the 1974 Codd and Bachman debate come mainly from participants and partisans, so no verdict on it is offered. The present-day survey figures describe one self-selected population of developers in 2025 and should not be read as market share. Interpretations in the section on philosophy, particularly the “pendulum” reading of the NoSQL era and the claim that the relational model survived because it promised nothing about hardware, are the author’s own synthesis of the evidence rather than claims made by the sources. The philosophical section draws parallels between database design and the work of Wittgenstein, Austin, Peirce, Putnam, Aristotle and others; except for Gray’s own contract and wedding analogies, the engineers involved did not cite these thinkers, and the parallels are the author’s interpretation offered to make the ideas understandable.

Footnotes

1. E. F. Codd, “A Relational Model of Data for Large Shared Data Banks,” Communications of the ACM 13, no. 6 (June 1970), p. 377: opening abstract, §1.1 (definition of data independence), §1.2 (ordering, indexing and access path dependence), the passage on the relational view’s “natural structure only”, the remark on the “connection trap”, and the statement that implementations “are not discussed.” ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸ ↩⁹

2. Jim Gray, “The Transaction Concept: Virtues and Limitations” (Tandem TR 81.3, June 1981; VLDB 1981), section “Update in Place: A poison apple?”, p. 7 of the report. ↩ ↩² ↩³ ↩⁴ ↩⁵

3. US Census Bureau, “The Hollerith Machine,” census.gov; see also “History and the Census: Herman Hollerith and Mechanical Tabulation” (January 2016). ↩

4. Charles W. Bachman, oral history interview by Thomas Haigh (Charles Babbage Institute, 2004), abstract, p. 2. ↩ ↩²

5. Computer History Museum, “Charlie Bachman,” computerhistory.org; release date 1964 for the GE 235 as given in standard references on the Integrated Data Store. ↩

6. Bachman oral history, transcript pp. 51–52 (no disk drive available during first programming; two tape drives used to simulate one; the database as virtual memory pages made of lines). ↩ ↩²

7. Bachman oral history, transcript pp. 64–65 (the backorder chain and the addition of a last-member pointer). ↩

8. IBM, “Information Management Systems,” ibm.com/history; dates for the start of work (1963 or 1965) and the commercial rename (1968 or 1969) differ between accounts. ↩ ↩²

9. Michael Stonebraker and Andrew Pavlo, “What Goes Around Comes Around… And Around…,” SIGMOD Record 53, no. 2 (June 2024), abstract and introduction, p. 21. ↩ ↩² ↩³ ↩⁴ ↩⁵

10. Charles W. Bachman, “The Programmer as Navigator,” Communications of the ACM 16, no. 11 (November 1973). Paraphrased from the published abstract, which frames the lecture around the 500th anniversary of Copernicus. ↩

11. Donald D. Chamberlin, “Early History of SQL,” IEEE Annals of the History of Computing 34, no. 4 (2012), p. 78. ↩ ↩² ↩³

12. The proceedings appeared as Proceedings of the 1974 ACM SIGFIDET Workshop on Data Description, Access and Control, vol. 2, Data Models: Data-Structure-Set versus Relational. For a participant’s recollection, see EnterpriseDB, “Those Who Forget the Past Are Doomed to Repeat It,” enterprisedb.com. ↩

13. Chamberlin, “Early History of SQL,” p. 79. ↩ ↩²

14. Chamberlin, “Early History of SQL,” p. 81. ↩ ↩² ↩³ ↩⁴

15. Chamberlin, “Early History of SQL,” p. 80. ↩ ↩²

16. P. Griffiths Selinger et al., “Access Path Selection in a Relational Database Management System,” Proceedings of the 1979 ACM SIGMOD International Conference on Management of Data. ↩

17. Gray, “The Transaction Concept,” Introduction, “What is a transaction?”, p. 1 of the report. ↩ ↩² ↩³

18. Theo Härder and Andreas Reuter, “Principles of Transaction-Oriented Database Recovery,” ACM Computing Surveys 15, no. 4 (1983): 287–317. ↩

19. Stonebraker and Pavlo (2024), §2.1 “MapReduce Systems” and §2.2 “Key/Value Stores,” pp. 22–23. ↩ ↩²

20. Giuseppe DeCandia et al., “Dynamo: Amazon’s Highly Available Key-Value Store,” Proceedings of the 21st ACM Symposium on Operating Systems Principles (October 2007), introduction; text as reproduced at allthingsdistributed.com. ↩ ↩² ↩³

21. Eric Brewer, “CAP Twelve Years Later: How the ‘Rules’ Have Changed,” Computer (IEEE Computer Society, 2012), republished at InfoQ. ↩ ↩² ↩³

22. Martin Fowler, “NosqlDefinition,” martinfowler.com. Accounts differ on who first suggested the hashtag. ↩

23. Stonebraker and Pavlo (2024), §2.3 “Document Databases,” p. 23. ↩

24. James C. Corbett et al., “Spanner: Google’s Globally-Distributed Database,” Proceedings of OSDI (2012), abstract; research.google.com/archive/spanner.html. ↩ ↩²

25. Stonebraker and Pavlo (2024), §2.3, p. 24. ↩ ↩²

26. Benoit Dageville et al., “The Snowflake Elastic Data Warehouse,” Proceedings of SIGMOD (2016). ↩

27. Michael Armbrust, Ali Ghodsi, Reynold Xin and Matei Zaharia, “Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics,” CIDR (2021), abstract. ↩

28. “The Origin Story of SQLite, the World’s Most Widely Used Database Software,” The New Stack, recounting Hipp’s 2021 interview on the CoRecursive podcast, thenewstack.io. ↩

29. SQLite, “Most Widely Deployed and Used Database Engine,” sqlite.org/mostdeployed.html. ↩

30. Stack Overflow, 2025 Developer Survey, Technology: Databases, all respondents (26,083 responses to this question), survey.stackoverflow.co/2025/technology. ↩

31. ISO/IEC 39075:2024, Information technology: Database languages: GQL (published April 2024); ISO/IEC 9075-16:2023, SQL Property Graph Queries (SQL/PGQ). ↩

32. Stonebraker and Pavlo (2024), §2.7 “Vector Databases,” pp. 25–26. ↩

33. Ludwig Wittgenstein, Tractatus Logico-Philosophicus (1921), propositions 1 and 1.1, trans. C. K. Ogden (1922). ↩

34. Raymond Reiter, “On Closed World Data Bases,” in H. Gallaire and J. Minker, eds., Logic and Data Bases (Plenum, 1978). ↩

35. David L. Parnas, “On the Criteria to Be Used in Decomposing Systems into Modules,” Communications of the ACM 15, no. 12 (1972). ↩

36. Hilary Putnam, “The Meaning of ‘Meaning’,” Minnesota Studies in the Philosophy of Science 7 (1975), section on the division of linguistic labour. ↩

37. Joel Spolsky, “The Law of Leaky Abstractions,” Joel on Software, 11 November 2002. ↩

38. J. L. Austin, How to Do Things with Words (Oxford University Press, 1962), Lecture I. ↩

39. Jim Gray, “Notes on Data Base Operating Systems,” in R. Bayer et al., eds., Operating Systems: An Advanced Course (Springer, 1978). ↩

40. Aristotle, Nicomachean Ethics, Book VI, chapter 2, quoting the poet Agathon. ↩

41. Regulation (EU) 2016/679 (General Data Protection Regulation), Article 17, “Right to erasure (‘right to be forgotten’)”, applicable from 25 May 2018. ↩

42. Charles S. Peirce, “How to Make Our Ideas Clear,” Popular Science Monthly 12 (January 1878), section IV. ↩

43. Leslie Lamport, “Time, Clocks, and the Ordering of Events in a Distributed System,” Communications of the ACM 21, no. 7 (1978); Lamport has described relativity as the inspiration in his commentary on his own papers. ↩

44. Geoffrey C. Bowker and Susan Leigh Star, Sorting Things Out: Classification and Its Consequences (MIT Press, 1999). ↩