Bigtable: What the Paper Means in Production

The paper in brief

"Bigtable: A Distributed Storage System for Structured Data" by Fay Chang, Jeffrey Dean, Sanjay Ghemawat and colleagues appeared at OSDI in 2006. It presents the storage system behind many Google services at that time. Bigtable is described as a sparse, distributed, persistent, multi-dimensional sorted map. Each value is indexed by a row key, a column key and a timestamp.

Key ideas

Rows sorted lexicographically. Data is ordered by row key, and reads and writes under a single row key are atomic. Row ranges are split into tablets, which are the unit of distribution and load balancing. Because rows are sorted, clients can choose keys so that related data sits together, for instance by reversing hostnames.

Column families. Columns are grouped into families, which are the unit of access control and configuration. Within a family, individual columns can be added freely, which makes the table sparse.

Timestamped versions. Each cell can hold multiple versions, with garbage collection rules such as keeping the last few versions.

Built from other systems. Bigtable stores its data in the SSTable file format on the Google File System. A tablet server keeps recent writes in an in-memory memtable, backed by a commit log, and periodically flushes to immutable SSTables, which are merged by compactions. Chubby is used to elect a master, track tablet servers and store schema information. This layering is a large part of the paper's value, because it shows how a storage system is composed from a file system and a lock service.

What breaks in practice

The data model is deliberately limited. There are no secondary indexes and no multi-row transactions in the paper's design; only single-row atomicity is offered. Teams that needed cross-row consistency had to build it themselves, and the Percolator system and later Spanner were built in part to address this.

Row key design is the whole game. A sequential key such as a timestamp prefix sends all new writes to one tablet, which becomes a hot spot while the rest of the cluster idles. Practitioners learn to design keys that spread load while still supporting the range scans they need. This is a data modeling problem that the system cannot solve for you.

Dependencies become failure modes. Bigtable relies on GFS and Chubby, and the paper reports that Chubby unavailability can make Bigtable unavailable. A storage layer is only as available as the services beneath it, and that coupling has to be understood before promising availability targets.

Compaction is background work with foreground effects. Merging SSTables consumes disk and network, and read performance depends on how many files must be consulted. Tuning compaction is a normal operational chore in any system of this family.

Modern descendants

Apache HBase is the open-source system closest to the paper, built on HDFS with ZooKeeper in the role Chubby played. Apache Cassandra borrows the Bigtable data model and storage engine design, combined with Dynamo-style distribution. The log-structured merge tree approach of memtable plus immutable sorted files is now found in many databases and embedded engines such as LevelDB and RocksDB. Google offers Cloud Bigtable as a managed service built on the same lineage.

If you work with any wide-column or LSM-based store, the paper explains why writes are fast, why reads can slow down, and why the shape of your keys matters more than the number of your nodes.

Key takeaways

• Bigtable is a sorted, sparse, versioned map, not a relational database.
• Only single-row operations are atomic, so cross-row consistency is the application's job.
• Row key design determines whether load spreads evenly or piles onto one tablet.
• The memtable, commit log and SSTable design is the root of today's LSM-tree databases.
• Reliance on GFS and Chubby shows how dependency chains limit availability.