Paper · Distributed Systems

Spanner: Google's Globally-Distributed Database

J. C. Corbett et al. · OSDI · 2012

Global transactions made possible by carefully bounded clock uncertainty.

Summary

Bigtable-style systems scaled well but made applications deal with weak consistency and no cross-row transactions. Spanner set out to provide distributed transactions with external consistency, meaning the order of commits matches real time, across data centres worldwide.

The key is TrueTime, an API that returns the current time as an interval with bounded uncertainty, backed by GPS receivers and atomic clocks. Spanner assigns commit timestamps and waits out the uncertainty before making a commit visible. Data is sharded and replicated with Paxos groups, and two-phase commit runs across groups. Timestamps also enable lock-free, consistent snapshot reads.

Spanner showed that strong consistency at global scale is practical. It influenced systems such as CockroachDB and YugabyteDB, which approximate its approach without specialised clock hardware.

Key ideas

  • TrueTime exposes clock uncertainty as a bounded interval.
  • Commit wait turns bounded uncertainty into external consistency.
  • Paxos-replicated shards with two-phase commit across shards.
  • Timestamp-based, lock-free snapshot reads.

More Distributed Systems papers