Paper · Transactions & Recovery

OLTP Through the Looking Glass, and What We Found There

S. Harizopoulos, D. J. Abadi, S. Madden, M. Stonebraker · SIGMOD · 2008

Where does the time actually go in a transaction? A surprising breakdown of overheads.

Summary

Traditional OLTP engines were designed when databases were much larger than memory. As main memory grew, many OLTP databases could fit entirely in RAM, which raised a question: where does a transaction's time actually go once disk I/O is no longer the bottleneck?

The authors instrument the Shore storage manager running a subset of the TPC-C workload and strip out components one at a time. They find that only a small fraction of instructions do useful work. Most of the time goes to buffer management, locking, latching and logging.

The result motivated a generation of main-memory OLTP systems, including H-Store and VoltDB, that remove these components through single-threaded partitions and other design changes. It is a clear lesson in measuring before optimising.

Key ideas

  • Systematically removing components to measure their cost.
  • Buffer management, locking, latching and logging dominate in-memory OLTP.
  • Useful work is a small share of total instructions.
  • Motivation for main-memory, partitioned OLTP engine designs.

More Transactions & Recovery papers