Paper · Storage & Indexing

The Design and Implementation of a Log-Structured File System

M. Rosenblum, J. K. Ousterhout · ACM Transactions on Computer Systems · 1992

Treating the disk as an append-only log. An idea that shaped storage design for decades.

Summary

The authors observed that growing memory sizes meant file caches would absorb most reads, so disk traffic would be dominated by writes. Traditional file systems scatter small writes across the disk, which wastes most of the disk's bandwidth on seeks.

Sprite LFS writes all changes, including data and metadata, sequentially to an append-only log. Inodes are located through an inode map, and a segment cleaner reclaims space by copying live data out of fragmented segments. The paper studies cleaning policies in detail, including a cost-benefit policy that considers how old and how full a segment is.

The ideas reach well beyond file systems. Log-structured storage, garbage collection of segments and the cost of cleaning appear again in LSM-trees, flash translation layers in SSDs, and log-structured key-value stores.

Key ideas

  • Write all updates sequentially to an append-only log.
  • An inode map locates the latest version of each file's metadata.
  • A segment cleaner copies live data to reclaim space.
  • Cleaning policy strongly affects performance.

More Storage & Indexing papers