The memory hierarchy is the arrangement of a computer’s storage into layers, ordered from fastest and smallest at the top to slowest and largest at the bottom. A typical hierarchy runs from processor registers, to one or more levels of cache, to main memory (RAM), and down to disk or solid-state storage. Each step down is larger and cheaper per byte but slower to reach. The goal is to keep the data the processor is actively using in the fast upper layers while parking everything else in the cheap lower ones.
This layering exists because no single memory technology is simultaneously fast, large, and cheap. Fast memory such as SRAM is expensive and small; dense memory such as DRAM is cheaper but slower; disk is enormous and cheap but slower still by orders of magnitude. Rather than choose one, designers stack them, so that the system behaves as if it had memory nearly as fast as the top layer and nearly as large as the bottom one.
Hennessy and Patterson’s textbook “Computer Architecture: A Quantitative Approach” treats the memory hierarchy as one of the foundational topics of the field, devoting its chapter on Memory Hierarchy Design and a review appendix to how the levels interact. The quantitative framing matters: the textbook teaches designers to reason about average access time as a weighted blend of hit times and miss penalties across the levels, rather than treating any one memory as a single fixed speed.
The hierarchy only delivers its promised speed because programs exhibit locality of reference. If accesses were truly random across all of memory, the small fast layers would almost never hold the right data and the whole structure would collapse to the speed of its slowest layer. Because real programs reuse data and touch nearby addresses, each layer can capture most of the requests aimed at the layer below it.
The same principle that builds a cache between the CPU and RAM repeats at every boundary. The cache is a fast layer over main memory; virtual memory uses main memory as a fast layer over disk; and even disk controllers and operating systems buffer data in RAM. The memory hierarchy is therefore not a single mechanism but a recurring design pattern, applied wherever a fast, scarce resource must stand in for a slow, abundant one.
The idea traces back at least to Maurice Wilkes’s 1965 slave-memory paper, which proposed pairing a fast store with a slow one so the effective access time stayed near the fast memory. That two-level scheme is the seed from which the modern multi-level hierarchy grew.