When you first learn to program, memory seems pretty straightforward. You have a bunch of bytes, each with an address. If you want to read or write something, you specify the address, and the computer goes to that place and does your bidding. Sure, some addresses might be reserved for hardware or the operating system, but the general idea is simple: memory is a big array of bytes.
Just like we pretend that a wire is a perfect conductor, the big array of memory model used to be a slight twisting of the truth. On a modern computer, however, it is almost entirely a lie. The address you use probably isn’t a physical address. The data you read might not come from RAM at all. Even if it does, it might be sitting in one of several caches. And if you follow the rabbit hole far enough, you discover that your SSD is arguably just another level in the memory hierarchy.
The Need For Speed
The fundamental problem is that processors have gotten much faster than memory. A modern processor can execute several instructions per clock cycle, but accessing ordinary DRAM can take hundreds of cycles. If the processor had to wait for RAM every time it needed data, much of that impressive computing power would go to waste.
The solution is caching. Keep copies of frequently used data in smaller, faster memories closer to the processor. Most modern desktop processors have at least three levels of cache. The L1 cache is tiny but extremely fast, and often split between instructions and data. L2 is larger and somewhat slower, while L3 is even larger and usually slower as well.
