Tác giả giữ toàn bộ bản quyền (All Rights Reserved) — không phải giấy phép mở. Bạn vẫn xem và học miễn phí trên OpenER; muốn tái sử dụng cần xin phép tác giả.
DAN GARCIA
Hey, welcome back. In this series of slides, we're going to take a look at what we understood the memory hierarchy to be, which before it was just I got registers in memory.
Are there more elements that we were hiding from you? And it turns out we were. So we're going to reveal some of the layers of the onion to you in this set of lectures.
So again, let's go back to our library analogy. I'm writing a report using the library books and going to the stacks. It's some English class.
I'm a freshman, and I've got to report the works of JD Salinger. So I could go to the library.
What I'd probably do — by the way, what I probably do is I go to the library, I look up the books, I fetch all of the books from the stacks.
I have a stack — you know, everybody sees that the picture of people walking back after they collect all the books they're supposed to get.
And then I'll find a disk in the library near enough that if I need to go back it's not too far. And then what I'll do is I'll open all the books.
And I'll have them all up on my table, and I'll be able to reference this book. And you've probably seen this.
You've probably seen your classmates have that, where you have the books all around the table, and you're kind you're writing a report on JD Salinger. Yep. I remember writing that last year.
Yep. Thanks so much. That kind of thing. You're having those conversations with your friends.
Now, if you need more, you go check them out. Now, you check them out until your disk kind of gets full.
And maybe they say you, nope, sorry, you can't stack the books or something. So maybe there is a limit.
So then you have to put some of those books back, and you have kind of a working set. In a way, you have a working set of books that are all laid out there.
If they don't let you, cause damage to the books, that means you can't stack the books up. So you have a certain fixed limit, a certain size on your disk.
But you're able to really save a lot of time. Imagine if I were to make much smaller desks, how much more painful that would be for you to only have one or two books.
And I have to keep running back and forth to the stack. So actually, we like the idea of having a really large desk. I can open more of these things.
I actually like the idea that they can let me stack them, but let's say I couldn't do that.
So this idea, this analogy, of maybe that the 10 books that I have on the desk are enough to cover most of what I need.
And in the rare case I need an 11th and 12th book, I'll be able to do that.
But for the most part, my working set of books that I had, working set of data that I have, is able to be very, very efficient.
And the time to reach it is — bloop, got it.
That's how fast it took me to grab something, rather than get up, go look it up again, go find it, walk the stacks, come back, get it, and bring it back, and do the same.
That would be painful. So this distance, the time — ready, 1, 2, 3, grabbing it. And that was about a second to grab the book and open it up.
That was great. We like that. We like the idea of having stuff that we're going to use often close to us, and that's a great analogy. Close to us.
And I say close to us physically, but close to us in time as well. It's great stuff. That's the idea. Memory caching is exactly the same idea.
That desk is in a way a local — it's a very-fast-to-access but smaller — certainly not smaller than all the books in the library — collection of the data that I'm going to be using.
And so, based on the idea that the CPU versus DRAM speed curve just got larger and larger — that was crazy — we don't want that.
We're going to introduce the idea of a memory cache. And this is, by the way, a big idea in computer science.
If you talk about computational thinking, and in fact I'm on some national committees to talk about computational thinking, my perspective, what computational thinking is, it's thinking like a computer scientist.
It's thinking of the world in a lens, in the glasses, that computer scientists all walk around with all the time. And caching is one of those big ideas.
I have it on my phone. You have a favorites list. You have a recently called list. Favorites is where you get to specify what's in your faster access.
But the recently called list, if it only shows 10, that is, in a way, recently called — it's exactly the idea of what we're talking about here.
It's like the recently used books that I had.
So typically, the way we implement — we're going to build these caches — are going to be done with the same IC technology, the same processing technology as the CPUs, and even sometimes on the same ship.
That's great. Same chip, really fast. Even physically, the farther you go from the CPU, from the registers, the farther you go physically in distance, the slower it's going to be to get there.
Probably the bigger you're going to have, but the slower it's going to be to get there.
Again, my hard drive, with a long cable across the room, I got a network of computers in the cloud. All those things farther way, larger, but really small and fast.
So you want to have the caches be on chip if you can. If you can do that, you get great speed.
The important idea but — boy, we put it in yellow here, and I can't emphasize it enough — a cache — and this is, by the way, where the library analogy falls down — I grab the single copy of the book, and now it's on my table, so it wasn't there.
But in digital versions, it's not like one version of those bits. So the cache is always a copy, a subset of what memory had.
Memory has, in a way, the reference copy, the reference values, and the cache is always a copy — it's always a copy.
In fact, for the first couple of lectures, we're going to talk about cache reads, where I'm only reading values. Only when you have storage do you have conversations of keeping the copy consistent.
So for now, just think they're always consistent until I tell you otherwise, and I'm only ever reading from a cache. Just for now, let's make it easy, okay?
Important. So say that to yourself — cache is a subset, and it's a copy of what was in memory. That's the important idea.
And by the way, this is also something we're going to see when I show you pictures of a CPU later, most processors actually separate caches for instructions and data.
You remember, in the place where code lived, instructions were down here and data was up here, right?
Whether the data is static, growing up from the heap, or going down from the stack, that still a different place than the instructions.
So rather than kind of corrupt them and put them all in one place, it actually makes a little bit more sense to have separate caches for only instructions, which are all the stuff down here, and the data, which could be a large group — data could come from a large range of memory addresses, where the instructions might be coming from a smaller.
But separating them actually keeps it more clean. It's kind of interesting. So think about having two caches, at least an instruction and a data cache. Pretty cool.
This is the picture. This is the picture of memory hierarchy that you've seen up till now. You've got a CPU. The core is there.
The smallest, fastest, and most expensive storage there are registers. They are amazingly fast, blistering speeds, but they're really small — not that many of them.
And they're really close to the CPU, so I can get to them in easily one clock cycle — fractions usually have a clock cycle. Farther away and down in this triangle is physical memory.
The DRAM chip — when you say, I want to upgrade the memory, you're talking about that. The DRAM is the Dynamic RAM, and dynamic means you have to refresh it.
Dynamic means that you kind of have to actually keep pinging it to — hey, remember that it was a 1, remember it was a 0. You got to keep refreshing it.
So that's why it's called dynamic RAM, versus static RAM where you don't do that. And reasonably fast, but price more reasonably than, boy, every bit for registers cost a lot.
Those are farther away. I go to Fry's Electronics — if that still exists anymore — and I buy up some DIMMs, and I plug them in — Dual Imprint Memory Module.
So you could have a larger space. Because they're external, I can now swap them in and out, which is great. That's nice.
Sometimes they're burned on the board and you can't move them, but most of the time you can — most desktop computers you can swap them in them out and increase.
By the way, if you want to make your computer run faster — you're going to learn this when you learn about VM, Virtual Memory — have more memory.
So talking about swapping it in and out. Ready for this? Revealing the layer of the onion — the layer the onion! We're going to put clashes in the middle.
And they're typically going to be a different process.
They're not going to be — sometimes they can be on the CPU, which is great — on the same core, on the same chip as the CPU.
They are, as they've shown of this triangle — it's a beautiful triangle, and it's consistent.
And by the way, if ever you come up with a technology that is kind out of place, the market will sort it out.
Meaning, well, what if I had this — if this could be bigger than memory? What if this could actually be bigger than memory without slowing it down?
Well, then it would move down there and memory would be above it. The market will move things around to automate that. It's really interesting.
So it is — the three parameters of this — faster than memory but slower than registers, It is more expensive than memory but less expensive than registers — same thing — and it's larger.
It's smaller — remember the triangle — than memory but larger than the registers above it. So those three parameters, it fits right in the middle of all those three parameters.
Jim Gray was a Berkeley alumnus. He got his bachelor's and PhD at Berkeley.
He came up with this beautiful analogy, thinking about, if you had a fact or a piece of paper with a secret thing on it, like a password or something, and you need to get it, how long would it take?
And making analogy — so again, that was the library analogy.
Now, what if you actually had the same idea but in terms of relative speed to how long it would take to get to something.
So registers, the model that you've seen before, is registers I can get to as fast as I can — about a minute.
If you say a register is a minute, even though it's like one nanosecond, one clock cycle for 1 gigahertz clock is 1 nanosecond.
Imagine if the analogy in a physical piece of paper is about a minute to get something. I'm thinking, oh, what was that number again? Oh, I got it.
So that's about a minute.
Now, memory, the equivalent time for how long it would take you to get something from memory, we're going to talk about something in the hundreds to maybe even 1,000 times that speed.
Usually it's a several hundreds times that. So what's several hundred minutes? Well, we're talking about a trip to Sacramento.
So that means you left a piece of paper — how annoying would that be — in Sacramento, and you got to go grab that. That would be just terrible.
And maybe that's a book you're talking about. Maybe that's a piece of paper with a password on it.
Caches are in the middle, right — in the middle of that triangle in the memory hierarchy. So the analogy there is it's in this room somewhere. Oh, it's in the room?
Okay, I'll find it. Did I leave it under that? No, it's not under. Under the plate?
No, it's no there. But searching the room isn't too bad. It's about two minutes. So we love caches.
They are bigger than registers. Remember registers, there's on;y a few of them. Now, they're bigger than registers, but they're about the same speed as registers.
They're not that far away in terms of distance, at least the closest cache. We're going to learn there's more than one cache.
But for now, that's pretty cool to be able to have something that's much larger than registers but just about as fast as them.
A little slower, but not that bad — maybe one or two clock cycles. That's great. So we love this.
We love this analogy, and we're going to actually see there are things way above Sacramento as we try to go to disk. How far is disk away? You're going to see.
You'll be surprised at how far disk is away in this analogy in terms of how long it takes for how many clock cycles would it take to go to disk.
So again, I think I mentioned before in my other memory hierarchy, as we look at this triangle, here are some of the parameters about this. I think I might have said this.
As you get farther away from the processor, you end up having increased access time. So you're going to be slower. It's just slower to go farther away from the CPU, okay?
And what I mean is, it's not just like it's the same speed but father kind of, like in the library analogy, just farther distance.
No, you're going to a different technology, so it actually kind of is like walking through quicksand, and then walking through quicksand with a big weight behind you, and then walking — so it gets slower and slower.
The farther you go you get to different processes, and you end up having different speeds as you get there. And that's just the same speed but farther.
Also, the relative size of memory at each level gets larger. So as I move away from the processor, I get bigger and bigger. So I have caches smaller than memory.
We're talking about memory and secondary memory here. So cache memory, secondary memory, which might be disks. So secondary memory we often call disks, which might be flash-based or might be spinning cylinders.
However we decide, that what we call secondary memory. And if we include that in this conversation, the farther you get away from the CPU, it's still consistent. It is slower access time.
Larger, but, we didn't mention here, it's cheaper. The other factor here, it's much cheaper the farther you go away. So the price for a single bit on disk, man, nothing.
Price for a single bit in memory, still pretty good. Price for a single bit in cache, whoa. Price for a single bit on the register, boy, that's pretty expensive.
I mean, even in today's dollars, that's pretty expensive, thinking about what a single bit costs relative to a single bit on a disk drive. Interesting. Okay.
The other important thing is always it's the inclusivity. It's always a subset. The smaller is always a copy of the larger.
And every single layer, from the processor working on something right now, it's computing something right now to the registers, which has more sent.
But if they're actually processors, I had some data flowing in the data path, I'm working on two numbers — okay, that's — okay, smaller.
And a copy of what was in the registers, a copy of what's in my cache, a copy what's in memory, and a copy, in some sense, of what's in disk.
Remember that the loader does? The loader grabs — remember that — from disk and loads into memory. Memory is then a copy of what was on disk. Same idea.
Pretty cool, right? Okay.
So the trick — here's the whole beauty. This a little longer of a video, but this is the beauty of this idea. The beauty is this abstraction.
Abstraction is the key idea in this whole course. It's the abstraction that you're living at — forget the cost for a moment, okay?
But you're living at the speed of the smaller guy but at the size of the bigger guy. That's it. That's the big idea.
It's like having — imagine having the disk available to you at the speed of registers. That's the idea.
And the idea of cache is, is what if you had all of main memory — which is big — but at the speed of a process that's in some sense the registers, but something faster than that?
It's really the speed of the cache. But what if you had access to all the memory at the speed of the cache? It's not registers, but it's pretty close.
That's the idea. So this whole, beautiful idea, look at the numbers here. So the speed in seconds, I want to circle some of these things.
So how fast is it to get to the reg file? Half of our clock cycle. Well, how about to get to the caches?
About 1's — in the single digits of clock cycles to get to the cache. Well, how about main memory? We talked about this. About 1,000-ish.
That's Sacramento. How about disk? Boy, you don't even want to know how far away disk is, okay? And this is a size.
The highest is the highest cost in the small guy and the lowest cost here. And the size is roughly like that [INAUDIBLE].. Well, how big do you mean this triangle?
What's the width of that triangle? In the order of 100. So they had 32 registers, something small. In the order of like 10,000-ish for caches, memory, you know what size memory is.
Memory is in the GiB. Usually it's some several GiB. And disk is typically the TiB. So it's interesting to see what the width is typically for computers and technology today.
And that number is changing every year. So those are the on-chip components. The things that are on chip are the cache. That's the idea.
So some part of the cache is on chip, and that's just great.
So summary of the whole idea — if the level is closer to the process, the higher up you get in that triangle, the memory hierarchy, it is smaller, faster, and more expensive per bit.
That's clear. And also, the important thing is a subset of the lower levels. I think I said that four times now.
I want to make sure that you don't get that question wrong on the exam — it's always a subset of copy of the lower levels.
And the idea of the memory hierarchy is the illusion — it's the illusion.
Woody Allen has a line that says, I put some backlights — I had backlights here in my studio — behind me to give me to give me the illusion of three dimensions.
So same idea. The idea here is, in this hierarchy I have the illusion of the size of the lower levels at the speed of the upper levels. That's it.
That's the big idea. We'll talk about how to actually make this work in the next lectures. We'll see you there.
Đăng nhập để đánh giá học liệu này.
Đăng nhập để bình luận — chưa ai bình luận về học liệu này.
Không có khóa học liên quan.