Video
Đăng bởi Ban biên tập OpenER ·
Tác giả giữ toàn bộ bản quyền (All Rights Reserved) — không phải giấy phép mở. Bạn vẫn xem và học miễn phí trên OpenER; muốn tái sử dụng cần xin phép tác giả.
And welcome back. This is our final lecture on the parallelism topic, and we're going to be talking about data centers and cloud computing. How do you actually run that MapReduce and Spark job?
Well, on a data center or a cloud computing model. So let's think about that. We'll first start with a little history, the eras of computer hardware. So let's first start with a review.
Where are we? Six great ideas of computer architecture. We've hit the layers of representation and interpretation.
C is an abstraction above assembly level, which is an abstraction above machine code, which is an abstraction above the hardware that takes to run it.
Each particular piece of hardware is an abstraction about how that's built, all the way down to the gate level, all the way down to the abstraction above electrons moving around and filling holes.
Pretty neat. We've seen Moore's law. Moore's law has been what has fueled the growth of the computer manufacturing industry for many years, for decades.
It's a trillion-dollar industry. It's very exciting. We've seen that Moore's law is continuing unabated so far, fingers crossed here, but that some things are changing.
In 2005, we saw a flattening of the curve of clock frequency and the single thread performance.
But then a resilience of the industry was we're going to go multicore, and so you saw the number of cores per machine continue to increase.
We've seen the principle of locality memory hierarchy, caches, virtual memory, disk, tape — cloud computing is even on that conversation somewhere down in the lower levels as well.
All right? Very interesting. And we've seen some parallelism. We've seen thread-level parallelism, data-level parallelism.
There was five different layers. We've seen all five of those layers. One of the big ones is data-level parallelism.
How do we actually work at a level — how does Spark and MapReduce actually work? Well, it works on a cloud system. But let's — still going back.
In the 1950s and '60s, before I was born, these were the machines people were using to compute. They didn't exist everywhere. They existed in big companies and at big research labs and big universities.
You might have one computer that the whole campus used back in the early, early days. If you look at this, it's pretty incredible. That's the CPU. And that's a person, by the way.
Just so you know, for scale here, these are people, okay? They might be actors and they might play one on TV.
But in terms of scale, it was several size of a human for the central processing unit. And you see all these other kind of person-sized boxes. What are they?
All I/O, all Input/Output.
Magnetic tape reading, control console, card punch system, card reader, control console, high-speed printers, disk storage, tape controller, magnetic tape units, all — I mean, just incredible how big things were.
If you think about all the engineering that has gone to taking that and making that as small as today's laptop, today's cell phones, today's watches, it's incredible what we've done.
IBM was leading this. IBM, UNIVAC, Digital, they were companies that put together $1 million computers business. COBOL was being used as the language, FORTRAN as another programming language, and a timesharing OS.
Remember, timesharing a little slice of time to everybody — OS for all their users.
In the '70s — when I was kind of coming up, I was still young at my time — but they had Digital, HP, Prime were out there, IBM was still out there as well, putting out bigger computers.
These are about $10,000 computers, so they weren't million-dollar computers. These were labs, universities. They were running C and Unix. So things that you're doing, they were doing in the '70s.
It's pretty nice. Then the '80s hit. Mid-'80s to the mid-2000s was the PC era. And there was a huge boom for individuals owning computers.
Pretty nice, that was exciting. I bought my first computer in college. That was really fun — which I'd had one when I was in high school.
And many of you have had it in high school as well. That says about a $1,000 computer. I've noticed that computers cost about 2 and 1/2 thousand thousand continually.
So that's kind of a funny thing that they've been — that's not the same price forever. [LAUGHS] My first computer was like — and this in back in 1980s dollars — $2,500.
It's still $2,500-ish, $2,000 to $3,000. Now, you can get cheaper ones like that. But a higher-end computer, that's what you're paying now.
For one person, you're running BASIC, you're running Java, you're running Windows OS, Mac OS. Nice. We call the next phase is the Post-PC era. This is the late 2000s.
This was the era of a smart device, a personal mobile device relying on wireless networking to make it work. Apple, Nokia, Google made lots of these, about $500 smartphones.
Although, the higher ones now are above $1,000.
And the programming languages we use to program those, Objective, C, Swift, Java, Android, OS, and iOS with the underlying operating systems — served by — drum roll, please — cloud computing.
So the idea of local area networks, Amazon, Google, Facebook, they built massive warehouse-scale computers with 100,000 servers in them serving the data on your personal mobile devices.
So if I go to my watch — let's try it here. What's the weather today? Let's see here how long it takes. Come on.
Give me some love. AUTOMATED VOICE: It's currently partly cloudy and 61 degrees. DAN GARCIA: Okay, that's pretty amazing. How did that happen?
Personal mobile device, the cloud was serving that data. Pretty incredible. How do you program those devices? MapReduce and Ruby on Rails.
So that's the first part of this lecture, is a little history lesson. And the next lecture, we'll talk about warehouse-scale computing. We'll see you there.
Đăng nhập để đánh giá học liệu này.
Đăng nhập để bình luận — chưa ai bình luận về học liệu này.
Không có khóa học liên quan.