Video
Đăng bởi Quản trị viên OpenER ·
Tác giả giữ toàn bộ bản quyền (All Rights Reserved) — không phải giấy phép mở. Bạn vẫn xem và học miễn phí trên OpenER; muốn tái sử dụng cần xin phép tác giả.
[MUSIC PLAYING] PROFESSOR: Hello, and welcome back to our Dependability module.
The main theme of the module is that we would like to use redundancy to improve dependability.
But remember, in every computer system, we need to measure something to know whether and how much have we improved it. So let's take a look at what are our dependability metrics.
So let's first understand how does a compute system operate. So in normal operation, it runs some services for us.
But it may happen that it will fail. So the fault — and this is very different than the page fault — is a failure of a component in a compute system.
That component may or may not lead to a system failure, depending whether it's being used or not and whether it is redundant. And there is something else that can take over, for example.
So the system will operate essentially between the two states. In one state, It operates normally so the services are accomplished. In the other state, there is a service interruption.
Moving from the normal operation to the service interruption is a failure.
And going back from the system interruption, that's a deviation from the specified service back to the normal operation, is the restoration.
What will matter here is, how frequently does this happen, and how long does it take us to get back?
When we talk about redundancy, we are going to see this use of redundancy in time and in space.
When we talk about spatial redundancy, that means that we have multiple copies of something. We may have multiple copies of compute units or, more frequently, we're going to have multiple copies of data.
Multiple copies of data may exist in DRAM or in disks.
In DRAM, more commonly, we are going to have some kind of an algebraic way of adding redundant bits such that we can recover from temporary errors.
In hard disks, we are often keeping redundant devices that are going to help us recover from any kind of a failure.
So replicas of data are essentially spatial redundancy. The other form of redundancy is temporal redundancy.
And temporal redundancy, essentially, what does that mean is that if there is a failure, if there is a temporary failure of a compute system and we have time to repeat the computation, we may as well do that.
So if we detect that the computation has failed or the service is not completed, we go again.
And if we complete that and we recover without any consequences, we are all good. So that is an example of temporal redundancy.
Let's take a look at some of these dependability measures.
How do we know, what kind of a yardstick do we use to measure dependability? So the first one and most commonly used one is the Mean Time To Failure or, for short, MTTF.
That basically measures, how long does it take us for a device to fail? Or between the two failures of the same device, if they are recoverable.
The other metric is the service interruption. And that service interruption is essentially, how long do we stay in that interrupted state? That is the mean time to repair.
Mean time between the failures is the sum of mean time to failure and mean time to repair. And then we can define something that is also very important, which is availability of a system.
Availability of a system is simply mean time to failure divided by the sum of the mean time to failure and the mean time to repair.
How can we improve availability of our system? Well, we can either work on increasing the numerator or decreasing the denominator.
What does it mean to increase mean time to failure? So the components are going to stay up for a longer time.
Usually, we are going to use higher quality, more reliable hardware and software, and we are generally going to have some kind of a redundancy there.
In addition to that, we will be adding some kind of a fault tolerance such that if a component fails, our system does not crash.
The other approach in reducing the denominator is to reduce the MTTR, Mean Time To Repair.
Basically, it's about improving the tools that have detected that something is not working quite right in the system and bring it back to the normal mode operation.
So that's about dependability measures. The next thing that we would like to define are these availability measures, sort of availability metrics.
Availability, as we have defined it already, is the ratio of time to failure divided by the sum of mean time to failure and the mean time to repair.
But it is generally expressed as a percentage of time. And both mean time to failure and mean time between the failures are measured in hours.
Now, compute systems, modern computer systems rarely go down — at least the systems that we are used to — rarely go down.
So we generally assume that it is going to be up by a very large percentage of time, and that large percentage of time meaning 99.99-some percent of a time.
That number of 9's is generally measured as number of 9's availability per year. What does that mean?
So if something is available 90% of a time, in one year, that means it'll be down for repairs for 36 days.
If it is available 99% of a time, that means that it would be down for 3.6 days. It would have 3.6 days of repair per year. That's very annoying.
Very few people would tolerate something like that.
Although, in some other systems, transportation, that may be okay. That's how many days do buses that operate every day, spend in a shop per year.
Three 9's means that the system would be down 526 minutes for repairs per year. That's annoying, but getting closer to being acceptable. Four 9's is 53 minutes of repairs per year.
And five 9's is 5 minutes of repair per year. That is getting closer to what we expect.
And that is something that we often encounter in these services that we actually use on the internet. I mean, of the order four to five 9's is the availability of, say, YouTube.
As you have noticed when watching these videos, YouTube sometimes goes down.
Other systems are designed to have six or seven 9's, things that we depend much more on, and there is essentially a much higher price for that system going down.
Examples are the losses of YouTube when YouTube went down for a few hours some years ago. And that is basically losses of millions per minute.
Okay, so that's about availability measures. Let's talk about reliability measures.
Generally, how we like to measure differently, this reliability is through Annualized Failure Rate, or AFR.
So how many failures do we have in a system that has many components per year?
For example, let's say we operate a data center that has 1,000 disks, and each disk is characterized to have 100,000 hour in mean time to failure.
So let's see. What does that mean? How many disks actually are going to fail per year in this kind of a system?
So how do we translate one reliability measure of MTTR to another reliability measure, which is annualized failure rate?
This is something that is very important for data centers, and data centers actually keep track of that.
So we have 8,760 hours in a year, and then we have 1,000 disks that are operating for the whole year.
Our mean time to failure is expressed in hours, so we are going to divide that number with 100,000 hours.
And what we end up with here is 87.6 failed disks per year on the average.
So we have to round that up because it's not 0.6 disks that generally fails, so it's 88 failed disks.
This translates to 8.8% annualized failure rate. Well, that's not that crazy.
That number actually is close to what we see, actually, in practice.
Google published a study in 2007 that found that the actual AFRs of individual drives range from 1.7% in the first year and as the age to go to 8.6% for three-year-old drives.
Kind of interesting thing to see. There are interesting websites.
One of these online backup services, Backblaze, actually publishes the failure rates of their drives.
And that may be useful information that you can take a look at when you would like to buy a new drive. And they're publishing these AFRs.
And they turn out to be pretty good for modern drives.
They are either single-digit or below-single-digit percentages, translating, essentially, to about a million hours of the order, or a million hours between the failures.
And another piece of good news — I mentioned this before — that these drives actually are getting better. They have less failures over time.
Slowly but surely, the AFR is decreasing. There is a decreasing trend over here.
There is another reliability measure that is out there that is used. This was used for different kinds of systems, but computer systems have to adhere for that.
It is Failures In Time, or FIT rate.
That has been used, for example, for cars.
And Failures In Time, or FIT rate of a device is the number of failures that can be expected in one billion device hours of operation.
That sounds like a lot. I mean, one billion hours is something that sounds unimaginable. But imagine that we have many of these devices operating at the same time.
So that means 1,000 devices for one million hours or one million devices for 1,000 hours each would end up failing.
Or one billion devices, which is approximately how many smartphones we have out there, would be failing every hour. And that sounds about right.
I mean, if you are looking at the whole world, I think the failure rate of cell phones out there is probably greater. There is more than one cell phone that fails per hour.
So MTBF is equal to 1 billion times 1 over the FIT rate.
This is relevant because computers are getting more and more into our cars, and all the specifications for reliability there are expressed in FIT rates.
So you'll see that, especially if you get into that kind of business, you're going to hear about this Automotive Safety Integrity Level, ASIL, that defines different fit rates for different classes of components in vehicles.
Okay. So design principles of dependability are as follows.
When we are designing a system that we should depend on, there should be no single point of failure.
And the knowledge of that, in common terms, is that the chain is only as strong as its weakest link. There is a pretty good corollary here of Amdahl's law.
It's always going to be the longest pole in the tent that is going to be sticking out.
So no matter — we can keep improving the dependability of one component, but the one that is the least dependable is going to dominate.
So we generally have to be measuring that for the whole system and make sure that the least reliable component, or least dependable component, is dependable enough.
All right, so we are going to make a quick break here. And then we are going to take a look into the ways how do we actually detect these faults or failures.
See you after the break.
Đăng nhập để đánh giá học liệu này.
Đăng nhập để bình luận — chưa ai bình luận về học liệu này.
Không có khóa học liên quan.