New hard drive reliability data from Backblaze

[leenux_tux]

Folks,

Some interesting information from Backblaze regarding hard drive reliability.

https://www.backblaze.com/blog/hard-drive-reliability-update-september-2014/

Some of the those numbers on the chart look a bit scary !

 

[cyberjock]

I haven't read that link yet, but they were flamed into oblivion with their last reliability report. So I'm not going to hold my breath on this one....

 

[Sir.Robin]

I'm not a Seagate fanboi... but most storage solutions i have come across deploy Seagate drives.

I have 144x 10k 600GB Seagates in my primary SAN. 3 of them has died within the first year.

In one of my secondary SAN's i have 48x 4TB SATA Seagates. 2 of these also died within the first year.

I have an aggregate of 70x 1TB SATA's from 2009'ish... NOONE has died so far.

In my FreeNAS'es i have 6x ES.2 1TB's... all are soon rounding 40.000 hours runtime. No deaths.

In my secondary FreeNAS i have 6x ES 750GB... running over 50.000 hours and steadily between 45-55 Celcius.
1 died yesterday.

I won't stop buying Seagates... nor WD's for that matter. :)

EDIT: Oh i forgot... i had a XYRATEX SOLUTION in production with the Seagate 750GB's... in that SAN they died like flies...

 

[Ericloewe]

The only drives I go out of my way to avoid are the Deathstars. Yes, they have little in common with the true Deathstars, but I firmly believe the name is cursed for all eternity and those who evoke their marketing-given name will suffer eternal damnation in the form of being forced to admin a datacenter containing only Deathstars (the genuine ones).

 

[Z300M]

The only drives I go out of my way to avoid are the Deathstars. Yes, they have little in common with the true Deathstars, but I firmly believe the name is cursed for all eternity and those who evoke their marketing-given name will suffer eternal damnation in the form of being forced to admin a datacenter containing only Deathstars (the genuine ones).

By "Deathstar" I assume you mean the Deskstar (IIRC, originally IBM, then Fujitsu[?], and now HGST, which belongs to WD). Admittedly the number of responders is smaller, but the HGST 4TB NAS drives have a very good feedback rating on NewEgg: all 4-star and 5-star reviews, whereas the corresponding Seagate has a significant number of 3-star, 2-star, and even 1-star feedback scores.

Edit: I now see that 28% of the NewEgg feedback scores for the 4TB WD NAS drive are only one star -- even worse than the 4TB Seagate. The 2TB WD NAS drives fare better than the 4TB ones.

 

[leenux_tux]

I have been trying to keep costs down by purchasing second hand drives from ebay.

The oldest drives I have (Samsung 1TB X 4) have been the most reliable so far. They used to be in my old FreeNAS system(s) and have been moved twice (kept the same install of FreeNAS, new server cases as my storage demands have grown) with no issues. I am however having SMART errors on two of my ten WD Greens, two new drive are in the post (ebay of course!). One of the errors is for spin up time, the other for current pending sector count. The spin up time issue has been there for over a year now. I have not been overly concerned about those. the pending sector count I AM concerned about, hence new drives on the way.

 

[jgreco]

In an incredibly unscientific and extremely small sample size, I will note that one of my four drive 1U chassis has four ST3000DM001's in it, shucked out of external USB's. They've got about 16000 hours on them. Of those, one has failed and been replaced, and another has been reporting a slightly varying number of errors for a few months...

Code:

Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65512 Offline uncorrectable sectors
Device: /dev/ada3, 65512 Offline uncorrectable sectors
Device: /dev/ada3, 65520 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65520 Offline uncorrectable sectors
Device: /dev/ada3, 65520 Offline uncorrectable sectors
Device: /dev/ada3, 65520 Offline uncorrectable sectors
Device: /dev/ada3, 65520 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65520 Offline uncorrectable sectors
Device: /dev/ada3, 65520 Offline uncorrectable sectors
Device: /dev/ada3, 65512 Offline uncorrectable sectors
Device: /dev/ada3, 65512 Offline uncorrectable sectors
Device: /dev/ada3, 65520 Offline uncorrectable sectors
Device: /dev/ada3, 65520 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65528 Offline uncorrectable sectors
Device: /dev/ada3, 65520 Offline uncorrectable sectors

I was like ... "wot?" when I saw the number *decrease* a few weeks back. Looks like it decreased again in the most recent report.

I'm kinda trying to hold off buying a replacement, because I'd really love to have 8's in there. Though I suppose I could do 6's and live with it. The pool has just finally passed 80% utilization

But anyways that's like a 50% problem rate on those 3TB drives. Ewww.

 

[jgreco]

By "Deathstar" I assume you mean the Deskstar (IIRC, originally IBM, then Fujitsu[?], and now HGST, which belongs to WD). Admittedly the number of responders is smaller, but the HGST 4TB NAS drives have a very good feedback rating on NewEgg: all 4-star and 5-star reviews, whereas the corresponding Seagate has a significant number of 3-star, 2-star, and even 1-star feedback scores.

No, Fuji was acquired by Toshy. I've got a pile of MAW3300NC's that I've been pounding on for 9 years with only very modest failure rates. Once they get past about two years they seem to live nearly forever.

Much of my work is done in support of clients (or my own businesses) running stuff in data centers.

I once had a client quip that if I wasn't located so far away from Ashburn VA, they could probably employ me full time to roll up and down the aisles all day long replacing failed drives. There was definitely a bunch of Deathstar in there but also a pile of rotting fish eggs IIRC.

 

[cyberjock]

Decreasing is not uncommon, but normally once a drive starts going bad it increases so rapidly from one SMART check to the next that it you usually don't notice it. Most hard drives are limited to 65k entries (minus some space for indexes and stuff) so your drive might be MUCH MUCH worse than you think it is. That disk is effectively useless though. :P

 

[jgreco]

Decreasing is not uncommon, but normally once a drive starts going bad it increases so rapidly from one SMART check to the next that it you usually don't notice it. Most hard drives are limited to 65k entries (minus some space for indexes and stuff) so your drive might be MUCH MUCH worse than you think it is. That disk is effectively useless though. :p

No, it's not. Bear in mind you're talking to someone who designed one of the early major SATA storage appliance strategies... scaled up to many thousands of drives. I'd been too lazy/busy/wotevr to go see if I can coerce a Seagate desktop drive into doing a selective self-test to get past the bad blocks, but I know they're pretty far out there in LBAland, which when taken with other symptoms isn't consistent with imminent failure but rather more of media defect or head crash or something. Opinion of someone who's seen /lots/ of drive failures.

Related: I'm in the process of modernizing our old disk array test script for possible posting here, it's pretty handy for burning in if you're like me and too lazy to babysit things. I've actually been testing it on the first part of that fail-y disk (plus the other three) without problems.

Speaking of which, let me PM you a copy of it. I'd like feedback.

 

[SirMaster]

Don't worry, I balance out your average :)

I have 6 ST3000DM001 in my pool for more than a year now and no issues.

Lately I have been buying 4TB Reds though.

 

[Starpulkka]

I just had failing ST3000DM001 2 years old hdd in windows machine and i replaced it in another ST3000DM001 of course i put latest firmware in it. and after 2 weeks it is even worse shape than that older drive. Just had a super deal on WD RED 3TB drive, im hoping it isnt a too good to be true deal.
Edit: Also looked weird that another seagate did not have any seeing epoxy on under where motor "bearing" tap is.
What the.. wd red failure rate 8.8%.

 

[rs225]

Does anybody know what the firmware update (circa July 2013) for the ST3000DM001 is supposed to do? I have applied it, but I could never find any notes on what the purpose was. I hoped it was to fix their high failure rate, or at least make pre-failure detection more likely.

 

[jgreco]

No. AFAICR the problems with these drives are specific to one hardware rev (somewhat identifiable by firmware rev). I suspect you can Google firmware update issues for that drive and come up with discussions on the hardware revs, but it has been awhile and I don't recall more-specifics.

 

[jgreco]

In an incredibly unscientific and extremely small sample size, I will note that one of my four drive 1U chassis has four ST3000DM001's in it, shucked out of external USB's. They've got about 16000 hours on them. Of those, one has failed and been replaced, and another has been reporting a slightly varying number of errors for a few months... [...] But anyways that's like a 50% problem rate on those 3TB drives. Ewww.

Anyways turns out I lied. Two of the drives were shucked, fw CC43, and two were retail drives, fw CC4B. Interestingly it is the two retails that failed.

I replaced them all with 6TB ST6000DX000's which have a very impressive max speed of around 200MB/sec read.

 

[Starpulkka]

Is it true that ST6000DX000 seagate has lsi branded controller, and western digital red has realtek controller ?
edit:heh wd it's marvell controller, i have quickly find that website where it looked and text did say realtek, lol

 

[jgreco]

HAAAAAAAAHAHAHAHAHAHA!!!!