New FreeNAS box - Heaps of drive issues. Head = Exploding.

[firewallwhisperer]

Hey all,

I'm building out a new freenas box. I ordered 12 used 8tb HGST 7200 SAS drives. 5 of them are giving me loads of trouble. I've been at this for hours to no avail. Someone PLEASE help, I'm losing it.

Here's the hardware:
- Chassis: Supermicro CSE-847BE1C-R1K28LPB 4U 36 x 3.5” Drive Bays
- Motherboard: [X10DRi-T4+]
- Integrated IPMI 2.0 Management
- Backplane: 2x Backplane:
- SAS3-846EL1 24-port 4U SAS3 12Gbps single-expander backplane,
- SAS3-826EL1 12-port 2U SAS3 12Gbps single-expander backplane
- Processor: 2x Intel Xeon E5-2620 V3 6 Core 2.4.Ghz
- Memory: 48GB ECC DDR4 (4 x 8GB REG 2133)
- Hard Drives: 12 x 8tb HGST SAS 7200 drives
- RAID: M1250 12Gbs HBA
- NIC: Onboard integrated Intel X540 Qad Port 10GBase-T

I first noticed the issue when attempting to SMART test all the drives. Notice the "device is NOT READY" error below.

Code:

root@freenas[~]# smartctl -i /dev/da1
smartctl 7.0 2018-12-30 r4883 [FreeBSD 11.3-RELEASE-p9 amd64] (local build)

=== START OF INFORMATION SECTION ===

Vendor:               HGST
Product:              HUH728080AL4200
Revision:             A515
Compliance:           SPC-4
LU is fully provisioned
Rotation Rate:        7200 rpm
Form Factor:          3.5 inches
Logical Unit id:      0x5000cca23b0fadf4
Serial number:        2EG8M9WR
Device type:          disk
Transport protocol:   SAS (SPL-3)
Local Time is:        Wed Jul  1 16:02:22 2020 PDT
device is NOT READY (e.g. spun down, busy)
A mandatory SMART command failed: exiting. To continue, add one or more '-T permissive' options.

I attempted to do a quick wipe of the drives and received the following error:

Code:

[EFAULT] Command gpart create -s gpt /dev/da1 failed (code 1): gpart: provider: Operation not supported by device

Some additional info:
I have tried moving the drives to different bay's same issue. Thus it is seemingly not a cable/HBA card/backplane,etc. issue (at least as far as I can tell)
Bios SATA settings are set to ACHI
I'm installed on two 32gb mirrored USBs
Yes I've tried turning it off and on again lol


What is going on here? I find it very hard to believe that 5 of 12 drives are bad. (6 really, one drive is not being detected at all. I'm treating that as a separate issue for the time being.)

HALP!

 

[Samuel Tai]

I find it very hard to believe that 5 of 12 drives are bad. (6 really, one drive is not being detected at all. I'm treating that as a separate issue for the time being.)

Believe it. As these are used drives, and have been jostled over their lifetimes and via shipping to get to you, odds are they're physically damaged. Don't waste time trying to revive them. Just replace them.

 

[firewallwhisperer]

Believe it. As these are used drives, and have been jostled over their lifetimes and via shipping to get to you, odds are they're physically damaged. Don't waste time trying to revive them. Just replace them.

You're probably right. However, i soon realized the other half of the seemingly healthy drives, don't allow extended SMART tests. This made me extremely curious. I have since found that these drives were previously used by EMC VNX storage (old name CLARiiON storage). They appear to have custom firmware. I am hoping to update the firmware and see if it allows extended SMART tests. I am wondering if the other issues are related to custom firmware as well.

All a pain in the ass, and perhaps a waste of time.. But these drives are cheap. If I can determine that they are relatively safe to use, it would be good to save some cash.

 

[HoneyBadger]

Regarding your one drive failing to spin up, check the 3.3v "Power Disable" pin - some of these might be getting set off this way.

I wonder if these drives also weren't set back to 512-byte sectors; if they came from an EMC they're likely still using T10 protection (520-byte sectors)

I am hoping to update the firmware and see if it allows extended SMART tests. I am wondering if the other issues are related to custom firmware as well.

Don't have a lot of hope for this - more recent EMC firmware bundles are signed (so you'll likely need to load them from an EMC storage array) and won't be readily available either.

It's very likely that your other issues are related to the EMC FW as well - even if SMART tests do run I would expect to see a lot of "UNKNOWN_ATTRIBUTE" as well.