smartd issues with FreeNAS 11

[MasterTacoChief]

I'm having a couple issues getting smartd running.

The first issue appears in the server messages when smartd attempts to start:


Jun 14 22:12:02 nas smartd[15523]: Unable to register SCSI device /dev/da7 at line 12 of file /usr/local/etc/smartd.conf
Jun 14 22:12:02 nas smartd[15523]: Unable to register device /dev/da7 (no Directive -d removable). Exiting.
Jun 14 22:12:02 nas root: /usr/local/etc/rc.d/smartd: WARNING: failed to start smartd


/dev/da7 is a hardware RAID disk on a PERC H710.

If I try to disable SMART tests for any hardware RAID disks through the GUI, the serial number reported for these disks is greater than 30 characters. Now this shows an error when trying to make changes:

Ensure this value has at most 30 characters (it has 32).

Not much I can do, since this field cannot be edited, so now I can't change disk settings (power/SMART).

To get smartd to start, I had to manually remove the offending lines in /usr/local/etc/smartd.conf.

 

[MasterTacoChief]

And of course, ix-smartd sneaks in and overwrites the changes. So I need to manually remove them again after making a few other changes.

 

[DrKK]

I'm sorry, did you say you were FreeNAS 11 on top of *HARDWARE RAID*?

 

[MasterTacoChief]

I'm sorry, did you say you were FreeNAS 11 on top of *HARDWARE RAID*?

Yes, I know it's not ideal within FreeNAS since I can't fully take advantage of the features, but my system currently includes one hardware RAID controller. A majority of the system is software RAID, and the plan is to eventually convert the hardware RAID to software once HBA adapter is purchased to replace the H710.

While it's shunned upon, previous versions were at least tolerant of a hardware RAID controller in the system. Now because I have this hardware in the system, one important feature that provided monitoring of my system doesn't work properly. I'd hope a fix could be implemented to at least allow the system to be tolerant of this configuration.

 

[MasterTacoChief]

Learning a bit more about smartd.conf, it appears that the configuration is relying on the -d default (auto), which for some reason isn't identifying the device correctly (I assume it thinks it's removable?). If I modify the lines with a -d option:

Code:

/dev/da7 -a -n never -W 0,45,65 -m youremail@email.mail -M exec /usr/local/www/freenasUI/tools/smart_alert.py -d SCSI

Then smartd will start. Of course this is temporary since the script will overwrite it again eventually.

Devices as they appear in camcontrol:


root@nas:~ # camcontrol devlist
<ATA HGST HUS724040AL AA70> at scbus0 target 2 lun 0 (pass0,da0)
<ATA HGST HUS724040AL AA70> at scbus0 target 3 lun 0 (pass1,da1)
<ATA HGST HUS724040AL AA70> at scbus0 target 4 lun 0 (pass2,da2)
<ATA HGST HUS724040AL AA70> at scbus0 target 5 lun 0 (pass3,da3)
<ATA HGST HDN724040AL A5E0> at scbus0 target 6 lun 0 (pass4,da4)
<ATA Micron_M600_MTFD MU03> at scbus0 target 7 lun 0 (pass5,da5)
<DELL PERC H710 3.13> at scbus1 target 0 lun 0 (pass6,da6)
<DELL PERC H710 3.13> at scbus1 target 1 lun 0 (pass7,da7)
<DELL PERC H710 3.13> at scbus1 target 2 lun 0 (pass8,da8)
<SanDisk SDSSDA120G Z32080RL> at scbus6 target 0 lun 0 (pass9,ada0)
<16GB SATA Flash Drive SFDDA01A> at scbus8 target 0 lun 0 (pass10,ada1)
<SanDisk Cruzer Glide 1.27> at scbus10 target 0 lun 0 (pass11,da9)


 

[hokan]

Smartd won't work with RAID because the individual disks are hidden behind the RAID controller. However, your disks are presenting as da0-da9 so you may have them set in HBA mode. If that's so, using the newer driver for the RAID controller might help.

In System >> Tunables, set variable "hw.mfi.mrsas_enable" to value "1" type "loader", reboot, and see if that helps.

 

[MasterTacoChief]

Smartd won't work with RAID because the individual disks are hidden behind the RAID controller. However, your disks are presenting as da0-da9 so you may have them set in HBA mode. If that's so, using the newer driver for the RAID controller might help.

In System >> Tunables, set variable "hw.mfi.mrsas_enable" to value "1" type "loader", reboot, and see if that helps.

In 9.10.x the disks showed up as /dev/mfid0-2, whereas now they are appearing as /dev/da6-8. I tried adding the tunable, but still having the same issues and the disks still appear as /dev/da6-8.

This appears to be configuration of the devices from dmesg:

Code:

AVAGO MegaRAID SAS FreeBSD mrsas driver version: 06.709.07.00-fbsd
mrsas0: <AVAGO Thunderbolt SAS Controller> port 0xec00-0xecff mem 0xd9ffc000-0xd9ffffff,0xd9f80000-0xd9fbffff irq 35 at device 0.0 numa-domain 0 on pci2
mrsas0: Using MSI-X with 8 number of vectors
mrsas0: FW supports <16> MSIX vector,Online CPU 8 Current MSIX <8>
mrsas0: MSI-x interrupts setup success
da7 at mrsas0 bus 0 scbus1 target 1 lun 0
da7: <DELL PERC H710 3.13> Fixed Direct Access SPC-3 SCSI device
da7: Serial Number 00aacbbe06c6dbc020002809eaa0a38c
da7: 150.000MB/s transfers
da7: 3814912MB (7812939776 512 byte sectors)
da6 at mrsas0 bus 0 scbus1 target 0 lun 0
da6: <DELL PERC H710 3.13> Fixed Direct Access SPC-3 SCSI device
da6: Serial Number 0096c161056fc42e20002809eaa0a38c
da6: 150.000MB/s transfers
da6: 571136MB (1169686528 512 byte sectors)
da8 at mrsas0 bus 0 scbus1 target 2 lun 0
da8: <DELL PERC H710 3.13> Fixed Direct Access SPC-3 SCSI device
da8: Serial Number 00fc1def080e21bd20002809eaa0a38c
da8: 150.000MB/s transfers
da8: 3814912MB (7812939776 512 byte sectors)

As far as I could tell, there is no IT firmware for the PERC H710. There's some mention of enabling JBOD mode using the MegaCli utility, so I might try that once I migrate some VMs off one of the HW RAID disks. I pulled the trigger on another HBA adapter to replace it though, so it will ultimately be going up for sale soon and I'll be 100% software RAID.

 

[Ericloewe]

/dev/mfid0-2

Eeewww.

/dev/da6-8

Good.

But wait, SAS 2208-based controller? Ooh, mrsas now supports those, too. Cool.

I pulled the trigger on another HBA adapter

That's better, because it's much more tested than the HW RAID controller + mrsas solution, but you could probably stick with the H710 and the mrsas driver.

 

[MasterTacoChief]

That's better, because it's much more tested than the HW RAID controller + mrsas solution, but you could probably stick with the H710 and the mrsas driver.

Not sure how I could stick with it right now with the two problems I originally posted (unless they're eventually fixed). One of the "disks" is a virtual disk of three physical disks, but the other two are single disks configured as individual RAID0 VDs.
And while there is a command to supposedly enable JBOD, from what I've read it isn't necessarily reliable on this particular controller, plus I get this when I check the device properties:


root@nas:~ # MegaCli -AdpAllInfo -a0
...
Supported PD Operations
================
...
Enable JBOD : No
...


I don't know if that means it's actually not supported at all, or just that because it's not enabled that it isn't currently supported. Either way, I don't need the "features" of the H710, and they're going for a lot more eBay than the other card I bought.

 

[Ericloewe]

Not sure how I could stick with it right now

By destroying all the hardware RAID crap and using the disks as they are shown.

Yes, you still need to move the data off of the pool and move it back to a sane pool, but the controller itself is usable.

 

[MasterTacoChief]

I guess I'm missing something. I migrated everything off the RAID VD that consisted of three physical disks, destroyed the virtual disk so the drives now show as available in the RAID startup configuration. When FreeNAS boots it cannot see those drives though. The only way I've been able to make drives visible to FreeNAS through this controller is to create a RAID VD, which results in the issues I posted about.

Sorry if it's something obvious, but what am I missing here?

 

[Ericloewe]

Try updating the firmware on the controller.