S.M.A.R.T. service failed to start.

[dashtesla]

Found a small issue with my FreeNAS 11.3, fresh install.

SMART won't start, I looked up online some people said i should disable any drives that do not support it, however doesn't seem to be the case since 11.2 that wasn't an issue running on the exact same enviroment/hardware.

The boot drive is a Hyper-V virtual drive, and I thought maybe why not just try to disable and reboot the entire system but that didn't work, all the other drives show up as it should and you get their serial numbers and everything else in every other way is fine, the pool is healthy no warnings of any kind I only noticed that because I went to the Services page and noticed that smart was set to start at boot but wasn't running and then i tried to start manually and it didn't work.

I checked the log and:

configuration file /usr/local/etc/smartd.conf parsed but has no entries

As usual any help is appreciated, if this is some kind of a bug let me know anything I can check on my system and I will post here.

 

[dashtesla]

Some updates.

The conf file is actually empty, i found this command online and the first time i used it actually worked but after a reboot doesn't work anymore, i also tried setting them as post init scripts as a dirty workaround but no luck.

cp /usr/local/etc/smartd.conf.sample /usr/local/etc/smartd.conf /usr/local/etc/rc.d/smartd-daemon start

Copying the same conf plus starting the service.

Also if you remove the command after a reboot the original conf file will be empty again.

 

[dashtesla]

I also just checked my server in a datacentre that's running freenas inside hyper-v with drives passthrough and i have the same problem, so seems to be an issue with 11.3, I'm gonna try to install 11.2 into a test vm with the sole purpose of finding a conf file from a working system and see what the conf file is supposed to look like i suspect the sample file doesn't have all the correct entries and the empty file is just empty and the system is not creating the correct file for some reason.

 

[pnunn]

Hi @dashtesla, did you manage to get this working? I've just discovered my FreeNAS-11.3-U3.2 install has exactly the same problem.

Any clues?

 

[dashtesla]

Hi @dashtesla, did you manage to get this working? I've just discovered my FreeNAS-11.3-U3.2 install has exactly the same problem.

Any clues?

Not really no, i've just been mostly ignoring the problem at this point since I have 0 time to deal with it sadly :/

 

[Fredda]

What happens if you run the smartctl commands by hand? Does this fail, too?
The output of smartctl -H /dev/daX or smartctl -a /dev/daX might bring up some clues on the issue.
(replace X with the numbers of your drives)

@dashtesla, you write you're running on Hyper-V, you have read this post?

"Absolutely must virtualize FreeNAS!" ... a guide to not completely losing your data.

[---- 2018/02/27: This is still as relevant as ever. As PCIe-Passthru has matured, fewer problems are reported. I've updated some specific things known to be problematic ----] [---- 2014/12/24: Note, there is another post discussing how to deploy a small FreeNAS VM instance for basic file...

www.ixsystems.com

 

[dashtesla]

da0 - boot drive/virtual disk vhdx

root@vm7[~]# smartctl -H /dev/da0
smartctl 7.0 2018-12-30 r4883 [FreeBSD 11.3-RELEASE-p9 amd64] (local build)
Copyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART Health Status: OK

root@vm7[~]# martctl -a /dev/da0
zsh: command not found: martctl
root@vm7[~]# smartctl -a /dev/da0
smartctl 7.0 2018-12-30 r4883 [FreeBS
Copyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Vendor: Msft
Product: Virtual Disk
Revision: 1.0
Compliance: SPC-3
User Capacity: 10,737,418,240 bytes [10.7 GB]
Logical block size: 512 bytes
Physical block size: 4096 bytes
LU is thin provisioned, LBPRZ=0
Logical Unit id: 0x600224802343a21914c34910d4799bd5
Device type: disk
Local Time is: Mon Jul 20 15:52:57 2020 CEST
SMART support is: Unavailable - device lacks SMART capability.

=== START OF READ SMART DATA SECTION ===
Current Drive Temperature: 0C
Drive Tripm Temperature: 0C

Error Counter logging not supported

Device does not support Self Test logging

 

[dashtesla]

da5 is one of the data drives.

root@vm7[~]# smartctl -H /dev/da5
smartctl 7.0 2018-12-30 r4883 [FreeBSD 11.3-RELEASE-p9 amd64] (local build)
Copyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org


Probable ATA device behind a SAT layer
Try an additional '-d ata' or '-d sat' argument.

root@vm7[~]# smartctl -a /dev/da5
smartctl 7.0 2018-12-30 r4883 [FreeBSD 11.3-RELEASE-p9 amd64] (local build)
Copyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org


Probable ATA device behind a SAT layer
Try an additional '-d ata' or '-d sat' argument.

 

[dashtesla]

About the post you mentioned yes i did read it but it's from 2013 I don't think it still applies 7 years later for hyper-v things have changed since then, and if my current setup is proof that it does work fine i suppose that argument is no longer valid, the only problem i've had was with smart really, performance-wise the system is quite fast and i can passthrough the drives individually including the nvme for l2-arc cache something proxmox would require passthrough of an entire pci-e device and they're connected to 3 individual controllers and the tape drive i want presented to the host itself not a vm and other drives are assigned to different vms or the host so pci-e passthrough really not an option for this server.

 

[Fredda]

You have no real discs. So SMART tests don't work. Your signature says you have PCIe passthrough enabled, but I am not really sure if that really worked or if it does not work as expected in Hyper-V.

None of your smartctl outputs shows usable SMART data, that is the reason the SMART service does not start.

You could try to add the "-d " switches like the smartclt output suggested. That helped me to get the SMART data on some of my Linux servers, which are equipped with a Hardware-RAID card. Are you sure the card is flashed to IT?

The original post is from 2013 but it has been updated several times, so I'd guess not everything is outdated.

 

[dashtesla]

Name
Pool
Serial
Disk Size

da0
Boot Pool
10 GiB

da1
VM7-SPA-Z2
WCJ1AWM5
4.55 TiB

da2
VM7-SPA-Z2
WCJ2H0KK
4.55 TiB

da3
VM7-SPA-Z2
WCJ2GYYQ
4.55 TiB

da4
VM7-SPA-Z2
WCJ0QJP0
4.55 TiB

da5
VM7-SPA-Z2
WCJ27Z16
4.55 TiB

da6
VM7-SPA-Z2
WCJ2BLWK
4.55 TiB

da7
VM7-SPA-Z2
WCJ0QVHP
4.55 TiB

da8
VM7-SPA-Z2
WCJ0RG17
4.55 TiB

da9
VM7-SPA-Z2
0025_3851_91B0_57AF.
465.76 GiB

 

[dashtesla]

root@vm7[~]# smartctl -a /dev/da5
smartctl 7.0 2018-12-30 r4883 [FreeBSD 11.3-RELEASE-p9 amd64] (local build)
Copyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org


Probable ATA device behind a SAT layer
Try an additional '-d ata' or '-d sat' argument.

root@vm7[~]# smartctl -a /dev/da5 -d ata
smartctl 7.0 2018-12-30 r4883 [FreeBSD 11.3-RELEASE-p9 amd64] (local build)
Copyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org

Read Device Identity failed: Inappropriate ioctl for device

A mandatory SMART command failed: exiting. To continue, add one or more '-T permissive' options.
root@vm7[~]# smartctl -a /dev/da5 -d sat
smartctl 7.0 2018-12-30 r4883 [FreeBSD 11.3-RELEASE-p9 amd64] (local build)
Copyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org

Read Device Identity failed: scsi error unsupported scsi opcode

A mandatory SMART command failed: exiting. To continue, add one or more '-T permissive' options.

 

[dashtesla]

The drives are shucked from Seagate Backup Plus Portable 5TB so they are originally USB but they're running straight off SATA/SAS.

 

[Fredda]

I doubt the drives itself are the problem. I suspect it's either the virtualisation or the PCIe passtrough which makes problems.
What is the output of sas2flash -list?
I'm no expert on virtualisation, but I'd expect with PCIe passtrough enabled this command should work.

 

[dashtesla]

I doubt the drives itself are the problem. I suspect it's either the virtualisation or the PCIe passtrough which makes problems.
What is the output of sas2flash -list?
I'm no expert on virtualisation, but I'd expect with PCIe passtrough enabled this command should work.

sas2flash is for LSI controllers, i'm passingthrough drives indivudally some are even nvme like the 970 evo there's no SAS controller in between, I'm not sure how hyper-v presents the drives to the operating system but i do get all the other information such as serial number drive model correct size anything you would expect really. Also some of the drives are connected to the onboard controllers, one being intel another LSI i think but SATA mode only despite being SAS controller.

I'm having some personal struggles with housing and finances so i'm in a hotel right now trying to figure out where to go and what to do and i'm not exactly able to debug or help do a lot of tests from here, and yes i carry my server in a samsonite suitcase because that's the only option i have right now.. but when i'm a little more stable i will look into it further.

 

[Samuel Tai]

What OS are you running on the Hyper-V host? According to https://docs.microsoft.com/en-us/wi...supported-freebsd-virtual-machines-on-hyper-v, a 11.3 FreeBSD guest with DDA pass-through is only supported on Server 2016 and 2019. If you're just passing through disks, then SMART is one of the things you give up, because FreeNAS expects direct access to hardware. Passing through an HBA would allow SMART to work.

 

[dashtesla]

What OS are you running on the Hyper-V host? According to https://docs.microsoft.com/en-us/wi...supported-freebsd-virtual-machines-on-hyper-v, a 11.3 FreeBSD guest with DDA pass-through is only supported on Server 2016 and 2019. If you're just passing through disks, then SMART is one of the things you give up, because FreeNAS expects direct access to hardware. Passing through an HBA would allow SMART to work.

You can see the specs on my signature, 2019 Datacenter. I can't passthrough an entire HBA because of the way i have my drives and some of my ports specially onboard ones are sata 2 and the hba are sas 2/sata 3 i also have 5 860 evos, sata 2 is fine for the 5TB drives but not the ssds, i also use the server for game streaming with my quadro p4000 and many other things other than freenas including windows vms and linux and the ability to hook up usb devices directly to windows.

I don't have enough pci-e for a third hba as i also have some pci-e fans to help cool down the hbas that get extremely hot in that case along with the quadro. I'm sure I could make it work somehow with 2 but i also need cables and afaik there is no option to passthrough the entire HBA with hyper-v and to top it all off there's the nvme passthrough which is already a pci-e device anyway.

I use Gen 1 VM for FreeNAS and other than the lack of SMART everything is running smoothly including data scrubs which keep everything safe since we're not dealing with hardware raid here.

This server physically has:

8x 5TB > SATA
5x 500GB SSD > SATA
3x 4TB > SATA
1x LTO-6 > SAS
1x 500GB NVME > PCIE

I wanted to use a SAS expander initially but mine didn't work as expected so i went for two hbas which give better bandwidth anyway since i'm running pci-e gen 2 x8 (i believe).

 

[Samuel Tai]

OK, then in your setup, you'll need to run SMART on the host.

 

[dashtesla]

OK, then in your setup, you'll need to run SMART on the host.

Yes I can run crystal disk info and fetch the smart info from the host plus see the usage of the drives in real time with task manager, it's not even all that bad but it doesn't have that integration with freenas, technically if this was a more common usage one could build a daemon for windows that sends the smart info of all drives every x minutes back to freenas but I don't think many people will do what i do.

What I still don't understand is why on 11.2 I didn't have that problem, so was 11.2 just accepting the lack of SMART?

 

[Fredda]

sas2flash is for LSI

Accoding to the spec https://i.dell.com/sites/doccontent...heets/Documents/dell-perc-h310-spec-sheet.pdf
it uses an LSISAS2008 chip.