Freenas destroys all my boot devices

[asimov-solensan]

Hello,

I totally need help on this because I don't know where the problem may be. I want to have freenas running from USB so I can save SATA ports.

But in two years it managed to screw three USB thumb drives and two ssd drives. All this in only two years. I didn't read any specific inconvenient on using USB drives, besides performance maybe but this is a home setup.

The thing is that on April this year and tired of replacing thumb drives i replaced both drives with SSD disks connected again through USB and everything went fine.

Then I leave for couple of weeks and get and email telling me that boot raid got degraded (no problem, I got a mirror). Then I get home, a lot of things not working, try to reboot and the server doesn't boot.

Connecting the disks on my laptop I can see that indeed both disks have hardware errors:
Disk1

Code:

[ 7602.427385] print_req_error: I/O error, dev sdb, sector 64
[ 7602.427413] sd 3:0:0:0: [sdb] tag#3 FAILED Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[ 7602.427419] sd 3:0:0:0: [sdb] tag#3 CDB: Read(10) 28 00 00 00 00 80 00 01 80 00
[ 7602.427422] print_req_error: I/O error, dev sdb, sector 128
[ 7602.432544] sd 3:0:0:0: [sdb] Synchronizing SCSI cache
[ 7602.682900] sd 3:0:0:0: [sdb] Synchronize Cache(10) failed: Result: hostbyte=DID_ERROR driverbyte=DRIVER_OK

Disk2 (I connect this one and sometimes doesn't give me problems still can't boot freenas from it)

Code:

 7864.710590] print_req_error: critical target error, dev sdb, sector 0
[ 7864.710599] Buffer I/O error on dev sdb, logical block 0, async page read
[ 7864.710626] Dev sdb: unable to read RDB block 0
[ 7864.710837] Buffer I/O error on dev sdb, logical block 0, async page read
[ 7864.712259] Buffer I/O error on dev sdb, logical block 0, async page read
[ 7864.712508] Buffer I/O error on dev sdb, logical block 3, async page read
[ 7864.713008]  sdb: unable to read partition table
[ 7864.713948] sd 3:0:0:0: [sdb] Unit Not Ready
[ 7864.713961] sd 3:0:0:0: [sdb] Sense Key : Data Protect [current]
[ 7864.713976] sd 3:0:0:0: [sdb] Add. Sense: Logical unit access not authorized
[ 7864.715482] sd 3:0:0:0: [sdb] Attached SCSI disk
asimov@charcoal2:~$

What am I doing wrong here? It may have been just bad luck that both disks died the same week?

I absolutely don't know what try next and any suggestion is welcome. Just want to avoid replacing boot disks every two months.

 

[Jailer]

If you are short on SATA ports get a HBA for your data drives and hook your boot SSD directly to the motherboard.

 

[ThreeDee]

this all using same USB port? .. maybe you have a bad port?

 

[asimov-solensan]

@Jailer that's the way to go on the long term, I am aware of that. But the thing is I don't find people complaining about booting from USB, seems to be a pretty usual way to go.

@ThreeDee. Indeed I'm using an internal USB in the motherboard with a USB hub to connect both disks. Do you think this may be the problem? I guess that something like that may lead to data corruption but not unrecoverable hardware failure on disks.

Is there any metric I may check to know if boot disks are being overused somehow? I do an extensive use of virtual machines and jails but checking CPU and memory usage are quite low. I pay a lot of attention to this.

 

[danb35]

I don't find people complaining about booting from USB

Then you really aren't looking.

seems to be a pretty usual way to go

Not nearly so much since the release of 9.3 over five years ago, which made the boot device a live ZFS pool rather than a UFS image that was loaded into a RAMdisk on boot. Since then, there's been a meteoric rise in reports of USB stick failures. USB sticks are no longer recommended as boot devices, and haven't been for some time.

 

[asimov-solensan]

That totally explains why I had so many problems with USB sticks. I began using freenas in 9.10.

But now why SSD disks also failed? The problem is then using USB ports? No matter if it's an SSD or a even a mechanical disk?

Is there a sensible way to use USB as boot device? Or is it absolutely not advised? I expected some performance hit but still cannot understand why the hardware failures.

 

[asimov-solensan]

Sorry for the double post but this totally blew my mind:

FreeNAS - Creating Installation Media

FreeNAS is now TrueNAS CORE. TrueNAS CORE is a better, faster, and safer version with all the same features you love plus more! Download TrueNAS CORE at http...

www.youtube.com

Official video from freenas, for 11.2 installation. It says it can be installed on USB sticks only warning that they may be less reliable than SSD.

Don't know what to think now.

 

[ThreeDee]

That totally explains why I had so many problems with USB sticks. I began using freenas in 9.10.

But now why SSD disks also failed? The problem is then using USB ports? No matter if it's an SSD or a even a mechanical disk?

Is there a sensible way to use USB as boot device? Or is it absolutely not advised? I expected some performance hit but still cannot understand why the hardware failures.

Welp .. a common denominator is your USB hub in both mirrored USB drives and your SSD plugged in via a USB adapter of some sort.. so maybe there is your problem. I use a 128GB SSD with a USB to SATA adapter. The USB port powers the SSD so no external power needed and everything is nice and neat inside my case. This isn't a recommended hardware configuration, but it's been running without issue since I implemented it about 6'ish months ago. I don't have anything mission critical either.. just Plex stuff and Unifi controller plug in.

 

[danb35]

Don't know what to think now.

What about this is confusing you? Yes, you can still install on USBs. No, they aren't a good idea--as you've experienced, they just aren't very reliable. The risk is pretty small--as long as you have a current backup of your config file, you can bring up an identical system with very little effort--but it's also very easily avoided in most cases by using a small SSD.

still cannot understand why the hardware failures.

USB sticks generally are, well, pretty crappy, and constant writes are pretty will guaranteed to destroy them in short order. And other storage controllers over USB have proven to be pretty unreliable over time.

 

[msbxa]

I lost my USB stick just last few days they’re not reliable anymore because of this note from the documentation:

Current versions of FreeNAS® run directly from the operating system device. Early versions of FreeNAS® ran from RAM, but that has not been the case for years.

Use SSD’s and the larger size the better.

 

[asimov-solensan]

The thing is that I'm using SSD and they died even faster than USB sticks. Of course I have got up to date backups, so nothing dramatic here, but

I really wanted to hear about someone like @ThreeDee. He seems to be doing fine with an SSD over USB. And I totally agree with him on blaming the hub I'm using. Still I find strange to get broken disks instead of simple data corruption.

Next I will try is to run same configuration without the hub.

Anyhow I want to remark a question I added before. How do I get statistics of disk usage to Identify if there is something that is overusing my disks? I fear that there may be something wrong with my configuration.