Implications of moving disks to new/different HBA

[Curtipus]

I have a testbench freenas server that I wanted to test upgrading from the onboard SATA controller (Intel 3400 series) to the LSI 9211-8i on 2x2TB disks (striped vdevs; I should hope RAID level is irrelevant in determining compatibility).

I created a new pool/dataset and threw some data onto them before installing the LSI card. I powered off the machine, put the disks on the LSI card and booted it back up. The pool comes up, and I see the data in the dataset (via SMB share), however I get a red alert from freenas that the "volume is ONLINE: One or more devices has experienced an unrecoverable error", similar to this post: https://forums.freenas.org/index.ph...has-experienced-an-unrecoverable-error.20939/ however my drives show up just fine in SMART tests.

Some of the data on the dataset seemed to come up corrupted, as in, SMB copies would fail or streaming a video file would halt at certain points. This did however not happen to all of the data. Switching back to the onboard SATA controller resolves this issue.

Is this expected behavior? What should one expect when moving disks to new hardware? What can I do to properly migrate the disks without error?

 

[cyberjock]

Any corruption pretty much means you have a hardware failure or hardware malfunction. Unfortunately there's not enough info posted to know exactly what is going on. So you're on your own to identify the culprit unless you'd like to provide a debug file (system -> Advanced -> Save Debug).

Edit: Please attach the debug with the LSI card being used and then again with the onboard SATA.

 

[SweetAndLow]

You don't have to do anything special when moving the disks to a proper hba. What you should make sure is that your hba card is supported and is flashed to the correct firmware.

 

[Curtipus]

You don't have to do anything special when moving the disks to a proper hba. What you should make sure is that your hba card is supported and is flashed to the correct firmware.

Thanks for the direct answer. The firmware is v20 IT mode and I know the 9211 seems to be very popular on these forums.

I will test the card on a fresh pool and see what happens.

 

[Stux]

In theory, all drives in a ZFS pool have a label and a copy of the array configuration. Thus when the drives are all recogized, they can all be identified as part of each others array, and it should just work. No matter which controller or port etc, the drive actually is physically connected, thus you can shuffle hot swap bays in a chassis, etc.

In theory.

 

[depasseg]

In theory, all drives in a ZFS pool have a label and a copy of the array configuration. Thus when the drives are all recogized, they can all be identified as part of each others array, and it should just work. No matter which controller or port etc, the drive actually is physically connected, thus you can shuffle hot swap bays in a chassis, etc.

In theory.

I can confirm that in my tests, that it's not just theory. The labels are there and pools import just fine. The biggest issue is to ensure ZFS has complete access to the drives (IT mode) and that there isn't any HW RAID in the mix. Which sounds like is all good for the OP.

 

[sfcredfox]

For what it's worth, I have done the same thing as OP. I have two HBAs, and moved a pool from one to the other. FreeNAS imported it at boot with no problems. The only thing I noticed was the drives got re-ordered (IE - da48 became da0 or something like that).

My best guess is the onboard controller wasn't allowing direct access and the LSI controller is, but now the disk looks 'different' to the OS?

 

[Stux]

The drive names are essentially random.

 

[HoneyBadger]

Welcome to ZFS, where the drive labels are made up and the order doesn't matter!