Mirroring Boot Device With More Than 2 USB Drives

[pab49162]

I recently had a mirrored USB boot device go bad and bought four new USB drives to use as a replacement and future spares. After installing one of the new USB drives, I re-established the boot mirror without any issue.

As I have 3 more of the identical USB drives, it got me wondering about mirroring the boot drive on more than two devices. I checked the FreeNAS Users Guide and this forum, but couldn’t find anything related to using more than two devices.

So I was wondering if anyone had any thoughts/comments/experience relative to adding one or two more of these new USB drives as additional mirroring boot devices.

Thanks in advance for any replies on this topic.

 

[jgreco]

It should work but be prepared for the possibility that the middleware might not really expect it.

 

[pschatz100]

I don't see much point in having more than two mirrored drives. It's good to keep the extras on hand, though.

And don't forget to back up your configuration file to a safe place. Mirrored drives are not a replacement for a proper backup.

 

[jgreco]

I don't see much point in having more than two mirrored drives.

The normal reason for having more than two drives in a mirror is where the operational requirements are that a drive loss not compromise redundancy.

Your system shows that you have a RAIDZ2 array. Why not RAIDZ1?

Same thing.

 

[pschatz100]

Of course, what you say is true about RAIDZ1 versus RAIDZ2. But the chance of catastrophic loss of data from a boot drive problem is low, and rebuilding a boot drive is a lot less risky and painful than rebuilding a data volume.

 

[jgreco]

Of course, what you say is true about RAIDZ1 versus RAIDZ2. But the chance of catastrophic loss of data from a boot drive problem is low, and rebuilding a boot drive is a lot less risky and painful than rebuilding a data volume.

For your use model, perhaps. But you're characterizing this as a data loss issue, which is far from the only concern.

Other people might be doing things like VM hosting where having to do an unscheduled/unplanned reboot to fix a boot device problem would impact the client environment - and dumping/restarting potentially hundreds of running VM's just because the frickin' NAS had a boot device issue would be considered unacceptable in most business settings. Remember, without an operating boot device, FreeNAS will panic. And if ZFS has no place to recover a corrupt block from, it can return the famous all-zero "error" block. It isn't that hard to picture a situation where a critical bit of the system image becomes unreadable on a boot device -- we see it often enough here on the forums and just tell people that the system needs to be reinstalled -- so if you are running two-way mirror boot, one of the mirror components fails, and the other experiences a single error in a bad location such as in the middle of /bin/sh or libc.so or other critical bits, you're going down hard. If you provide the third mirror component, you survive, and have some opportunity to make it right.

This is what I mean when I say "operational requirements" and where requirements such as "drive loss shall not compromise redundancy," which probably seem excessive to the home hobbyist, are nevertheless things you see in the real world.

 

[pab49162]

Thank you all for the comments on my question.

As my FreeNAS server is only used by myself, there isn’t a downtime issue if both USB boot devices were to somehow simultaneously fail. I regularly make backups of the config file, so system restoration wouldn’t be an issue if this did happen.

Assuming the middleware would allow me to use more than two devices, the downside I see to using more devices was that I would be putting “usage hours” on the additional devices with a very remote chance I would ever use the increased redundancy. Conversely, if I stick with only two devices, the unused ones wouldn't accumulate “usage hours” until I used one a replacement for a failed drive in the mirrored pair of devices.

The botton line is that I am going to stick with just using a mirror pair of USB drives and save the other 3 drives as future replacements when one of the current pair go bad.