Hot, Warm, and Cold Spares once again

[Monkey_Demon]

I've read Sections 10.2.5 & 10.2.6 of The Fine Manual and perused the numerous entries about spares on this forum, but I still have questions.

My system runs FreeNAS-11.2-U7 and has 5 x 4TB WD Red drives configured as 1 pool (Volume 1) consisting of 2 x 8 TM mirrored vdevs containing 8 TB (i.e., 2 drives x 4 TB) each. It has 2 hot-swap cages that can hold 4 drives each for a total of 8 hot-swap drives.

The fifth drive is the spare. Currently it's configured to go into standby mode after 5 minutes and Level 1 (minimum) power usage. I believe these settings qualify the drive as a "warm" spare, and I plan to set it up as a warm spare.

The system is used at home to store music, videos, and a library of electronic documents containing much of my life's work. At least once per year I travel away from home from a week to almost two months. While away, I often use the server for entertainment (music & videos) or work (the library). While not entirely critical, not having access to this material can be a major inconvenience.

So here are my questions:
1. What, if any, are the drawbacks of this configuration?
2. Are the standby settings of the spare worthwhile in the sense they keep it from spinning needlessly, even though it has to spin up from standby?
3. I'm confused by the documentation in 10.2.5, which says, "To add a device to an existing pool, Extend that pool." This is because Section 12.2.6 on extending a pool seems mainly addressed to increasing a pool's storage. Specifically, regarding extending a mirrored pool like mine it says, "to extend a ZFS mirror, add the same number of drives." Does this mean I can't just use one drive as the warm spare?

Any further suggestions or comments from persons more knowledgeable and experienced about this would be greatly appreciated.

 

[PhiloEpisteme]

The system is used at home to store music, videos, and a library of electronic documents containing much of my life's work. At least once per year I travel away from home from a week to almost two months. While away, I often use the server for entertainment (music & videos) or work (the library). While not entirely critical, not having access to this material can be a major inconvenience.

What do you do for backups? This may be the most important thing to consider, especially if you don't keep backups currently.

My system runs FreeNAS-11.2-U7 and has 5 x 4TB WD Red drives configured as 1 pool (Volume 1) consisting of 2 x 8 TM mirrored vdevs containing 8 TB (i.e., 2 drives x 4 TB) each. It has 2 hot-swap cages that can hold 4 drives each for a total of 8 hot-swap drives.

I don't understand what you're saying exactly, can you clarify? Do you mean that you have 4 x 4TB drives in 1 pool made up of 2 vdevs and each vdev is a mirror vdev? Is the 5th drive officially configured as a spare in FreeNAS or is it just sitting there plugged into the computer but not configured?

1. What, if any, are the drawbacks of this configuration?

If you're gone for long periods of time but have access to the GUI I think this setup makes some sense. I'm not sure if configuring a hot spare is exactly worth it. Hot spares only kick in in the event a drive goes down completely and generally it is a good idea to replace a drive before that which required manually telling the system to replace the drive and thus precluding the system's automatic use of the hot spare. So having a drive plugged in and ready to go in the event you see the need for replacing a drive is a great idea. If you were to not have GUI access for long periods of time I would suggest you consider using hot spares configured in the GUI.

2. Are the standby settings of the spare worthwhile in the sense they keep it from spinning needlessly, even though it has to spin up from standby?

You want to reduce the number of times it is spinning up and down. If it remains spun down until you need it, great. But if it spins up and down and up and down it might wear the drive out faster.

3. I'm confused by the documentation in 10.2.5, which says, "To add a device to an existing pool, Extend that pool." This is because Section 12.2.6 on extending a pool seems mainly addressed to increasing a pool's storage. Specifically, regarding extending a mirrored pool like mine it says, "to extend a ZFS mirror, add the same number of drives." Does this mean I can't just use one drive as the warm spare?

To add a spare you also click "extend" and then on the bottom row of buttons there is one for "spare". You click that and it opens a box to the right which you can move the drive into as a spare. This would be a hot spare and as above I think this would be most useful if you are leaving the system for long periods of time without GUI access. If you have GUI access this may not be necessary as just having the drive plugged into the system will likely be all you need.

 

[Monkey_Demon]

I've interspersed replies.

What do you do for backups? This may be the most important thing to consider, especially if you don't keep backups currently.

I'm still working on this, but I plan to use FreeNAS's Cloud Sync feature to back up to Google Drive. So far the main problem I've had is getting a certificate from G Suite, but I understand from the rclone forum there are ways around this. I hope to try them next week.

I don't understand what you're saying exactly, can you clarify? Do you mean that you have 4 x 4TB drives in 1 pool made up of 2 vdevs and each vdev is a mirror vdev? Is the 5th drive officially configured as a spare in FreeNAS or is it just sitting there plugged into the computer but not configured?

Yes about the pool. Here's what it looks like:

Pool (Volume 1)
>MIRROR (vdev 1)
>>drive 1 (4 TB)
>>drive 2 (4 TB)
>MIRROR (vdev 2)
>>drive 3 (4 TB)
>>drive 4 (4 TB)

Regarding the spare, it's just sitting there plugged in but not configured.

If you're gone for long periods of time but have access to the GUI I think this setup makes some sense. I'm not sure if configuring a hot spare is exactly worth it. Hot spares only kick in in the event a drive goes down completely and generally it is a good idea to replace a drive before that which required manually telling the system to replace the drive and thus precluding the system's automatic use of the hot spare. So having a drive plugged in and ready to go in the event you see the need for replacing a drive is a great idea. If you were to not have GUI access for long periods of time I would suggest you consider using hot spares configured in the GUI.

Even when I'm in places far, far away (e.g., Tibet) I have had Internet access. If not, since I'm the main one using the NAS, there's little point in worrying about this since the only reason I would need it is to keep the server operative so I can access what's on it while I'm away.

However, I've had some difficulty setting up TLS using Let's Encrypt. So I haven't had the chance to test this from a remote location yet.

But I did have an almost brand new drive fail and replaced it (under warranty -- thank you WD). I don't recall having any warning. So I replaced the bad drive only after I got notice saying the pool was degraded, and then replacing the drive took some time to obtain an RMA and get the replacement.

My nightmare is having the whole thing crash while I'm away and then not only having no access, but also losing data if both vdevs degrade before I have the chance to replace the bad drives.

I don't quite understand all of your comment. Are you suggesting keeping the drive physically installed and "Unused" and using the GUI via long distance to activate the drive manually if one of the other drives appears to be failing? Is this what you mean by "plugged in and ready to go"?

You want to reduce the number of times it is spinning up and down. If it remains spun down until you need it, great. But if it spins up and down and up and down it might wear the drive out faster.

Agreed. But how would I tell what it's doing? Put differently, wouldn't the drive configuration settings described in my OP keep it tranquil most of the time? In other words, I expect the drives to go to sleep when I'm not using them; and given my usage pattern, I see the spare drive waking up only when I'm accessing the NAS (which should be at most about a 4-hour daily period).

To add a spare you also click "extend" and then on the bottom row of buttons there is one for "spare". You click that and it opens a box to the right which you can move the drive into as a spare. This would be a hot spare and as above I think this would be most useful if you are leaving the system for long periods of time without GUI access. If you have GUI access this may not be necessary as just having the drive plugged into the system will likely be all you need.

Thanks. I had no idea this is how Extend works.

 

[PhiloEpisteme]

In order to do any maintenance on the server you'll want GUI access. You should probably use a vpn of some sort to get access to your home network while away. It is a bad idea to expose FreeNAS GUI to the web, it wasn't intended for that.

But I did have an almost brand new drive fail and replaced it (under warranty -- thank you WD). I don't recall having any warning. So I replaced the bad drive only after I got notice saying the pool was degraded, and then replacing the drive took some time to obtain an RMA and get the replacement.

early failures can happen. If you didn't, you should always follow the full burn in procedures. You can find great detailed info about that on these forums. The idea is that you stress test the drives when you first get them to weed out those early failures.

I don't quite understand all of your comment. Are you suggesting keeping the drive physically installed and "Unused" and using the GUI via long distance to activate the drive manually if one of the other drives appears to be failing? Is this what you mean by "plugged in and ready to go"?

Yes, exactly.

Agreed. But how would I tell what it's doing? Put differently, wouldn't the drive configuration settings described in my OP keep it tranquil most of the time? In other words, I expect the drives to go to sleep when I'm not using them; and given my usage pattern, I see the spare drive waking up only when I'm accessing the NAS (which should be at most about a 4-hour daily period).

A typical setup is to leave the drives spun up. It takes relatively little power to keep a drive spinning and reduces stress.

My nightmare is having the whole thing crash while I'm away and then not only having no access, but also losing data if both vdevs degrade before I have the chance to replace the bad drives.

You should take 3 steps to prevent this. First, keep a backup of your data. This is the most important of the three steps. Second, run regular scrubs of your data. This can be configured via the GUI. Third, configure regular smart tests of your drives, this too can be configured in the GUI. Smart tests will often "pass" but indicate troubling values which you can be notified of via email. These notifications give you the chance to replace a drive before it fully fails.

 

[Monkey_Demon]

Thanks very much. This is extremely helpful.

To follow up, I have just two questions.

1. Regarding your first point, exactly how do you suggest shielding the FreeNAS GUI from the web? I've struggled to get TLS security up and running w/ Let's Encrypt, but expect to devote a few more days to get it going. And I generally use a VPN when on untrusted networks: when overseas, in coffee shops, airports, hotels, etc. But do you suggest doing more than this? What? No port forwarding?

2. So overall, to address my original question, you recommend just configuring the spare drive normally and keeping it spinning. This, even though during the estimated 4 hrs/day that I actually access the NAS I won't be using the spare unless it's replacing one of the other drives. And avoiding the stress caused by spin-up is more important than the additional power and (of more concern to me) wear caused by constant spinning. Is this correct?

Thanks again.

 

[PhiloEpisteme]

1. Regarding your first point, exactly how do you suggest shielding the FreeNAS GUI from the web? I've struggled to get TLS security up and running w/ Let's Encrypt, but expect to devote a few more days to get it going. And I generally use a VPN when on untrusted networks: when overseas, in coffee shops, airports, hotels, etc. But do you suggest doing more than this? What? No port forwarding?

You can run a VPN on your home network and give yourself access that way. I'm not an expert in this area so perhaps you might post another thread or search around for the best way to do this.

2. So overall, to address my original question, you recommend just configuring the spare drive normally and keeping it spinning. This, even though during the estimated 4 hrs/day that I actually access the NAS I won't be using the spare unless it's replacing one of the other drives. And avoiding the stress caused by spin-up is more important than the additional power and (of more concern to me) wear caused by constant spinning. Is this correct?

You could certainly take that approach, yes. What I did not explain well above was that if the spare drive is just sitting in your system awaiting you to use it configuring the drive to stay spun down is not a bad idea. It will save you a bit of power and because the disk is not part of a pool it won't get spun up so long as you have nothing configured to use the disk. However, you will want to be sure that once you use the drive as a replacement disk that you change the settings so that it does not spin down. You want to avoid your drive spinning up and down several times a day every time you use your NAS. The trouble will be in making sure nothing is causing your system to spin up that disk.