BUILD Designing a server for cold storage, low availability?

[star]

Hello FreeNAS,

I'm looking to build an offline backup server that will only be used for weekly-monthly incremental backups of around 2.5TB of personal files.

I do not need much capacity beyond the 2.5tb, so my current plan is to buy 4 Hitachi Deathstars and mirror them all at the disk level. I know that's only 25% storage efficiency, but my goals here are longevity and redundancy.

My questions pertain to using FreeNAS as a 'cold storage' solution:

• Does my backup system need to be up 24/7? This system would only receive periodic read/writes. Is it better for the disks to stay up 24/7, or could I have a scheduled task to turn on the system every X days?
• Is FreeNAS even designed for this sort of backup scenario? Is there a better OS to use for long-term storage?
• Are there any hardware considerations to take in account when designing a long-term backup server?
  

Here is the motherboard I'm considering: SUPERMICRO MBD-A1SRi-2758F-O

 

[Bidule0hm]

It's usually better for the drives to run 24/7 but in your case you'll spin-up the drives at most once a week so it's better to shutdown the server. Personally for me the limit is once a day, if you need to spin-up the drives more than that then the server should be on 24/7 ;)

I don't see any problem. ZFS protects against bit rot (as long as you use correct hardware, especially ECC RAM) so I think it's well designed for this task, no problem.

Use proper server grade hardware, ECC RAM, at least 8 GB of RAM, ... see the link "hardware recommendations" in my signature for more details ;)

 

[gpsguy]

Ditto what BiduleOhm said.

One thing you'll need to consider is how you run your SMART tests and scrubs. You may need to run them on demand as opposed to scheduling them with a cron job.


Sent from my phone

 

[star]

Is there any documentation out there pertaining to this sort of low availability, long-term backup design? Again -- is FreeNAS even a suitable solution for long-term backups?

Basically my question is if you don't need your system available 24/7, how often should the server be on? 8 hours a week for SMART checks?

I don't mind keeping my system on 24/7, but I wonder if I'll get more life out of my disks if they are only powered on every so often.

 

[mjws00]

I'll give you the opposite viewpoint. I think FreeNAS is a pretty poor choice for cold-storage or offline storage. It is really designed around SAN or NAS needs where things are expected to be available 24/7. It happily assumes the box will be up for maintenance processes.

It does a poor job of power management. Doesn't spin down drives correctly without work. Also doesn't do WOL very well out of the box. All of those things are pretty much the point of cold-storage. So meh. Sure you can manually boot/reboot screw around... but you'd be better off with a system that has the flexibility to tweak settings and wake-up built in. This appliance does not target that niche, imho. To much hassle on a system designed to work as intended and not be modified at its base.

A number of the linux based NAS options have these things baked in. Of course raw BSD or Linux would work nicely as well.

 

[star]

I'll give you the opposite viewpoint. I think FreeNAS is a pretty poor choice for cold-storage or offline storage. It is really designed around SAN or NAS needs where things are expected to be available 24/7. It happily assumes the box will be up for maintenance processes.

It does a poor job of power management. Doesn't spin down drives correctly without work. Also doesn't do WOL very well out of the box. All of those things are pretty much the point of cold-storage. So meh. Sure you can manually boot/reboot screw around... but you'd be better off with a system that has the flexibility to tweak settings and wake-up built in. This appliance does not target that niche, imho. To much hassle on a system designed to work as intended and not be modified at its base.

A number of the linux based NAS options have these things baked in. Of course raw BSD or Linux would work nicely as well.

Thanks mjws, this is the sort of feedback I'm looking for.

Now, I'm very new to this whole world -- can you recommend any linux based NAS options that are better suited to this sort of cold storage?

Is there a specific name for this type of storage? I call it 'cold storage', but from what I've found, that can mean many different things. Any suggestions for search parameters so I can Google this a little better?

And dare I open up this can of worms -- am I going to expect any more longevity from my system having it boot X hours every Y weeks vs. 24/7 uptime?

 

[mjws00]

The thing is, cold-storage is typically used in very large systems where we can save 1000's of kilowatts of power. When you are talking about 4 drives and a modern power sipping system... what is there to be saved? Maybe a few dollars a year. Unless you fire up like 1 hr per month or something... but even then it is not what we generally refer to. Offline backups "completely cold" can be useful to avoid a crypto virus or catastrophic power event that might damage many drives.

Spin-up / Spin-down wear is a pretty hard thing to quantify. If you spin up once or less a day... there is possibly less wear and increased drive life. But there is no real data with a valid sample size. Where is that magic number? Is it once a week? Once a month? Every 6 hours? It varies by drive, and by manufacturer... so we can only guess. I would spin down a once a week backup, but likely not power-off. But my bias is towards letting things run, I hate reboots on servers. The good news is power management is getting pretty darn amazing, but not on FreeNAS.

4 drives on an Avoton I'd likely just let spin. But I'd be tempted to fiddle with spin-down if accessed only once a week for an hour. Most of the linux based nas's qnap, xpenology etc do pretty well at this. FreeNAS has other strengths, just a matter of prioritizing what is right for you. Plus ZFS will get in your blood if you let it.

 

[star]

I'm not really concerned about power savings, as I agree, they are negligible at this scale.

Are there examples of user configurations for low access backup servers on FreeNAS? If not FreeNAS, what should I go with?

How would you design a server to keep 3tb safe for a long time? (Assuming this data is already stored in other physical locations)

 

[mjws00]

FreeNAS doesn't target low access so I've never seen a thread on a good config. Occasionally there is a power zealot after his tree-fiddy. They run into trouble quickly.

If we bring long-time into the picture, it is fair to consider bit-rot. So ZFS wins. For me. I'd grab 3 4TB drives and likely run a 3 way mirror. That puts reliability WAY to the forefront. You have literally chopped usable capacity to 33%. But for that tiny amount of data we can afford to. In addition you need to think of off-site storage as well. The size is manageable, and fire, theft, act-of-god should be considered. That is about where I'm at for my valuable (irreplaceable) data, plus there are likely a half dozen more copies on different drives and multiple systems, with long version histories. Unfortunately cloud is a little too impractical for me at this point, for some it is great.

 

[star]

FreeNAS doesn't target low access so I've never seen a thread on a good config. Occasionally there is a power zealot after his tree-fiddy. They run into trouble quickly.

If we bring long-time into the picture, it is fair to consider bit-rot. So ZFS wins. For me. I'd grab 3 4TB drives and likely run a 3 way mirror. That puts reliability WAY to the forefront. You have literally chopped usable capacity to 33%. But for that tiny amount of data we can afford to. In addition you need to think of off-site storage as well. The size is manageable, and fire, theft, act-of-god should be considered. That is about where I'm at for my valuable (irreplaceable) data, plus there are likely a half dozen more copies on different drives and multiple systems, with long version histories. Unfortunately cloud is a little too impractical for me at this point, for some it is great.

Yes, multiple physical locations is the key :)

Any recommendations on motherboards + cpus for a build like this? I have access to basically unlimited ECC DDR3, and even more DDR2...

 

[mjws00]

Avoton is nice, with a bit of a premium. You want 2750 not 2758. Basically it is Supermicro x10 for an optimal solution. You could likely get by with a celeron, drkk has one in his sig. The hardware stickies here are great. Don't even think about non-ecc. i3 had some recent ecc confusion... so I'd steer clear and xeon, celeron, avoton.

The TS140 is amazingly cost effective for a system that doesn't need many drives as well. For 3TB and no huge need of performance even an off the shelf NAS might be a good solution. But that would likely sacrifice ZFS.

 

[Robert Smith]

One thing you may want to consider is the FreeNAS version 9.2.0. It uses the previous major version SAMBA, which does not interfere with drives going to sleep, if you configure logging to memory only.

 

[depasseg]

My questions pertain to using FreeNAS as a 'cold storage' solution:

• Does my backup system need to be up 24/7? This system would only receive periodic read/writes. Is it better for the disks to stay up 24/7, or could I have a scheduled task to turn on the system every X days?
• Is FreeNAS even designed for this sort of backup scenario? Is there a better OS to use for long-term storage?
• Are there any hardware considerations to take in account when designing a long-term backup server?

I don't see your needs that different from most others looking for a safe place to keep data.
1. Yes, I would leave it up 24x7 to take advantage of SMART tests, pool scrubs and event alerting.
2. I think it's perfect for this scenario. You have the data protection piece (Mirrors) and ZFS (with ECC RAM) give you the silent bit rot protection.
3. If it were me, I would get the 4 drives (probably 4TB, to give you room for your incremental backup jobs), burn them in and then create a 3 way mirror and put one on a shelf as a spare.

Have you considered using a backup service like crashplan as well?

 

[star]

I don't see your needs that different from most others looking for a safe place to keep data.
1. Yes, I would leave it up 24x7 to take advantage of SMART tests, pool scrubs and event alerting.
2. I think it's perfect for this scenario. You have the data protection piece (Mirrors) and ZFS (with ECC RAM) give you the silent bit rot protection.
3. If it were me, I would get the 4 drives (probably 4TB, to give you room for your incremental backup jobs), burn them in and then create a 3 way mirror and put one on a shelf as a spare.

Have you considered using a backup service like crashplan as well?

Is there any research into bit rot you can point me towards? I'd like to know how likely I am to encounter it in non-ZFS solutions.

I'm leaning heavily towards your suggestion. Do hot spares work in FreeNAS?

And yes, I understand that this server will not protect me from all data loss, etc. etc.

 

[pirateghost]

I see zero benefit to powering your server up only once in a while. If this is a backup server, leave it on and let your other servers/clients back up to it frequently.

You will gain nothing in terms of hard drive longevity (in the grand scheme of things), by powering off/on your drives frequently.

You cannot use hot spares in FreeNAS, but you dont need them. Keep an extra drive in the system and don't configure it for anything. When you get an email that a disk has failed, you just go in and swap them out.

 

[gpsguy]

Supposedly it was added sometime after the original release of 9.3.

"Support for hot spare drive replacement has been added. If you have spare drives in your pool, and a drive fails, FreeNAS® should automatically remove the failed drive from the pool and replace it with the spare."

You cannot use hot spares in FreeNAS, but you dont need them.

 

[pirateghost]

Supposedly it was added sometime after the original release of 9.3.

"Support for hot spare drive replacement has been added. If you have spare drives in your pool, and a drive fails, FreeNAS® should automatically remove the failed drive from the pool and replace it with the spare."

Hmm. I was just asked about hot spares the other day at work and spent a few minutes researching it. I didn't see that anywhere

 

[gpsguy]

I try to generate a PDF manual every 7-10 days or so. It was mentioned in "1.2 What’s New Since 9.3-RELEASE" of a PDF I generated on 5/11. It wasn't in my 4/24 version.

 

[depasseg]

I don't have any specific info on bit rot. Just do a little googling. I remember reading a lot about how ZFS helps.

As for hot spares, I wouldn't bother unless you have multiple pools that the spare could be used in. If you are leaning towards using a spare with a single pool, just make it part of the pool in the first place.

 

[Ericloewe]

Also, for NAS, the C2750 boards are a better choice. C2758 is a networking SoC. The former boosts to higher clocks, the latter has some sort of acceleration for network tasks like VPNs.