Backup solution sought

[Dom93950]

I am new to FreeNAS and I'm building my first NAS server. I've read the documentation and searched the forums but it is still not clear to me what would be the best backup strategy. The documentation describes Periodic Snapshots and Replication Tasks and I get that but it doesn't describe how these snapshots would be used to restore or re-create a NAS volume if it was lost. Furthermore, I've read about setting up a pair of drives with mirroring but this is claimed to not be a substitute for a backup strategy. So what is a good backup strategy?

What I would like to do is to schedule periodic cloning of the NAS to another drive in case the original NAS fails. This way, the clone is a complete backup and can be swapped in for the failed drive. Is this possible with mirroring? If not, is there another way to schedule periodic cloning of the NAS so a replacement drive can be swapped in if there is a failure of the original NAS drive?

If the best strategy is Periodic Snapshots and Replication Tasks as described in the documentation, can someone point me to how these backups would be used to re-create an NAS in the event that the original NAS drive failed?

Any other suggestions for a backup strategy would be greatly appreciated.

-Dom.

 

[garm]

When your server eventually fail or is replaced, you install FreeNAS on the new server. Then you will have a new pool as well and you simply run your backup in reverse to restore your data. Then you reload the configuration to restore your settings. This schema works independently of the backup solution you have.

 

[Arwen]

It's best if you can tell us;

• Your intended use of your NAS.
• How important is the data.
  
• Amount of storage you need.
• Hardware you have / want for your NAS.

A couple of things;

• Periodic Snapshots can be used as point in time for backups. Or used to recover deleted files, which can give the impression of being a backup.
  
• Mirroring and RAID are not a backup, but a way to keep your data safe and available WHEN a disk drive fails.
• Backups are intended to keep your data safe, if your NAS dies completely. Or is stolen. Or you have too many disk drives fail. Can be used to recover deleted files, IF they are in the backup.

So, simple scheme;

• NAS with 2 disks Mirrored.
• A pair of disks same size as the Mirrored disks, will be used for backups.
• Once a week, (or twice a month, depending on what you desire), you install one of your backup disks as a 3rd Mirror. Then use ZFS to split the pool, allowing you to remove the backup disk.
• Next time you run a backup, you use the other backup disk. This is called a backup rotation set. Here is only 2, but you can have more.
• If you have a disk failure in your Mirrored set, you can use your older backup disk as a replacement. Or order another disk. Or both.

My own backups are similar. I have 8TB of usable space in my NAS, (4 x 4TB disks in a RAID-Z2). So I bought a single 8TB disk to make a backups, (but not as a 3rd Mirror described above). I also have another disk, 750GB, which can backup everything except the video media. It's in my backup rotations, but I understand the limitations of my scheme. I just have no other use for the 750GB drive, except as a backup disk.

 

[pro lamer]

Once a week, (or twice a month, depending on what you desire), you install one of your backup disks as a 3rd Mirror. Then use ZFS to split the pool, allowing you to remove the backup disk.

Sounds like a very nice plan comparing to detaching a whole pool:

• No need to deal with replication tasks (I've heard they don't work well when destination pool is absent)
• Less pools = less RAM for ARC needed

I guess there are some cons too, for example it works only for mirrors BAM.

EDIT: and I am happy to hear it works in practice :) :)

Sent from my mobile phone

 

[kdragon75]

I am new to FreeNAS and I'm building my first NAS server. I've read the documentation and searched the forums but it is still not clear to me what would be the best backup strategy. The documentation describes Periodic Snapshots and Replication Tasks and I get that but it doesn't describe how these snapshots would be used to restore or re-create a NAS volume if it was lost. Furthermore, I've read about setting up a pair of drives with mirroring but this is claimed to not be a substitute for a backup strategy. So what is a good backup strategy?

What I would like to do is to schedule periodic cloning of the NAS to another drive in case the original NAS fails. This way, the clone is a complete backup and can be swapped in for the failed drive. Is this possible with mirroring? If not, is there another way to schedule periodic cloning of the NAS so a replacement drive can be swapped in if there is a failure of the original NAS drive?

If the best strategy is Periodic Snapshots and Replication Tasks as described in the documentation, can someone point me to how these backups would be used to re-create an NAS in the event that the original NAS drive failed?

Any other suggestions for a backup strategy would be greatly appreciated.

-Dom.

I can appreciate your relizastion that they are TONS of guides about making backups but little information is ever given about recovering when you need to. I plan to use replication from my striped mirrors (poor use of space but fast!) to my old box with *GASP* RAIDz1. If my desktop, main server, and backup server all get trashed? I guess I didn't need the data as bad as the house that burned down/flooded/got sucked into a black hole. I do plan to add cloud backup is my ISPs ever pull their heads out of... congresses arses... and provide decent upload speeds.... Seriously... You have to get their 1gbs download to get 35mbps up! the next step down (on upload) is only 10mbps! They do offer a 2gb symmetric plan but thats $400usd per month! Anyway I got a bit off subject...
I'm all about replication as it gets the data outside of the source box.

 

[Dom93950]

It's best if you can tell us;

• Your intended use of your NAS.
• How important is the data.
  
• Amount of storage you need.
• Hardware you have / want for your NAS.

A couple of things;

• Periodic Snapshots can be used as point in time for backups. Or used to recover deleted files, which can give the impression of being a backup.
  
• Mirroring and RAID are not a backup, but a way to keep your data safe and available WHEN a disk drive fails.
• Backups are intended to keep your data safe, if your NAS dies completely. Or is stolen. Or you have too many disk drives fail. Can be used to recover deleted files, IF they are in the backup.

So, simple scheme;

• NAS with 2 disks Mirrored.
• A pair of disks same size as the Mirrored disks, will be used for backups.
• Once a week, (or twice a month, depending on what you desire), you install one of your backup disks as a 3rd Mirror. Then use ZFS to split the pool, allowing you to remove the backup disk.
• Next time you run a backup, you use the other backup disk. This is called a backup rotation set. Here is only 2, but you can have more.
• If you have a disk failure in your Mirrored set, you can use your older backup disk as a replacement. Or order another disk. Or both.

My own backups are similar. I have 8TB of usable space in my NAS, (4 x 4TB disks in a RAID-Z2). So I bought a single 8TB disk to make a backups, (but not as a 3rd Mirror described above). I also have another disk, 750GB, which can backup everything except the video media. It's in my backup rotations, but I understand the limitations of my scheme. I just have no other use for the 750GB drive, except as a backup disk.

First, I want to thank everyone who replied to my question. I learned something from everyone even though I am replying to Arwen.

I should have included more information about my intended use of NAS. My intention is to use it primarily as a media server (photos, movies, music) using Plex. I have two 8 TB drives. My intention was to use one as the source drive for media and the other drive as backup. I'm running FreeNAS 11.1 on an Intel NUC with the two 8 TB drives connected by USB 3.0. I've experimented before with smaller capacity drives and it works fine but I want to have a backup plan before I install the complete media library. The media library on the NAS will be a copy of files on other storage but the format of the files will be different so, in theory, the data on the NAS could be regenerated from the files on other storage but this would take a lot of time to transcode, etc. So, the data on the NAS is of moderate importance to avoid a laborious regeneration of the library hence the desire to have a complete backup of the NAS.

The scheme you describe sounds very good (2 mirrored disks plus a third for periodic backups). But, I'm wondering if I need to have the third disk for periodic backups. If I set up the two disks as mirrored can't one of the disks be considered a backup if the other drive fails? Then, all I would need to do is to get a replacement for the failed drive to recover all the data. I've never had a RAID setup before so I guess this comes down to a question about the value of a mirrored RAID versus a backup. It seems to me that the value of a mirrored RAID is that the data is immediately available in the event that one drive fails. An alternative would be to have a single media drive with no RAID and to have the second drive for periodic backups as you describe by periodic mirroring. The difference would be that the second drive is not being used all the time and the data would not be available immediately in the event of a drive failure (I would need to obtain a replacement drive which could take a couple of days).

Am I thinking about this correctly? Are there other factors I should consider? I would appreciate your thoughts about my current plan to have a single media drive and use the second drive to make periodic backups via mirroring.

-Dom.

 

[Dom93950]

I can appreciate your relizastion that they are TONS of guides about making backups but little information is ever given about recovering when you need to. I plan to use replication from my striped mirrors (poor use of space but fast!) to my old box with *GASP* RAIDz1. If my desktop, main server, and backup server all get trashed? I guess I didn't need the data as bad as the house that burned down/flooded/got sucked into a black hole. I do plan to add cloud backup is my ISPs ever pull their heads out of... congresses arses... and provide decent upload speeds.... Seriously... You have to get their 1gbs download to get 35mbps up! the next step down (on upload) is only 10mbps! They do offer a 2gb symmetric plan but thats $400usd per month! Anyway I got a bit off subject...
I'm all about replication as it gets the data outside of the source box.

kdragon75,

I completely agree with your point about the restrictions on upload speed. Since the early days of "public" access to the internet the ISPs restricted access to maximize profits from commercial users with "tiered plans". We could discuss this forever but I think its very unlikely this will change in the forseeable future.

Your point about offsite location for backup is well taken. If I were to do this I would probably not choose a cloud service due to the amount of data that I need to back up. I understand that the upload speed would still be a problem but I would probably opt for a storage system at a physically separate family residence. My question is: How would you perform the periodic backups over the internet? Would you do this using a Linux command? Mirroring? Another app or service?

Assuming you have a remote backup, how would you restore your NAS if your source drives were destroyed?

 

[kdragon75]

How would you perform the periodic backups over the internet?

I would still use replication. This big thing it seeding the backup. Run the first replication job locally and then move the unit off site. This way we only have to transfer the changed data.

how would you restore your NAS if your source drives were destroyed?

If I needed to rebuild the entire array, I would pick up the remote machine and reverse the replication on the new system. Obviously there are other considerations like reinstalling the same version of freenas, restoring the config, setting up the new sync job, etc. But I think you get the gist of it. Just take some time and read the User Guide about replication jobs and test it out in a lab/VM environment while taking notes. Be sure to print the notes and keep them with both units. There is no point in having a DR plan if you cant find/access the DR plan when you need it!

 

[Arwen]

First, I want to thank everyone who replied to my question. I learned something from everyone even though I am replying to Arwen.
...

Glad my post was helpful. I did write about how I do my backups in a Resource post, here in the FreeNAS forums. (See the Resource tab at the top of the web page.) But, it's specific to my needs and is just a general example. I probably should write a companion Resource on backups.

I should have included more information about my intended use of NAS. My intention is to use it primarily as a media server (photos, movies, music) using Plex. I have two 8 TB drives. My intention was to use one as the source drive for media and the other drive as backup. I'm running FreeNAS 11.1 on an Intel NUC with the two 8 TB drives connected by USB 3.0. I've experimented before with smaller capacity drives and it works fine but I want to have a backup plan before I install the complete media library. The media library on the NAS will be a copy of files on other storage but the format of the files will be different so, in theory, the data on the NAS could be regenerated from the files on other storage but this would take a lot of time to transcode, etc. So, the data on the NAS is of moderate importance to avoid a laborious regeneration of the library hence the desire to have a complete backup of the NAS.
...

In general, USB is not a reliable connection method for NAS data disks. I know many home NASes have or use USB ports, so it seems like an easy choice. However, we, (long time computer users), have found that USB tends to be slow and unreliable. Some of that changed with USB 3.0, and UAS, (USB Attached SCSI), but still the connector can be "bumped" and dis-connected too easily.

So, we generally recommend "real NAS" hardware of SATA or SAS type disk connections. It's not that we are purists, it's that we mostly want reliable, fast access and some certainty that it will last for years without much trouble.

The scheme you describe sounds very good (2 mirrored disks plus a third for periodic backups). But, I'm wondering if I need to have the third disk for periodic backups. If I set up the two disks as mirrored can't one of the disks be considered a backup if the other drive fails? Then, all I would need to do is to get a replacement for the failed drive to recover all the data. I've never had a RAID setup before so I guess this comes down to a question about the value of a mirrored RAID versus a backup. It seems to me that the value of a mirrored RAID is that the data is immediately available in the event that one drive fails. An alternative would be to have a single media drive with no RAID and to have the second drive for periodic backups as you describe by periodic mirroring. The difference would be that the second drive is not being used all the time and the data would not be available immediately in the event of a drive failure (I would need to obtain a replacement drive which could take a couple of days).

Am I thinking about this correctly? Are there other factors I should consider? I would appreciate your thoughts about my current plan to have a single media drive and use the second drive to make periodic backups via mirroring.

-Dom.

You are missing one serious factor to consider. Data deletion / destruction. For example;

• You attempt to move data around, (to organize your video collection). But, in the process, you delete things instead.
• An update to your media finds that the updated media is unusable, (perhaps you encoded to H.265 but your media player won't play H.265, only H.264)
• A virus encrypts all your files and demands payment.

All these can be solved reasonably quick by restoring from backups.

That said, a single drive for media can be an option. It's just one we won't recommend.

In someways, FreeNAS may not be the best option for your hardware and needs. We tend to want rock stable results so we steer users to reliable hardware, (not a NUC, and not USB attached data disks).

 

[Dom93950]

Arwen, Thanks again for your helpful advice. It's good to know what the best practices are. Since this is my first venture into NAS for a media library I thought I would try FreeNAS with hardware I already have. When I have more experience I may want to upgrade to "real NAS" hardware. I am curious what you mean by "real NAS" hardware: Do you mean dedicated units like Synology or QNAP or FreeNAS boxes? Or do you mean CPUs with external drives connected via SATA?

 

[Dom93950]

Arwen, no need to reply to my last question. I read your backup recommendations in the Resources section and see that you and others have posted hardware recommendations there as well. Thanks again for your help!