Remote Replication for offsite backups

[MrNaz]

Hi all,
I will be creating two FreeNAS instances, one with 30tb and the other with 4tb.

• The 4tb one will be in an office environment. Due to the data retention requirements, it will have hourly snapshots running on it with a 3 month retention.
• The 30tb one will be in a remote location.
• We would like to back up the 4tb one to the 30tb one. If possible, we would like to sync all snapshots to the 30tb unit as well, such that if the 4tb unit dies, it can be replaced snapshots and all.
• The internet connection between the two is a little unreliable, and low speed. However, an initial sync can be done with the two on the same LAN, and then they can be moved to their respective locations.
• The volume of data changing on a daily basis will be low, in the order of a hundred MB or so.

What is the best way to achieve a full replication of the 4tb unit to the 30tb unit?

Thanks in advance people.

 

[DrKK]

The standard ZFS replication, which is built in to FreeNAS, accomplishes all of your requirements as part of its normal operations. It will copy only the delta in the data moving forward, and automatically enumerates all the snapshots on the receiving end.

You will find the quick config outlined in the FreeNAS documentation to be more than sufficient for setting it up.

 

[toadman]

I agree with KK that standard replication will probably be the best bet.

With two potential issues I suppose. With the snapshot schedule obviously up to 1 hour of changed data may be lost if the primary server goes down. (Which you can easily change if desired.) Second, depending on how reliable the internet connection is, may be more could be lost as it may not have transferred to the backup server. I say that because a ZFS replication fails completely when it fails. i.e. if the connection drops the current "zfs send/zfs receive" will fail for that snapshot. So if it's a REALLY flaky connection you could have issues vs some other solution which allows "partial" transfers. Highly unlikely it's an issue, but not sure since "a little" is kinda subjective. I suppose this can be monitored for a bit and you can see if it's an issue or not.

 

[MrNaz]

I agree with KK that standard replication will probably be the best bet.

With two potential issues I suppose. With the snapshot schedule obviously up to 1 hour of changed data may be lost if the primary server goes down. (Which you can easily change if desired.) Second, depending on how reliable the internet connection is, may be more could be lost as it may not have transferred to the backup server. I say that because a ZFS replication fails completely when it fails. i.e. if the connection drops the current "zfs send/zfs receive" will fail for that snapshot. So if it's a REALLY flaky connection you could have issues vs some other solution which allows "partial" transfers. Highly unlikely it's an issue, but not sure since "a little" is kinda subjective. I suppose this can be monitored for a bit and you can see if it's an issue or not.

Does this mean that if the link between the FreeNAS boxes is broken for a period, there will be permanent data gaps, or will the replicant be brought up to full sync again when the connection is restored, even if the outage is multiple days?

 

[toadman]

The latter. When the connection is back up, the system should "catch up" and there should be no data gaps assuming all the snapshot sends are completed.