A solution to backup data outside network?

[jamesrussell]

Hi,


I am looking to make a secondary backup of the company data out of the company network to make it more safe. We are looking at online backup and recovery solutions. The entire company data will be more than 7TB at least, which makes the already out of consideration due to security concerns free cloud storage, even more out of the map. Can anyone suggest any good solution, preferable based on their experience. High storage capacity, scheduled backup, easy recovery functionality, safety and security and good customer support are the major features I am looking for. I’d appreciate any kind of help.

 

[enemy85]

ZFS replication on a second FreeNas?

 

[Robert Trevellyan]

High storage capacity, scheduled backup, easy recovery functionality, safety and security and good customer support are the major features I am looking for.

Based on your priorities, you might consider rsync.net. The cost per GB month is high, but here's an example from their FAQs:

Will you support my use of X ?

Yes. If you are using X with an rsync.net filesystem, we will support and troubleshoot your use of it, regardless of how complex or esoteric that usage or application is. All such support will be handled by a real live Unix engineer, and will never be dealt with by a ticket system, autoresponder, or first level / junior "technician".

Really ?!

Yes. We are not just providing offsite filesystems, we are providing complete, end to end, personal customer support.

 

[DrKK]

At the other end of the spectrum is Amazon Glacier. The cost is $10/TB-month, (i.e., $0.01 per GB per month). But that is raw block storage---and requesting that all 7TB be brought down for a recovery would be brutal (the low cost to store has the catch that if you actually DOWNLOAD the data, they charge you to the bejesus belt). This will certainly be your cheapest option for 7TB, but, you also will get about as much professional help with this as you would ordering a value meal at Taco Bell. Amazon Glacier is what I recommend for nuclear option recovery. i.e., this is a backup that you don't believe you will ever have to use, barring an asteroid impact.

Generally speaking, 7TB of hosted, full service, cloud recovery backups, is going to be brutally expensive.

 

[Robert Trevellyan]

At the other end of the spectrum is Amazon Glacier.

Google's Nearline Storage costs the same per GB-month, but without the incomprehensible billing calculations and retrieval delays. However, I'm not aware of an easy way to utilize either platform with FreeNAS (unless you count s3cmd).

 

[DrKK]

Google's Nearline Storage costs the same per GB-month, but without the incomprehensible billing calculations and retrieval delays. However, I'm not aware of an easy way to utilize either platform with FreeNAS (unless you count s3cmd).

Well, for the record, and I didn't check this out myself, Matt was under the impression that the Amazon plugin that they had in FreeNAS could be trivially adapted for use on Glacier by a sufficiently enterprising person.

 

[alexg]

Crashplan offers unlimited storage, but it is slow. However, it is a cost effective solution for " if my house burns down" coverage.

 

[SweetAndLow]

Crashplan offers unlimited storage, but it is slow. However, it is a cost effective solution for " if my house burns down" coverage.

Crashplan maxes out my upload at 10mb/s I wouldn't call that slow. It does uses lots of CPU resources for dedup but that's a good thing because you end up uploading less information.

 

[Robert Trevellyan]

Matt was under the impression that the Amazon plugin that they had in FreeNAS could be trivially adapted for use on Glacier by a sufficiently enterprising person.

All you have to do is set an appropriate policy on the bucket you're using and Amazon will automatically move whatever you put in it to Glacier. I don't think the plugin needs to be modified, unless the goal is to make it set the policy for you (as done by Arq Backup for Mac.)

Hm, maybe retrieval is the issue, since there's a 4 hour delay before you can begin downloading from Glacier. Again, that's where Google's Nearline Storage would be a win. It throttles downloads but it doesn't impose an initial delay.

 

[Fiber]

Wouldn't it be possible to use gsutil to backup to google nearline? Or maybe Symantec NetBackup if you wanted a commercial option.

Also relevant

 

[Robert Trevellyan]

Using something like gsutil would be pretty limiting because it has no built-in differential backup capability, so you'd either be sending a full backup every time or doing something smarter by hand.

 

[Fiber]

Using something like gsutil would be pretty limiting because it has no built-in differential backup capability, so you'd either be sending a full backup every time or doing something smarter by hand.

I don't think think that's true. gsutil can utilize rsync for differential backups.

 

[Robert Trevellyan]

gsutil can utilize rsync for differential backups.

Oh, thank you for pointing this out, I did not look very closely, but I certainly will now.

 

[Joshua Parker Ruehlig]

I use zfs replication from my house to a pfsense server with a zfs harddrive at my parents house. I have about 250GB data which I initially replicated locally. My freenas sends about 8GB of new data weekly.

It's been working well enough for me, even with my 10mbps upload (which equates to almost 2 hours of sending per week). I'm sure a similar solution could work for your company if you had multiple locations.

Because some of my snapshots aren't initiated by/on my freenas I am using my own scripts to accomplish this.
https://www.jruehlig.com/owncloud/s/GI8xDYs5KhTxIbg

 

[Robert Trevellyan]

10mbps upload

I wish...

gsutil can utilize rsync

I took a closer look and as far as I can tell from the docs, it's a file-based sync. Much better than a full upload, and looks to be easily the best direct-to-inexpensive-cloud option available for FreeNAS, but not nearly as efficient as the rolling checksum differential transfer of true rsync.

 

[DaveFL]

So is the most popular approach to use crash plan? Or are people using google near line? Also any thoughts on using a tool such as arq and just uploading to cloud drive?

 

[Robert Trevellyan]

any thoughts on using a tool such as arq and just uploading to cloud drive?

Do you mean mounting the storage on a client machine and running Arq there, or are you thinking of running Arq in a VM on your FreeNAS?

 

[DaveFL]

Do you mean mounting the storage on a client machine and running Arq there, or are you thinking of running Arq in a VM on your FreeNAS?

Arq on VM running in ESXi with access to say FreeNAS NFS share.

 

[Robert Trevellyan]

OK, well I've been using Arq for some time now and the experience has been great, but I only use it to backup from 3 Macs to FreeNAS. I tested it with Amazon Glacier and Google Nearline and it worked well, but in the interests of spreading the risk, I decided not to use one solution for both local and cloud backups.

 

[DaveFL]

OK, well I've been using Arq for some time now and the experience has been great, but I only use it to backup from 3 Macs to FreeNAS. I tested it with Amazon Glacier and Google Nearline and it worked well, but in the interests of spreading the risk, I decided not to use one solution for both local and cloud backups.

I'm basically in the process of getting everything off of laptop/desktop onto network share. And I'm also doing laptop/desktop mirror to network share. Now I want to figure out a means to take whatever I consider to be really important that i store on ZFS to the cloud. Just not sure what the tight approach is.

I've got everything running smoothly, just need to setup my snaps and figure out what to do with my backup.