FreeNAS backup to webspace using FTP

[takkischitt]

I have a FreeNAS setup and I recently had a hard drive fail in it. It wasn't too serious as I did have a backup, although the backup was a little out of date due to the backup drive being kept offsite and only updated every couple of weeks.

I've decided I want to have the system do a regular (maybe every 30 mins) automatic backup to an external location and due to having 'unlimited' webspace, I would like to have the system automatically back up to that location. I'm a little unsure about which would be the best way to go about this, FTP, Rsync, etc.

Can anyone offer some advice on this as I can't seem to find anything definitive on the forums recently.

 

[anodos]

I have a FreeNAS setup and I recently had a hard drive fail in it. It wasn't too serious as I did have a backup, although the backup was a little out of date due to the backup drive being kept offsite and only updated every couple of weeks.

I've decided I want to have the system do a regular (maybe every 30 mins) automatic backup to an external location and due to having 'unlimited' webspace, I would like to have the system automatically back up to that location. I'm a little unsure about which would be the best way to go about this, FTP, Rsync, etc.

Can anyone offer some advice on this as I can't seem to find anything definitive on the forums recently.

This is what rsync was designed for.

 

[takkischitt]

This is what rsync was designed for.

Would the web server not need to have some rsync software installed on it also? So it can check the difference between the files?

 

[depasseg]

Correct, as well as handle the stream of data properly. I can't see using FTP, unless you had a very small amount of data, since you would need to copy everything every time, or write some scripting to only send new files.

 

[takkischitt]

There is approx 150GB of data at the minute, but that will probably increase over time.

So is having offsite backup via FTP not really viable? What would be the most practical way of having a regularly updated offsite backup?

 

[depasseg]

Best way would be to have another ZFS capable server on the remote side and use somthing like this:

host1# zfs send -i snap1 tank/dana@snap2 > ssh host2.example.net zfs recv newtank/dana

Or, if that isn't possible, then have rsync on the remote side. In that case, I would use ZFS to send to a file, and then use rsync to send to the remote side.

 

[takkischitt]

hmmm, that seems to be pushing the limits of my skills a bit!

I might just set it up with my backup software and set it to do a daily backup when my computer is on. Should keep it more up to date than it currently is, yet it's quite easy to set up.

 

[danb35]

@takkischitt, before we can offer suggestions of the best way to do this, you'll have to identify the methods that your host has available for you to transfer data. Without knowing that, we can only speculate. But no, FTP really isn't a practical way to do this, mainly because there's no way to tell FTP to transfer changes since a certain point.

If FTP is the only method available from your host, then the best method that comes to mind is to install some kind of backup software in a jail that's capable of (1) generating backup files, and (2) doing incremental backups. One package that has this capability is dar; I'm sure there are others. You could then script an FTP upload of those files to your host. Although I doubt it's consistent with your webhost's TOS to use your "unlimited" space for file storage.

Edit: or just use Crashplan. It's a PITA to set up, but once set up it seems to work pretty well. They do charge for their storage, but it's on the order of $50/year.

 

[anodos]

Would the web server not need to have some rsync software installed on it also? So it can check the difference between the files?

I just logged into a crappy shared hosting account I have access to (godaddy - doesn't get much worse than that). It has ssh and rsync. I would probably check the TOS for the site. Look into crashplan if you need truly 'unlimited' storage for backup purposes.

Although FTP is often a solution to a problem, it is rarely a good solution. There are much, much better ways of doing this than FTP.

 

[DJ9]

You might want to take a look at Tarsnap. I've never used the service yet, but Allan Jude seems to like it.
http://www.tarsnap.com/

 

[danb35]

tarsnap is great, but kind of price-y. I wrote a post earlier this year about installing and configuring it, and it's easy enough, but $0.25/GB transferred + $0.25/GB/mo adds up if you're backing up a lot of data.

 

[anodos]

@takkischitt, before we can offer suggestions of the best way to do this, you'll have to identify the methods that your host has available for you to transfer data. Without knowing that, we can only speculate. But no, FTP really isn't a practical way to do this, mainly because there's no way to tell FTP to transfer changes since a certain point.

If FTP is the only method available from your host, then the best method that comes to mind is to install some kind of backup software in a jail that's capable of (1) generating backup files, and (2) doing incremental backups. One package that has this capability is dar; I'm sure there are others. You could then script an FTP upload of those files to your host. Although I doubt it's consistent with your webhost's TOS to use your "unlimited" space for file storage.

Edit: or just use Crashplan. It's a PITA to set up, but once set up it seems to work pretty well. They do charge for their storage, but it's on the order of $50/year.

My lazy approach to crashplan is to just install it in a centos VM with a gui and point it at my shares.