Remote backup

[Lars Kluken]

I want to setup an automatic backup to an offsite NAS (Zyxel 540).
I tried rsync and was able to pull data from the remote NAS, but I was not able to push data to the zyxel NAS.
Seems like the rsync implementation is poor with no option to create a rsync module.

Is there any other way to create a routine backup task from my freenas server to the zyxel NAS?

 

[enemy85]

have you tried as described in the manual?

http://doc.freenas.org/9.3/freenas_tasks.html#rsync-tasks

 

[Lars Kluken]

have you tried as described in the manual?

http://doc.freenas.org/9.3/freenas_tasks.html#rsync-tasks

I have tried, but the problem is that I can't create an rsync module on the zyxel NAS. According to their support that feature is not available.

 

[Lars Kluken]

From what I have found it seems that the Zyxel 540 does not actually have rsync. Instead it runs a service called zysync, which can only partly communicate with rsync. Feeling cheated :(

In the meantime I have managed to setup rsync with another zyxel 325 unit. This was made possible by installing the fonz fun_plug script where rsync is included.
To my knowledge it is not possible to install fonz fun_plug on the 540 NAS.
Does anybody know if there is a way to install rsync on that unit?

If its not possible to installing rsync I am back to my original question.
Is there another way to create a routine backup between my freenas server and the zyxel 540?
The 540 does have ssh capabillity if that helps.

 

[SweetAndLow]

you should be able to configure rsync over ssh here is a good post about how to do that: https://www.digitalocean.com/community/tutorials/how-to-copy-files-with-rsync-over-ssh

 

[Lars Kluken]

you should be able to configure rsync over ssh here is a good post about how to do that: https://www.digitalocean.com/community/tutorials/how-to-copy-files-with-rsync-over-ssh

I agree that's a very good post on setting up rsync over ssh. I got the passwordsless authentication working which is nice.
Problem is that rsync requires rsync on both ends, and I can't install rsync (or any other backup service) on the remote NAS.

I'm wondering if the problem can be solved by mounting the remote NAS shares and then performing backup on the freenas side as if it was two local folders.
Does anyone know if that is possible?

 

[SweetAndLow]

you could use a combination of tar and scp. But you would be storing fully copies of your data everytime.

 

[Lars Kluken]

you could use a combination of tar and scp. But you would be storing fully copies of your data everytime.

Thank you for coming back at me.
Tar and scp might be the best solution.

Since last post I have mounted a remote NFS share to my freenas and used rsync for backup. There will be no benefit of incremental backups, but existing files will not be transferred.
The problem is that I had to DMZ the remote NAS to be able to mount it. Also, I'm not sure how secure this method is even without DMZ.

I have also used the same approach with sshfs on ubuntu with success. This would be my preferred solution if sshfs was available on freenas. That is unless somebody has some good arguments against it.
Have anybody tried installing sshfs on freenas?

 

[Lars Kluken]

Since last post I have mounted a remote NFS share to my freenas and used rsync for backup. There will be no benefit of incremental backups, but existing files will not be transferred.
The problem is that I had to DMZ the remote NAS to be able to mount it. Also, I'm not sure how secure this method is even without DMZ.

I have also used the same approach with sshfs on ubuntu with success. This would be my preferred solution if sshfs was available on freenas. That is unless somebody has some good arguments against it.
Have anybody tried installing sshfs on freenas?

In case someone stumbles over this tread.
After some reading I have found that mounting a remote volume and then performing the rsync operation does not decrease the data transfer.
From my understanding rsync will have to transfer the whole file on the remote volume in order to calculate the checksum and then transfer the changes if any. This means that the amount of data to be transferred increases and that unnecessary cpu resources are used.

 

[SweetAndLow]

Here is a good article with another link in it to another tutorial.

http://www.admin-magazine.com/Articles/Using-rsync-for-Backups

Also I don't think it will transfer the file twice just to check the checksum. When I use it on a 30GB file the second time I run it, it only takes a couple seconds.

 

[pirateghost]

One of the features of rsync is it's ability to NOT have to transfer things it has already transferred....

 

[Lars Kluken]

Thanks for the article.
When I experimented with a remote volume mounted over sshfs I found that the time it took for each rsync run was approximately the same as one full transfer.
I also thought it made sense if rsync needs the whole file to calculate the checksum. But I haven't worked much more on this because I got help installing rsync on my remote nas.

Could it be that the checksums are stored as long as the files are no changed?

 

[Lars Kluken]

One of the features of rsync is it's ability to NOT have to transfer things it has already transferred....

That' definitely the case when rsync is installed on both ends, but I'm not 100% sure it works when its not.

 

[pirateghost]

That' definitely the case when rsync is installed on both ends, but I'm not 100% sure it works when its not.

The rsync verification should take place on the end that runs the executable. It should verify by hash and file name

 

[Lars Kluken]

The rsync verification should take place on the end that runs the executable. It should verify by hash and file name

That would be consistent with SweetAndLow's findings. In that case I must have something wrong when I experimented with this.

 

[Robert Trevellyan]

After some reading I have found that mounting a remote volume and then performing the rsync operation does not decrease the data transfer.
From my understanding rsync will have to transfer the whole file on the remote volume in order to calculate the checksum and then transfer the changes if any. This means that the amount of data to be transferred increases and that unnecessary cpu resources are used.

When you use rsync that way, there's only one instance of rsync running, so the local system has to read the remote file over the wire to calculate the checksum and compare it with the local file. So yes, you'll transfer a lot more data over the network than you would by running rsync on the remote.

Could it be that the checksums are stored as long as the files are no changed?

No, rsync does not store any checksums.