rsync driving me bonkers

[Robert Thomspon]

OK, so...

Ive been fighting with rsync trying to do a complete dump of 1 truenas machine to another (machine 2 is just a backup of machine 1). Machine 1 is being decommsioned and rebuilt (the drives are being reused, but going into a clean TrueNAS build with new hardware).

Anyway, im trying to rsync from Machine 1 to Machine 2. and for the live of me, its driving me crazy

if i try to run rsync (CLI) as root (no user) it prompts me for a password (which, apparently i dont have and cant create) (i read the documentation that says dont enter a password, but that doesnt help either. If i DO run as a user, rsync tries to copy my data into a /root/su foldet aht doesnt exist so rsync fails out with "no space left"

Here is the current command i am running.. WHAT am i doing wrong? (FYI, ive tried "--inplace" but that yields the same results

(run from TrueNAS shell)
rsync -avh --progress /mnt/Storage/Movies/1BR su -robert@192.168.2.191:/mnt/Storage/Movies/1BR

(this is just a test to copy 1 directory. target machine is 192.168.2.191. Pool is named Storage. NFS share is enabled as is SMB (though, that shouldnt matter.)

 

[rvassar]

That simply isn't a valid rsync command. You can't insert the "su -" in the middle of the rsync, it interprets that as the destination. What you likely want is:

Code:

 rsync -avh --progress /mnt/Storage/Movies/1BR  robert@192.168.2.191:/mnt/Storage/Movies/1BR 

Rsync will then ssh into 192.168.2.191 as user "robert". But the first steps are:
1. Make sure you can ssh as root from Machine 1 to "robert@192.168.2.191".
2. Make sure "robert" has the correct ownership & permissions to write to 192.168.2.191:/mnt/Storage/Movies/1BR.

 

[Robert Thomspon]

That simply isn't a valid rsync command. You can't insert the "su -" in the middle of the rsync, it interprets that as the destination. What you likely want is:

Code:

 rsync -avh --progress /mnt/Storage/Movies/1BR  robert@192.168.2.191:/mnt/Storage/Movies/1BR 

Rsync will then ssh into 192.168.2.191 as user "robert". But the first steps are:
1. Make sure you can ssh as root from Machine 1 to "robert@192.168.2.191".
2. Make sure "robert" has the correct ownership & permissions to write to 192.168.2.191:/mnt/Storage/Movies/1BR.

well, thats what i get for using bing :)

 

[Robert Thomspon]

That simply isn't a valid rsync command. You can't insert the "su -" in the middle of the rsync, it interprets that as the destination. What you likely want is:

Code:

 rsync -avh --progress /mnt/Storage/Movies/1BR  robert@192.168.2.191:/mnt/Storage/Movies/1BR 

Rsync will then ssh into 192.168.2.191 as user "robert". But the first steps are:
1. Make sure you can ssh as root from Machine 1 to "robert@192.168.2.191".
2. Make sure "robert" has the correct ownership & permissions to write to 192.168.2.191:/mnt/Storage/Movies/1BR.

however, when using the command as youve adjusted it, i get:
Could not chdir to home directory /nonexistent: No such file or directory
sending incremental file list
rsync: mkdir "mnt/Storage/Movies/1BR" failed: No such file or directory (2)
rsync error: error in file IO (code 11) at main.c(665) [Receiver=3.1.3]

(adding --inplace flag give the same error)

 

[Robert Thomspon]

fixed the issue with not being able to do it with root (had to enable login with root under ssh)

Still same exact problems as listed above when copying as root, except no chdir error

 

[rvassar]

Do the parent dataset & directories exist?

Ala:
1. You've created a pool called "Storage".
2. You've created a dataset called "Movies" on pool "Storage".
3. Either "root" or "robert" can write to dataset "Movies".

On Edit... Try:

Code:

rsync -avh --progress /mnt/Storage/Movies/1BR  robert@192.168.2.191:/mnt/Storage/Movies/

 

[Robert Thomspon]

On the upside, i did get it working:
rsync -rvh --progress /mnt/Storage/ 192.168.2.191:/mnt/Storage

 

[Robert Thomspon]

Do the parent dataset & directories exist?

Ala:
1. You've created a pool called "Storage".
2. You've created a dataset called "Movies" on pool "Storage".
3. Either "root" or "robert" can write to dataset "Movies".

On Edit... Try:

Code:

rsync -avh --progress /mnt/Storage/Movies/1BR  robert@192.168.2.191:/mnt/Storage/Movies/

the folders did not, i wanted rsync to create everything except the pool...

 

[Robert Thomspon]

It seems to be working and creating everything just fine. Though, it now seems to be god awfully slow (~10-15MB/s)

 

[rvassar]

It seems to be working and creating everything just fine. Though, it now seems to be god awfully slow (~10-15MB/s)

Rsync has a lot of overhead early on, it's trying to avoid transferring things that are already there... A case that doesn't apply here. You'd likely do better doing a /bin/tar piped thru ssh, but that would be another headache for you to get right in the cli.

The good news... It should speed up.

 

[Robert Thomspon]

Rsync has a lot of overhead early on, it's trying to avoid transferring things that are already there... A case that doesn't apply here. You'd likely do better doing a /bin/tar piped thru ssh, but that would be another headache for you to get right in the cli.

The good news... It should speed up.

Its been a good long while since I used rsync :)

iirc, theres a way to start rsync in a terminal, close the terminal and rsync will continue to run... you can then open another terminal and check the progress...

any chance you know how thats done? (my google fu isnt leading me in the right direction)

 

[rvassar]

Its been a good long while since I used rsync :)

iirc, theres a way to start rsync in a terminal, close the terminal and rsync will continue to run... you can then open another terminal and check the progress...

any chance you know how thats done? (my google fu isnt leading me in the right direction)

You're probably thinking of the "screen" command. You can close the shell, and it keeps it alive/running, and then you ssh back in and do a "screen -r" to reconnect to it. I think the more modern version is "tmux".

I use screen all the time in cloud VM's, but haven't tried it on TrueNAS.

 

[Spearfoot]

Its been a good long while since I used rsync :)

iirc, theres a way to start rsync in a terminal, close the terminal and rsync will continue to run... you can then open another terminal and check the progress...

any chance you know how thats done? (my google fu isnt leading me in the right direction)

I wrote some rsync wrapper scripts a few years ago, they're available on GitHub:

GitHub - Spearfoot/scripts-for-using-rsync: Shell scripts for using rsync

Shell scripts for using rsync. Contribute to Spearfoot/scripts-for-using-rsync development by creating an account on GitHub.

github.com

These disable SSH encryption and compression, which helps speed up transfers. The caveat is -- only use them on secure systems.

tmux works well and is supported on TrueNAS/FreeNAS.

EDIT: I address this on the GitHub repository, but I should have mentioned that you will get much faster transfer rates if you set up an rsync module on the receiving server.

 

[Robert Thomspon]

I wrote some rsync wrapper scripts a few years ago, they're available on GitHub:

GitHub - Spearfoot/scripts-for-using-rsync: Shell scripts for using rsync

Shell scripts for using rsync. Contribute to Spearfoot/scripts-for-using-rsync development by creating an account on GitHub.

github.com

These disable SSH encryption and compression, which helps speed up transfers. The caveat is -- only use them on secure systems.

tmux works well and is supported on TrueNAS/FreeNAS.

EDIT: I address this on the GitHub repository, but I should have mentioned that you will get much faster transfer rates if you set up an rsync module on the receiving server.

I actually tried setting up the rsync modules in truenas but it kept failing/complaining about needing access... that was before i realized target machine needs to have it explicitly set to allow root to log in with password (i always forget that that is a setting that needs set)

 

[winnielinnie]

I actually tried setting up the rsync modules in truenas but it kept failing/complaining about needing access...

Using the Rsync modules approach doesn't use SSH. (Faster directory listing, no encryption overhead, yet "less secure", which isn't an issue for a personal LAN at home.)

For future reference:

• Double-check your trailing slashes in your rsync commands. They are crucial, and change the behavior of the command.
• Like @Robert Thomspon mentioned, use tmux, so that you don't need to keep the terminal open.
• Perhaps next time use ZFS-to-ZFS replication, rather than a file-based method, such as Rsync?

@Spearfoot
UPDATE: I personally use (and greatly prefer) the Rsync modules (Rsync Service) method over the SSH method. I'm much happier with it. I highly recommend others to enable "Usage collection" under the menu System -> General, if you are using the Rsync Service.

iXsystems is considering dropping the Rsync Service / Modules from future versions of TrueNAS, based on what I gathered from this bug report:

https://jira.ixsystems.com/browse/NAS-111949

We need to track rsync mod in usage stats.

We will need it to decide if we will remove that feature or not.

 

[pschatz100]

You're probably thinking of the "screen" command. You can close the shell, and it keeps it alive/running, and then you ssh back in and do a "screen -r" to reconnect to it. I think the more modern version is "tmux".

I use screen all the time in cloud VM's, but haven't tried it on TrueNAS.

Because RSYNC runs in the foreground, once you start it you won't be able to use the console until the sync completes. It also means that if you start it over SSH and lose your connection, the sync will be canceled if it has not completed. The answer to this is to use "tmux":

Enter "tmux" (without the quotes)

You should now see a green stripe at the bottom of the screen. You can then enter any commands you want, such as rsync. If you close the session or lose connection from a remote machine, the job will continue to run​

Enter "tmux attach" to resume your session.

Tmux is a great utility, definitely worth learning about if you run commands remotely or from the CLI. Check the documentation. For instance, it is possible to start multiple Tmux sessions at one time - which is handy for tasks such as burning in new drives before deployment, etc.

 

[Robert Thomspon]

follow up to this, and i think im probably going to start a new thread about it as its kind of a wholly separate idea...but:

when rsync'ing, how do i stop jails from having their mapped drives copied over? As of now, my radarr is causing my tv shows collection to duplicate. Once in its folder on the NAS itself, and then seemingly following the drive mapping to duplicate that as well (i thought they were symlinks, but a -l doesnt stop them from being created/copied)

 

[winnielinnie]

Perhaps the -x (or preferably -xx) flag will help?

It tells rsync to stay within the same filesystem and not cross over.

Which means if the directory /mnt/media/movies is really a mount to another filesystem, then the "directory" movies will be ignored (even if it falls within the "tree" to be rsync'd.)

 

[Robert Thomspon]

Perhaps the -x (or preferably -xx) flag will help?

It tells rsync to stay within the same filesystem and not cross over.

Which means if the directory /mnt/media/movies is really a mount to another filesystem, then the "directory" movies will be ignored (even if it falls within the "tree" to be rsync'd.)

I'll try it in a little bit. I actually ended up just removing the path mapping as I'm moving to another server.