rsync keeps sending data with no changes between files

[simdim]

Hi All,
I am setting up rsync between TrueNAS-12.0-U8.1 and Ubuntu 18.04.6 LTS as a backup of truenas pool
Test run on a small set was successful and once complete every consecutive run has no files transferred if there are no changes .
On a large set (about 6TB) initial rsync fas successful, however with no changes to the files next run transfers something since used space on the target grows.

rsync is set as push via ssh

/usr/local/bin/rsync --delay-updates -r -t --progress --exclude=.BIN --exclude=.Bin --exclude=.AppleDB --exclude=.AppleDesktop --exclude=.AppleDouble --exclude=.com.apple.timemachine.supported --exclude=.dbfseventsd--exclude=.DocumentRevisions-V100* --exclude=.DS_Store --exclude=.fseventsd --exclude=.PKInstallSandboxManager --exclude=.Spotlight* --exclude=.SymAV* --exclude=.symSchedScanLockxz --exclude=.TemporaryItems --exclude=.Trash* --exclude=.vol--exclude=.VolumeIcon.icns --exclude=Desktop DB --exclude=Desktop DF --exclude=hiberfil.sys --exclude=lost+found --exclude=Network Trash Folder --exclude=pagefile.sys --exclude=Recycled --exclude=RECYCLER --exclude=System Volume Information --exclude=Temporary Items --exclude=Thumbs.db --omit-dir-times -e ssh -p 22 -oBatchMode=yes -o StrictHostKeyChecking=yes /mnt/BigDisk/NetworkShare storage@x.x.x.x:/media/user/Elements


If there are no changes what is it copying?

 

[Patrick M. Hausen]

Run with -v, -vv, -vvv ... and watch what it does?

 

[simdim]

Should I just run it from shell instead of "Run now" in "Tasks" ? Or it created a log as it goes?

 

[Patrick M. Hausen]

Exactly. Run from shell and increase verbosity.

 

[simdim]

The errors I am getting:

rsync: mkstemp failed: Operation not permitted (1)

sounds like a permission issue to me but where?

 

[winnielinnie]

Might this possibly be the reason? You're jumping between ZFS and I'm assuming ext4?

I have found under linux that when transferring files across different fs types, the mkstemp call can fail due to a valid name on the source system being invalid on the target system.

This means even a touch of the filename will fail & not all fs seem to accept escape sequences to avoid this. I verify that I can touch into the target folder, but it fails when I incorporate the first invalid character (eg, ':') even when escaped.

What if you initiate a "dry-run" (with the -n flag). Does it show a list of files that will be transferred? You'll have to remove the verbosity options so that it only displays changes, not every single file.

 

[simdim]

I kept external disk NTFS. This is pure portable backup - something that I can remove and take in case of total disaster.
I can format it to something different but wanted to keep it readable on UX MacOS and Windows, so my choices are somewhat limited.
Journaling is not critical for this use case so maybe exFat will behave better?

 

[winnielinnie]

I kept external disk NTFS.

This probably explains it.

I can format it to something different but wanted to keep it readable on UX MacOS and Windows, so my choices are somewhat limited.
Journaling is not critical for this use case so maybe exFat will behave better?

exFAT would be "better", but still might introduce issues if there is an incompatibility with attributes and/or filenames between the source (ZFS) and destination (exFAT).

There's also the problem of the "modified time" accuracy. You might have to include another rsync option:
--modify-window=1

I'm not sure if it's needed for exFAT (it is for FAT32), but I haven't played around with it recently.


If you're going to use exFAT, make sure to format the drive with Samsung's version. (Usually named "exfatprogs" in the distro's package manager, to differentiate it from "exfat-utils".)

EDIT: Unfortunately, exfatprogs is only available for Ubuntu 21.04 and newer. You're stuck with the FUSE version (exfat-utils). (Officially in mainline since kernel 5.7).

 

[simdim]

Thank you all will try with exFAT

 

[winnielinnie]

If it means anything, Ubuntu 22.04 LTS was released recently.

It includes exfatprogs in its repository.