Hello,
I hope someone can help me understand what went wrong in my situation.
Last night, after running my first rsync task for almost three days, the process was interrupted (an error of some sort). This left hundreds of thousands of files orphaned in .~tmp~ folders, but none of them had been moved into place.
FreeNAS uses the --delay-updates option which tries to make the entire process more atomic. All the downloaded files are left in .~tmp~ until the transfer is done. When my task terminated early, rsync never moved the good files into place.
I looked at the process list and saw that FreeNAS is invoking rsync with the --delay-updates switch, and I suspect that's what's biting me here.
Furthermore, upon reading, --delay-updates disables rsync's 'incremental recursion', which forces it to know the entire file list. I sometimes have millions of files. It seems like --delay-updates is a bad option to have hardcoded on by default, but maybe I am not understanding something.
Is there a way to remove it? Am I approaching this wrong? I know a lot of this problem is me trying to rsync a large initial seed. Maybe I can look into doing a straight up file copy for the first run.
I hope someone can help me understand what went wrong in my situation.
Last night, after running my first rsync task for almost three days, the process was interrupted (an error of some sort). This left hundreds of thousands of files orphaned in .~tmp~ folders, but none of them had been moved into place.
FreeNAS uses the --delay-updates option which tries to make the entire process more atomic. All the downloaded files are left in .~tmp~ until the transfer is done. When my task terminated early, rsync never moved the good files into place.
I looked at the process list and saw that FreeNAS is invoking rsync with the --delay-updates switch, and I suspect that's what's biting me here.
Furthermore, upon reading, --delay-updates disables rsync's 'incremental recursion', which forces it to know the entire file list. I sometimes have millions of files. It seems like --delay-updates is a bad option to have hardcoded on by default, but maybe I am not understanding something.
Is there a way to remove it? Am I approaching this wrong? I know a lot of this problem is me trying to rsync a large initial seed. Maybe I can look into doing a straight up file copy for the first run.
