Rsync.. then ZFS Send?

[Firebug24k]

I just rsync'd, successfully, about 21TB of data from one storage pool to another. It took a very long time, and I'd like to use zfs send/receive in the future. If I organize all the files on storage server "B" into a pool structure identical to "A", then try an incremental ZFS send, is it smart enough to recognize that the data is already there (since I did a rsync)? Or will it want to start from square one again?

 

[cyberjock]

It'll start from square one.

 

[Firebug24k]

Crap, that's what I thought, but I was hoping to get lucky. Shouldn't have done rsync in the first place.

 

[cyberjock]

I will tell you that when I did zfs send versus rsync zfs send will max out your Gb port assuming your hardware can keep up. Rsync was 20-30MB/sec on my hardware. So you may find that the zfs send will be 3-4 times faster :P

 

[Firebug24k]

Yeah, it took about six days to rysnc it. It sat at 100% CPU usage the entire time, because of "double" encryption - both SSH, and then an encrypted FS write.

I didn't initially want to do ZFS send, because I was nervous about sending from a different pool version (Solaris 11) to FreeBSD, but I tried it on a smaller pool (500GB) just now and it worked just fine. I will probably wipe the whole thing and do it again with ZFS send, because Rsync is taking many many hours to generate the incremental file list (there's millions of files total, a very large image database with metadata) even whenthe total changes are just a few GB, making it impractical to keep doing backups this way.

Both machines are virtual machines on one beefy ESXI box, so if I use netcat and an internal VM network that doesn't go through the physical switch, I should get pretty good performance. Will test it tonight.

 

[cyberjock]

I will probably wipe the whole thing and do it again with ZFS send, because Rsync is taking many many hours to generate the incremental file list (there's millions of files total, a very large image database with metadata) even whenthe total changes are just a few GB, making it impractical to keep doing backups this way.

And now you know why rsync isn't useful for large quantities of data. It has some limitations that get messy at large volumes. Small volumes it works great though!