Local ZFS Replication to SAN-volume

[Phuncz]

For my education I have access to some hardware I've configured like this:

I have one FreeNAS server configured, with 4 data disks in a single RAIDz2 ZFS-volume and 2 small disks in a RAID1 (via controller) volume as a boot volume.
There is an HP MSA1500 24-disk SAN-solution, connected with fibre to a FC SAN Switch, the FreeNAS server has 2 fibre HBA's connected to the same switch.
I managed to get a LUN from the MSA1500 connected to FreeNAS using multipath and configured a second ZFS-volume on that one.

For future reference: to get the HBA's working on FreeNAS 8.30, they need to have their Boot ROM active, but they can be set to not allow booting from them if that is a problem. You can do this with Ctrl+Q during BIOS POST.

I'm now trying to get ZFS Replication to work locally, although the SAN isn't locally. I want to mirror the ZFS volume from the local disks to the SAN.
I'd figure since I'm able to use Fibre Channel straight from the FreeNAS server to the SAN would be a better idea then approach this over Gbit Ethernet, to allow more bandwidth when in use and have a centralized backup.

I've read topics about this on the forum, using "localhost" and the public key, but I'm still getting "broken pipe" errors, which I don't see solved anywhere. I've seen them talk about the replication user but there is no such user and I don't see any way of changing that user.

The problem is I'm stuck trying to figure out what is wrong. Permissions on both the ZFS volumes are the same, both root/wheel as owner/group-owner, both Windows ACL (maybe this should be Unix ?).

 

[Phuncz]

Any thoughts on this ? I know people have successfully done local replication, but I can't get it to work ("broken pipe"-errors).

 

[paleoN]

Any thoughts on this ? I know people have successfully done local replication, but I can't get it to work ("broken pipe"-errors).

Not really. Looks like they decided to pipe zfs send through 2 dd's in r12814:

Code:

/sbin/zfs send %s%s%s | /bin/dd obs=1m | /bin/dd obs=1m | %s -p %d %s "/sbin/zfs receive -F -d %s

 

[Phuncz]

Sorry, couldn't find what /bin/dd is exactly, but I guess you are saying it can't be done with replication.

Is there another way of duplicating a ZFS array ? Or am I forced to use ZFS on a RAID-array ?

 

[paleoN]

Sorry, couldn't find what /bin/dd is exactly, but I guess you are saying it can't be done with replication.

I'm not saying anything of the sort. You apparently did a terrible job searching for dd by the way. It's been around for some time and is a well known utility. The FreeNAS devs are using dd for some type of buffering, but I don't know if that will help for your problem as I don't know what they were fixing.

Is there another way of duplicating a ZFS array ? Or am I forced to use ZFS on a RAID-array ?

Are you trying to mirror the local devices to the SAN or do you want to backup the local pool to the SAN? If the latter, you can always use rsync if zfs send isn't working for you.

 

[Phuncz]

I'm not saying anything of the sort. You apparently did a terrible job searching for dd by the way. It's been around for some time and is a well known utility. The FreeNAS devs are using dd for some type of buffering, but I don't know if that will help for your problem as I don't know what they were fixing.

Apparently dd is well-known at Google as I was expecting it to return mostly bogus results and that's why I searched for /bin/dd, but I now understand what it is.

Are you trying to mirror the local devices to the SAN or do you want to backup the local pool to the SAN? If the latter, you can always use rsync if zfs send isn't working for you.

I'd like to mirror the entire, single ZFS volume, which is comprimised of 4 disks in RAIDz2, onto the SAN.
I'll try to get rsync running tomorrow, thanks for the advice.