SOLVED Replication to FreeBSD fails via GUI but not CLI

[monotok]

Hi All,

I hope someone can help me with this replication issue. I can't seem to find anyone else having the same error messages.

I have an HP Microserver Gen 8 with a Intel(R) Celeron(R) CPU G1610T @ 2.30GHz (Xeon coming soon), 16 GB ECC RAM, 1x4TB WD Red (Will be getting another very soon) and FreeNAS-9.10.2 (a476f16). The remote host is a personal dedicated server from Online.net with 1 TB disk and 4 GB non ECC RAM (I know it should be ECC but I have backups of the data on USB HDD at work too so if it goes bang then I will rebuild it from the Microserver) and running FreeBSD 11.

This works well except for replication, I followed the manual and the CLI commands work within the troubleshooting section. I ran the below command it succeeded in creating a snapshot on the remote host. I have done this for other volumes too although it seems compression didn't copy.

zfs send OneDrive/RaspPhotos@auto-20161222.0900-2m | ssh -i /data/ssh/replication remotehost.com -p 22 zfs receive backup/RaspPhotos@auto-20161222.0900-2m

zfs list -t snapshot (PUSH)

NAME USED AVAIL REFER MOUNTPOINT
OneDrive/RaspDocuments@auto-20161222.0900-2m 8.34M - 7.28G -
OneDrive/RaspDocuments@auto-20161223.1200-2m 8.64M - 7.33G -
OneDrive/RaspDocuments@auto-20161227.1331-2m 0 - 7.45G -
OneDrive/RaspGeneral@auto-20161222.0900-2m 2.28M - 176G -
OneDrive/RaspPhotos@auto-20161222.0900-2m 16.5M - 129G -
OneDrive/RaspPhotos@auto-20161223.1200-2m 6.86M - 129G -
OneDrive/RaspPhotos@auto-20161227.1331-2m 0 - 129G -
OneDrive/Software@auto-20161220.0900-2m 192K - 75.7G -



zfs list -t snapshot (PULL)

NAME USED AVAIL REFER MOUNTPOINT
backup/RaspDocuments@auto-20161222.0900-2m 0 - 9.06G -
backup/RaspGeneral@auto-20161222.0900-2m 0 - 198G -
backup/RaspPhotos@auto-20161222.0900-2m 0 - 132G -

Running this via the GUI produces several errors. If I didn't have a dataset on the remote side with the same name then it refused to create it due to permissions which I solved by using "root" although the user had zfs allow permissions. I don't mind logging in manually to create the datasets anyway as there are not many. Currently I am using root to do this.

If the dataset exists then I get this.

python /usr/local/www/freenasUI/tools/autorepl.py
warning: cannot send 'OneDrive/Software@auto-20161220.0900-2m': signal received

Dec 27 23:52:27 nas autorepl.py: [tools.autorepl:304] Checking dataset OneDrive/Software
Dec 27 23:52:27 nas autorepl.py: [common.pipesubr:66] Popen()ing: /sbin/zfs list -H -t snapshot -p -o name,creation -r -d 1 "OneDrive/Software"
Dec 27 23:52:27 nas autorepl.py: [tools.autorepl:336] ds = Software, remotefs = backup
Dec 27 23:52:28 nas autorepl.py: [tools.autorepl:344] Unable to create remote dataset backup: cannot create 'backup/Software': dataset already exists
Dec 27 23:52:28 nas autorepl.py: [common.pipesubr:66] Popen()ing: /usr/local/bin/ssh -i /data/ssh/replication -o BatchMode=yes -o StrictHostKeyChecking=yes -o ConnectTimeout=7 -p 22 remotehost.com "zfs list -H -t snapshot -p -o name,creation -d 1 -r 'backup/Software'"
Dec 27 23:52:28 nas autorepl.py: [tools.autorepl:138] Sending zfs snapshot: /sbin/zfs send -V -p OneDrive/Software@auto-20161220.0900-2m | /usr/local/bin/lz4c | /bin/dd obs=1m 2> /dev/null | /bin/dd obs=1m 2> /dev/null | /usr/local/bin/pipewatcher $$ | /usr/local/bin/ssh -i /data/ssh/replication -o BatchMode=yes -o StrictHostKeyChecking=yes -o ConnectTimeout=7 -p 22 remotehost.com "/usr/bin/env lz4c -d | /sbin/zfs receive -F -d 'backup' && echo Succeeded"
Dec 27 23:52:29 nas autorepl.py: [tools.autorepl:157] Replication result: env: lz4c: No such file or directory
cannot receive: failed to read from stream
Failed to write to stdout: Broken pipe
Error 35 : Write error : cannot write compressed block
Dec 27 23:52:29 nas autorepl.py: [tools.autorepl:625] Autosnap replication finished

Even doing an incremental on the CLI works fine.

zfs send -v -i OneDrive/RaspDocuments@auto-20161222.0900-2m OneDrive/RaspDocuments@auto-20161227.1331-2m | ssh -i /data/ssh/replication remotehost.com -p 22 zfs receive backup/RaspDocuments
send from @auto-20161222.0900-2m to OneDrive/RaspDocuments@auto-20161227.1331-2m estimated size is 905M
total estimated size is 905M
TIME SENT SNAPSHOT
23:59:23 2.23M OneDrive/RaspDocuments@auto-20161227.1331-2m
23:59:24 3.35M OneDrive/RaspDocuments@auto-20161227.1331-2m
23:59:25 4.35M OneDrive/RaspDocuments@auto-20161227.1331-2m
23:59:26 5.36M OneDrive/RaspDocuments@auto-20161227.1331-2m

 

[m0nkey_]

Replication result: env: lz4c: No such file or directory

Make sure you have lzip installed on the destination box.

 

[monotok]

Make sure you have lzip installed on the destination box.

Brilliant! Thank you, Lzip wasn't the package I needed but that led me down the correct path. The package I was missing was liblz4-131.

It is now running happily :D