ZFS dataset missing after replication task failed to unmount share

Fab Sidoli

Contributor
Joined
May 15, 2019
Messages
114
Hi All,

During a ZFS send/receive a users share did not unmount cleanly which has resulted in their share appearing to be missing on the receiving box. Below is the redacted log for the transaction.

I'm not entirely clear on the processes involved during a ZFS s/r but I feel the result of a failed unmount should abort more cleanly and remount the share on the receiving box.

Is this a bug or expected behaviour?

[2020/07/28 00:20:14] INFO [replication_task__task_6] [zettarepl.replication.run] For replication task 'task_6': doing push from 'store/home/myUSER' to 'store/home/myUSER' of snapshot='auto-2020-07-28_00-00' incremental_base='auto-2020-07-27_00-00' receive_resume_token=None
[2020/07/28 00:20:14] DEBUG [replication_task__task_6] [zettarepl.transport.base_ssh.root@IPADDR.shell.12.async_exec.33076] Running ['zfs', 'umount', 'store/home/myUSER']
[2020/07/28 00:20:14] DEBUG [replication_task__task_6] [zettarepl.paramiko.replication_task__task_6] [chan 2580] Max packet in: 32768 bytes
[2020/07/28 00:20:14] DEBUG [Thread-34] [zettarepl.paramiko.replication_task__task_6] [chan 2580] Max packet out: 32768 bytes
[2020/07/28 00:20:14] DEBUG [Thread-34] [zettarepl.paramiko.replication_task__task_6] Secsh channel 2580 opened.
[2020/07/28 00:20:14] DEBUG [Thread-34] [zettarepl.paramiko.replication_task__task_6] [chan 2580] Sesch channel 2580 request ok
[2020/07/28 00:20:14] DEBUG [replication_task__task_6] [zettarepl.transport.base_ssh.root@IPADDR.shell.12.async_exec.33076] Reading stdout
[2020/07/28 00:20:14] DEBUG [Thread-34] [zettarepl.paramiko.replication_task__task_6] [chan 2580] EOF received (2580)
[2020/07/28 00:20:14] DEBUG [replication_task__task_6] [zettarepl.transport.base_ssh.root@IPADDR.shell.12.async_exec.33076] Waiting for exit status
[2020/07/28 00:20:14] DEBUG [Thread-34] [zettarepl.paramiko.replication_task__task_6] [chan 2580] EOF sent (2580)
[2020/07/28 00:20:14] DEBUG [replication_task__task_6] [zettarepl.transport.base_ssh.root@IPADDR.shell.12.async_exec.33076] Error 1: "cannot unmount '/mnt/store/home/myUSER': Device busy\n"
[2020/07/28 00:20:14] DEBUG [replication_task__task_6] [zettarepl.transport.local.shell.1.async_exec.33077] Running ['zfs', 'send', '-V']
[2020/07/28 00:20:14] DEBUG [replication_task__task_6] [zettarepl.transport.local.shell.1.async_exec.33077] Error 2: 'missing snapshot argument\nusag.... list, run: zfs allow|unallow\n'
[2020/07/28 00:20:14] DEBUG [replication_task__task_6] [zettarepl.transport.local.shell.1.async_exec.33079] Running ['sh', '-c', 'exec 3>&1; eval $(exec 4>&1 >&....] && exit $pipestatus1; exit 0']
[2020/07/28 00:20:16] DEBUG [replication_task__task_6.async_exec_tee.wait] [zettarepl.transport.local.shell.1.async_exec.33079] Success: None
[2020/07/28 00:20:16] DEBUG [replication_task__task_6.monitor] [zettarepl.transport.local.shell.1.async_exec.33079] Stopping

Thanks,
Fab
 

curtishall

Dabbler
Joined
Feb 7, 2020
Messages
13
Did you fix this? I have a similar issue:


[2020/11/26 23:41:38] DEBUG [replication_task__task_3] [zettarepl.transport.base_ssh.root@10.10.1.210.shell.10.async_exec.156] Waiting for exit status
[2020/11/26 23:41:38] DEBUG [replication_task__task_3] [zettarepl.transport.base_ssh.root@10.10.1.210.shell.10.async_exec.156] Error 1: "cannot unmount 'store/backup': not currently mounted\n"
[2020/11/26 23:41:38] DEBUG [replication_task__task_3] [zettarepl.paramiko.replication_task__task_3] [chan 7] EOF sent (7)
[2020/11/26 23:41:38] DEBUG [replication_task__task_3] [zettarepl.transport.local.shell.1.async_exec.157] Running ['zfs', 'send', '-V']
[2020/11/26 23:41:38] DEBUG [replication_task__task_3] [zettarepl.transport.local.shell.1.async_exec.157] Error 2: 'missing snapshot argument\nusag.... list, run: zfs allow|unallow\n'
[2020/11/26 23:41:38] DEBUG [replication_task__task_3] [zettarepl.transport.local.shell.1.async_exec.159] Running ['sh', '-c', 'exec 3>&1; eval $(exec 4>&1 >&....] && exit $pipestatus1; exit 0']
[2020/11/26 23:41:38] DEBUG [replication_task__task_3.async_exec_tee.wait] [zettarepl.transport.local.shell.1.async_exec.159] Error 255: None
[2020/11/26 23:41:38] DEBUG [replication_task__task_3.process] [zettarepl.transport.local.shell.1.async_exec.158] Error 255: "cannot resume send: Unknown er....11-15_21-12': signal received\n"
[2020/11/26 23:41:38] DEBUG [replication_task__task_3.monitor] [zettarepl.transport.local.shell.1.async_exec.159] Stopping
[2020/11/26 23:41:38] WARNING [replication_task__task_3] [zettarepl.replication.run] For task 'task_3' at attempt 2 recoverable replication error RecoverableReplicationError("cannot resume send: Unknown error: 1030\nclient_loop: send disconnect: Broken pipe\nwarning: cannot send 'tank/test@auto-2020-11-15_21-12': signal received")
[2020/11/26 23:41:38] INFO [replication_task__task_3] [zettarepl.replication.run] After recoverable error sleeping for 2 seconds
[2020/11/26 23:41:40] DEBUG [replication_task__task_3] [zettarepl.transport.local.shell.1.async_exec.160] Running ['zfs', 'get', '-H', '-p', 'type', 'tank/test']
[2020/11/26 23:41:41] DEBUG [replication_task__task_3] [zettarepl.transport.local.shell.1.async_exec.160] Success: 'tank/test\ttype\tfilesystem\t-\n'
[2020/11/26 23:41:41] DEBUG [replication_task__task_3] [zettarepl.transport.base_ssh.root@10.10.1.210.shell.10.async_exec.161] Running ['zfs', 'get', '-H', '-p', 'type', 'store/backup']
[2020/11/26 23:41:41] DEBUG [replication_task__task_3] [zettarepl.paramiko.replication_task__task_3] [chan 8] Max packet in: 32768 bytes
[2020/11/26 23:41:41] DEBUG [Thread-21] [zettarepl.paramiko.replication_task__task_3] [chan 8] Max packet out: 32768 bytes
[2020/11/26 23:41:41] DEBUG [Thread-21] [zettarepl.paramiko.replication_task__task_3] Secsh channel 8 opened.
[2020/11/26 23:41:41] DEBUG [Thread-21] [zettarepl.paramiko.replication_task__task_3] [chan 8] Sesch channel 8 request ok
[2020/11/26 23:41:41] DEBUG [replication_task__task_3] [zettarepl.transport.base_ssh.root@10.10.1.210.shell.10.async_exec.161] Reading stdout
[2020/11/26 23:41:41] DEBUG [Thread-21] [zettarepl.paramiko.replication_task__task_3] [chan 8] EOF received (8)
[2020/11/26 23:41:41] DEBUG [replication_task__task_3] [zettarepl.transport.base_ssh.root@10.10.1.210.shell.10.async_exec.161] Waiting for exit status
[2020/11/26 23:41:41] DEBUG [replication_task__task_3] [zettarepl.transport.base_ssh.root@10.10.1.210.shell.10.async_exec.161] Success: 'store/backup\ttype\tfilesystem\t-\n'
[2020/11/26 23:41:41] DEBUG [replication_task__task_3] [zettarepl.paramiko.replication_task__task_3] [chan 8] EOF sent (8)
 
Top