Thibaut
Dabbler
- Joined
- Jun 21, 2014
- Messages
- 33
Hello,
Our company is active in the video editing domain, we're making use of zvols that are shared through iSCSI using FreeNAS, one zvol being created for each project. Once the zvol is available through iSCSI, it is mounted on a workstation and formatted with a NTFS partition that is used for the video editing process.
Once a project is finished, the corresponding zvol is transferred to an archive server running ZoL on Debian (10). Since all transfers are made on our local network, we're using netcat to achieve faster transfer rates than with ssh, the effective transfer is done issuing commands on both the receiving (archive) system and the sending (FreeNAS server) system, starting on the receiving end to initiate the connection as follows :
On the receiving system (ZoL / Debian 10):
On the sending system (FreeNAS server 11.2-U8)
Once the zvol is available on the archive server, the original gets removed from the FreeNAS server.
One important thing to note is that, on the receiving server running ZoL, the NTFS partition becomes "visible", since the default vfs.zfs.vol.mode value is 1 (geom), whereas FreeNAS uses a default of 2 (dev). Thus, on the FreeNAS server, the zvol's corresponding /dev/ entry only displays the volume itself:
While the receiving system displays both the volume and it's partition:
We rely on this feature that allows us to mount the partition on the archive system to now and then access old jobs and recover some material, simply using a command like:
THE PROBLEM:
Recently we started observing that, using the exact same process described here above, which we've been using for years, the partition doesn't appear on the archive server anymore!
Although the volume's data is transferred, as can be seen by listing the volume:
I wasn't able to find a way to access the NTFS partition, using gdisk to recreate a GPT partition on top of the existing data didn't help, issuing a message when attempting to mount, stating:
In the end, the only way I found to finally access the partition's data, which DOES EXIST, was performing data recovery using testdisk on the gdisk re-created, still unmountable, partition.
I have absolutely no clue regarding what causes this new, very annoying, behavior i.e. not showing the partition and rendering it inaccessible in ZoL, although the data is well there as can be attested by the fact that testdisk is able to access it.
As a side note, we currently have volumes showing the "-part1" and others not showing it on the same archive pool, The only noticeable change that occurred since we observe the current situation was updating FreeNAS to 11.2-U8, although we cannot say for sure that the behavior changed right after the upgrade.
Any explanation, suggestion or pointing to a potential way to solve this situation would be greatly appreciated!
Thank you.
Our company is active in the video editing domain, we're making use of zvols that are shared through iSCSI using FreeNAS, one zvol being created for each project. Once the zvol is available through iSCSI, it is mounted on a workstation and formatted with a NTFS partition that is used for the video editing process.
Once a project is finished, the corresponding zvol is transferred to an archive server running ZoL on Debian (10). Since all transfers are made on our local network, we're using netcat to achieve faster transfer rates than with ssh, the effective transfer is done issuing commands on both the receiving (archive) system and the sending (FreeNAS server) system, starting on the receiving end to initiate the connection as follows :
On the receiving system (ZoL / Debian 10):
Code:
# nc -lp 44444 | \ mbuffer -q -s 128k -m 1G | \ pv -rtab | \ zfs receive -F ARCHIVEpool/JobID
On the sending system (FreeNAS server 11.2-U8)
Code:
# zfs set readonly=on FNASpool/JobID && \ zfs snapshot -r FNASpool/JobID@DATE && \ zfs send -R FNASpool/JobID@DATE | \ mbuffer -q -s 128k -m 1G | \ pv -rtab | \ nc -N xxx.xxx.xxx.xxx 44444
Once the zvol is available on the archive server, the original gets removed from the FreeNAS server.
One important thing to note is that, on the receiving server running ZoL, the NTFS partition becomes "visible", since the default vfs.zfs.vol.mode value is 1 (geom), whereas FreeNAS uses a default of 2 (dev). Thus, on the FreeNAS server, the zvol's corresponding /dev/ entry only displays the volume itself:
Code:
/dev/zvol/FNASpool/JobID
While the receiving system displays both the volume and it's partition:
Code:
/dev/zvol/ARCHIVEpool/jobID /dev/zvol/ARCHIVEpool/jobID-part1
We rely on this feature that allows us to mount the partition on the archive system to now and then access old jobs and recover some material, simply using a command like:
Code:
mount -t ntfs-3g /dev/ARCHIVEpool/JobID-part1 /mnt/JobID-HD
THE PROBLEM:
Recently we started observing that, using the exact same process described here above, which we've been using for years, the partition doesn't appear on the archive server anymore!
Although the volume's data is transferred, as can be seen by listing the volume:
Code:
# zfs list ARCHIVEpool/JobID NAME USED AVAIL REFER MOUNTPOINT JobID 180G 77.7T 180G -
I wasn't able to find a way to access the NTFS partition, using gdisk to recreate a GPT partition on top of the existing data didn't help, issuing a message when attempting to mount, stating:
Code:
NTFS signature is missing. Failed to mount '/dev/zd1280p1': Invalid argument The device '/dev/zd1280p1' doesn't seem to have a valid NTFS. Maybe the wrong device is used? Or the whole disk instead of a partition (e.g. /dev/sda, not /dev/sda1)? Or the other way around?
In the end, the only way I found to finally access the partition's data, which DOES EXIST, was performing data recovery using testdisk on the gdisk re-created, still unmountable, partition.
I have absolutely no clue regarding what causes this new, very annoying, behavior i.e. not showing the partition and rendering it inaccessible in ZoL, although the data is well there as can be attested by the fact that testdisk is able to access it.
As a side note, we currently have volumes showing the "-part1" and others not showing it on the same archive pool, The only noticeable change that occurred since we observe the current situation was updating FreeNAS to 11.2-U8, although we cannot say for sure that the behavior changed right after the upgrade.
Any explanation, suggestion or pointing to a potential way to solve this situation would be greatly appreciated!
Thank you.
