Snapshot replication on push "is running" but no snapshots on pull system present

[bramble]

Hi,

2 x physical FreeNas 9.10 Stable System
system1 holds productive data and is configured as iscsi target.
system 1 is replicating to System 2 every 30 minutes

On System1 replication is showed as running with no errors.
On System2 snapshots are visible on web-GUI as expected.

BUT when trying to mount a snaphot on System2 for example, there is no physical (mountable snapshot) file!
e.g.: system1: /mnt/vol1/dataset1/datafile
the directory structure on system2 is available except that "datafile" doesnt exist (even anywhere else on the filesystem)

how is it possible that system1 says that replication is running and system2 says it has snapshots at the specific directory
but the physical file (e.g. "datafile, or snapshot-clone") is missing.

 

[nojohnny101]

do you have the "recursive" box checked on the periodic snapshot task?

 

[bramble]

yes, "recursive..." and "delete stale..." are both checked

 

[depasseg]

BUT when trying to mount a snaphot on System2 for example, there is no physical file!

What do you mean by "mount a snapshot"? Can you provide the output of zfs list for both systems?

 

[nojohnny101]

yes please walk us through exactly what you are doing to try and access the files on the PULL machine.

You should be cloning the snapshot on PULL, setting up a share (whatever service you want), setting appropriate permissions on the cloned dataset, then browsing the files.

 

[bramble]

I hope i can explain it like that:

when i ssh connect to the pull system (system2) and i cd at the pull system to:
/mnt/vol1/dataset/

the directory where the copy of the "datafile" from the push system should be located
at the shell console i list with ls -al
the file(s) is not found .
also the hidden .zfs directory is empty

Although, zfs list output shows the files are there???

 

[depasseg]

If you look at the output of zfs list it will show mount points for each dataset. Can you also provide the output of zfs mount

 

[bramble]

above u see what the pull system shows for the latest snaphots

here ist the output of the zfs list command on the pull system:

vol1/dataset1 8.57T 5.39T 279K /mnt/vol1/dataset1
vol1/dataset1/dataset1 8.57T 5.39T 279K /mnt/vol1/dataset1/dataset1
vol1/dataset1/dataset1/ISCSI-Target1 8.57T 5.39T 4.85T /mnt/vol1/dataset1/dataset1/ISCSI-Target1
vol1/jails 279K 5.39T 279K /mnt/vol1/jails

although the filenames are different from my explanation above (i used generic names trying to explain the problem)

the file "ISCSI-Target1" does not not exist when i cd to /mnt/vol1/dataset1/dataset1/ !!!!

[root@freenas-101 /mnt/vol1/dataset1/dataset1]# ls -al
total 2
drwxr-xr-x 2 root wheel 2 Apr 28 17:00 .
drwxr-xr-x 3 root wheel 3 Apr 28 17:00 ..
[root@freenas-101 /mnt/vol1/dataset1/dataset1]#

 

[depasseg]

the file "ISCSI-Target1" does not not exist when i cd to /mnt/vol1/dataset1/dataset1/ !!!!

zvols don't get mount points and won't show up in ls. So did you add that manually, or did you create a subdataset also called iscsi-target1?

 

[bramble]

@depasseg, thx for your efforts

pls find a screenshot of the push system datasets:

pls find a screenshot of the pull system datasets:

 

[depasseg]

@depasseg, thx for your efforts

pls find a screenshot of the push system datasets:
View attachment 14375


pls find a screenshot of the pull system datasets:
View attachment 14373

That looks good to me.

 

[Stux]

You should be closing the snapshot on PULL

He means cloning

 

[bramble]

here is the error message that i get, when cloning a snapshot on the pull system:

 

[depasseg]

You can't mount a zvol.

 

[bramble]

You can't mount a zvol.

what do you mean?
the above error occurs while trying to clone a snapshot on the pull system via GUI icon

in my opinion it should be possible to mount a clone?
but anyway, still there is no "ISCSI-Target1" at the pull system, but it should be there, right?

how can i check that the needed files are correctly stored in the fs without GUI or ZFS utility?

 

[depasseg]

Clone and mount are 2 different things. And your error message explicitly states that. Clone was successful but mounting was not successful. You can clone a zvol, you can not mount a zvol. It's a block device. You can present it as an iscsi target.

but anyway, still there is no "ISCSI-Target1" at the pull system, but it should be there, right?

It is there. It's in the zfs list output and in the gui and the size is larger than the original (I'm guessing because you aren't pruning expired snapshots).

the file "ISCSI-Target1" does not not exist when i cd to /mnt/vol1/dataset1/dataset1/ !!!!

Which is correct. It shouldn't. Because it is not a file. And you can't see it on your push system either if you try to 'ls'.

 

[bramble]

hmm i am confused.
so, how can i get a snasphot clone mounted on the system where my replicated snapshots are stored?

e.g. the push system (my system1) is broken and i need the replicated snapshots to get my iscsi target back online!

please find a screenshot from my push system:
as you can see, there is a file named ISCSI-Storage1

 

[depasseg]

That isn't your zvol that's a filesystem mountpoint that exists because you tried to mount it. Your output of zfs list shows that mountpoint for that zvol.

Show the results of ls -la. or try renaming it and then running zfs list again.

Or zfs mount:

Can you also provide the output of zfs mount

 

[bramble]

here's the output of ls -la on push(system1)

here's the output of zfs mount on push:

 

[depasseg]

Compare the size of that iSCSI-Target1 to the size of the zvoll in the GUI. They are not the same thing.