Replication and out of space

[jms123]

TrueNAS 12.0-U6.1

2 TrueNAS servers - a primary and a backup server which I am trying to replicate a zvol to. The zvol is shared out with iSCSI to a cluster and the space used as reported by the cluster software is 246GB.

Please note I have used 80% of the pool space for the zvol. I know (from other posts on here :)) that I should use 50% max of the pool for the zvol) but this is a test environment and will never be used in production.

Primary -

Code:

root@truenas[~]# zfs list -o space,quota  TestVPS-Pool
NAME          AVAIL   USED  USEDSNAP  USEDDS  USEDREFRESERV  USEDCHILD  QUOTA
TestVPS-Pool   178G   721G        0B     96K             0B       721G   none
root@truenas[~]# zfs list -o space,quota  TestVPS-Pool/TestVPS-Zvol
NAME                       AVAIL   USED  USEDSNAP  USEDDS  USEDREFRESERV  USEDCHILD  QUOTA
TestVPS-Pool/TestVPS-Zvol   666G   721G        0B    233G           488G         0B      -

Backup -

Code:

root@truenas[~]# zfs list -o space,quota  TestVPS-Pool
NAME          AVAIL   USED  USEDSNAP  USEDDS  USEDREFRESERV  USEDCHILD  QUOTA
TestVPS-Pool   178G   721G        0B     96K             0B       721G   none
root@truenas[~]# zfs list -o space,quota  TestVPS-Pool/TestVPS-Zvol
NAME                       AVAIL   USED  USEDSNAP  USEDDS  USEDREFRESERV  USEDCHILD  QUOTA
TestVPS-Pool/TestVPS-Zvol   899G   721G        0B     56K           721G         0B      -

the backup server zvol has nothing it in at present, it is a new pool and zvol.

Can anyone help me understand why when I try to replicate it says it cannot snapshot as it is out of space and there seems to be more than enough space to copy the data across together with a snapshot.

Any pointers much appreciated

 

[Whattteva]

This is a zvol limitation. Look at the "REFER" column when you do zfs list -t volume <path-to-your-zvol>

If you don't have at least as much space as what is specified in the "REFER" column, you can't take a snapshot of it.

I believe, you can, however just straight zfs send/recv a ZVOL without needing a snapshot.

 

[jms123]

Code:

root@truenas[/dev/zvol/TestVPS-Pool]# zfs list -t volume
NAME                        USED  AVAIL     REFER  MOUNTPOINT
TestVPS-Pool/TestVPS-Zvol   721G   666G      233G  -

Unless I am misunderstanding I have 666GB available so it should work ?

I don't actually have 666GB available to the cluster I have 710GB - 246GB so 464GB but either way I have enough space as far as I can see.

That said if I could just send the data without snapshots that would suit me fine but whenever I try to setup a replication task it includes snapshots.

 

[Whattteva]

That said if I could just send the data without snapshots that would suit me fine but whenever I try to setup a replication task it includes snapshots.

Ah you may have to do this from the CLI using just plain zfs send | zfs recv.

 

[jms123]

I really appreciate the help but even on the CLI it still seems to want a snapshot.

I still can't understand why it says out of space as the data would just be copied across from one zvol to the other and the snapshot would be relatively small based on what I have seen so far in testing so completely baffled as to why it keeps reporting out of space.

I will reduce the disk usage on the cluster until it will replicate and then see if I can work out what the cut off point is but at the moment have to say it appears to be completely counterintuitive as to what it is doing.

My understanding of replication is the snapshot is only the metadata so when you replicate a snapshot is created and the actual data changes are copied across to the backup server together with the snapshot.

Perhaps I have got that wrong.