How to replicate data (send| recv) but not migrate properties?

[Stilez]

Suppose I have a pool with a bunch of nested zvols and datasets. The data in them was saved with various settings (dedup=on/off, compression=whatever, atime=whatever, casesensitivity=whatever).

I now want to run a zfs send -R Old_pool | zfs recv New_pool, but I want the new pool to have specific properties. SAo I create the datasets with those properties, and zfs receive the stream into it, using -x to prevent the old values in the stream having an effect. It doesnt work.

zfs send -R deletes and recreates datasets, if it runs without -I, and also replicates properties by default (implicit). There doesnt seem to be a way to recursively send that doesnt send properties as well. zfs recv -x should ignore properties in the sent stream but i find it deletes/overrides the properties in the recipient pool rather than leaving them untouched.

Is there a better way to do this, than manually adding "-o (list every damn property)" to the zfs recv command?

 

[Chris Moore]

Suppose I have a pool with a bunch of nested zvols and datasets.

I just did a send and receive between an older system and a newer system over the network and used this method:

Replication Stream Compression

Hi All, This is just a bit of knowledge sharing so hope somebody finds this helpful. For a long time now all my systems have been using LZ4 compression during replication as after testing (a long time ago) this seemed to be a good idea. I must add that all my datasets are LZ4 by default...

www.truenas.com

I only had a top-level dataset, but in the old pool it was set as LZ4 compression but in the new pool I had it set to the new Zstandard compression. That setting was not overwritten during the process.

Possibly worth looking at.

 

[Stilez]

I just did a send and receive between an older system and a newer system over the network and used this method:

Replication Stream Compression

Hi All, This is just a bit of knowledge sharing so hope somebody finds this helpful. For a long time now all my systems have been using LZ4 compression during replication as after testing (a long time ago) this seemed to be a good idea. I must add that all my datasets are LZ4 by default...

www.truenas.com

I only had a top-level dataset, but in the old pool it was set as LZ4 compression but in the new pool I had it set to the new Zstandard compression. That setting was not overwritten during the process.

Possibly worth looking at.

Not really, though I appreciate the help. The problem is that the options to replicate *recursively* also seem to insist on replicating the source's properties.

The code in that thread doesn't do any of that, its a bare send/receive of one dataset for speed, which doesn't implicitly create new sub datasets or their properties. So the question of whether those new destination subdatasets (created as needed, or pre-created but empty) get their properties from inheritance, or from the source subdataset being ZFS SENT is moot.

 

[ilmarmors]

Not really, though I appreciate the help. The problem is that the options to replicate *recursively* also seem to insist on replicating the source's properties.

Yes, -R implies -p by design. See https://openzfs.github.io/openzfs-docs/man/8/zfs-send.8.html

-R, --replicate
Generate a replication stream package, which will replicate the specified file system, and all descendent file systems, up to the named snapshot. When received, all properties, snapshots, descendent file systems, and clones are preserved.

-p, --props
Include the dataset's properties in the stream. This flag is implicit when -R is specified.

 

[Stilez]

Yes, -R implies -p by design. See https://openzfs.github.io/openzfs-docs/man/8/zfs-send.8.html

Well, we know this. The question is, what's the right way to copy a pool (or top level dataset) recursively, and not override properties that are desired on the destination (affecting blocks written during the ZFS receuve()

 

[Arwen]

Try something like this;

Code:

zfs send  -Rpv  ${MY_SRC_POOL}@${MY_SNAP} | \
  zfs receive -dFu -x copies ${MY_DEST_POOL}

The important part is "-x copies", or whatever option you are trying to avoid. So if your new pool uses ZSTD compression, but the source pool uses LZ4 you could try something like "-x compression". Then any newly created child datasets of the new pool would inherit ZSTD, (in theory... have not tested it).

 

[Stilez]

Try something like this;

Code:

zfs send  -Rpv  ${MY_SRC_POOL}@${MY_SNAP} | \
  zfs receive -dFu -x copies ${MY_DEST_POOL}

The important part is "-x copies", or whatever option you are trying to avoid. So if your new pool uses ZSTD compression, but the source pool uses LZ4 you could try something like "-x compression". Then any newly created child datasets of the new pool would inherit ZSTD, (in theory... have not tested it).

I've tried -x, and the problem seems to be that -R causes it to delete and recreate the target dataset, so that even if properties aren't copied over, the resulting replicate doesn't have the properties required.

As in, I created an empty TARGETPOOL/SUBDATASET3 with the properies desired (eg say dedup=sha512 or compression=zstd), and did exactly as you also thought would work, with -x dedup -x compression,.... but it ends up with a copy of the data migrating tio TARGETPOOL/SUBDATASET3 with property dedup=off.

Hence why "zfs recv -x should ignore properties in the sent stream but i find it deletes/overrides the properties in the recipient pool rather than leaving them untouched" in the OP.....

 

[Arwen]

@Stilez Hmm. I've not used or played with ZFS send / receive much, (except to script copying my root pool to alternate media).

Have you tried this?

zfs create TARGETPOOL/TOPDS
zfs set compression=zstd TARGETPOOL/TOPDS
zfs set dedup=sha512 TARGETPOOL/TOPDS
zfs create TARGETPOOL/TOPDS/SUBDATASET3

Verify that child dataset TARGETPOOL/TOPDS/SUBDATASET3 inherited the new values for "dedup" and "compression";
zfs get all TARGETPOOL/TOPDS/SUBDATASET3 | egrep "dedup|compression"

Replace XXX below with the proper parameters
zfs send XXX | zfs receive XXX -x dedup -x compression TARGETPOOL/TOPDS/SUBDATASET3

Not saying it will work... nor a great solution. But it might work.

 

[Stilez]

That is *precisely* what I was doing. Create TARGETPOOL and its sub datasets, with desired properties, verify them via get all | grep, and then send over the data using -x options.

Before the send, TARGET/SUBDATASET has dedup sha512, if I recheck after it starts receiving, that's now been changed to dedup=off.

My best guess is that recursive send can only be done with -R, -R seems to delete and recreate the target DS, and I have no idea what happens to inheritance and why that's not solving it. That's about the point where I was at when I came here to ask help

 

[Arwen]

@Stilez

No, what I mean is that the PARENT of the target must have the ZFS settings. Try it on the top level dataset, (named the same as the pool).

Are you using this?

zfs set dedup=sh512 TARGET
zfs set compression=zstd TARGET
zfs create TARGET/SUBDATASET

Then send to "TARGET/SUBDATASET" with the proper excludes.

Again not saying it will work, but inheritance is tricky.

 

[Stilez]

@Stilez

No, what I mean is that the PARENT of the target must have the ZFS settings. Try it on the top level dataset, (named the same as the pool).

Are you using this?

zfs set dedup=sh512 TARGET
zfs set compression=zstd TARGET
zfs create TARGET/SUBDATASET

Then send to "TARGET/SUBDATASET" with the proper excludes.

Again not saying it will work, but inheritance is tricky.

Hmm.
The top level dataset contains a bunch of sub-datasets. Are we saying you have to do a separate send -R /recv -x for every 2nd level object in the pool, rather than a single send -R / recv -x for the top level dataset? That replicating at 2nd level works but not at top level, for preserving properties? I'd be surprised, it's possible. What's the thinking? Can you explain what you have in mind and if I've understood you?

 

[Arwen]

I don't know if I can explain it. Since your target was a 2nd level, I suggested changing the top level with the parameters you wanted. If however, your intent is to copy pool to pool, then your target is wrong.

The one time I used "-x copies", it worked. My source had "copies=2" for some datasets, and I did not want that on the destination. It just worked.

It's possible you have found a bug.

 

[Stilez]

I don't know if I can explain it. Since your target was a 2nd level, I suggested changing the top level with the parameters you wanted. If however, your intent is to copy pool to pool, then your target is wrong.

It's possible you have found a bug.

I meant, copy pool to pool, but when it hits SUBDATASET this is what happens. I shortcutted that by saying send/recv POOL/SUBDATASET but what I really meant was send/recv POOL and then eventually see what happens when it gets to POOL/SUBDATASET. Maybe that confused things, its a good point.

Bug? Dont know. What do you think, now Ive added that info?

 

[Arwen]

Sorry, I am just out of my depth now.