Zvol compression options in 12?

diskdiddler

Wizard
Joined
Jul 9, 2014
Messages
2,374
Sorry to ask - I could've sworn I read this but no idea where.
Were there some new compression options coming to 12? LZ4 is long in the tooth and I've seen many an article about much superior options which are only marginally slower - think gzip compression with lz4 speed?

Anyone know? I really must find a comprehensive "these are the cool bits for 12" article.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,176
Zstd isn't ready yet. There are a bunch of thorny issues around send/recv that haven't been addressed yet. They may be biting people in the ass if they're using sha512 or skein, since the problems are similar, but the numbers there are small enough for these not to have popped up much before.
 

Yorick

Wizard
Joined
Nov 4, 2018
Messages
1,912
You are thinking of zstd compression. This, as well as persistent L2ARC, did not make it into TrueNAS 12.0. By virtue of not being ready yet upstream.
 

diskdiddler

Wizard
Joined
Jul 9, 2014
Messages
2,374
Zstd isn't ready yet. There are a bunch of thorny issues around send/recv that haven't been addressed yet. They may be biting people in the ass if they're using sha512 or skein, since the problems are similar, but the numbers there are small enough for these not to have popped up much before.

So is this not working in beta / alpha or as in this might be more of a 12.1 / 13 thing?
Thanks for reply - it sounds like the thing I was thinking of.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,176
Definitely a later version than 12.0.
 

diskdiddler

Wizard
Joined
Jul 9, 2014
Messages
2,374
Ok I'll go looking for more information on it to follow.
 

diskdiddler

Wizard
Joined
Jul 9, 2014
Messages
2,374
You know I've just found this

and I think I'll take my time worrying about following it, sounds like politics have got involved.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,176
Let's put it this way: it's a big feature and it treads old ground. Expect second system syndrome. LZ4 is full of "I wish we'd done X differently" details.
It's also a risky feature in the sense that it touches what gets written.
Encryption took a long time, too, but it had the advantage of being totally new, whereas zstd is a nice-to-have, encryption opens great new scenarios. Zstd just makes things that bit better (I say this with no intent of minimizing its usefulness, I have plans to trial it at work as soon as it is viable).
 

diskdiddler

Wizard
Joined
Jul 9, 2014
Messages
2,374
Well I look forward to it - but sometimes these political things go haywire in open source development.
I wish them luck but it's not a good sign - looks like over 3 years of mess!

Fingers crossed, zstd, upon further reading, sounds pretty nice! (and to get my real answer, it looks like using lz4 is going to be the way to go, for the new VM I'm about to create)
 

diskdiddler

Wizard
Joined
Jul 9, 2014
Messages
2,374

ornias

Wizard
Joined
Mar 6, 2020
Messages
1,458
Talking about that discussion 2 weeks back..........


Is this relevant guys?
Yes ZSTD has been merged and I already submitted the PR's to have it added to the TrueNAS GUI as well.

I hate the way Allan uses the work of others as some PR stunt to promote himself though. It's only about 50% his work and he didn't care anymore for years until he got paid to start working on it again. DISGUSTING.

He also send in "weekly updates" to the mailinglist the last two months while other people where trying the fix the shitton of bugs his additions created (because he doesn't like to test things it seems, some where really obvious test failures), after we spend about a month getting the many breaking bugs out of his first PoC which he abandoned. So yeah, I'm kinda done with Allan's PR messages on both Twitter and the Mailinglist.
 

ornias

Wizard
Joined
Mar 6, 2020
Messages
1,458
@diskdiddler Headsup, after getting ZSTD merged into ZFS yesterday, it got merged into ZoF and TrueNAS today, so it should be an UI option in the next TrueNAS version.
 

diskdiddler

Wizard
Joined
Jul 9, 2014
Messages
2,374
@ornias
Next version as in truenas12 ?

Because right now, 12 is by far the biggest leap of improvements and changes in a single release that I've seen in 6 years.
.
Exciting, scary
 
Last edited:

diskdiddler

Wizard
Joined
Jul 9, 2014
Messages
2,374

ornias

Wizard
Joined
Mar 6, 2020
Messages
1,458
As I told already, I PR'ed/merged it into TrueNAS (12, nightlies) so it should be in the RC1

I expect TrueNAS 12 to be released before FreeBSD 13, so TrueNAS will be released with OpenZFS (and ZSTD) support before FreeBSD RELEASE.
 
Last edited:

diskdiddler

Wizard
Joined
Jul 9, 2014
Messages
2,374
Oh I see sorry, I wasn't 100% sure on that. I suspected so but it wasn't fully clear to me.
Ok well that's very good and very rapid progress too!
 
Top