OpenZFS improvements coming to an OS near you

[Arwen]

While poking around at the ZFS on Linux native encryption, I saw a few interesting things;

• DRAID-{1,2,3}
  
• ZIL pipelining improvements, (this reduces locked code block sizes / times)
• ZFS Channel Programs, (atomic operations in the kernel)
• Exotic scrub bugs getting fixed
• Exotic resilver bugs getting fixed
• Device evactuation/removal, (but not the block pointer re-write type)
• Sequential Scrubs and Resilvers
• Plus, some ZFS native encryption additional support issues

To be clear, some if not most of these are many months, potentially greater than a year, before we would see them in FreeNAS. (If ever.)

For those interested, look at these 2 sites;

https://github.com/openzfs/openzfs/pulls
https://github.com/zfsonlinux/zfs/pulls

That DRAID is pretty out there, in the sense it takes time to wrap your brain around the concept. It's really only useful for configurations that use lots of disks, (greater than 15), and more or less requires integrated spare disk(s).

 

[Stux]

I wonder if the side-lining of btrfs will mean more focus goes to ZFS...

 

[Ericloewe]

DRAID is a weird thing. Basically, instead of a real spare drive, every disk has a bit of reserved space marked as spare. Once a disk is resilvered, it's resilvered onto the spare space on all disks - this supposedly speeds up rebuilds because the write load is distributed over the entire vdev. That uses up one of the virtual spares. To replace the spare, a new drive is inserted and populated like the old one, restoring the reserved space on all disks.

Sequential scrub/resilver is a non-obvious name for a simple speed improvement: Instead of scrubbing in the order that things are found on the filesystem, a cache is kept in RAM and large, contiguous blocks on disk are issued at once, turning scrubs/resilvers into a much more sequential workload.

Device removal is an ugly hack that creates a virtual vdev to where the contents of a mirror vdev are moved, so that it can be removed. Definitely not recommended. Also, it does not work at all for RAIDZ.

 

[Arwen]

I wonder if the side-lining of btrfs will mean more focus goes to ZFS...

I hope so. Based on how long BTRFS has been in development, I don't expect the RAID-5/6, (and potentially 3rd parity type), to be available AND production ready until 2024. That's 7 years. Except that it's been at least 5 years since they started working on the RAID-5/6 code.

Part of my thoughts is that ZFS has the concept of RAID-Zx vDevs. If you want 16 disks, you can use 2 x 8 disk RAID-Z2 vDevs. We all agree that a single 16 disk RAID-Z2/3 vDev is not a good idea. Except how is BTRFS planning on dealing with that amount of disks?

DRAID is a weird thing. Basically, instead of a real spare drive, every disk has a bit of reserved space marked as spare. Once a disk is resilvered, it's resilvered onto the spare space on all disks - this supposedly speeds up rebuilds because the write load is distributed over the entire vdev. That uses up one of the virtual spares. To replace the spare, a new drive is inserted and populated like the old one, restoring the reserved space on all disks.
...

You absolutely said it better than I could have done.

...
Device removal is an ugly hack that creates a virtual vdev to where the contents of a mirror vdev are moved, so that it can be removed. Definitely not recommended. Also, it does not work at all for RAIDZ.

Yes and agreed. But, if the disk to remove, is from a mistaken add, the amount of indirect references the ugly hack adds, should be minimal.

 

[JoeAtWork]

I wonder if the side-lining of btrfs will mean more focus goes to ZFS...

BTRFS is a joke, on one of our tier 1 email servers with SUSE Enterprise I have already ran into issues. not good

 

[JoeAtWork]

if we can get DRAID and the ability to use dedupe via a dedicated SSD pool for the metadata that would be a huge benefit.