ZFS Deduplication

Status
Not open for further replies.

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
Fiiiiiiine...... make me suffer through figuring out vBulletin, see if I care... but yeah I've written several things about this.

I should probably disclose that I'm the primary developer behind another open source project (and one that's even sort-of a storage product, albeit a highly specialized protocol-specific one), and one that, due to its nature, is primarily used by multi-gigabit service providers... so I'm more familiar (and more cynical) about the pros and cons of free software development and how that all gets leveraged in order to help earn a living, or not, as the case may be.


I think you guys are missing the reality here.

FreeNAS is fundamentally a wrapper around FreeBSD and a bunch of ports. It's a *nice* wrapper, but it's a wrapper.

ZFS v15 is what shipped with FreeBSD 8.2R, the latest stable RELEASE on the FreeBSD 8 branch. FreeBSD 8.3R is expected to be released in early 2012. ZFS v28 has already been committed to 8-STABLE, and will therefore be part of 8.3R.

http://lists.freebsd.org/pipermail/f...ne/011673.html

Now here's the thing. iXsystems adds value to their TrueNAS product by including v28 features. That much is absolutely true.

However, when FreeBSD 8.3R comes out, there will be an eventual need to move forward with the base system, which is how you get new device drivers, bug fixes, security updates, and the like. iXsystems is probably not interested in trying to fork FreeBSD in order to keep v28 out of the base system; there will be no more FreeBSD releases with v15. So there aren't realistic paths forward that do not include ZFS v28 as part of the FreeBSD release that FreeNAS will be based on.

Even if they tried, basically there are a lot of real clever people who would just take a more recent NanoBSD build and dump the FreeNAS stuff on top of it and start hacking and distributing it. So it doesn't seem likely that they'll try.

The iXsystems value-add will eventually need to be their support capabilities/opportunities, the hardware sales, and the desire of many businesses to be able to buy an appliance and support and not have to worry about "hacking."

Along the way, they benefit from the FreeNAS community, their more-or-less willing guinea pigs. Now you can either enjoy a nearly commercial grade product for free or you can go buy a commercial NAS product for cash; generally speaking, the quality of FreeNAS is sufficiently high that even experienced admins are willing to consider it as a viable option.

But basically, when someone offers free beer, and also sells snacks alongside, it's kind of rude to complain that the snacks aren't free too. Coders need to feed their families too.
 

t3h0th3r

Dabbler
Joined
Jul 29, 2011
Messages
14
There is a patch to include ZFS v28 in FreeNAS 8.1 http://support.freenas.org/browser/freenas/branches/8.1/patches/freebsd-zfsv28.patch?rev=6033
This patch can be applied to the current source of FreeNAS 8.0 (FreeBSD 8.2-RELEASE) and from there on you can build your own FreeNAS 8.0.2 with ZFS v28.
At least theoretically, i have no idea what the effects are on the finished product, but i got it to compile with the most recent svn revision, altough i had to remove the "tws" device (LSI raid?) from the nanobsd kernel configuration, because it would generate errors about it being an unknown device... // <- apparently the rest of the patches didn't get applied correctly. i have to fiddle a bit more with them, since the 2 zfs patches for 8.0 and the v28 patch seem to interfere. since i'm no kernel developer and am only copy pasting stuff, this might take a while :)
 

Gnome

Explorer
Joined
Aug 18, 2011
Messages
87
That was checked in by jpaetzel (FreeNAS developer) in the 8.1 branch. That means 8.1 will hopefully have ZFS v28.
 

Tekkie

Patron
Joined
May 31, 2011
Messages
353
That was checked in by jpaetzel (FreeNAS developer) in the 8.1 branch. That means 8.1 will hopefully have ZFS v28.
Did it make its way to trunk as well?
 

Tekkie

Patron
Joined
May 31, 2011
Messages
353

Gnome

Explorer
Joined
Aug 18, 2011
Messages
87
I'm not 100% sure how the FreeNAS SVN works because they don't have tags/branches for the latest stable releases (something I have always had everywhere I worked).

However if they do work on the normal structure the 8.1 branch is what will eventually be released as FreeNAS 8.1. Therefore if the 8.1 branch has the patch it means that 8.1 release will probably have it.

I'm not a FreeNAS developer however, so it is pure speculation based on my experience as a software developer.
 

t3h0th3r

Dabbler
Joined
Jul 29, 2011
Messages
14
The patch i mentioned is probably a bit too old, it wants to revert a lot of stuff which has already been fixed/changed in the FreeBSD source. I tried with the newest ZFS v28 patch for 8.2-RELEASE, but this only gave me some failure during the buildworld stage.
 

AlexP

Cadet
Joined
Oct 27, 2011
Messages
5
Hello,
I'm interested to try freenas for ZFS deduplication too, but I understand it's not supported yet in the free release.

Now that freebsd 9 is out do anyone knows if there is a plan to update ZFS support on freenas?
 

ProtoSD

MVP
Joined
Jul 1, 2011
Messages
3,348
Hi Alex,

It hasn't been announced officially, but I think the tentative plan is for later this year (Fall) to either upgrade FreeNAS's base to either FreeBSD 8.3 Release or less likely FreeBSD 9.0 Release. The pluses for 9.0 vs. 8.3 would possibly be better USB 3.0 support, and hot drive swapping among other things, unless those features get back ported to 8.3 Release when it comes out.

From what I've heard, Dedupe isn't all it's cracked up to be and is a huge memory/resource hog.
 

Gnome

Explorer
Joined
Aug 18, 2011
Messages
87
From what I've heard, Dedupe isn't all it's cracked up to be and is a huge memory/resource hog.

ZFS v28 is much more than just dedup tho, performance increases, RAID-Z3 as well as many fixes.
 

ProtoSD

MVP
Joined
Jul 1, 2011
Messages
3,348
ZFS v28 is much more than just dedup tho, performance increases, RAID-Z3 as well as many fixes.

Yes, I realize that, it just seems to be the feature a lot of people are after because they think of it as an alternative to spending more money on disks when they'll end up spending more money on RAM and a better processor.

Which version of ZFS is used on current freenas 8.0.x?

Its version 15, so it's a bit behind a lot of the competition, but it should catch up later this summer.

Anyone interested in trying FreeNAS with version 28 on FreeNAS 8.0.3-p1 can take a look at my blog here:

http://protosd.blogspot.com/2012/02/freenas-803-p1-multimedia-with-zfs-v28.html


This is an Unofficial build, so please don't open any tickets or ask for support here. Send me a PM if you have a question etc.
 
Status
Not open for further replies.
Top