kayot
Dabbler
- Joined
- Nov 29, 2014
- Messages
- 36
So I wanted to use deduplication. But everyone and their uncle is telling me it would be a colossal mistake. I'm not stupid enough to ignore that many people.
When I was first conceptualizing my server build, one question I kept asking was what OS should I use?
Windows Server 2012R2 or FreeNAS 9.2 (Now 9.3)
A feature I liked in Server was the asynchronous deduplication that is ran every so often, usually when the system sits idle. Rather than deal with absurd ram requirements, it just does it every now and then. That's perfect for a home server system that only stores data. It's something that gets turned on then ignored since it takes care of itself.
Unfortunately Server sucks almost everywhere else. Sure it's SMB 3.0 is multi-threaded, but that's just nice to have, not a requirement since FreeNAS can easily saturate a gigabit network one a single thread. It's Storage Spaces is a joke to humanity. If a bird sh|ts on a frog in a the woods, the SS will drop a drive for no reason other then %uck you, that's why! It's parity is slower than molasses in the winter time going up hill both ways. Sure, if I had money to blow, I'd just get a SAS Expander, setup a RAID6, and be done with it, but that's costs.
Then there is FreeNAS. Synchronous deduplication is instant, but requires a F*c^ ton of RAM. It's absurd. Sure it can use a Solid State drive and that's all nice and dandy, but that's a drive slot I could put another 3TB on. Base line I'd need 128GB of ECC ram to make a 30TB Raidz2 array (21TB available) even a remote possibility. Everything else on ZFS is nice. It handles JBOD arrays like a champ (no need to use expanders, just get another M1015 card) and with routine maintenance, data will never go bad.
So, how do you the users feel about this idea?
Have an asynchronous deduplication option.
The only downside I can see with this is fragmentation, so it would have to rewrite files to remove the holes it would make. This is fine since this would be done when the system sits idle. This is an option for home servers.
Addition:
I think they are looking into it already -> https://github.com/zfsonlinux/zfs/issues/1071
It's an older ticket, but it wasn't closed which is always a good thing.
When I was first conceptualizing my server build, one question I kept asking was what OS should I use?
Windows Server 2012R2 or FreeNAS 9.2 (Now 9.3)
A feature I liked in Server was the asynchronous deduplication that is ran every so often, usually when the system sits idle. Rather than deal with absurd ram requirements, it just does it every now and then. That's perfect for a home server system that only stores data. It's something that gets turned on then ignored since it takes care of itself.
Unfortunately Server sucks almost everywhere else. Sure it's SMB 3.0 is multi-threaded, but that's just nice to have, not a requirement since FreeNAS can easily saturate a gigabit network one a single thread. It's Storage Spaces is a joke to humanity. If a bird sh|ts on a frog in a the woods, the SS will drop a drive for no reason other then %uck you, that's why! It's parity is slower than molasses in the winter time going up hill both ways. Sure, if I had money to blow, I'd just get a SAS Expander, setup a RAID6, and be done with it, but that's costs.
Then there is FreeNAS. Synchronous deduplication is instant, but requires a F*c^ ton of RAM. It's absurd. Sure it can use a Solid State drive and that's all nice and dandy, but that's a drive slot I could put another 3TB on. Base line I'd need 128GB of ECC ram to make a 30TB Raidz2 array (21TB available) even a remote possibility. Everything else on ZFS is nice. It handles JBOD arrays like a champ (no need to use expanders, just get another M1015 card) and with routine maintenance, data will never go bad.
So, how do you the users feel about this idea?
Have an asynchronous deduplication option.
The only downside I can see with this is fragmentation, so it would have to rewrite files to remove the holes it would make. This is fine since this would be done when the system sits idle. This is an option for home servers.
Addition:
I think they are looking into it already -> https://github.com/zfsonlinux/zfs/issues/1071
It's an older ticket, but it wasn't closed which is always a good thing.