Quick Question ZFS vs EXT4

[rob90033]

I see that ZFS is better than EXT4 for storage, however, it seems there is no "strongly recommended" requirements for EXT4 to have ECC memory. Does this mean it (and devices that use it) are less or not susceptible to data corruption? I am curious, as I had a synology for about 6-7 years that ran 24/7 and was kept pretty busy. It never missed a lick and for what it was it did a great job. I assume that one reason for this was less or no need for ecc ram. I even upgraded the ram with some laptop memory a few years ago.

Thanks for all you do and for taking time out to help out a noob! :)

 

[danb35]

Bad RAM has the potential to hose your data with any filesystem, and you really should use ECC if you value your data, irrespective of the OS or filesystem you're using. There are a couple of reasons that it's even more strongly recommended with ZFS, though: (1) The filesystem is so robust that the lack of ECC leaves a really big and obvious gap in the data integrity chain (I recall one of the ZFS devs saying that using ZFS without ECC is akin to putting a screen door on a submarine). (2) The periodic scrubs, at least in theory, increase the risk from bad RAM. During a scrub, every block is read and compared to its checksum. If it doesn't match, ZFS does its best to correct the data (from parity or a mirror). But if the mismatch is a result of bad RAM, the "correction" may itself be incorrect, and the result is that data on disk that was correct before the scrub is now incorrect.

Honestly, I'm not convinced that the risk from #2 is that significant, as it seems like a pretty unlikely chain of events to lead to on-disk data corruption. But even so, "pretty unlikely" multiplied by several thousand (or million) users still works out to a much greater chance of it happening to someone. Although I think scenario #2 may be a little far-fetched, I still use ECC in my FreeNAS server (and, for that matter, my Linux server using ext4).

 

[SmallGuy]

In few words, ZFS is a countermeasure against the "write hole" (Google search for more details). But ZFS relies on data stored on RAM. This is a "weaknesses" of the ZFS robust file system, easly managed using ECC RAM.
As there isn't any tool like "chkdsk" or "fsck", Handle ZFS without ECC is risky (no recovery tool).

 

[SweetAndLow]

In few words, ZFS is a countermeasure against the "write hole" (Google search for more details). But ZFS relies on data stored on RAM. This is a "weaknesses" of the ZFS robust file system, easly managed using ECC RAM.
As there isn't any tool like "chkdsk" or "fsck", Handle ZFS without ECC is risky (no recovery tool).

ehh what you are saying doesn't really work the way you think it works. All filesystems are vulnerable to to bit rot, this has nothing to do with chkdsk or fsck. ZFS combats this problem by periodically reading all blocks of data and verifying their checksum with the one stored when the data was last written. With ecc memory you know for certain that if you find an error that it was found on disk and not from bad memory. Without ecc memory you can have a memory error that corrupts data and that data will get written to disk incorrectly but with ecc memory if you have a memory error that data can also be rebuilt if it was a single bit error, multibit errors will result in a hardware panic most of the time. All of this is just a way to guarantee the data that you wrote to disk stays consistent.

ECC memory has nothing to do with the write hole and isn't fixing a weakness in zfs. It is just a way to do hardware data verification. At one point all computers had ecc memory but manufactures got away from that because of cost and most people didn't care if there desktop crashed once in a blue moon.

 

[danb35]

ZFS does correct the write hole, but that's specifically an issue with traditional RAID (http://www.raid-recovery-guide.com/raid5-write-hole.aspx). Agreed that it doesn't really have anything to do with ECC.

 

[SmallGuy]

Think I have been misunderstood, trying give some key points without enough depth, too many assumptions about readers mind and probably a poor English.
-With ext4, if you want disk redundancy you need hw RAID (that's why I have talk about the 'write hole')
-ECC is for data integrity on RAM (critical with ZFS as it could result errors on disk)
-chkdsk and fsck are for data integrity on disk (not applicable with ZFS)
Sorry for the confusion.

 

[danb35]

-With ext4, if you want disk redundancy you need hw RAID (that's why I have talk about the 'write hole')

This depends on the OS more than the filesystem, but since ext4 is primarily used with Linux, and Linux has supported software RAID for quite some time, I believe this statement to be incorrect--unless I'm misunderstanding it.