DDR5 “ECC” and ZFS

Joined
Oct 17, 2023
Messages
8
ZFS does not have things like chkdsk or fsck because it is impractical to do these operations on a tiny computer, even if you think your 128 core 1TB machine is "huge". ZFS filesystems often range into the multipetabyte range. Consider this:

Let's pretend that the hypothetical 128C/1TB system I just speculated about has 4PB of storage. If we were to break that down into 4C/32GB systems, we would end up with 32 separate systems. (4C x 32 = 128C, 32GB x 32 = 1TB). Now if we split the 4PB up into 32 chunks, that'd be 128TB per system. Is 128TB even practical to fsck or chkdsk on a 32GB 4 core system? And if not (it's not), how is ZFS supposed to manage that trick for something 32x larger? You need a HUGE amount of memory and resources and it'd also take... forever. This only gets worse because ZFS has incredible CoW features such as snapshots and clones that substantially increases the metadata complexity, which means the system has to work much harder to validate the correctness of the structure.

ZFS works super hard to protect against introducing failures into the pool in the first place because once an error is introduced, it may be virtually impossible to expunge, especially if it is in something like metadata. This is why ZFS users need to be a little paranoid about things like using well-supported HBA's, and favoring ECC memory, because once crap data is flushed out to the pool, it could potentially do permanent damage to the pool, with the primary recovery option being "use your backups". On the flip side, if you play the ZFS game the way the designers intended, your data is quite safe compared to a conventional filesystem.


SO another words basically what you are implying is it woukd be better off to use a differnet file system if I do not use ECC RAM. But if I use ECC RAM with ZFS and TRueNAS, it is better protection than anything else really?

Though based on what you describe about ZFS and the reason it does not have chkdsk or fsck, is there any more risk to it even with EVV RAM than another file system like in event of hardware crapping out or disk controller failure and such?
 

Patrick M. Hausen

Hall of Famer
Joined
Nov 25, 2013
Messages
7,776
ECC is better than no ECC regardless of the file system. ZFS is better than any other file system existing in protecting your data - ECC or not.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,194

ChrisRJ

Wizard
Joined
Oct 23, 2020
Messages
1,919
[..] an older socket 2011/DDR3 system would likely suffice, but the power draw and heat output are things I am looking to avoid [..]
Also, starting with those systems, the old equation "TDP = average power consumption" is not valid anymore. If you look at my signature, my NAS takes around 100 W idle, while the CPU alone has a 130 W TDP.
 

Whattteva

Wizard
Joined
Mar 5, 2013
Messages
1,824
Also, starting with those systems, the old equation "TDP = average power consumption" is not valid anymore. If you look at my signature, my NAS takes around 100 W idle, while the CPU alone has a 130 W TDP.
Yeah, I think starting from Haswell/Broadwell generation (3rd/4th-gen Core), idle power consumption starts to really significantly improve. Unfortunately, Intel stopped producing ECC-capable i3's starting at 9th-gen Core i3.
 

Arwen

MVP
Joined
May 17, 2014
Messages
3,611
Yeah, I think starting from Haswell/Broadwell generation (3rd/4th-gen Core), idle power consumption starts to really significantly improve. Unfortunately, Intel stopped producing ECC-capable i3's starting at 9th-gen Core i3.
Yes, it is disappointing that Intel stopped the non-embedded Core i3 with ECC.

Intel does have i5 and others with ECC in the later generations, like this one, (somewhat picked at random);
 
Top