Register for the iXsystems Community to get an ad-free experience and exclusive discounts in our eBay Store.

ECC vs non-ECC RAM and ZFS

Status
Not open for further replies.

Ericloewe

Not-very-passive-but-aggressive
Moderator
Joined
Feb 15, 2014
Messages
16,048
Thanks
3,894
Noob question, so in reading the op I've come to the conclusion that in order to run ZFS, you need to have ECC Ram. I have an i5 on an ASUS mobo with 16GB non ecc ddr3@1600. I'm guessing even if I bought ECC ram for it, the ECC function would be unusable. So then, I would literally have to replace mobo,cpu,ram in order to support the "Data Integrity" function of my fileserver... and there isn't any way around it. Am I correct?
That's basically it, yes.
 

TXAG26

FreeNAS Experienced
Joined
Sep 20, 2013
Messages
290
Thanks
26
Yep, you need ECC ram (and a CPU & MB to support it). I actually have first-hand experience with ECC saving my butt. Had a stick of ram start going bad by throwing single bit errors. The ECC function caught and fixed it, and logged it into the bios system log (accessible through IPMI). The next time I took my machine down, the bios prevented it from booting back up because of the dying ram. I can guarantee you if I would have had non-ECC ram, my data would have been toast.

All of this is just a heads up and just something to be aware of next time you do a hardware upgrade and you may want to be even more vigilant regarding backups because of your current system.
 

DrKK

FreeNAS Generalissimo
Joined
Oct 15, 2013
Messages
3,630
Thanks
1,090
Noob question, so in reading the op I've come to the conclusion that in order to run ZFS, you need to have ECC Ram. I have an i5 on an ASUS mobo with 16GB non ecc ddr3@1600. I'm guessing even if I bought ECC ram for it, the ECC function would be unusable. So then, I would literally have to replace mobo,cpu,ram in order to support the "Data Integrity" function of my fileserver... and there isn't any way around it. Am I correct?
Just for the record. Everything these guys are saying to you is correct. The i5 CPU's from Intel, and the i7's, basically have ECC support disabled. You can see this on ark.intel.com for the various CPU's. They do this on purpose, because dollar-per-performance, the Xeon CPUs (which are what they want you to buy for server situations) are much, much more expensive, since of course, they are being bought by enterprises, who aren't as price conscious as you and I are. They want the enterprises to be forced to buy the Xeons. It makes sense from a business standpoint. The good news, however, is that the G3220/3240 type "Pentium" Haswells *do* strangely support ECC. So if you want ECC support on the modern Intels, you need to buy either the Xeons, certain i3's, or the lower priced Pentium'ish guys. But if you have an i5 or i7, you are SOL.

On TOP of that, you must have a motherboard and chipset that supports ECC RAM. Almost zero of the "desktop" or "gamer" boards do that. You specifically have to go finding a server board.

So yes, in your case, you are completely SOL, and what you have cannot be salvaged in any part, for an ECC setup.

You need a new CPU, a new mobo (most of us recommend the SuperMicro X10SLM or X10SL7 or similar to small/home users), and the proper ECC RAM.
 
Joined
Aug 17, 2014
Messages
95
Thanks
3
Thanks for the answers guys, is RAM speed any factor at all? 1333/1600? I'm guessing it's not. I don't know anything about ECC ram, are there any /better/ brands? Or something to specifically look for besides ECC? Do I want them to be registered?
 

Knowltey

FreeNAS Experienced
Joined
Jul 21, 2013
Messages
430
Thanks
23
Thanks for the answers guys, is RAM speed any factor at all? 1333/1600? I'm guessing it's not. I don't know anything about ECC ram, are there any /better/ brands? Or something to specifically look for besides ECC? Do I want them to be registered?
The difference between 1333 and 1600 won't be noticeable to a home user. If you were setting up the server for a big business though with many users accessing simultaneously it might come into play though.
 

Ericloewe

Not-very-passive-but-aggressive
Moderator
Joined
Feb 15, 2014
Messages
16,048
Thanks
3,894
Rule of thumb: if you have no idea if you want registered memory, you do not want registered memory.

If you're going with something like an Avoton, i3/Xeon E3 or similar platform, you can only use Unbuffered DIMMs.
 

jgreco

Resident Grinch
Moderator
Joined
May 29, 2011
Messages
11,799
Thanks
3,054
Thanks for the answers guys, is RAM speed any factor at all? 1333/1600? I'm guessing it's not. I don't know anything about ECC ram, are there any /better/ brands? Or something to specifically look for besides ECC? Do I want them to be registered?
RAM speed is not as significant a factor for a NAS as for other things. You're limited by the speed of disk and network I/O. DDR3-1066 ECC RAM is going to be more desirable than DDR3-2133 non-ECC simply due to the protection offered. Any minor speed bump from slightly reduced memory subsystem latency is dramatically offset by the risk.

Typically the cheapest vendors do not bother making ECC RAM, but the best strategy is to look at the motherboard manufacturer's compatibility page and select something like a Samsung or other actual memory manufacturer's part if possible.

Registered memory is not something you want unless your CPU and motherboard support it. There's a link in the hardware sticky that points to Patrick Kennedy's explanation of that, if I recall correctly.
 

jgreco

Resident Grinch
Moderator
Joined
May 29, 2011
Messages
11,799
Thanks
3,054
in order to run ZFS, you need to have ECC Ram.
Not "need", but "so strongly prefer for all the reasons outlined"

I have an i5 on an ASUS mobo with 16GB non ecc ddr3@1600. I'm guessing even if I bought ECC ram for it, the ECC function would be unusable. So then, I would literally have to replace mobo,cpu,ram in order to support the "Data Integrity" function of my fileserver... and there isn't any way around it. Am I correct?
You can check with ASUS on the status of ECC with your selected board. These days, since the memory controller is actually part of the CPU, it isn't much more than the addition of some traces on the mainboard to hook the "parity"/check bits from the CPU socket to the DIMM slot. However, there are some BIOS changes that probably need to be there (IIRC the ECC functionality is soft-configured in the processor by the BIOS, dependent on whatever is in the DIMM SPD data). Also you want BIOS support for reporting detected correction events, so that you know something went wrong. Plus you want that information to be available without having to down the server and look in the BIOS Setup screen.
 

Ericloewe

Not-very-passive-but-aggressive
Moderator
Joined
Feb 15, 2014
Messages
16,048
Thanks
3,894
Not "need", but "so strongly prefer for all the reasons outlined"



You can check with ASUS on the status of ECC with your selected board. These days, since the memory controller is actually part of the CPU, it isn't much more than the addition of some traces on the mainboard to hook the "parity"/check bits from the memory module to the DIMM slot. However, there are some BIOS changes that probably need to be there (IIRC the ECC functionality is soft-configured in the processor by the BIOS, dependent on whatever is in the DIMM SPD data). Also you want BIOS support for reporting detected correction events, so that you know something went wrong. Plus you want that information to be available without having to down the server and look in the BIOS Setup screen.
Intel, for some crazy reason, requires a chipset that supports ECC, even though there is no connection between PCH and RAM. Unless he's running a Server/Workstation board with a C2XX PCH, ECC is not supported.

I assume it's pure market segmentation enforced by stern warnings to motherboard manufacturers and BIOS developers, not in any way a physical limitation of the chipset.
 
Joined
Aug 17, 2014
Messages
95
Thanks
3
So this is what I went with, and the board has a C226 chipset.

SUPERMICRO MBD-X10SAE-O
Kingston 32GB (4x 8GB) 1600 UDIMM KVR16E11K4/32
Xeon E3-1220V3 Haswell

I was told in another thread that the Kingston ram might be an issue with the X10 board :\
 

TXAG26

FreeNAS Experienced
Joined
Sep 20, 2013
Messages
290
Thanks
26
That's a good board. I'm using one for my main workstation and it's been solid. It doesn't have IPMI, so you'll need a monitor and keyboard for initial config. If the Kingston ram doesn't work out, give the Crucial 1.35v ECC a try.

Remember, only 6 of the SATA ports are off the Intel C226 chipset. The other two are via a 3rd party chip.
 
Last edited:
Joined
Aug 17, 2014
Messages
95
Thanks
3
Thank you for that information!
 

Ericloewe

Not-very-passive-but-aggressive
Moderator
Joined
Feb 15, 2014
Messages
16,048
Thanks
3,894
That board is not ideal, since it doesn't have IPMI and has all kinds of Desktop stuff (it's a workstation board, not a server board).

It will work, however.

Additionally, 32GB of Kingston RAM is pretty much the configuration with the highest risk of not working.
 

TXAG26

FreeNAS Experienced
Joined
Sep 20, 2013
Messages
290
Thanks
26
It sounds like you've already ordered everything (most people post a build list first for comments). Like I said above, that's a good WORKSTATION board. You may want to see if you can still cancel that Newegg Order and pick up the X10SL-F and some of the approved ECC ram.

The X10SL-F is about $20 more than the X10SAE, but that $20 gets you a $250 LSI 8-port 6gbps HBA which works great in FreeNAS. Plus you also pick up IPMI and I think you were the one that keeps your FreeNAS box in a basement closet, why the heck wouldn't you want IPMI? IPMI is great for servers and you don't know what you're missing until you've tried it.
 

TXAG26

FreeNAS Experienced
Joined
Sep 20, 2013
Messages
290
Thanks
26
Joined
Aug 17, 2014
Messages
95
Thanks
3
So C222 > C226 PCH? Or are the differences minute.

Also for 7.2k 4TB hdds, SAS is $100 more / harddrive... which I don't think warrants the price increase for virtually the same drive with the same cache. Or am I wrong?
 
Last edited:

Z300M

FreeNAS Experienced
Joined
Sep 9, 2011
Messages
847
Thanks
32
So this is what I went with, and the board has a C226 chipset.

SUPERMICRO MBD-X10SAE-O
Kingston 32GB (4x 8GB) 1600 UDIMM KVR16E11K4/32
Xeon E3-1220V3 Haswell

I was told in another thread that the Kingston ram might be an issue with the X10 board :\
I now see that although Kingston still does not have any recommended 8GB modules for the X10SL7, they do have recommended 8GB modules (and 16GB and 32GB kits) for the X10SAE -- BUT the recommended 32GB kit is the KVR16E11K4/32G: note the "G" suffix; I don't know what that suffix indicates, but the "EF" suffix indicated that the chips were "Elpida Server F." So the actual chips on the modules you've bought may be different from the ones on the modules Kingston recommends for that board.
 
Joined
Aug 17, 2014
Messages
95
Thanks
3
Could I technically combine two of these kits with the X10SL7?

CT2KIT102472BD160B

(I already started an RMA for the other X10 and the Kingston ram in favor of the X10SL7 with hopefully the Crucial ram?)

Edit: talked to one of Crucial's reps because Crucial's part numbers and Newegg's part numbers are different. Newegg lists the base part number; he confirmed that the model I listed above is the kit for the X10SL7
 
Last edited:

Ericloewe

Not-very-passive-but-aggressive
Moderator
Joined
Feb 15, 2014
Messages
16,048
Thanks
3,894
Could I technically combine two of these kits with the X10SL7?

CT2KIT102472BD160B

(I already started an RMA for the other X10 and the Kingston ram in favor of the X10SL7 with hopefully the Crucial ram?)
If you use Crucial's tool on their website, it'll give you a list of DIMMs they recommend. The 1.35V 8GB DIMMs they recommend are the same that Supermicro tested under the Micron brand (I've confirmed this myself). Running fine over here.

As for the chipset, the C222 is inferior, but the only thing you lose are two USB 3.0 ports (you still have two) and four SATA ports are only 3Gb/s (not a bottleneck, especially since you have an LSI 2308 that can handle 6Gb/s from 8 ports if needed).
 
Status
Not open for further replies.
Top