RAM for Supermicro X11

[NASf@n]

Hi,

I have a question about ECC memory. I am building a FreeNAS server with Supermicro X11SSL-CF-O motherboard. The system will have 80 TB (10x 8TB WD Red disks) disk space total. So I want to buy 64 GB of ECC RAM. But I struggle to find any recommended ECC UDIMMs in stock from reputable vendors. The first place I looked was crucial.com, everything is out of stock except:
http://www.crucial.com/usa/en/x11ssl-cf/CT11004124
Any thoughts about this memory or any recommendations where I can buy ECC memory for X11 motherboard?

Thanks

 

[Chris Moore]

I would suggest going with an older system board that can use DDR3 ECC memory so you don't have to deal with the shortage.
Here is an excellent option:
https://www.ebay.com/itm/Supermicro...-2-6ghz-8-Core-128gb-24-Bay-JBOD/232604715524
CPU, Memory, Power Supply, the works, all for the less than you would spend on a new system and it will give you enough drive bays to use more drives to expand later.
This is actually a deal I would buy myself if I were in the market right now. If I had the cash, I would buy two of these.

 

[NASf@n]

I would suggest going with an older system board that can use DDR3 ECC memory so you don't have to deal with the shortage.
Here is an excellent option:
https://www.ebay.com/itm/Supermicro...-2-6ghz-8-Core-128gb-24-Bay-JBOD/232604715524
CPU, Memory, Power Supply, the works, all for the less than you would spend on a new system and it will give you enough drive bays to use more drives to expand later.
This is actually a deal I would buy myself if I were in the market right now. If I had the cash, I would buy two of these.

I know that these rack servers are very loud and there is no easy way to make them silent. My system is going to the living room so it will not work for me. But I agree that it is a very good deal. Any recommendations where to buy 64 GB of ECC memory for SUPERMICRO MBD-X10SRL-F ?

Thanks

 

[Johnnie Black]

These will work for the X11SSL:

http://www.memory4less.com/kingston-16gb-ddr4-pc19200-kvr24e17d8-16

 

[joeschmuck]

The system will have 80 TB (10x 8TB WD Red disks) disk space total.

This is a bit off topic for this thread but realize that I would recommend a RAIDZ3 here, RAIDZ2 at worst, assuming RAIDZ2 then you total storage capacity would be 58.2TB - 20% (11.64TB) = 46.56TB. Under RAIDZ3 it would be 50.9TB - 20% (10.18TB) = 40.1TB.

What was your design plan here? I just don't want you to see a huge drop in capacity and think to yourself that something is wrong. The 20% if you are not familiar with it is to maintain a healthy fast pool. If you plan to use iSCSI then the rule is 50% free.

 

[Chris Moore]

This is a bit off topic for this thread but realize that I would recommend a RAIDZ3 here, RAIDZ2 at worst, assuming RAIDZ2 then you total storage capacity would be 58.2TB - 20% (11.64TB) = 46.56TB. Under RAIDZ3 it would be 50.9TB - 20% (10.18TB) = 40.1TB.

What was your design plan here? I just don't want you to see a huge drop in capacity and think to yourself that something is wrong. The 20% if you are not familiar with it is to maintain a healthy fast pool. If you plan to use iSCSI then the rule is 50% free.

Good point, but you have changed. Looking a little blue.

 

[Chris Moore]

I know that these rack servers are very loud and there is no easy way to make them silent. My system is going to the living room so it will not work for me. But I agree that it is a very good deal. Any recommendations where to buy 64 GB of ECC memory for SUPERMICRO MBD-X10SRL-F ?

Thanks

This board still wants DDR4. There is a massive shortage right now that is running prices up through the roof, if you can even find DDR4...
That is why I suggested an older board that can use DDR3, the registered type, which is plentiful on the used market and much cheaper.
A board like this for example:
https://www.ebay.com/itm/NEW-Unopened-Super-Micro-Motherboard-LGA-2011-MBD-X9SRL-F-O/253300204890
If you went with the board I suggested, here is the RAM for it:
32GB: https://www.ebay.com/itm/32GB-2X16G...SRW-F-X9SRW-3F-X9SRL-F-X9SRL-B16/272519006563
64GB: https://www.ebay.com/itm/64GB-4X16G...SRW-F-X9SRW-3F-X9SRL-F-X9SRL-C19/282323196904

The other option, why not put it in a different room? There is no reason to have a FreeNAS system in the living room, it just sits there taking up space.

 

[joeschmuck]

Good point, but you have changed. Looking a little blue.

It's winter time, it's cold outside. Maybe I need a hat?

@NASf@n Why do you need 64GB of RAM? What is your use case for this system? If you are only using it for storage and Plex and maybe a few small VMs then start out with 32GB RAM, odds are it will be more than enough. If you plan to run a lot of VMs through FreeNAS then I'd still recommend 32 GB to start with and double that if you find out you need more RAM.

As for DDR4 RAM, yes, those are some high prices and in a FreeNAS system if you are not building fast VMs then DDR3 will be more than fine. We need to know what you are using your system for.

 

[Chris Moore]

It's winter time, it's cold outside. Maybe I need a hat?

Absolutely, I had to turn the heat on this morning.

I know that these rack servers are very loud and there is no easy way to make them silent.

The idea of making a system with 10 hard drives silent is a little odd to me. If you have that many drives, you are going to need fans to keep them cool. The drives plus fans equal sound of some sort, but it doesn't bother me. I have two rack servers in my office and they are quiet enough that I don't notice them most of the time.

 

[NASf@n]

This is a bit off topic for this thread but realize that I would recommend a RAIDZ3 here, RAIDZ2 at worst, assuming RAIDZ2 then you total storage capacity would be 58.2TB - 20% (11.64TB) = 46.56TB. Under RAIDZ3 it would be 50.9TB - 20% (10.18TB) = 40.1TB.

What was your design plan here? I just don't want you to see a huge drop in capacity and think to yourself that something is wrong. The 20% if you are not familiar with it is to maintain a healthy fast pool. If you plan to use iSCSI then the rule is 50% free.

I am considering RAIDZ2. Is 32 GB of RAM enough to run this rig? I did not make my mind if I am going to install Plex on this FreeNAS server also or not. I do have a powerful gaming PC in the same room. So perhaps a better choice is to use older motherboard with DDR3, which Chris suggested and use FreeNAS server only as a file server and run Plex server on gaming PC.

Thanks.

 

[NASf@n]

This board still wants DDR4. There is a massive shortage right now that is running prices up through the roof, if you can even find DDR4...
That is why I suggested an older board that can use DDR3, the registered type, which is plentiful on the used market and much cheaper.
A board like this for example:
https://www.ebay.com/itm/NEW-Unopened-Super-Micro-Motherboard-LGA-2011-MBD-X9SRL-F-O/253300204890
If you went with the board I suggested, here is the RAM for it:
32GB: https://www.ebay.com/itm/32GB-2X16G...SRW-F-X9SRW-3F-X9SRL-F-X9SRL-B16/272519006563
64GB: https://www.ebay.com/itm/64GB-4X16G...SRW-F-X9SRW-3F-X9SRL-F-X9SRL-C19/282323196904

The other option, why not put it in a different room? There is no reason to have a FreeNAS system in the living room, it just sits there taking up space.

I agree these prices are very reasonable. Which CPU do you recommend? I may install Plex server and use it for occasional streaming of up to two 1080P channels.

Thanks.

 

[NASf@n]

Absolutely, I had to turn the heat on this morning.

The idea of making a system with 10 hard drives silent is a little odd to me. If you have that many drives, you are going to need fans to keep them cool. The drives plus fans equal sound of some sort, but it doesn't bother me. I have two rack servers in my office and they are quiet enough that I don't notice them most of the time.

I had very good experience with Fractal Design R5 case in the past. This case can accommodate 10 drives easily and run very quietly. There is a room to put it next to my gaming PC in the living room.

Thanks.

 

[NASf@n]

It's winter time, it's cold outside. Maybe I need a hat?

@NASf@n Why do you need 64GB of RAM? What is your use case for this system? If you are only using it for storage and Plex and maybe a few small VMs then start out with 32GB RAM, odds are it will be more than enough. If you plan to run a lot of VMs through FreeNAS then I'd still recommend 32 GB to start with and double that if you find out you need more RAM.

As for DDR4 RAM, yes, those are some high prices and in a FreeNAS system if you are not building fast VMs then DDR3 will be more than fine. We need to know what you are using your system for.

If I go for DDR3 then I will probably just buy 64 GB of memory. The difference is not that big and it opens possibility for running Plex server also on this FreeNAS server.

Thanks.

 

[Chris Moore]

If I go for DDR3 then I will probably just buy 64 GB of memory. The difference is not that big and it opens possibility for running Plex server also on this FreeNAS server.

Thanks.

I run my Plex on my FreeNAS server (an x9 board) with 32GB of RAM and I did run it with 16GB before and it worked fine. The only reason I upgraded the memory was to be able to run a couple virtual machines. The board I pointed you at would be more than capable of running FreeNAS, Plex and a couple more virtual machines besides even with just 32GB of memory and with 64GB you would probably be set for the future as long as you wanted to use the system, because that board and the CPU below can be upgraded all the way up to 256GB of RAM using DDR3 ECC RDIMM (registered memory) like I previously linked you to.
I looked on the Supermicro page to verify the supported processors and double checked the Intel archive; this should work and be more processing power than you need.
https://www.ebay.com/itm/3-7GHz-Int...-Core-Ivy-Bridge-LGA-2011-Socket/192408197121

I agree these prices are very reasonable. Which CPU do you recommend? I may install Plex server and use it for occasional streaming of up to two 1080P channels.

My system very rarely exceeds 50% on the CPU and is usually 10% or so, even though it is a 3.2GHz v1 chip and the one I am suggesting to you is even better at 3.7GHz and a v2 chip in addition to which my Xeon is a E3 and the one this board requires is a Xeon E5 which is a better chip architecturally. You should be very easily able to do everything you need with this, but the Plex server does not also do playback.
I have a smart-TV with a Plex module that connects to the server over the network, but there are plenty of other options. The server can be anywhere there is a network connection.

 

[joeschmuck]

Plex will run fine on a FreeNAS system wit 8GB RAM.

Your RAM amount needs to be based on how you use your FreeNAS. If it's just storage and a few small jails, and Plex, 8GB will work, however I'd recommend 16GB as you will likely want to do something else with it. Do not base RAM size based on how much storage you have, not if you are just using the NAS as simple storage.

 

[NASf@n]

I run my Plex on my FreeNAS server (an x9 board) with 32GB of RAM and I did run it with 16GB before and it worked fine. The only reason I upgraded the memory was to be able to run a couple virtual machines. The board I pointed you at would be more than capable of running FreeNAS, Plex and a couple more virtual machines besides even with just 32GB of memory and with 64GB you would probably be set for the future as long as you wanted to use the system, because that board and the CPU below can be upgraded all the way up to 256GB of RAM using DDR3 ECC RDIMM (registered memory) like I previously linked you to.
I looked on the Supermicro page to verify the supported processors and double checked the Intel archive; this should work and be more processing power than you need.
https://www.ebay.com/itm/3-7GHz-Int...-Core-Ivy-Bridge-LGA-2011-Socket/192408197121

My system very rarely exceeds 50% on the CPU and is usually 10% or so, even though it is a 3.2GHz v1 chip and the one I am suggesting to you is even better at 3.7GHz and a v2 chip in addition to which my Xeon is a E3 and the one this board requires is a Xeon E5 which is a better chip architecturally. You should be very easily able to do everything you need with this, but the Plex server does not also do playback.
I have a smart-TV with a Plex module that connects to the server over the network, but there are plenty of other options. The server can be anywhere there is a network connection.

Thanks for advice on CPU. I guess the last question left is about Supermicro 4U Server chassis before I commit. Is there any way to make this chasis quiet? I can replace stock fans (three front and two in the back) by Noctua 80mm fans (will they fit in?), but I am not sure what to do about power supply fans. I can probably find a quiet CPU fan also.

 

[NASf@n]

Plex will run fine on a FreeNAS system wit 8GB RAM.

Your RAM amount needs to be based on how you use your FreeNAS. If it's just storage and a few small jails, and Plex, 8GB will work, however I'd recommend 16GB as you will likely want to do something else with it. Do not base RAM size based on how much storage you have, not if you are just using the NAS as simple storage.

I see, thanks for advice.

 

[Chris Moore]

Thanks for advice on CPU. I guess the last question left is about Supermicro 4U Server chassis before I commit. Is there any way to make this chasis quiet? I can replace stock fans (three front and two in the back) by Noctua 80mm fans (will they fit in?), but I am not sure what to do about power supply fans. I can probably find a quiet CPU fan also.

The supermicro 24 bay server chassis comes with a thicker than standard fan that is designed for high static pressure to be able to get a volume of air through the drive cage. The fans are great quality with ball bearings and I wouldn't suggest replacing them, but here are a few things to think about, with regard to keeping the drives cool. Any drive slots where you don't have a drive, needs one of these plastic fillers in it to restrict the airflow so you get enough air over the drives that are installed instead of sucking all the air through the open holes where the empty drive brackets are:
https://www.ebay.com/itm/Used-3-5-D...-XX00C104-Dummy-Drive-sn-78670-6/253163127680
A new server comes with those, but lots of places throw them away when they put the drives in, so the decommissioned servers don't always come with them. I had to buy some for each of my servers.

My servers came with the high pressure / high speed 5000 RPM fans. They are really fantastic fans but they are very loud. No need to replace them though, you can slow them to a much more reasonable sound level with these:
https://www.ebay.com/itm/4-Pin-Comp...ow-Noise-Extension-Cable-Adaptor/222335215715
https://www.ebay.com/itm/3Pin-Compu...se-Extension-cord-Adapter-cable-/192044686343
There is a 3 pin and a 4 pin model, so you might need to wait to see what your server is equipped with before you order these.
On my server, the rear exhaust fans were plugged to the system board, but the fan-wall fans were plugged to the SAS backplane. The fans were all 3 pin in my system, but I know they are not all the same, depends on when they were built. Either way, this unit just goes in-line where the fan plugs in.

The problem with slowing the fans down, then they don't provide enough air-flow for the original passive heat-sync that the system comes with. I had to change the CPU heat-sync to a model that includes a fan. I used one like this:
https://www.ebay.com/itm/Dynatron-F...esktop-2500rpm-TDP-160W-Aluminum/161981526757
You will need to verify the mounting bracket requirements though because they make two different LGA2011 brackets.

You might be able to find a different model power supply that is quiet, but I didn't do anything to the power supplies in my servers because the sound from those 40mm fans is high pitched and it is easy to just block the sound by putting something between it and where you are. My server is eight feet from me and I can't discern the sound of it over other background sounds in the room.

It isn't too hard, but take your time to make it right and it will last for years.

 

[Chris Moore]

PS. I thought I would show you a little graph from my system so you can see the impact of running a 1080p stream on Plex.
The red circle is where I started the movie.

 

[NASf@n]

PS. I thought I would show you a little graph from my system so you can see the impact of running a 1080p stream on Plex.
The red circle is where I started the movie.
View attachment 22046

What about energy efficiency? Skylake+motherboard+DDR4 combo is more expensive, but with 14 nm litography Skylake is very energy efficient (compared to 22 nm Ivy Bridge+DDR3). In the long term, it may very well worth paying extra for Skylake.