8HDD Freenas Build

[HK416]

Below is a parts list for a freenas build. I need a fresh pair of eyes on these components. I was originally going to go with the Asrock C2750D4I as my mainboard but I have heard many who suffer with the marvel controllers on that board when doing a build above six HDDs. Has anyone any experience with a similar set of parts? Any insight is welcome.

PS: I was originally going to go with the E3-1230v3 (socket 1150) processor but the one below is supported by the board and on promo now.

- X2 SanDisk USB 3.0 Ultra Fit 32GB
- Fractal Design Define R5 (Black)
- Fractal Design Newton R3 600W
- X2 Crucial DDR3 PC12800/1600MHz ECC 2x8GB
- X8 WD Red WD60EFRX 64MB 6TB
-Intel Xeon E3-1241v3 3,5GHz
-Supermicro X10SL7-F

 

[DrKK]

One problem, one warning:

Problem: If you set up 8x6TB of hard drives, you are way way way way way way way under our suggested approximate 1GB per 1TB raw (or pool, depending on who you ask) hard drive space with only 16GB of RAM. I do think, however, off the record, because I know how off the record this forum post is, that you'll more than likely be OK, and I think you can safely try running this with 16GB without risking your pool---if you have performance issues as you trim up the appliance, then you can add more RAM.

Warning: E3-1241v3 probably won't work unless the BIOS on the unit you get is the most up to date. I think. There is no way to upgrade the BIOS without either a functioning CPU, or by paying SuperMicro a bit extra to use the IPMI-based upgrade firmware.

 

[HK416]

One problem, one warning:

Problem: If you set up 8x6TB of hard drives, you are way way way way way way way under our suggested approximate 1GB per 1TB raw (or pool, depending on who you ask) hard drive space with only 16GB of RAM. I do think, however, off the record, because I know how off the record this forum post is, that you'll more than likely be OK, and I think you can safely try running this with 16GB without risking your pool---if you have performance issues as you trim up the appliance, then you can add more RAM.

Warning: E3-1241v3 probably won't work unless the BIOS on the unit you get is the most up to date. I think. There is no way to upgrade the BIOS without either a functioning CPU, or by paying SuperMicro a bit extra to use the IPMI-based upgrade firmware.

Thanks for the heads up. You raise some pretty insightful points there. The RAM was a typo on my part. I intend to max out the entire 32GB capacity of that board. But given how you describe the RAM reqs, I might still fall short so I might have to just ramp up the build some more to a board that can take more RAM

I started out wanting to go with the top end board: X9SRH-7F But the suppliers out here are limited and those that have it seem to want to an arm and a leg for it.

 

[Ericloewe]

Warning: E3-1241v3 probably won't work unless the BIOS on the unit you get is the most up to date. I think. There is no way to upgrade the BIOS without either a functioning CPU, or by paying SuperMicro a bit extra to use the IPMI-based upgrade firmware.

I fully expect most current stock at large retailers to be at the 2.0 BIOS or later, by now. Of course, there's always the occasional straggler who ends up in the hands of the poor schmo without a compatible CPU lying around.+

- Fractal Design Newton R3 600W

The prices for that thing are well within Seasonic territory. Consider instead a Seasonic Platinum 650 instead.

 

[HK416]

I fully expect most current stock at large retailers to be at the 2.0 BIOS or later, by now. Of course, there's always the occasional straggler who ends up in the hands of the poor schmo without a compatible CPU lying around.+


The prices for that thing are well within Seasonic territory. Consider instead a Seasonic Platinum 650 instead.

Your above expectation on that motherboard was the exact reason I went for it in the first place. Seems I might have to write in to the retailer before placing the order. I do have a CPU around on another machine but can do without the hassle

As for the Newton R3, I already had one unopened laying around the house (Got it for free I think).

 

[DrKK]

Sounds like you are in good shape.

One more thing:

If you have 32GB of RAM, as you said, then that is almost CERTAINLY going to be sufficient for an 8x6TB pool. Once you have 32GB, you are usually good to pools up to 64TB+ raw storage, no problem. Once you get to 32GB RAM, the field is wide open, and the "1GB per 1TB" rule is greatly relaxed...

With 8x6TB in presumably a RAID-Z2 or RAID-Z3, and without anything fancy (dedup, e.g.) going on:

8GB RAM: Definitely dodgy. Might work, we wouldn't recommend it. Well below the recommendation. Probably could get away with it for very simple file storage only. We will make fun of you.
16GB RAM: Probably will work. We wouldn't recommend it, but you could expect that to work in many cases, with possible performance issues at times. We will not make fun of you, but if you can afford 8x6TB, then you can afford 32GB.
32GB RAM: Definitely sufficient. You won't have a performance problem here. You're fine.

Unless you have special use cases beyond basic file storage and a few jails and a handful of users.

 

[HK416]

Sounds like you are in good shape.

One more thing:

If you have 32GB of RAM, as you said, then that is almost CERTAINLY going to be sufficient for an 8x6TB pool. Once you have 32GB, you are usually good to pools up to 64TB+ raw storage, no problem. Once you get to 32GB RAM, the field is wide open, and the "1GB per 1TB" rule is greatly relaxed...

With 8x6TB in presumably a RAID-Z2 or RAID-Z3, and without anything fancy (dedup, e.g.) going on:

8GB RAM: Definitely dodgy. Might work, we wouldn't recommend it. Well below the recommendation. Probably could get away with it for very simple file storage only. We will make fun of you.
16GB RAM: Probably will work. We wouldn't recommend it, but you could expect that to work in many cases, with possible performance issues at times. We will not make fun of you, but if you can afford 8x6TB, then you can afford 32GB.
32GB RAM: Definitely sufficient. You won't have a performance problem here. You're fine.

Unless you have special use cases beyond basic file storage and a few jails and a handful of users.

Thanks a lot for laying out the RAM facts. It helps me trust this build more and makes me ready to pull the trigger on it. What made me a lil nervous was the 32GB limit of this board but your explanation and some other info I was able to find has put those fears to rest. As for use case, the box shall be mainly used by me. It shall carry all my workstation NFS mounts (Synology 409+ is showing its age with these 6 mounts). All family computers for a small household of 3 persons shall backup to this box. I intend to run a few jails (plex sabnzbd Syncthing). Hope with that load it shall still feel snappy.

 

[DrKK]

Thanks a lot for laying out the RAM facts. It helps me trust this build more and makes me ready to pull the trigger on it. What made me a lil nervous was the 32GB limit of this board but your explanation and some other info I was able to find has put those fears to rest. As for use case, the box shall be mainly used by me. It shall carry all my workstation NFS mounts (Synology 409+ is showing its age with these 6 mounts). All family computers for a small household of 3 persons shall backup to this box. I intend to run a few jails (plex etc). Hope with that load it shall still feel snappy.

You are in good shape.

You are "go", I say again "go" for launch.

 

[HK416]

Quick update all went well. initial build is done memtest86 went great. Testing now with two drives I had lying around while waiting for the 8drives to come. LSI 2308 is flashed to IT mode and seems to work fine. Shall post how it goes as I move along. Thanks guys for all the help.

This build was so much fun to do after brainstorming with you guys in here. I knew what to expect and every single thing went as expected. For those that may be curious about the same board, mine arrived with all firmware and bios at the latest (That was a relief) was prepared for work but there was none to do.

As an aside, I'm never buying another board without IPMI or similar tech (even for home use). I added the IPMI interface to my management VLAN and just like that I can do ANYTHING with the machine.

 

[Pheran]

I'd love to hear how it goes - this is nearly identical to my build except you are using larger drives and a faster Xeon. The rest of my parts should be arriving this week.

 

[diedrichg]

What is your critique of the Define R5? I'm seriously considering switching to that case.

 

[HK416]

What is your critique of the Define R5? I'm seriously considering switching to that case.

The Define R5 works great. I have only one issue which does not matter very much (Mostly aesthetics). The position of the main power connectors in the board I'm using make me only able to route the main power supply cable over the front of the motherboard. I would have loved to route all the cables behind the board so everything looks squared away. Aside from that, I could not have picked a better case. Its very well thought out, silent and has good airflow.

 

[HK416]

I'd love to hear how it goes - this is nearly identical to my build except you are using larger drives and a faster Xeon. The rest of my parts should be arriving this week.

Your list looks great and I sense you shall have a great build. I have not tested mine with the larger drives yet (on account of they have not arrived). However I'm thinking I might not have needed such a powerful CPU (We shall see if this is true when the drives get here). As the build stands now, The CPU does not reach even 5% load. Hopefully when fully built and loaded with files I might see enough load to justify such a powerful CPU. I have two drives in it in a mirror and The CPU does not break a sweat despite my best efforts.

 

[HK416]

I have one hitch though under alerts, I get the following:

Code:

 WARNING: Firmware version 19 does not match driver version 16 for /dev/mps0

Don't know what to make of it. Any help is welcome.

 

[Ericloewe]

Don't know what to make of it. Any help is welcome.

Google is your friend. That has to be the single most-asked question lately.

Flash your LSI SAS2 controller to P16 IT mode.

 

[Pheran]

Your list looks great and I sense you shall have a great build. I have not tested mine with the larger drives yet (on account of they have not arrived). However I'm thinking I might not have needed such a powerful CPU (We shall see if this is true when the drives get here). As the build stands now, The CPU does not reach even 5% load. Hopefully when fully built and loaded with files I might see enough load to justify such a powerful CPU. I have two drives in it in a mirror and The CPU does not break a sweat despite my best efforts.

For pure FreeNAS, you certainly do not need that much CPU horsepower. The only way you are going to stress that CPU is with CPU-intensive plugins/jails, like using Plex to transcode multiple simultaneous 1080p streams.

 

[HK416]

Google is your friend. That has to be the single most-asked question lately.

Flash your LSI SAS2 controller to P16 IT mode.

Thanks- That's sorted now.

 

[HK416]

For pure FreeNAS, you certainly do not need that much CPU horsepower. The only way you are going to stress that CPU is with CPU-intensive plugins/jails, like using Plex to transcode multiple simultaneous 1080p streams.

There shall be some jail use and some trans-coding with plex. But it still suprised the hec out of me how little it is used when at rest. I guess its a good thing, Its very efficient and silent.