BUILD Need you help/opinion on a build/configuration idea

[ctpfaff]

Hallo FreeNAS community

I am currently thinking about building my first own NAS and to give FreeNAS a first try. Below I list the hardware configuration that I have in mind to give you an overview.

Case: Lian Li PC-Q26b
Board: ASRock C2750D4I (Octacore)
RAM: Kingston Value RAM DIMM Kit 32 GB DDR3-1600 CL11 ECC (KVR16E11K4/32)
HDD: 10 x 6 TB WD Red WD60EFRX
Backplanes: 4 x Lian Li BP2Sata (only one is preinstalled in the case)
USB Stick: SanDisk Cruzer Fit 8GB (for FreeNAS)
PSU: ??? (Any suggestions are welcome) maybe this one:
- Bee Quite: Straight Power 10-CM 500W ATX 2.4 (E10-CM-500W/BN234)


With the power supply I am not sure. It would be great if you can help me find a suitable power supply so that the build will run smoothly. Another thing I do not know is if the ram will be enough to handle ZFS with that amount of storage. I actually wanted to have 64 GB RAM but as the board only supports unregistered RAM (what a pity) there is no reasonably expensive set of RAM that I could use. So this is actually something that I hope to get a better idea on in discussion with you.

The box will mainly be used as a data grave for home-videos and pictures and it will run a Plex server to provide access to my media for various devices of my family. Further more I want to use encryption to prevent data loss in case of hardware theft. I have no experience in setting up a FreeNAS and configuring the pools. Thus I thought I would just set up all disks in one pool using a z2 configuration. That might not be wise and suggestions regarding this are highly welcome.

Important questions for me are:

* Will that build run smoothly or should I forget about it?
* If the build is not good what can be improved (name alternatives, board, ram ....)?
* What is a good (well dimensioned) power supply for the build?
* What is the best way setting up the 10 drives as raid

I am looking forward to your answers and to the discussions with you.

EDIT:

The revised hardware collection at one glance

* Case: Lian Li PC-Q26b
* Backplanes: 4 x Lian Li BP2Sata (One is preinstalled in the case)
* PSU: Seasonic G-550w (PCHG Edition: Redesigned and more silence)
* Board: E3C224D4I-14S
* CPU: Intel Xeon E3-1231 v3
* Cooler: EKL Alpenföhn Brocken Eco (bxhxt: 126x150x88)

* RAM: Crucial 32GB DDR3 (4x8GB) 1600 ECC UDIMM 240 pin
* HDD: 10 x 6 TB WD Red WD60EFRX
* Drive for OS: SanDisk Cruzer Fit 8GB
* Connectors: (All neccessary comes with the board 2xsata and 2xsas2sata)

 

[BigDave]

Hallo FreeNAS community

I am currently thinking about building my first own NAS and to give FreeNAS a first try. Below I list the hardware configuration that I have in mind to give you an overview.

Case: Lian Li PC-Q26b
Board: ASRock C2750D4I (Octacore)
RAM: Kingston Value RAM DIMM Kit 32 GB DDR3-1600 CL11 ECC (KVR16E11K4/32)
HDD: 10 x 6 TB WD Red WD60EFRX
Backplanes: 4 x Lian Li BP2Sata (only one is preinstalled in the case)
USB Stick: SanDisk Cruzer Fit 8GB (for FreeNAS)
PSU: ??? (Any suggestions are welcome) maybe this one:
- Bee Quite: Straight Power 10-CM 500W ATX 2.4 (E10-CM-500W/BN234



Important questions for me are:

* Will that build run smoothly or should I forget about it?
* If the build is not good what can be improved (name alternatives, board, ram ....)?
* What is a good (well dimensioned) power supply for the build?
* What is the best way setting up the 10 drives as raid

That board has a Marvel controller which has some documented issues with the number of
drives you are proposing. My recommendation would be this model instead E3C224D41-14S.
You need Xeon with this board that has AES New Instructions for the encryption you plan.
This board has maximum 32GB RAM capacity, but has the proven LSI controller. For home
use 32GB should be OK. RAIDz2 will be fine for 10 drives as well.

I love the case and the back planes.

Around our forum, Seasonic PSUs are highly recommended.
For 10 drives, I would go with one of the two nice G-550w or 650w units.

Some other forum members might be a bit more knowledgeable about what is available in Germany.

 

[joeschmuck]

My advice isn't from a hardware perspective but from the encryption choice you are making. Read up on how the encryption works, take very good notes and I'd write up a single page document on how to replace a failing hard drive. Next, I'd test it out. Build up your NAS, use 5 of the 10 hard drives (keep 5 brand new and unused). Create your encrypted pool and add some data for the heck of it, maybe a few videos for the heck of it. Make sure it works fine.

The Test: Disconnect one of the SATA cables on one of your drives. Run a Scrub (not needed but it will show you the pool is degraded and give you a more realistic example of what the future holds). Even though you know which drive you disconnected, try to figure out which drive has failed by the serial number (this is very important, you never want to pull a working drive by accident, especially an encrypted drive). Now following the replacement instructions in the User Manual, replace the drive you unplugged with a new drive. If you are able to do it right (most people fail the first time I'd say) then your pool will no longer be degraded.

My point is, If you can encrypt your data by other means, you should. Such as using PKZIP with encryption, or maybe one of a few backup programs which will encrypt on the fly. Trust me, if you do not go through this routine while you have nothing to risk, you will be biting your fingernails when you try to recover from a failed drive replacement when you did have data on the system. We see way too many people lose data because they didn't have the correct encryption keys backed up or they pulled the wrong drive (well the wrong drive thing happens to non-encrypted systems too).

This exercise also allows you to play around with FreeNAS before finalizing what you want to do with all those Terabytes of data.

Since you are getting the hotswap backplanes, I would still highly recommend against swapping drives with the power on. Turn it off before removing and inserting a drive, you will thank me later.

EDIT: BTW, nice looking case.

 

[BigDave]

@joeschmuck has given you some stellar advice and I heartily recommend you follow it.
As a matter of fact, IMHO this subject should be refined and stickied here in the forums.
WTG old man;)

 

[joeschmuck]

@BigDave Thanks for the complement. You know we see these issues too often and I'm never going to use an encrypted pool in my NAS solution. The risk is too high for failure, even when you "think you know" what you are doing. I think a sticky on a subject like this is a good idea, not sure I'm the one to write it because I my test rig is now my new Sophos UTM firewall thing so I don't have something to play around with so I could actually write up something honest. Maybe I could try it in a VM.

But if someone else came up with it, I have no problem making it a sticky if it were well written.

 

[ctpfaff]

He together

Thanks for the precious input. I think there is no way around with the encryption as I actually do not only want some parts of my data encrypted, I want everything encrypted in a way so that I have to provide a key on start-up to be able to use the drives. But I will definitely try things out and get familiar with the routines first before I go productive with the system. For the hardware. The board you suggested is not available to buy anywhere but there is another one with an additional "I" in the name "E3C224D4I-14S" is that the one you mean or is it fine as well if not? I read some comments about that board where people state to have problems with RAM errors on this board and had to clock down the RAM to circumvent the errors. This board is limited to 32 GB of RAM. You say that this must be fine. I have think to remember I have read about a rule of thumb with RAM and ZFS that states that it is good to have 1 GB RAM per TB of storage space. This would mean in my configuration 10x6TB in raid z2 => 48 GB HDD => 16 GB more RAM would be good. Can it happen that I encounter noticeable performance issues with only 32 GB RAM or run into other problems even worse like a degraded pool???

Another thing that comes up as a question when I use this board. Which processor actually to put into it (xeon but which one?). Any suggestions here? My basic requirements to the CPU are simple. It should handle the media content with Plex fine. That means for example transcoding of multiple 1080p streams at the same time. Ore more generally it should be as fast as possible handling transcoding with Plex. That would be great as Plex offers that nice possibility to transcode things down to a lower quality so you can take a video with you to go on your mobile device. It would be cool to not have to wait too long for that process. But I have to say that I prefer a CPU that is also not using too much energy. This is also an important criteria for me. One more thing that just came to my mind is. If it could be passively cooled like the CPU of the ASRock C2750D4I it would amazing.

All the best

 

[ctpfaff]

Intel Xeon E3-1231 v3 @ 3.40GHz would be a possible choice I think. As of its benchmarks it should handle up to 4 1080p streams seamlessly. Plex states it needs approximately 2000 passmarks per 1080p stream. The CPU has 9588 in average. Still remains the question on effective cooling. Passive would be amazing if possible.

 

[ctpfaff]

Just another question regarding the marvel controller on the ASRock C2750D4I. Might be a stupid question but I have no idea. Are these controllers problematic in general or only if the hardware raid functionality of the board is used? I read that the hardware raid thing can be disabled after a firmware update. Does this help to circumvent problems?

 

[BigDave]

there is another one with an additional "I" in the name "E3C224D4I-14S" is that the one you mean

Yes, that is the board I recommended, it was a typo on my part, sorry.

I have think to remember I have read about a rule of thumb with RAM and ZFS that states that it is good to have 1 GB RAM per TB of storage space. This would mean in my configuration 10x6TB in raid z2 => 48 GB HDD => 16 GB more RAM would be good. Can it happen that I encounter noticeable performance issues with only 32 GB RAM or run into other problems even worse like a degraded pool???

The "rule" you mentioned is some what antiquated since FreeNAS now uses compression by default.
Under your proposed configuration (and usage/load), 32GB of RAM should be good enough for home use.

I read some comments about that board where people state to have problems with RAM errors on this board and had to clock down the RAM to circumvent the errors.

I tend to think alot of these bad comments come from people who don't use the proper brand/model of RAM
or maybe don't provide proper cooling. Thus clocking it down, it runs cooler and stops the errors? I always check how
many times the UEFI gets a version upgrade from the manufacturer, sometimes these issues have been fixed?

Another thing that comes up as a question when I use this board. Which processor actually to put into it. Any suggestions here? My basic requirements to the CPU are simple. It should handle the media content with Plex fine. That means for example transcoding of multiple 1080p streams at the same time. Ore more generally it should be as fast as possible handling transcoding with Plex. That would be great as Plex offers that nice possibility to transcode things down to a lower quality so you can take a video with you to go on your mobile device. It would be cool to not have to wait too long for that process. But I have to say that I prefer a CPU that is also not using too much energy. This is also an important criteria for me. If it could be passively cooled like the CPU of the ASRock C2750D4I it would amazing.

Intel Xeon E3-1231 v3
http://ark.intel.com/products/80910/Intel-Xeon-Processor-E3-1231-v3-8M-Cache-3_40-GHz
You can get faster, but you will spend alot more money. This will do the job...
Unfortunately passive cooling is not an option for this CPU.
All the socket 1150 processors are great when it comes to saving power/energy, BUT when you
need the power, it will be there.

 

[BigDave]

Passive would be amazing if possible.

Remember, you are putting 10 hard disks in a shoe box, cooling these will be anything but passive:p

 

[BigDave]

Just another question regarding the marvel controller on the ASRock C2750D4I. Might be a stupid question but I have no idea. Are these controllers problematic in general or only if the hardware raid functionality of the board is used? I read that the hardware raid thing can be disabled after a firmware update. Does this help to circumvent problems?

AnandTech had this to say about the board, there are many others, just Google for more...

 

[joeschmuck]

I'd take one of these systems if I could spend the money for it without getting placed in the dog house.

 

[BigDave]

My better half would prob. take the dog house (along with everything else) in the divorce

 

[tyronebiggums]

Guten abend!

As stated on here in countless other threads, there are known issues with using Kingston memory. Crucial seems to be the defacto standard for FreeNAS builds. If you go with that ASRock motherboard that @BigDave recommended, this is the kit that you would need for it: https://www.lambda-tek.de/Crucial-C...gbaseDE22.16?gclid=CNjmsOjhvccCFUc8gQod--QD8Q

 

[ctpfaff]

Hello together

Thank you for the further input. As passive cooling is no option I was thinking about having something that is as silent as possible. Thus, cooler wise, I was thinking into the following direction:

* EKL Alpenföhn Brocken 2 (BxHxT: 146x165x98mm)
* Be Quite Pure Rock (L x B x H 87.5 x 121 x 155mm)

However non of these fit into the case. It also might be possible that a cooler that is as big as the case allows (150 mm height) might be over-dimensioned and that a silent cooling solution for that CPU might be possible with something much smaller but I do not know. Some input on that would be great. I can also imagine that a cooler that big might hinder the rest of the air flow in that small case a bit. What cooler would you use if you would plan to place this machine in your living room somewhere?

Another question that arose while checking the board you suggested (E3C224D4I-14S). I am not familiar with SAS and the specs state

8 x SAS2 from 2 x mini SAS 8087 connector by LSI 2308

So in my scenario I would start using the 4 SATA ports and then I would connect the rest of the drives via adapters to the SAS ports? Is this the right adapter here:

http://www.amazon.de/Delock-mini-SAS-Kabel-36-polig-SATA/dp/B001HC4NDA

All the best

 

[ctpfaff]

After a bit of research regarding a cooling solution I found the

* EKL Alpenföhn Brocken Eco (bxhxt: 126x150x88)

It has a 120 mm fan which should be quite silent and the specifications say that the cooler fits exactly
into the case (150 mm height). I read somewhere that this big tower coolers can be used in a semi passive cooling strategy
(at least in some scenarios). Now I am wondering if this might work out for my case. But I think here
I will have to try and measure the temperature of the CPU. The case has 4 120 mm fans that should
generate a nice air flow. My only worries remaining with this cooler is that it intrudes the space reserved
for the 4 DIMMS and the other thing is that it might not fit into the case
maybe just because of a millimetre or so that I won't be able close it. That would be a pity.

All the best

 

[ctpfaff]

And one more question regarding the PSU. The one I picked comes with 6 SATA cables. How to I best connect the rest of the HDDs to the PSU? Using adapters splitting a connector into two? Or is this something that is not recommended and there is a better solution?

 

[BigDave]

What cooler would you use if you would plan to place this machine in your living room somewhere?

I would just use the stock cooler on the cpu, if it's too loud for your living room, do like most of us have done and move it to a location where the noise isn't
a problem. The noise from the fans required to cool 10 hard disks will not be fit for the living area IMHO

Another question that arose while checking the board you suggested (E3C224D4I-14S). I am not familiar with SAS and the specs state

8 x SAS2 from 2 x mini SAS 8087 connector by LSI 2308

As far as I know the retail boxed version of this board comes with the mini SAS to SATA cables, no need to buy others unless you're worried about quality.

And one more question regarding the PSU. The one I picked comes with 6 SATA cables. How to I best connect the rest of the HDDs to the PSU? Using adapters splitting a connector into two? Or is this something that is not recommended and there is a better solution?

No better solution is needed, the back planes you will have in the case take four pin Peripheral connections as well as SATA
From the Lian Li website

Edited for content and spelling

 

[ctpfaff]

He

Thank you so much for your valuable input. Seems like I am good to go now and buy the hardware soon.I will update the first post to list the revised list of components at one glance.

All the best

 

[ctpfaff]

He

I have some remarks on the assemblage. The case is really nice however getting in all of the components is pain in the hat. There is really not much space to use your hands to fix stuff in the case. I have to confess that I decided to prefer the big cooling tower (EKL Alpenföhn Brocken Eco) over the boxed cooler which just makes the space situation worse. In fact it took me some time with trail and error to find the right order of assemblage. Things need to be done in a more or less specific order otherwise you get stuck and have to go back again which can be painfull. Thus I provide the order that I figured out working in the end.

* Mount CPU
* Mount RAM
* Mount CPU cooler (without fan)
* Attach the two sas to sata cables on the motherboard (with fan mounted hard to get them in)
* Attach the two sata cables (next to the sas connectors if you want to fill all the 10 bays with hdds)
* Mount the CPU cooling fan (blowing into the direction of the exhaust fan located on top of the case)
* Open both sides of the case
* Mount the backplane for the motherboard into the case (where the connectors go)
* Mount the motherboard into the case
* Attach the connectors from the frontpanel to the motherboard (Fans + Power Butten etc.. really annoying)
* Attach the CPU cooler power supply to the motherboard
* Assemble the PSU. Attach peripherial power supply cables to the PSU (distribute the hdd popwer supply equally over as many cables as you can)
* Mount the PSU in the case and connect the four pin and the other power connector to the motherboard
* Organize the power cables first then also the sas to sata cables as well as the sata cables
* Mount the HDD backplanes into the case
* Attache the sata connectors and the peripherial power cables to the backplanes.
* Organize your cables a bit more (use straps and the cable channel in the case) so that you can close one side of the case
* Then you just have to add your HDDs and are done

EDIT:

Just one more word about the cooling strategy. The PSU does its own job. Sucks in fresh air from below and blows it out in the back. Fresh air is streaming in to the front over the HDDs getting warm slowly and rising up. With the fan on the CPU cooler mounted to blow into the direction of the top exhaust we just increase the speed of warm air rising up anyway. The strategy seems to work quite well. The CPU does hardly reaches 50 degree celsius under full utilization.