BUILD New year, new resolutions, new build

[ponas]

Hi guys, I’ve been lurking and reading these forums for a while now and I think I’m finally ready to make the jump and build my first NAS. I’m pretty much a noob when it comes to hardware and I’m quite new to the whole NAS thing. So, sorry already if some of my questions are way noobish.

After looking around the forums I settled on 2 different configurations. The first one uses the X10 mobo from SuperMicro and the second one the X11 platform. I understand that there’s a lot more support and documentation for the older X10, however I want to build something that will last me for a while and also I guess having the “latest and greatest” would be nice. From what I gather, the main issue for installing FreeNAS on the X11 mobo will be the XHCI issue. Is there something else I might have overlooked? From your experience, how probable is it that I will run in hard/very hard to debug problems due to using this newer platform?

I plan on using my server in RAIDZ2 for media storage, plex streaming/transcoding as well as a personal cloud and windows backup machine.

Here’s what will be common between the 2 configurations:

Case: Fractal Define R5

Power Supply: Seasonic G650 (this is probably overkill (from what I gather of the sticky on PSU sizing) but is there a risk I don’t know in running a bigger-than-needed PSU?)

HDD: WD Red 4To x 6 (but might be expended later to 8 or 10 drives)

Now for the differences;

1st configuration:

Motherboard: SuperMicro X10SL7-F

RAM: Crucial CT102472BD160B x 4 or Samsung M391B1G73QH0-YK0 x 4 (is one usually more recommended than the other? or since they are both on the “tested memory list” they are equivalent?)

CPU: Xeon E3-1230v3

Boot device: SanDisk 16 Gb Cruzer Fit SDCZ33-016G-B35 x3

2nd configuration:

Motherboard: SuperMicro X11SSH-F (I was also looking at the X11SSH-CTF, however is there any word on how long it will take for the new drivers?)

RAM: Samsung M391A2K43BB1-CPB x2 (is there any other RAM that would also be compatible with this board other than the Hynix one ?(because I cannot find it where I live))


CPU: Xeon E3-1230v5


Boot device: SuperMicro SATADOM 32Gb SSD-DM032-PHI x2 OR Transcend 32Gb SSD TS32GSSD370 x2


Am I missing something in these lists for my build? Will I need to buy extra fans? Looking at the X11SSH-F board specs, it says that it comes with 8 SATA III ports as well as 2 SuperDOM are the 2 SuperDOM ports included in the count of 8 SATA ports?(because looking at the board’s picture I only see 6 SATA white SATA ports and 2 yellow ports that I assume are the SATADOMs) Finally, if I go with the X11 board is booting from SATADOM and SSD equivalent?, is it a big issue if I don’t have the recommended 3 boot devices (1 for boot, another for update and the last one for oops), is there something I can do to mitigate this ?

Thank you so much for helping! And sorry for the wall of text and question. Have a wonderful new year!

 

[Ericloewe]

Power Supply: Seasonic G650 (this is probably overkill (from what I gather of the sticky on PSU sizing) but is there a risk I don’t know in running a bigger-than-needed PSU?)

Not on any modern high-quality PSU (within reason, of course).

At that power level, you can also consider the Seasonic X-650 or Seasonic Platinum 660 (they were the same price on Amazon.de last time I checked - I needed to buy two and the Platinum 660 was out of stock, so I ordered to X-650s. Of course, the next day, the Platinums were back in stock just after they'd shipped the X-650s )

Motherboard: SuperMicro X11SSH-F

RAM: Crucial CT102472BD160B x 4 or Samsung M391B1G73QH0-YK0 x 4 (is one usually more recommended than the other? or since they are both on the “tested memory list” they are equivalent?)

CPU: Xeon E3-1230v3

You've got the wrong board listed here. ;)

Anyway, Crucial or Samsung should both be fine, as would Hynix.

Motherboard: SuperMicro X11SSH-F (I was also looking at the X11SSH-CTF, however is there any word on how long it will take for the new drivers?)

Hard to say. 10GbE shouldn't take too long, since it's a derivative of the already-supported Intel 10GbE stuff.

LSI SAS3 is a bit trickier. P9 is out now, but nobody's actually validated it and said "Yup, works fine!". Expect bugs.

The X11SSH-F is an odd board - the only difference is that one PCI-e slot is gone, replaced with 2x PCI-e 3.0 M.2 slot (the other two lanes are unused - they're for the extra NICs in the X11SSH-LN4F).

RAM: Samsung M391A2K43BB1-CPB x2 (is there any other RAM that would also be compatible with this board other than the Hynix one ?

There's little data so far. I recommend you stick to the QVL as much as possible. I have one of the Samsung DIMMs on order, along with an X11SSM-F. Try geizhals.eu to see if any of your usual stores sell the RAM you're looking for.

SuperMicro SATADOM 32Gb SSD-DM032-PHI x2 OR Transcend 32Gb SSD TS32GSSD370 x2

A bit overkill. A single SSD would probably be enough. In any case, cheap consumer SSDs are a better deal than the DOMs - they're not as easy to use or as cute, but they're much cheaper.

are the 2 SuperDOM ports included in the count of 8 SATA ports?

Yes, they're the yellow ones.

is booting from SATADOM and SSD equivalent?

From the system's point of view, yes.

is it a big issue if I don’t have the recommended 3 boot devices (1 for boot, another for update and the last one for oops), is there something I can do to mitigate this ?

Not at all, that's old advice. Upgrades are in-place, using ZFS snapshots. You still have access to your previous environments.

If something goes wrong and you need to troubleshoot, with an X11 board you will probably need a second SATA device, though. If it were an X10 board, a USB drive would be fine. (Just realized this myself, I'll have to prepare myself for that...)

 

[ponas]

Thanks a lot for the super fast answer!

You've got the wrong board listed here. ;)

Ahahah indeed, I meant the X10SL7-F

At that power level, you can also consider the Seasonic X-650 or Seasonic Platinum 660 (they were the same price on Amazon.de last time I checked - I needed to buy two and the Platinum 660 was out of stock, so I ordered to X-650s. Of course, the next day, the Platinums were back in stock just after they'd shipped the X-650s )

I just checked but both the X version and the platinum one are 50-60 bucks more. Are the extra efficiency and the fan control worth the added cost in your opinion ?

The X11SSH-F is an odd board - the only difference is that one PCI-e slot is gone, replaced with 2x PCI-e 3.0 M.2 slot (the other two lanes are unused - they're for the extra NICs in the X11SSH-LN4F).

I chose the X11SSH-F board because I wanted more SATA ports but also because it is the one I saw people use most in the forums. However it's true that the X11SSM-F also looks good (and has the same number of SATA). Since I'm not totally clear on what everything is for on the boards, what would be the advantage for me to have the extra 2 M.2 ports of the X11SSH-F or the PCI-e slot ? what are these usually used for ?(sorry for the noob question)

Concerning the SSD would you have any recommendation for some cheap models that would fit the bill ? And would i need extra fans with the case I chose or are the two 140mm fans that come with it enough to insure sufficient cooling ?

thanks again!

 

[Ericloewe]

I just checked but both the X version and the platinum one are 50-60 bucks more. Are the extra efficiency and the fan control worth the added cost in your opinion ?

Probably not. For me, it was mostly an availability question.

hat would be the advantage for me to have the extra 2 M.2 ports of the X11SSH-F

You don't get two, you get a single M.2 port. It allows for small PCI-e SSDs (probably best for boot devices or L2ARC).

the PCI-e slot ?

Larger PCI-e SSDs, SAS controllers, 10GbE NICs...

Concerning the SSD would you have any recommendation for some cheap models that would fit the bill ?

The cheapest models you can find from Crucial or SanDisk are fine.

And would i need extra fans with the case I chose or are the two 140mm fans that come with it enough to insure sufficient cooling ?

I'm not familiar with the case, but more fans are almost always better.

 

[BigDave]

Concerning the SSD would you have any recommendation for some cheap models that would fit the bill ?

I use these Intel drives for my boot devices in two Freenas machines and my pfSense box. If you don't mind giving up the sata port to hook them up,
the flash chips in these things are Single Layer Cell (SLC) and therefore give a level of reliability WAY beyond a USB flash drive. Their use for FreeNAS has been
discussed in many posts here in the forum, and are used by more than a few members. If you decide to go this way, try to purchase a unit with the 8860 firmware
as shown in the linked auction.

 

[ponas]

So after much time I think I'm almost decided on the configuration but wanted to do a last sanity check and still had a couple questions.
Case: Fractal Define R5 (already ordered)
HDD: 8 x WD Red 4To
CPU: Xeon E3-1230v5
RAM: 2 x Samsung M391A2K43BB1-CPB
Power Supply: SeaSonic X-650 (I'm going with the X series since it has 10 SATA and I might want to add SSDs later on for jails. However is the 650W going to be enough for the 8 drives + potential 2 SSD etc. ? or should I go with the 750W ?)
Motherboard: X11SSH-F this is where I'm hesitating. I am thinking of going with the X11SSH-F since it has an M.2 port and I was thinking of using it for a small SSD boot device in order to save the 8 SATAs for the HDDs. Is this a good idea ? and if so how would I go about installing FreeNAS on the M.2 SSD ? (I am not too clear on how i could use the IPMI for that) Furthermore, I can relatively easily find the X11SSH-F board where I live whereas the X11SSM-F is much harder to find.
Otherwise, I would go with the X11SSM-F with the 2 PCiE ports and would probably need to replace one of the HDDs with a SATA SSD as a boot device since PCiE SSDs seem to be quite expensive.
Boot device: whether Transcend 32GB M.2 SSD or SanDisk ReadyCache 32GB

Now for the final questions: My case came with two 3pins fans but from what I read around here, I would be better off having 4pins fans and since I would anyways need to add at least 2 extra fans for cooling and mixing 3 and 4 pins was not recommended, does anybody have a recommendation for good 140mm or 120mm 4pins fans ? And last, does the CPU come with heatsink and fan or should I buy that separately ?

Thanks a lot for all your help again

 

[Bidule0hm]

The X-650 is more than enough for 8 drives. Plus SSD don't care about the +12 V rail and they don't eat too much power anyway ;)

Search on the forum for the X11 boards. In short they are very new so there's maybe some bugs not found yet.

The Arctic F12 PWM CO (there's also the F14 PWM that is 140 mm instead of 120 mm but there's no CO version for this one) fans are very silent, good quality and relatively cheap but they don't have too much static pressure (but the Define R5 is not a restrictive case anyway), that's what I use and I'm very happy with them ;)

 

[Ericloewe]

(I am not too clear on how i could use the IPMI for that)

Uhmm... Same way you'd install to any device.

Transcend 32GB M.2 SSD

That's a SATA M.2 SSD, not a PCI-e SSD.

 

[ponas]

That's a SATA M.2 SSD, not a PCI-e SSD.

I know. Maybe that was not very clear from my previous post, but if i go the X11SSM-F way I'd use the Transcend 32GB M.2 SATA SSD connected to a SATA port on the motherboard as a boot device and have only 7 HDDs since PCI-e SSDs are quite expensive.

Would there be any differences in using the M.2 port vs the SATA for the boot device ? is there any caveats I should be aware of using an M.2 boot device ?

The Arctic F12 PWM CO (there's also the F14 PWM that is 140 mm instead of 120 mm but there's no CO version for this one) fans are very silent, good quality and relatively cheap but they don't have too much static pressure (but the Define R5 is not a restrictive case anyway), that's what I use and I'm very happy with them ;)

Thanks! I'll look into those.

 

[Ericloewe]

Would there be any differences in using the M.2 port vs the SATA for the boot device ?

Well, you'd have one more SATA port available.

is there any caveats I should be aware of using an M.2 boot device ?

I don't expect any.

 

[ponas]

Thank you Eric:)

The Arctic F12 PWM CO (there's also the F14 PWM that is 140 mm instead of 120 mm but there's no CO version for this one) fans are very silent, good quality and relatively cheap but they don't have too much static pressure (but the Define R5 is not a restrictive case anyway), that's what I use and I'm very happy with them ;)

Just looked around for those but they are not sold by the retailers I'm using :( However they sell some Noctua fans. Do you think four Noctua NF-F12 PMW will be enough to cool my case (they seem to be weaker than the Arctic ones your recommended) or should I go with four of the more powerful Noctua NF-A14 3000 PMW ? (I'm a bit worried about the noise level)

 

[Bidule0hm]

Noctua is a good brand but I don't know about their specs, I guess it should be ok.

But, I was thinking about one thing: if it's like the Define R4 the R5 comes with 2 fans (personally on my R4 I moved the fan from the rear to the front to be in full positive pressure to avoid dust) so why do you want more fans?

 

[ponas]

if it's like the Define R4 the R5 comes with 2 fans (personally on my R4 I moved the fan from the rear to the front to be in full positive pressure to avoid dust) so why do you want more fans?

Indeed, the R5 did come with 2 fans. However these are 3pins fans and I wanted to have 4pins since it seemed like the better choice when using the SuperMicro board. Also I wanted to add a couple extra fans to be sure I can really cool everything well(and mixing 3 and 4 pins doesn't seem to be a good idea according to the forums). I don't really have an idea though if 4 fans is way overkill for my setup or if it is reasonable. Would you have an idea ?

 

[Bidule0hm]

Ah yes, I read that the other day, sorry, crappy day at work...

Personally I'd start with two fans in the front and then add more fans if you have a thermal issue but I think two fans will be enough ;)

And I recommend to mount the PSU with the fan on top; I know, the PSU will suck "warm" air from the case instead of cool external air as if it was mounted with the fan below but in this case it will also not suck dust and the air path is far less restricted, moreover the "warm" air isn't that warmer than the external air. The bottom oriented thing is the kind thing that in theory is a good idea but in practice is not so good... :)

It has also the advantage of providing a minimum air flow through the PSU even if his fan fails, and as the PSU is probably the most important thing in your server it's always a good thing to care about it...

 

[Ericloewe]

Ah yes, I read that the other day, sorry, crappy day at work...

Personally I'd start with two fans in the front and then add more fans if you have a thermal issue but I think two fans will be enough ;)

And I recommend to mount the PSU with the fan on top; I know, the PSU will suck "warm" air from the case instead of cool external air as if it was mounted with the fan below but in this case it will also not suck dust and the air path is far less restricted, moreover the "warm" air isn't that warmer than the external air. The bottom oriented thing is the kind thing that in theory is a good idea but in practice is not so good... :)

It has also the advantage of providing a minimum air flow through the PSU even if his fan fails, and as the PSU is probably the most important thing in your server it's always a good thing to care about it...

I prefer to have them facing down, to minimize the chance of screws falling into them.

...I'm a firm believer in Murphy's Law.

 

[Bidule0hm]

Mmh... not a bad argument but what screws are we talking about? because I only see MB screws falling at this place and only if you unscrew them (they don't fall on their own) which you can do with the case on it's side so no screws can fall in the PSU this way :)

 

[Ericloewe]

Mmh... not a bad argument but what screws are we talking about? because I only see MB screws falling at this place and only if you unscrew them (they don't fall on their own) which you can do with the case on it's side so no screws can fall in the PSU this way :)

There's not much in my server that could fall in there, but my workstation is quite a different matter. I can't get anything done in this monstrosity without removing several screws. Except swapping drives, but I only remove them for cleaning.