Sanity check my first build please

[Naskolnikov]

Cheers.

I had a great plan to do some research before buying a new computer and replacing my old Netgear ReadyNAS when FreeNAS 10 arrived. Yesterday i logged on to my ReadyNAS, while doing said research, to check some settings and lo and behold, one disk had died. Tried replacing it with my spare disk but it did not work. Figured my spare disk had been lying around dead on arrival for two years. Went to the store today and bought a new disk but that did not work either. I guess one of the remaining three disks isn't as healthy as it seems. While this is not a catastrophe, the stuff i really want safe exists on backups, it means my plan to wait for FreeNAS 10 will be abandoned.

It would be splendid if someone with a bit of experience (when it comes to hardware i am hopelessly out of the loop and even though i have *nix experience i have never used FreeNAS) took a look at what i have come up with, because even if i have looked at some other builds and read up on a thing or two, a fresh pair of eyes is never a bad thing. The box will run Plex, Squeezebox (or whatever Logitech calls it nowadays) and Transmission.

The idea is to use the following:
Motherboard: Supermicro X10SDV-4C-TLN2F.
CPU: Intel® Xeon® processor D-1521, embedded CPU, a fan will be added to the heatsink.
Memory: 2x16GB Samsung M393A2G40DB0-CPB, DDR4, registered ECC, 2133MHz.
PSU: Seasonic G-550.
Case: Fractal Design Node 304.
Disks: 6xWD Red 3TB, this time with two drives for parity.
Boot device: When it comes to this i am at a loss since i get conflicting information. Is using a single M.2 device really a better solution than mirrored USB in any other aspect than speed?
The whole shebang will cost ~1600€, including shipping to Sweden, a tad bit more than my old friend the ReadyNAS but since it has worked so tirelessly for me it deserves some rest.

Any kind of input is greatly appreciated.

//Naskolnikov

 

[Ericloewe]

Motherboard: Supermicro X10SDV-4C-TLN2F.
CPU: Intel® Xeon® processor D-1521, embedded CPU, a fan will be added to the heatsink.
Memory: 2x16GB Samsung M393A2G40DB0-CPB, DDR4, registered ECC, 2133MHz.
PSU: Seasonic G-550.
Case: Fractal Design Node 304.
Disks: 6xWD Red 3TB, this time with two drives for parity.

Thumbs up!

Is using a single M.2 device really a better solution than mirrored USB in any other aspect than speed?

Yeah, though PCI-e M.2 can be rather expensive - but there are few alternatives with that board.

USB boot devices work, they're just generally unreliable (unless you stay with known-good models) and very slow (which you won't notice outside of updates and other boot device manipulations).

To cut costs, you could start with 16GB of RAM (which will almost certainly be plenty) and use UDIMMs, which should be cheaper than RDIMMs (dunno if there are any UDIMMs on the QVL for that board, though).

 

[Ericloewe]

Disks: 6xWD Red 3TB, this time with two drives for parity.

Oh, nitpick: With RAIDZ, parity is distributed evenly over the drives. There are no dedicated parity drives.

From the user's perspective, the end result is the same.

 

[Naskolnikov]

Thumbs up!

Yeah, though PCI-e M.2 can be rather expensive - but there are few alternatives with that board.

USB boot devices work, they're just generally unreliable (unless you stay with known-good models) and very slow (which you won't notice outside of updates and other boot device manipulations).

To cut costs, you could start with 16GB of RAM (which will almost certainly be plenty) and use UDIMMs, which should be cheaper than RDIMMs (dunno if there are any UDIMMs on the QVL for that board, though).

If speed is the only issue, as long as you stick to good models, i guess there is no reason for me to go down the M.2 route.
That QVL sure is elusive, i haven't found it so in fact i do not know for certain that the Samsung sticks will work. I guess it is worth looking more deeply into though, if it can save me a couple of bucks.

Oh, nitpick: With RAIDZ, parity is distributed evenly over the drives. There are no dedicated parity drives.

From the user's perspective, the end result is the same.

Ah, okey. You live and learn.

Thanks for the input. It's a good thing my work assignment tomorrow will be the control room, so i can research the quality of USB sticks and compatibility of UDIMM:s, while the rest of the lot actually works.

 

[Ericloewe]

That QVL sure is elusive, i haven't found it so in fact i do not know for certain that the Samsung sticks will work.

Shows up fine for me on Supermicro's product page. Anyway, here's the link: http://www.supermicro.com/support/resources/mem_exclude_DDR3_135V.cfm

 

[Naskolnikov]

Shows up fine for me on Supermicro's product page. Anyway, here's the link: http://www.supermicro.com/support/resources/mem_exclude_DDR3_135V.cfm

Of course...
When i looked at that the first time i did not note the upper tab where you could select different sizes. Considering i thought they listed only one registered and one unregistered model i assumed that this was not the QVL but something like a Prefered VL.
I guess 2x8GB Samsung M391A1G43DB0-CPB (saving ~50€) or 2x16GB Samsung M391A2K43BB1-CPB (saving ~25€) are my options for unregistered memory then.

 

[Ericloewe]

Don't get anything smaller than 16GB, to keep your upgrade options open.

 

[joeschmuck]

When selecting a USB flash drive, stick to USB 2.0 models, do not get a USB 3.0 model no matter how well it's rated. I don't want to put words into @Ericloewe 's mouth but I assume that is what he meant by a slow USB model.

Disks: 6xWD Red 3TB, this time with two drives for parity.

I think we all assume you mean RAIDZ2.

Boot device: When it comes to this i am at a loss since i get conflicting information. Is using a single M.2 device really a better solution than mirrored USB in any other aspect than speed?

Did you read the user manual for the motherboard? It states that it shares the SATA0 port, what I'm curious of is if it means if you use the M.2 card that the SATA0 port can no longer be used or if it somehow magically does multiplex the two drives. Probably need to find out more about it. The reason for mentioning it is a SSD is much more reliable than a USB Flash drive as a boot device so if adding an M.2 SSD at a later date is possible, that would be good to know.

 

[Naskolnikov]

Don't get anything smaller than 16GB, to keep your upgrade options open.

Right, 2x16GB Samsung M391A2K43BB1-CPB it is then. I'd rather spend a couple of bucks extra and have way too much RAM than save a couple and regret not getting more RAM from the beginning.

When selecting a USB flash drive, stick to USB 2.0 models, do not get a USB 3.0 model no matter how well it's rated. I don't want to put words into @Ericloewe 's mouth but I assume that is what he meant by a slow USB model.


I think we all assume you mean RAIDZ2.

Did you read the user manual for the motherboard? It states that it shares the SATA0 port, what I'm curious of is if it means if you use the M.2 card that the SATA0 port can no longer be used or if it somehow magically does multiplex the two drives. Probably need to find out more about it. The reason for mentioning it is a SSD is much more reliable than a USB Flash drive as a boot device so if adding an M.2 SSD at a later date is possible, that would be good to know.

The plan is to use an adapter on an internal USB header and stick 2xSandisk Cruzer Fit (i seem to remember those being recommended on this forum but please correct me if that is wrong) in the adapter.

Yeah, RaidZ2 is what i mean, i am just not used to the lingo. A raid where two out of six disks can crash without dataloss.

The manual isn't crystal clear, but since the only real issue with USB:s seems to be speed the M.2-option is no longer interesting.

 

[Ericloewe]

if it means if you use the M.2 card that the SATA0 port can no longer be used or if it somehow magically does multiplex the two drives

There's no such thing as magic. Only one of them will work at one time. However, it might be wired for PCI-e, too, in which case, it won't take up a SATA port (when used with PCI-e M.2 drives, obviously).

 

[Ericloewe]

Right, 2x16GB Samsung M391A2K43BB1-CPB it is then. I'd rather spend a couple of bucks extra and have way too much RAM than save a couple and regret not getting more RAM from the beginning.

But you can start with a single 16GB DIMM and add more as required. It's trivial to do so later.

 

[joeschmuck]

There's no such thing as magic. Only one of them will work at one time. However, it might be wired for PCI-e, too, in which case, it won't take up a SATA port (when used with PCI-e M.2 drives, obviously).

The manual only specified it was SATA but who knows. The manual also states it multiplex's the two connections which leads me to believe both could be active but it could be a poor choice of words in the manual. I have seen manuals which clearly state it's ether one active and not both.

 

[Naskolnikov]

But you can start with a single 16GB DIMM and add more as required. It's trivial to do so later.

This is a good option of course, which will save me ~75€. I guess i will consult with my bank account before i deciede if i should try with "only" 16GB first.

There's no such thing as magic. Only one of them will work at one time. However, it might be wired for PCI-e, too, in which case, it won't take up a SATA port (when used with PCI-e M.2 drives, obviously).

The manual only specified it was SATA but who knows. The manual also states it multiplex's the two connections which leads me to believe both could be active but it could be a poor choice of words in the manual. I have seen manuals which clearly state it's ether one active and not both.

The manual is a bit cryptic.

Code:

The X10SDV series deploy an M key
only dedicated for SSD devices with the ultimate performance capability in a PCI
Express 3.0 X4 interface for native PCIe SSD support. The X10SDV M.2 is mux
with the I-SATA0 port for legacy SATA SSD devices.

 

[Ericloewe]

It implies that it does both SATA and PCI-e.

 

[Naskolnikov]

It implies that it does both SATA and PCI-e.

Yeah, it would probably work splendid to use a M.2 PCIe as a boot device with 6xSATA HDD:s but it seems like an awful expensive way to get faster updates.

 

[joeschmuck]

While in your situation I would be hesitant on purchasing an M.2 card, the reason for using a SSD has nothing to do with speed or faster updates, it's purely a longevity thing over USB Flash drives. But true, updates are considerably faster.

 

[Bl4ckShadow]

Very informative thread! :)
One question: Should the SSD boot drive be mirrored in any form to get extra safety?
I am about to build my first FreeNAS-server as well and was about to use two little bite more expensive mirrored USB (3.0) sticks for the OS, but that plan just changed. :D
But since a failure of the boot SSD - for whatever reason - would be catastrophic in terms of the whole data stored in the RAID-array, I am thinking about buying a secondary OS-drive.

 

[Mirfster]

Should the SSD boot drive be mirrored in any form to get extra safety?

If so desired. To some it is overkill, to me if you have the spare ports and space it is not a bad idea. Just keep regular backups of the configuration file. If you have that re-installing and uploading the config file is pretty trivial.

But since a failure of the boot SSD - for whatever reason - would be catastrophic in terms of the whole data stored in the RAID-array, I am thinking about buying a secondary OS-drive.

Failure of the boot device does not affect the data stored on the hard drives. They are intentionally separated for this very reason (among other reasons).

 

[Mark Holtz]

Boot device: When it comes to this i am at a loss since i get conflicting information. Is using a single M.2 device really a better solution than mirrored USB in any other aspect than speed?

Make sure you get the right m.2 drive. The motherboard specs call for a "M.2 PCI-E 3.0 x4". I use one myself in my FreeNAS build because I wanted something reliable, and all eight SATA ports were being used for hard drives. I ended up using a Samsung SM951 which cost me $100 USD. The really big gotcha is that the smallest of these m.2 drives is 128GB, which is several times bigger than what is needed for FreeNAS (at most 16GB). The other considerations for using a m.2 drive, such as faster access and faster boot times, do not apply to a FreeNAS build. It is rare, once you have FreeNAS running, that you will reboot the server beyond system updates.

On the other hand, it beats figuring out what the best USB drive to use on your FreeNAS build.

 

[Ericloewe]

which is several times bigger than what is needed for FreeNAS (at most 16GB)

16GB is actually a reasonable target for 9.10, hardly a maximum. FreeNAS 10 is said to recommend at least 32GB boot devices.

I'd say that 128GB is a nice size that will work well, but not excessive overkill.