BUILD Time for a new machine

[adrianwi]

I've been 'playing' with FreeNAS for about a year now after moving up to a HP Microserver (N40 and then N54) from a pretty ancient Netgear ReadyNAS NV+

It's been a huge learning curve, with lots of frustration and equal amounts of satisfaction when things are running just as I wanted. This forum is fantastic, and full of great guides and help when required. Anyway, the N54 is pretty much approaching capacity and is starting to struggle a little with all the things I'm throwing at it now, so though it was time to build something to last me another few years.

I've checked out the hardware recommendations, and this is pretty much based on that. Just wondering whether I'm going a little overboard and should look at other options. I'm approaching 10TB of storage (mainly media files and backups from a number of Macs) and have jails running Plex Media Server, PlexConnect, ownCloud, openVPN and unbound DNS. Might look at running some VMs with a little more space and horsepower, but it's best described as a home media server with some small home business use.

Anyway, here's what I was thinking:

Fractal Design Define R4 Case
Corsair Builder Series CXM 600W Modular 80 PLUS
Supermicro MBD-X10SL7-F-O Motherboard
Intel BX80646E31230V3 Quad-Core Processor
Crucial CT4KIT102472BD160B 32GB (4x 8GB) Memory
Seagate ST4000VN000 4TB NAS HDD SATA III 3.5 inch

Plan was to use the 4x4TB Seagate drives in my N54 and add another to create a bigger RAIDZ1 pool. I'll carry on using the N40 as a nightly backup (this has 4x4TB drives striped) so I have some redundancy and a ZFS replicated copy of everything should the worst happen. Will probably upgrade my UPS too as thats getting a little old now.

Anyway, thoughts on the build much appreciated...

Thanks

 

[Bidule0hm]

I don't recommend Corsair PSU (excepted the AX series) for server usage. Personally I recommend SeaSonic, but there's also others good brands ;)

Don't use RAID-Z1, and even less with 4x 4 TB drives, see the link in my sig about the RAID-Z1/RAID5 problem :)

You might want to start with 16 GB of RAM and only add 16 more GB if the performance are too low, no need to buy 32 GB if it's not useful ;)

 

[BigDave]

The board's sata ports will be wasted with only four drives (if I read your intentions correctly).
Buy a lesser X10, put only 16GB of RAM (I agree with BiduleOhm) and use $ for a six drive RAIDz2 setup;)

 

[DrKK]

The board's sata ports will be wasted with only four drives (if I read your intentions correctly).
Buy a lesser X10, put only 16GB of RAM (I agree with BiduleOhm) and use $ for a six drive RAIDz2 setup;)

Yeah what Dave said. If you're only using four drives, you'll want to get a Supermicro X10SLM or X10SLL.

 

[DrKK]

Also just for the record, this whole "RAID-Z1 is dead" shit is really overblown. RAID-Z1 is perfectly acceptable for small arrays with medium-capacity drives.

I wouldn't say 4x4TB drives in RAID-Z is "bad". It's on the cusp. You are living a bit dangerously, but it's hardly insane. 4x3TB on RAID-Z I would do all day without a second thought. 4x6TB I'd never do on RAID-Z. I'd say 4x4TB RAID-Z is a borderline case, and I'd personally go with it if I were money-constrained and had irreplaceable data backed up.

 

[GrumpyBear]

As others have pointed out you can get support for up to 6 disks withe the X10SLL or SLM versions so unless you are planning for future expansion past 6 drives you could save some money there.

The E3-1230v3 is still available but the 1231 is out and retails for the same price in most markets and is 100MHz faster.

600w is a lot of power for just 4 disks. You can use a 450w supply for 8 disks and still only run it at 50% under startup and peak loads.

The Define R4 is a nice case but is being replaced by the R5 which has some improvements. Again, unless you are planning on expanding a 6-bay case such as the Define or Arc Minis might be better.

 

[adrianwi]

Thank for all the comments.

I have given RAIDZ1 v RAIDZ2 some thought and might reconsider and add another disk into the mix.

The one common factor with all my previous NAS devices was a 4 disk limit, which meant I was constantly chasing and replacing larger drives. Think my ReadyNAS started with 4x500GB and went through 1TB and 2TB drives (the maximum it supported) before moving to FreeNAS. I've done the same with the N40/54 though, and started with 3TB drives and now have 4x4TB! I thought having room to add more drives and stick at the 4TB level was probably a more sensible approach to jumping up to 5TB and then potentially 6TB.

If I stick with RAIDz1 then a 6 device combination still gives me some room for growth, but RAIDZ2 with 6 drives would leave me in a similar position of needing to increase capacity rather than another drive, so I'll probably stick with the larger case and motherboard. Will look at a smaller power supply (must admit, I just picked one and didn't think about how much I actually needed) and probably start with 16GB RAM.

 

[Bidule0hm]

You know you can't just add a disk to an existing vdev? you need to backup the data, destroy the current RAID-Z1 and create a new RAID-Z2 then copy the data back ;)

 

[adrianwi]

Haha :D Yes, I've learned that much in the past year ;)

As I'll have a backup, adding another disk to either a RAIDz1 or RAIDz2 pool shouldn't be too much of a problem, and probably quicker overall than the process of replacing individual drives and resilvering to increase capacity. I don't have that option with a 4 disk box!

 

[Bidule0hm]

Ok, no problem then ;)

You can add a 5th disk to the existing 4 no?

 

[adrianwi]

I think I probably could add another disk into the optical drive space in the N54 although that doesn't solve my performance issues and will probably only last another 6-9 months.

I'm looking for something that will expand with me for the next 2-4 years

 

[Bidule0hm]

Ah ok, it's the case the problem. Well, with a hammer you can fit pretty much anything into anything... :D

 

[DataKeeper]

If you're at 10TB already and want to be able to expand more over the years I'd be looking for more than 4-6 drives. Something with 8-10 or even 12, even if not immediately used. With a 12 bay setup you could populate them all with smaller/cheaper drives or start with a raidz2 x6 drives with room to add another 6 drives at a later date. Lots of 8 bay cases out there and an 8 drive raidz2 with 3-4TB drives will give you the space you need now and can always be increase later.

Agree with everyone else about starting with 16GB ram, the power and raidz2. Remember, you really do not want to be using 50-60% or your space from the start and you don't want to max out a xfs system. Spend more on hard drives now and be done with it for a couple years. Then simply order a new larger drive every other month and replace the drives one at a time until they all get replaced.

I've seen a number of raid5 arrays fail when a drive was replaced and restoring from a backup always sucks. Use raidz2 and rest easier at the expense of one extra drive.


Edit.. Btw why replace the UPS? Is it just old, sending errors or not holding a charge as long? I've seen many UPSs last 10 years without hardware issues. If it works it works. Most can be opened and batteries replaced easily and cheaply enough.. much cheaper then replacing the unit. Just something to think about as it's a shame how many get tossed due to age yet still function perfectly fine. For home use I'll usually order a used APC without batteries for free/cheap shipping and then simply buy brand new batteries for it. You're able to get a large unit with much more runtime for 1/4 the price usually. Of course I'd not do this in a production environment but for home use it's great.

 

[adrianwi]

Thanks again for all the comments and suggestions. After giving this a bit more thought, and some further reading I'm now leaning toward this:

Fractal Design Define R4/5 Case
Seasonic S12G-450 450W
Supermicro MBD-X10SL7-F-O Motherboard
Intel XEON E3-1231V3 3.40GHZ
Crucial 2x8GB DDR3 PC3-12800 Unbuffered ECC 1.35V
Seagate ST4000VN000 4TB NAS HDD SATA III 3.5 inch

Going to bite the bullet and buy more drives than I'd originally thought and build some future proofing from the start. Thinking 7x4TB in RAIDZ2 which should give me and extra 6-7TB of space to grow into without having to add more disks and rebuild the pool. It also give me the extra disk redundnacy that most seem to favour.

Looking at the motherboard manual though, I noticed it only has 6 SATA ports (2x3 and 4x2) but a similar number of SAS ports. Can one of these be used for a SATA drive?

Priced up at ~£1250 on amazon.co.uk but am sure I can knock that down with a little shopping around. Might take a while to pull everything together but will let you know how it compares to the N54 :D

Oh, and another thought! I can't see anything about onboard graphics on the motherboard. Do most people just have a cheap PCI card inside should you need to connect up to a monitor?

As you can probably tell, it's been a while since I actually built a computer (last one was probably a Pentium 3, although having taken a few iMacs apart in the meantime - which is a much more delicate job - it can't be that different!)

 

[marbus90]

SAS ports can be used for SATA drives as well.

The mobo includes a graphics chip. As well as IPMI. So you only plug in a LAN cable into the IPMI port and you can control everything trough the browser, including remote Keyboard/Video/Mouse.

 

[GrumpyBear]

.. Looking at the motherboard manual though, I noticed it only has 6 SATA ports (2x3 and 4x2) but a similar number of SAS ports. Can one of these be used for a SATA drive?

Most people just use the SAS ports for their pools and save the SATA ports for other things like SSDs. You MUST flash the on-board LSI2308 into IT (JBOD) mode using version 14 for use with FreeNAS 9.3. Eric Lowe has written an excellent article on the X10 Boards which includes hints on how to do this (search the forms and you will find specific instructions).

Don't forget to post pictures too in the Build Pic Thread.

 

[Bidule0hm]

It's version 16 AFAIK ;)

 

[Ericloewe]

Thanks again for all the comments and suggestions. After giving this a bit more thought, and some further reading I'm now leaning toward this:

Fractal Design Define R4/5 Case
Seasonic S12G-450 450W
Supermicro MBD-X10SL7-F-O Motherboard
Intel XEON E3-1231V3 3.40GHZ
Crucial 2x8GB DDR3 PC3-12800 Unbuffered ECC 1.35V
Seagate ST4000VN000 4TB NAS HDD SATA III 3.5 inch

Going to bite the bullet and buy more drives than I'd originally thought and build some future proofing from the start. Thinking 7x4TB in RAIDZ2 which should give me and extra 6-7TB of space to grow into without having to add more disks and rebuild the pool. It also give me the extra disk redundnacy that most seem to favour.

Looking at the motherboard manual though, I noticed it only has 6 SATA ports (2x3 and 4x2) but a similar number of SAS ports. Can one of these be used for a SATA drive?

Priced up at ~£1250 on amazon.co.uk but am sure I can knock that down with a little shopping around. Might take a while to pull everything together but will let you know how it compares to the N54 :D

Oh, and another thought! I can't see anything about onboard graphics on the motherboard. Do most people just have a cheap PCI card inside should you need to connect up to a monitor?

As you can probably tell, it's been a while since I actually built a computer (last one was probably a Pentium 3, although having taken a few iMacs apart in the meantime - which is a much more delicate job - it can't be that different!)

The Seasonic G-450 (not S12G) should be quite a bit better but not too much more expensive.

Most people just use the SAS ports for their pools and save the SATA ports for other things like SSDs. You MUST flash the on-board LSI2308 into IT (JBOD) mode using version 14 for use with FreeNAS 9.3. Eric Lowe has written an excellent article on the X10 Boards which includes hints on how to do this (search the forms and you will find specific instructions).

Don't forget to post pictures too in the Build Pic Thread.

Well, the SAS 2308 has twice the PCI-e bandwidth of the PCH, so, all things considered, I'd place SSDs on the LSI SAS 2308 instead of the PCH.

 

[adrianwi]

Parts ordered :)

Just need to read up on the various threads for updating the BIOS and LSI2308.

What do people do for their USB boot drives? The N54L has one internal USB port, so the stick is nicely hidden away inside. Was going to use 2 sticks mirrored, but ideally keep them inside the case.

 

[Ericloewe]

Parts ordered :)

Just need to read up on the various threads for updating the BIOS and LSI2308.

What do people do for their USB boot drives? The N54L has one internal USB port, so the stick is nicely hidden away inside. Was going to use 2 sticks mirrored, but ideally keep them inside the case.

I use the existing Type A USB 3.0 port plus an adapter that converts the USB 3.0 header into two internal Type A ports. Note that only one of the header's ports is actually connected to anything, since the real port is routed to the Type A socket.