BUILD 32 threads for cheap

[lukyjay]

This post is long, so I have put my questions in bold for those who don't want to read my dribble.

Hello, I am new to this community and have not used FreeNAS before. I have been searching through these forums and have found some great articles and information. (Don't use non-ECC, don't use "gaming" hardware, don't have less than 8gb RAM, don't be a tool, etc.)

I have experience with Linux, Open Media Vault (FreeNAS looks better and more powerful) and currently run a home server on an Intel i5 Nuc with headless ubuntu. Considering the Nuc doesn't have sata ports, I have turned it in to a NAS by connecting a USB raid controller and have 4 drives hooked up. As I write this, my girlfriend is complaining about how Plex isn't working because the raid controller has crashed again.

I want to upgrade to FreeNAS and I know that if I attempt to do this on a Nuc, two things will happen:

1. The USB 3 ports probably wont work on BSD
2. The man with the green frog avatar will (rightfully) tell me off because it's waaayyy under spec

At the moment I am low on savings but have a steady income. What I want to do is buy the basics for the time being and try use some of what I already have, then add more RAM and harddrive space in the coming months. I still want a powerful machine though and in the mid-near future intend to have 32gb of RAM, if that's enough to run between 10-20tb?

So after reading this forum, Reddit and the general internet, I have a few questions about ZFS and would appreciate some guidance on a motherboard.

The primary uses of this server are:

• Plex (Being able to support 4k content would be nice, though not critical)
• Usenet + Torrent + Sonarr + Couch Potato
• Virtual Machine host (If FreeNAS has a plugin for this.. I'm not sure if it does, but if it does, I'll be hosting a windows OS to run some CCTV cameras off)
• And possibly other things in the future.

I am very interested in the e5-2670 CPUs because they're very good bang for buck, and are Xeons - so I know you guys should adore them. I am not sure what motherboard to use to house two e5-2670 in. I have seen a lot of dual socket boards but they're in the $500~ range when looking at 10~ SATA ports, and I'm worried I'll get the wrong one.

Is there any good value motherboards that this community can recommend, which will support ECC RAM, 2x e5-2670, 10~ sata ports and have some pci-e ports?

In regards to storage, I'm going to carry over my current hard drives (4x 2tb). With ZFS and RaidZ (I think that's the most popular option).

Can I add new hard-drives to ZFS at any time? Do the new harddrives have to be the same size as the old? Is it possible to remove and upgrade existing drives (remove 4x 2tb drives and insert 4x 4tb drives)?

I have an existing 750w PSU, but I'm not sure if I'll need a special one to power a dual socket CPU motherboard? It's a gaming PSU, but I don't think it will matter as much, so long as it has enough plugs, right?

I intend to run this machine in a tower, because racks appear expensive and noisy (I'm in Australia, so it's like $100 shipping on top of the chassis itself). Plus I already have a tower chassis, and maybe I'll upgrade to racks when I have more savings to do so.

I am sorry that I've written so much, I'm very excited to learn about FreeNAS servers and look forward to hearing everyone's recommendations and thoughts on my post. Thank you for reading.

Kind regards,
Jay

 

[Mirfster]

The man with the green frog avatar will (rightfully) tell me off because it's waaayyy under spec

I think you have mistaken the Grinch for a frog.. which is already going to be a strike against you... :p

I still want a powerful machine though and in the mid-near future intend to have 32gb of RAM, if that's enough to run between 10-20tb?

Should be good for a start, but you need AT LEAST 8 GB of RAM just for FreeNas. So don't even try to run with less. Since you mentioned a lot of other things, I would say you want 16 GB minimal to begin with.

Virtual Machine host (If FreeNAS has a plugin for this.. I'm not sure if it does, but if it does, I'll be hosting a windows OS to run some CCTV cameras off)

You may want to ensure your system is capable of handling more RAM to future proof it...

In regards to storage, I'm going to carry over my current hard drives (4x 2tb). With ZFS and RaidZ (I think that's the most popular option).

What RaidZ are you considering? RaidZ2?
RaidZ1 is rarely recommended, especially if you have drives larger than 1 TB. For me (and probably most others), RaidZ1 is dead and not worth ever considering..

Can I add new hard-drives to ZFS at any time? Do the new harddrives have to be the same size as the old? Is it possible to remove and upgrade existing drives (remove 4x 2tb drives and insert 4x 4tb drives)?

Have you read cyberjock's: Slideshow explaining VDev, zpool, ZIL and L2ARC for noobs! ?
As far as hardware, I am a "Rack Mount" person so I will leave that to others. But, would suggest that you not count it out entirely.

It is good that you are taking time to "think this through" since to me the most important thing is to have a solid foundation to build from. There are several links in my sig that will really help you get a good understanding. If you have the time, I would suggest you go through them.

 

[Dice]

Is there any good value motherboards that this community can recommend, which will support ECC RAM, 2x e5-2670, 10~ sata ports and have some pci-e ports?

Many guys around here are fans of SuperMicro. So am I after getting my first.
For dual slots, on LGA2011, there are a few options.

One aspect to keep in mind is the distinction between workstation boards and server boards. You could spot which one is which by looking at parameters as 'sound card' (typically on workstation) and 'IPMI' (never on workstation models, often on server models. It is absolutely lovely to have).
Beyond that it'll be neat to 'future proof' the rig by aiming for models with high count in ram slots.

At the end, availability decides what to get.

Cheers /

 

[joeschmuck]

Yes, FreeNAS does support VMs but it should do it much better in FreeNAS 10 which is still several months from being released, but everything else should be fine. As for CPUs, I'd stick with a single CPU solution unless you really think you need the extra horsepower, which isn't a bad option to have in your back pocket if you have the money. RAM, buy a board which can support at least 64GB RAM but more is of course better, but I'd only start with 32GB RAM unless again, you have lots of money and then you could fill it up.

Don't waste any money on a ZIL or SLOG, even if tempted. Based on your requirements that is not needed and I'd fill up the RAM to maximum capacity before you get into that kind of hardware.

As for Supermicro boards, you might want to stay away from the X11 boards since they are the latest and greatest and as I can attest to, it wasn't a smooth ride as I needed firmware updates and some tweaking. Being on the cutting edge is not always fun. And X9 or X10 board would do fine and there are many to choose from, just start looking and find the one you think has all the options you desire. When looking you may find out that you now desire 10Gb NIC connectivity, well that is an option as well and is usually cheaper than an add-on card, the down side is your connections on your network need to be 10Gb also so it's not a cheap upgrade, but just something to think about. Once you have made a choice on components you can add a post to ask if they look reasonable and you will get answers. Don't go spending money on untested hardware unless you're a risk taker like myself.

Read the link that was provided above to Cyberjock's guide, but the short answer is no, you can't just add a new hard drive to expand your pool (storage).

Your PSU may be fine but it depends on if its a good quality PSU. Many are over rated and don't provide good power when under heavy load. If you pop in 10 hard drives, well it could cause issues. And I'm not saying it will either but you didn't specify the model of the PSU. But these things are usually rooted out during the burn-in testing.

Just a personal note... If the above requirements are all you are looking for, a single CPU solution with 32GB RAM should be fine. The troubles come in when you start talking about adding VMs as you need to plan resources for that.

 

[Jailer]

If you're going to get a supermicro board you'll need an X9 series board. The X10 series supports the 2011v3 CPU's and not the E5-2670 you are looking at.

A dual CPU system is going to draw a lot of power. Are you sure you need 2 CPU's?

 

[pirateghost]

If you're going to get a supermicro board you'll need an X9 series board. The X10 series supports the 2011v3 CPU's and not the E5-2670 you are looking at.

A dual CPU system is going to draw a lot of power. Are you sure you need 2 CPU's?

??

My X10 is socket 1150.

 

[Dice]

??

My X10 is socket 1150.

This is a tad confusing I agree.
E3 XEON most recent socket are 1151, X11.
The previous E3 XEON socket was 1150 hosting the X10 boards.

Yet, the "corresponding" E5 xeons are one step behind it seems.
The E5's are on LGA2011 (x9, x8 too?) or LGA2011-r3 for the latest version where X10 is the name of the series. There is no X11 on LGA2011-r as of now.

 

[Jailer]

??

My X10 is socket 1150.

http://www.supermicro.com/products/Product_Naming_Convention/Naming_MBD_Intel_UP.cfm

 

[pirateghost]

http://www.supermicro.com/products/Product_Naming_Convention/Naming_MBD_Intel_UP.cfm

That doesn't help me at all.

http://www.supermicro.com/products/motherboard/Xeon/C220/X10SL7-F.cfm

 

[Jailer]

X10 = 10th gen. Xeon®
S = Single
L = Lynx Point (C222)
7 = LSI SAS 6Gbps

 

[pirateghost]

X10 = 10th gen. Xeon®
S = Single
L = Lynx Point (C222)
7 = LSI SAS 6Gbps

Yeah.

I figured that out just fine.

My comment was originally that you said they would want an X10 board and I didn't know they made x10 boards in socket 2011. When I bought my board, I never saw any 2011 socket X10 boards. I had no idea they used the same lines to handle different sockets. It's a stupid ass naming scheme.

Your comment originally sounded to me as if all X10 boards were socket 2011, when that is not the case.

 

[Jailer]

Yeah.

I figured that out just fine.

My comment was originally that you said they would want an X10 board and I didn't know they made x10 boards in socket 2011. When I bought my board, I never saw any 2011 socket X10 boards. I had no idea they used the same lines to handle different sockets. It's a stupid ass naming scheme.

Your comment originally sounded to me as if all X10 boards were socket 2011, when that is not the case.

Yeah I wasn't real clear on my response. I was referring to X10 vs X9 in just the socket 2011 boards in my response.

 

[pirateghost]

Yeah I wasn't real clear on my response. I was referring to X10 vs X9 in just the socket 2011 boards in my response.

Yeah. I was confused. I thought X11 = 2011 and X10 was 1150 only. I see they use the same moniker across many different sockets. It really is a stupid ass naming scheme.

 

[Jailer]

I agree, it's a bit confusing. Notice the prolific use of "or" in all of their product naming guides. The same naming convention is even used on their embedded product line.

I'm sure @Ericloewe will be along shortly with his Supermicro secret decoder ring to make sense of it for the rest of us mere mortals.

 

[pirateghost]

Ugh

 

[Mlovelace]

Yeah. I was confused. I thought X11 = 2011 and X10 was 1150 only. I see they use the same moniker across many different sockets. It really is a stupid ass naming scheme.

X11 is for the LGA1151 socket.

 

[pirateghost]

X11 is for the LGA1151 socket.

Oh jeez

 

[Jailer]

X11 is for the LGA1151 socket.

Not necessarily: http://www.supermicro.com/products/motherboard/X11/X11SBA-LN4F.cfm

Key Features

1. Intel® Pentium® Processor N3700,
Socket FCBGA 1170; CPU 6W

 

[Robert Trevellyan]

Can I add new hard-drives to ZFS at any time?

Not in the way many people expect. After you read the ZFS introduction slideshow, you'll understand that you can't grow a vdev by adding a drive. What you can do is grow a pool by adding a vdev.

Do the new harddrives have to be the same size as the old?

No, but unbalanced vdevs are considered sub-optimal.

Is it possible to remove and upgrade existing drives (remove 4x 2tb drives and insert 4x 4tb drives)?

Yes, this is how you grow storage when drive bays and/or ports are limited: http://doc.freenas.org/9.3/freenas_storage.html#replacing-drives-to-grow-a-zfs-pool

 

[lukyjay]

I think I unintentionally started a debate on CPU comparability. Sorry for that!

I'm still really confused to what motherboard from SuperMicro I should look at. Lots of people recommend a single socket, but I'm hoping that with dual socket I will have more luck with 4k content on Plex. Also the CCTV cameras in VM (when FreeNAS 10 comes out) utilise a huge amount of CPU power. My i5 was maxed on all cores with 5 cameras running due to the motion detection. The software they use (Blue Iris) isn't very efficient. For these reasons, I think the dual socket CPU will be important to me down the track.

I found one motherboard I like, but it's a little on the expensive side: http://www.supermicro.com/products/motherboard/xeon/c600/x9drh-if.cfm
Anything similar for $100~ cheaper?

I found the model number naming guide very useful, and thank you for pointing out I need an X9 motherboard, I was looking at and planning to buy X10, which would've been a disaster considering the money I would have wasted. :(

Not in the way many people expect. After you read the ZFS introduction slideshow, you'll understand that you can't grow a vdev by adding a drive. What you can do is grow a pool by adding a vdev.

But I can replace the drives in a specific udev, as long as they are all the same, right? I'll start with 2tb drives for now, and look in to 6tb drives in the future. Will be sufficient for the short term, and flexible for the long

Just a personal note... If the above requirements are all you are looking for, a single CPU solution with 32GB RAM should be fine. The troubles come in when you start talking about adding VMs as you need to plan resources for that.

Yeah I plan to go for the dual cpu as it will only be around $200-300 more, and hopefully will aid me in 4k content on Plex in the future.

I'm unsure of the PSU model, but it's a corsair 750w. It has an orange sticker, so I believe tx750w may be the model. I know Corsair is a really good brand, so here's to hoping there are no problems

Should be good for a start, but you need AT LEAST 8 GB of RAM just for FreeNas. So don't even try to run with less. Since you mentioned a lot of other things, I would say you want 16 GB minimal to begin with.

You may want to ensure your system is capable of handling more RAM to future proof it...

I was going to start with 1x 16gb dimm, or 2x 8gb dimms (what ever is cheaper) and add another down the track. With using a server board, it shouldn't be too difficult to reach 512gb support - not that I would ever go that far.

What RaidZ are you considering? RaidZ2?

Raidz2 should fit my needs and allow large capacities.

Have you read cyberjock's: Slideshow explaining VDev, zpool, ZIL and L2ARC for noobs! ?

Have now :D I find the wording confusing, but this helps a lot. I don't feel that I require a ZIL or L2ARC. Also it helps me understand that I can replace harddrives with larger ones. I can expand, but I cannot shrink. Very useful slideshow

As far as hardware, I am a "Rack Mount" person so I will leave that to others. But, would suggest that you not count it out entirely.

If I were to go down the rack mount route, I'd be selecting parts that fit in a tower anyways, such as an E-ATX or ATX motherboard, so the only thing that would change for me is the chassis I put it in :)