Working on a server build, just some questions

[LaxSlash1993]

Hi all,

I'm working on a new project.

Very long story short, I'm looking to design a FreeNAS server capable of being both a file server and a virtualization server. This is my basic list of what I've come up with to start:

- MBD-X10DRL-I ATX
- 32GB Modules of DDR4 RDIMM RAM
- Intel E5-2620 6-Core Processor (Starting off with just one, will add the second one later on)
- 4U Rack Case
- Crucial M4 2.5" 256GB SATA III MLC Internal SSD (For hosting the FreeNAS Operating system... or would just using a flash drive be alright?)
- 4TB WD Red for NAS Data Storage
- 2TB WD Black or WD Red for VM Files

The purpose of the server would be:
- Host OS would be used for both NAS and for Virtualization.
- One VM for hosting a WordPress website and a forum
- One VM for hosting a mail server
- One VM for hosting a server for the database for the website
- One VM for hosting a database server for other services
- One VM for hosting a custom built application
- One VM for hosting a MailScanner server
- One VM for hosting various control panels

And as requirements come up, I'd like head room to setup multiple other servers as well.

Anyways... here are my questions.

- How much CPU should I leave dedicated to FreeNAS? Would I be better off just getting the second processor?
- Going off of that... should I just go a step further and get a quad socket?
- Using the above specifications, I should be able to get a total of 16 VMs to run, assuming leaving two cores dedicated to FreeNAS. Does this sound about right?
- I read somewhere last night while researching all of this (and it figures I lost my source because bookmarking is apparently overrated according to myself) that each VM should only have one core per socket. Is this accurate?
- For the VM Files drive. Should I go with WD Black or WD Red?

Curious for thoughts! Some recommendations would be nice as well if possible. I'd like to order at least some parts asap for the Newegg and Amazon sales. As far as my budget... I realize that I won't be doing this for <$1000, but if I can stay with a ceiling of maybe $1500 to start, that'd be ideal. My rough estimate for the above is $1200.00 USD.

Thanks for any input!

 

[Spearfoot]

Hi all,

I'm working on a new project.

Very long story short, I'm looking to design a FreeNAS server capable of being both a file server and a virtualization server. This is my basic list of what I've come up with to start:

- MBD-X10DRL-I ATX
- 32GB Modules of DDR4 RDIMM RAM
- Intel E5-2620 6-Core Processor (Starting off with just one, will add the second one later on)
- 4U Rack Case
- Crucial M4 2.5" 256GB SATA III MLC Internal SSD (For hosting the FreeNAS Operating system... or would just using a flash drive be alright?)
- 4TB WD Red for NAS Data Storage
- 2TB WD Black or WD Red for VM Files

The purpose of the server would be:
- Host OS would be used for both NAS and for Virtualization.
- One VM for hosting a WordPress website and a forum
- One VM for hosting a mail server
- One VM for hosting a server for the database for the website
- One VM for hosting a database server for other services
- One VM for hosting a custom built application
- One VM for hosting a MailScanner server
- One VM for hosting various control panels

And as requirements come up, I'd like head room to setup multiple other servers as well.

Anyways... here are my questions.

- How much CPU should I leave dedicated to FreeNAS? Would I be better off just getting the second processor?
- Going off of that... should I just go a step further and get a quad socket?
- Using the above specifications, I should be able to get a total of 16 VMs to run, assuming leaving two cores dedicated to FreeNAS. Does this sound about right?
- I read somewhere last night while researching all of this (and it figures I lost my source because bookmarking is apparently overrated according to myself) that each VM should only have one core per socket. Is this accurate?
- For the VM Files drive. Should I go with WD Black or WD Red?

Curious for thoughts! Some recommendations would be nice as well if possible. I'd like to order at least some parts asap for the Newegg and Amazon sales. As far as my budget... I realize that I won't be doing this for <$1000, but if I can stay with a ceiling of maybe $1500 to start, that'd be ideal. My rough estimate for the above is $1200.00 USD.

Thanks for any input!

Welcome to the forums!

What do you plan to use for a hypervisor? VMware ESXi? Xen? ProxMox? Or FreeNAS itself via bhyve/jails?

All-In-One (AIO) systems built with FreeNAS running as a virtual machine (VM) on VMware ESXi are stable and well-understood; but require a pretty steep learning curve and careful design. For one thing, you need to passthrough one (or more) Host Bus Adapters (HBAs) to the FreeNAS VM so that it has direct control of the hard drives it uses - this means you can't use the 10 SATA ports on the motherboard you've selected with FreeNAS; you need to get one or more HBAs such as an LSI 9210/9211, IBM M1015, or Dell H200/H310 instead. You need to reserve/lock the memory assigned by ESXi to the FreeNAS VM. And so one... there is a wealth of information about AIO systems here on the forum and on the internet in general. A particularly good article by forum member @Benjamin Bryan is available here: "FreeNAS 9.10 on VMware ESXi 6.0 Guide".

There are other hypervisors besides ESXi, but I'm not familiar with them and they're not in common use like ESXi.

FreeNAS is a great NAS, but it's not a great virtualization platform: its virtualization support is too new and untested for real use, IMHO. Some folks here on the forum will disagree with me about this...

However you choose to proceed, 32GB won't be enough RAM. And I'm certain you won't be able to stay within a budget of $1200.00 USD, especially if you buy new equipment.

Have you considered buying used equipment on eBay? I recently purchased a system nearly identical to this one ("Supermicro 4U 846BA-R920B 2x E5-2660 128GB RAM 24x TRAYS 3x LSI 9210-8i JBOD"). I bought directly from the listing seller because I wanted a custom memory configuration.

Good luck!

 

[LaxSlash1993]

Welcome to the forums!

What do you plan to use for a hypervisor? VMware ESXi? Xen? ProxMox? Or FreeNAS itself via bhyve/jails?

All-In-One (AIO) systems built with FreeNAS running as a virtual machine (VM) on VMware ESXi are stable and well-understood; but require a pretty steep learning curve and careful design. For one thing, you need to passthrough one (or more) Host Bus Adapters (HBAs) to the FreeNAS VM so that it has direct control of the hard drives it uses - this means you can't use the 10 SATA ports on the motherboard you've selected with FreeNAS; you need to get one or more HBAs such as an LSI 9210/9211, IBM M1015, or Dell H200/H310 instead. You need to reserve/lock the memory assigned by ESXi to the FreeNAS VM. And so one... there is a wealth of information about AIO systems here on the forum and on the internet in general. A particularly good article by forum member @Benjamin Bryan is available here: "FreeNAS 9.10 on VMware ESXi 6.0 Guide".

There are other hypervisors besides ESXi, but I'm not familiar with them and they're not in common use like ESXi.

FreeNAS is a great NAS, but it's not a great virtualization platform: its virtualization support is too new and untested for real use, IMHO. Some folks here on the forum will disagree with me about this...

Thanks! The plan was to use FreeNAS itself as the host. Of course, I'm not opposed to looking into other options as well. Keeping an open mind about this all, I'll be sure to check out that article.

However you choose to proceed, 32GB won't be enough RAM. And I'm certain you won't be able to stay within a budget of $1200.00 USD, especially if you buy new equipment.

The price comes from ballparking average prices for the listed items above from sources like Google, Newegg, and Amazon. In regards to over 32GB of RAM... I can go to 64GB if needed. Plus, I plan on building this up over a period of a few months... I'm just trying to get to a good starting point.

Have you considered buying used equipment on eBay? I recently purchased a system nearly identical to this one ("Supermicro 4U 846BA-R920B 2x E5-2660 128GB RAM 24x TRAYS 3x LSI 9210-8i JBOD"). I bought directly from the listing seller because I wanted a custom memory configuration.

I did lightly consider going with a used system.

Good luck!

Thanks!

 

[tvsjr]

Unless your VMs are extremely lightly loaded, you're going to be disappointed with this configuration. You've picked a rather low-clocked 6 core chip. Think of it this way... assuming average load (such as during updates/virus scans/any sort of routine system maintenance), you've provided about 1/2 of a 2GHz core to each system, including FreeNAS.

BTW, I run something similar... and I'm doing it on a 3-node vSphere cluster (all with dual E5-2670s and 128GB RAM) plus FreeNAS on a dedicated system. My biggest limitation is memory on the vSphere nodes.

 

[Mark Fox]

Not to suggest that your server build isn't justified but if you can use containers (jails in the FreeBSD world), you could probably get away with a lot less.

I've done close to what you've described, minus the NAS functions, using containers on a pretty modest VPS. Back when I was using virtualization I would have spec'd a pretty beefy box, or three, for what you've described.

Apologies, but I've spent the last half of my career working under too-tight resource constraints. You may have more freedom.

 

[Stux]

Sounds a helluva lot like a docker install..,

 

[Mark Fox]

Sounds a helluva lot like a docker install..,

It should. ;)

Docker started as a thin wrapper for Linux LXC containers. Its a bit more now, but its still containers underneath. I've always used LXC or LXD directly, and jails, of course.

They all reduce the memory overhead compared to a VM, to what it would cost to run all applications directly on a single host directly. Our experience is that we can run 2-10 times as many "virtual" machines, depending on the memory and CPU needs of the underlying applications.

But I'm starting to sound like a new Christian.

 

[Stux]

You can run dockers quite successfully in a lightweight Linux vm on FreeNAS