First Real FreeNAS Build. Have Some Questions.

[capn783]

Hello everyone. I have a small home lab/home server environment in my house. I am kind of new to FreeNAS. I have it running on a microserver right now but as a backup server. Here are some images of what I am currently running. Basically, I am running out of space on the FreeNAS box so I decided since I am going to have to spend money anyway that I will do it right this time and migrate my DAS storage to a new FreeNAS storage server.

I am planning on purchasing a Dell R510 Gen II. Specs will be:

CPU: 2x Xeon X5675 3.06Ghz or 2x Xeon 5560 2.80Ghz (Not sure if the extra money for the 12-core is worth it/necessary)

RAM: 32 or 64GB DDR3 RAM 10600R

CONTROLLER CARD: H200 and I will have to watch a guide on how to flash it

HDDs: Going to use NAS drives. Was looking at WD Reds 3 or 4TB. Think I am going to need 10 but that’s part of one of my questions.

So this server will be my main storage array. It will be serving media for Plex and direct playback through shares. It will be hosting home folders for AD users (less than 10 users). It will also just be storing general data (application installs, drivers, music, etc. etc.) I have a few questions:

1. How does this build look? Should I spring for the higher core and RAM setup or am I overkilling it? Also, the R510 has two onboard Gbit NICS. Will that suffice or should I put in a Quad NIC card?

2. I read some of the material on this forum and went through the PowerPoint. I believe I understand everything (vdevs, zpools, etc.). I am still confused as to what would be the best setup for my use case. What would you guys recommend? Should I just run one large raid z2 or z3 array or do striped mirrored z2/z3 vdevs? Is raid z2 sufficient or should I be aiming for z3. Also, it seems like doing an SSD for ZIL might be unnecessary for what I am doing? Would you agree? Big thing here is my SA120 DAS RAID 6 array has 14.5 TB usable capacity so I need to at least have that same amount of higher.

3. With the NAS drives should I be spending the extra money to make sure they are 7200 rpm or will the 5400 rpm drives work for what I am doing?

4. I might move plex to a jail in FreeNAS. I am curious if there are others here who do this if they think it runs better from the NAS directly or from a separate machine. Currently I have a Windows Server 2016 VM running plex and other utilities (deluge, sickrage, etc.). I am wondering if I will see a performance difference if I have Plex running from the NAS directly compared to how I am doing it now?

Sorry for the long post just trying to get as much information as go into this upgrade. I appreciate your time and help.

 

[tvsjr]

First, keep in mind that an R510, and most any other rack server, is designed to live in a data center. It's going to be hot and noisy. Make sure you have somewhere to put it that it will be sufficiently cooled and won't bother you.

1. Probably overkilling it, CPU anyway, unless you're going to run a LOT of stuff. My FreeNAS box has dual E5-2670s and it's waaaaay overkill. Where memory is concerned, you can't have too much. 32 will be quite reasonable unless you're running a ton of jails. A multi-port NIC only helps you if you are connecting to a switch that supports link aggregation, and then you only get 1Gbps to a single client. If you aren't serving dozens of clients, you won't see any real benefit.

2. Striped mirrors and RAIDZ are orthogonal. Striped mirrors involve striping the data and then mirroring it, which is the recommended configuration for high-performance needs and VM datastores. RAIDZ is parity. If you're just storing files, RAIDZ is the right answer. As far as the configuration, you have to determine what your risk tolerance is. I assume you're doing a 12-bay R510 chassis. Assuming you want room for future expansion, I would actually consider doing 6 6TB drives in RAIDZ2. This gives you 17.18TiB usable (filled to 80% capacity) and you can add another 6-drive vdev down the road to double your capacity. You will have a ZIL by definition, but it'll be on the pool. A SLOG (moving the ZIL to a separate device) is overkill.

3. For the purpose of file storage, the 5400/5900RPM drives will do just fine and run cooler. Faster drives become useful in high-performance/VM applications.

4. This is a religious debate. Lots of people are running all you list and more directly on their FN box. FN10 will bring even better support for virtual machines. Personally, I have a 3-node vSphere cluster where all my VMs live... the FN box just runs FN. But that adds a huge layer of additional complexity.

 

[capn783]

First, keep in mind that an R510, and most any other rack server, is designed to live in a data center. It's going to be hot and noisy. Make sure you have somewhere to put it that it will be sufficiently cooled and won't bother you.

1. Probably overkilling it, CPU anyway, unless you're going to run a LOT of stuff. My FreeNAS box has dual E5-2670s and it's waaaaay overkill. Where memory is concerned, you can't have too much. 32 will be quite reasonable unless you're running a ton of jails. A multi-port NIC only helps you if you are connecting to a switch that supports link aggregation, and then you only get 1Gbps to a single client. If you aren't serving dozens of clients, you won't see any real benefit.

2. Striped mirrors and RAIDZ are orthogonal. Striped mirrors involve striping the data and then mirroring it, which is the recommended configuration for high-performance needs and VM datastores. RAIDZ is parity. If you're just storing files, RAIDZ is the right answer. As far as the configuration, you have to determine what your risk tolerance is. I assume you're doing a 12-bay R510 chassis. Assuming you want room for future expansion, I would actually consider doing 6 6TB drives in RAIDZ2. This gives you 17.18TiB usable (filled to 80% capacity) and you can add another 6-drive vdev down the road to double your capacity. You will have a ZIL by definition, but it'll be on the pool. A SLOG (moving the ZIL to a separate device) is overkill.

3. For the purpose of file storage, the 5400/5900RPM drives will do just fine and run cooler. Faster drives become useful in high-performance/VM applications.

4. This is a religious debate. Lots of people are running all you list and more directly on their FN box. FN10 will bring even better support for virtual machines. Personally, I have a 3-node vSphere cluster where all my VMs live... the FN box just runs FN. But that adds a huge layer of additional complexity.

Thank you for your reply tvsjr. I do have my rack in the basement in a cooled room (Central Air). I doubt this will be as noisy as my SA120 which I am removing anyway. God that thing was like a jet turbine spinning up when I first turned it on lol.

1. So for the CPU maybe I will run 2x L5640 (this is still a 12 core setup but the processor speed is a little lower 2.26Ghz. I actually have the same chip in my DL380 G6). The L series chips have a lower power draw I think. I do have a switch that supports LAG (Cisco 2960S PoE+ 24 port gbit managed switch). Internally in the house I am probably only serving directly less than 5 clients but on my plex server I probably have 8-10 users that have access to my libraries from remote. Would the plex server necessitate upgrading to a quad nic or should I just try the two first and scale up if I need it?

2. I understand now. I think I stated wrong what I was looking to do anyway. I should have said two 5 or 6 drive vdevs in one zpool. I guess my question here is the same as the last question. If I am running plex for like 8 - 10 users (definitely not all streaming at the same time and if that happens its wildly rare) would that necessitate the high performance setup or not? I will look into the 6TB NAS drives as well. I definitely wont have FreeNAS running as a massive VM system. That is what my Windows Server 2016 (DL 380) host is for.

3. Same question as the other two. Does my plex setup put in that high performance bracket or will the lower rpms suffice for my use case (sounds like it will I just forgot to give the specifics of my plex use in the original post).

4. My setup will be kind of similar to yours. I don't have a cluster at the moment (although I would like to add another server to do that down the road) but my Windows Server 2016 host is my main VM server. I am running only 3 right now but I plan to add some more (next one I think I am doing is another DC running ADCS so I can play around with certificates and setup openvpn in pfSense). My only question here is what would be the best option to serve the data to my plex VM. Is ISCSI the best for performance or just straight up widows CIFS shares? Or is there a better option I am not seeing?

Thank you again for your time and help tvsjr.

 

[tvsjr]

The lower power chips are a false economy. They just cap the maximum performance. If the CPU isn't being run wide-open, it throttles back anyway. So, I wouldn't worry about this.

As for your networking configuration, there's nothing that would necessitate quad NICs or aggregation of any time. Think about it... how fast is your Internet connection? It's probably not faster than a gig, so that's your limiting factor.

You should be just fine running RAIDZ2. Plex will be long linear reads with minimal/no writes, an ideal use case for parity RAID.

Higher speed drives give you better IOPS and slightly lower latency. Neither is critical for an application like Plex, unless you start serving hundreds of users.

If you're just hosting the video content, serving that up via CIFS will work fine. iSCSI/NFS comes into play primarily when you run the entire VM from FreeNAS storage. In my case, my vSphere hosts don't have local storage (beyond a pair of small 40GB SSDs for the hypervisor) and *everything* comes from FN. This necessitates the striped mirrors, SLOG, etc. configuration.

 

[capn783]

The lower power chips are a false economy. They just cap the maximum performance. If the CPU isn't being run wide-open, it throttles back anyway. So, I wouldn't worry about this.

As for your networking configuration, there's nothing that would necessitate quad NICs or aggregation of any time. Think about it... how fast is your Internet connection? It's probably not faster than a gig, so that's your limiting factor.

You should be just fine running RAIDZ2. Plex will be long linear reads with minimal/no writes, an ideal use case for parity RAID.

Higher speed drives give you better IOPS and slightly lower latency. Neither is critical for an application like Plex, unless you start serving hundreds of users.

If you're just hosting the video content, serving that up via CIFS will work fine. iSCSI/NFS comes into play primarily when you run the entire VM from FreeNAS storage. In my case, my vSphere hosts don't have local storage (beyond a pair of small 40GB SSDs for the hypervisor) and *everything* comes from FN. This necessitates the striped mirrors, SLOG, etc. configuration.

Ok thank you again tvsjr. I am going to message the seller on ebay today. Its like a build your own 510 thing but he doesnt have the h200 as raid card that i can pick only the h700 which is useless to me so I am going to ask him if he can custom build it and put an H200 in it for me. Thanks again for all your help.