BUILD Building a server for FreeNas primary for Plex

[AltecBX]

Hi guy’s and girls I’m new to the community and looking to build my first time Freenas server. I’m planning on using Freenas as my OS of choice. My primary service is going to be the Plex service. I’m going to anticipate about 8-10 concurrent sessions connecting to the Plex server remotely. And 3-5 concurrent sessions connecting locally on the LAN. So about 15 in total when in maximum use. I know the server will have to transcode most of these connections as well. Another service of the server will be syncing data to a remote server as well. It will also shoot and receive files remotely and locally. It will be backing up 4 machines locally as well. Here’s the list of things I’m anticipating what the server will do at its full capacity.

- Plex service (15 concurrent session transcoding)
- Syncing data to a remote server offsite
- Backing up 4 local machines (2 pc & 2 macs)
- Retrieving and sending data to 4 machines and 3 wireless devices (Map drives & iSCSI connection)
- Host remote files
- Host website

So now that you know primary what I’m using the server for, here is the hardware list I’m planning on purchasing.

Processor - Intel Core i7-5820K Haswell-E 6-Core 3.3GHz LGA 2011-v3
Motherboard -ASRock X99 Extreme4 LGA 2011-v3 Intel X99
Memory - ADATA XPG Z1 32GB (4 x 8GB) 288-Pin DDR4 SDRAM DDR4 2133
Main Drive - Plextor M6e PX-G128M6e M.2 2280 128GB PCI-Express 2.0 x2
Case - LIAN LI PC-A75 Black Aluminum ATX Full Tower Computer Case
Power supply – Thermaltake SMART Series 850-Watt
Hard drives – 10 WD Red WD60EFRX (6TB each)

My reason with going with the MB is because it has 10 native 6Gbs sata ports.
My reason with going with the processor is because it supports the x99 chipset.
My reason with going with the M.2 drive is because it has a faster bandwidth than the sata ports
My reason with going with the 10 WD Reds 6TB is because I’m planning on using the ZFS file system and doing a 9+1 setup. 9 drives as storage & 1 drive as parity.

Questions:
Is the list of items enough for what I’m doing?
Do I need ECC memory instead?
Do I need more than 32GB of memory for ZFS with this setup?
What are your suggestions in hardware?
What is the best file system structure to do with this hardware?
Does the main drive speed and port matter for the Freenas OS?

Thanks for your inputs in advance!

 

[cyberjock]

Let me be short and sweet... You should spend more time reading and researching. Clearly you didn't read our stickies or are choosing to ignore them. Do you think we're going to change our mind? No, expect responses to be straight off the stickies.

I'm not sure which is worse... that you may have read the stickies and think that if you list hardware that isn't on the sticky we'll agree or the fact that you may not have even read our stickies before posting.

 

[mjws00]

Heh. I'll give you the gentle version.

That gear is nice. But it isn't well suited to FreeNAS. Specifically, you want something that supports ECC RAM. CJ's sig has an awesome hardware write-up. There are no DDR4 builds on the list. The e3 v3 xeons don't take it. The e5 v3's are still a little premium so not widely in use round here.

z1 on a 60TB single vdev is probably suicide. Again, see CJ's sig. You don't need a "main drive" A usb stick is fine, the system runs directly from RAM.

z2 on 10 6tb REDs should rock your world. You are pushing the memory limits with all the concurrent users and services... so you might have to jump to something that can support >32GB. This is going to be very dependent on actual use. Unfortunately you have no upgrade path if you're wrong and pick the e3. I'd probably look at the X9 Supermicros. Or suck it up and blaze a trail on the 2011-3 boards. You do want lots of clock speed for transcoding, and it will eat cores as well... so ante up ;)

Good luck. There is a pretty solid learning curve ahead.

 

[SweetAndLow]

My max local simultaneous transcodes on my xeon E3-1230v3 is 7 at max settings using 1080p is 7. Anything more than that started giving me shuttering. If you are going to have a combination of remote and local you could maybe squeeze a little more out of it but not much. So you are going to need something more powerful than that to get ~15. I think you are in the E5 territory so you can jump to 64Gig of ram.

 

[diedrichg]

That's going to be a hot machine too, you will need plenty of airflow and should consider a closed-loop CPU cooler that has a 240mm radiator such as the Corsair H100i that gets top billing at Maximum PC.

 

[Ericloewe]

I won't even start questioning the need for 15 streams (though I can imagine...).

If a Xeon E3-1230V3 can do 7, a high-end Xeon E5 should have no trouble doing 15. Expect to pay insane amount of cash, though.

 

[AltecBX]

Cyberjock great article on ECC. I just read the first page and your right. I can't imagine not using ECC memory on a ZFS file system. Thanks for the heads up, and sorry I didn't read the stickies prior to posting. I will do as much reading as possible prior to posting any questions.

Guys right now I'm mostly serving 2-5 connections on my main Windows 8 machine. Usually 2 remotely and 2 locally 95% of the time. I usually have no trouble serving them as I have an Intel 6-core with 32GB DDR3. For my Freenas setup, my pool of connection will go up as I will allow more remote users to connect. But I'm anticipating to still have about 5 concurrent connections 95% of the time. I'm anticipating to have more like 8-10 sat nights. I mention 15 connections is because thats the most connections I can see myself allowing in the next few years. So this is the maximum amount of users that will have access to the server through Plex but will I never think they will connect all at the same time.

I'll will continue reading this forum here everyday. This is all very interesting. But feel free to drop any suggestions as to what kind of hardware most are running with a similar setup in mind. I think the E3 is great for now but don't think it will hold up for my future goals.

 

[diedrichg]

If you are just looking to strictly run a Plex server, why not just stick with your current setup and add some more RAM and maybe other upgrades such as some monster storage disks since Plex can point to multiple sources. It's not like your movie and TV collection needs to be redundantly backed up, right? Then you can spend money on the other components that are going to bring your streams to a crawl such as switches, routers and your ISP speed.

 

[AltecBX]

That's the problem. My Windows 8.1 system has the current hardware: samsung 840 1gb ssd, 32GB DDR 2400 memory, Intel 6-core processor, Asus Rampage V extreme, and five 4TB drives for data.
I have 1 Netgear Nas 4 drive backup with 4TB in each bay. I also have a Drobo 5N with five 4TB in each bay. I'm ran out of space on the Drobo as the red light it's on in front of the case (40GB left). The Netgear has 350GB left. So I decided that I'm going to stop buying individual nas devices and just built one central location. Also, since I use this main pc for trading, browsing internet, print server, file sharing, and torrent download; My machine hangs when I download any torrent over 5GB. No matter if I have uTorrent pointing to the 4TB drives or to the SSD, it still manages to crash on torrentz over 5GB. So I get users calling me that the Plex is down, and I have to wait to get home and restart the machine. Sometimes I'm away for a few days and it's a problem for me as well as I can't remote in for my files and have to rely on Google Drive for backup. My Internet connection is verizon fios 300 down / 75 up.

 

[diedrichg]

Wow, that's allot of space requirements.

I use Teamviewer to restart Plex on my HTPC.

300 down and 75 up?! I hope your "clients" are paying you.

 

[mjws00]

I'll be just a tiny bit evil as my setup is similar. I'd be looking seriously at an ESXI server if I were you. Letting remote users, torrents, and plex pollute a trading box, or any other production workstation to the extent that it crashes is nasty. Break that crap out to small purpose based VM's and fire them up as you need them. Now one small glitch doesn't take out the whole thing and you have centralized management on one excellent hardware platform that can scale.

Unfortunately that complicates the game, and may not be suitable for your skill sets. But honestly if you add another 10 x 6TB and you are around 120TB of raw space which is a significant amount of data to manage. While FreeNAS can "manage" those other services it isn't as elegant as running the "servers" on the platforms they are primarily designed for. It's a different game if you are a BSD wizard... but I saw a windows box and a bunch of plug and play nas's.

The hook with FreeNAS is ZFS. But your requirements for the system as a whole extend well beyond only data integrity. In fact SAFE is probably no where near the primary goal. You need a level of fault tolerance, something flexible, and stable. So I'd look at the overall design and see where you land. Don't even think about the e3, imho. They max out instantly (RAM) and there is a high probability you will want 'more'.

The EVIL bit is simply because I'd have you think about a solution that requires more expertise to execute perfectly. You can certainly build a big FreeNAS box and make it work as intended. But your significant investment may be able to serve you better.

It's two bits. With great power comes great responsibility. So be careful.

 

[borisnet]

15 concurrent session transcoding ? Is that for some sort of training session? Do you know roughly the bitrate expected by the 15 plex clients and the format of your 'source' files?

 

[Ericloewe]

Is that for some sort of training session?

I think the less we know, the better.

 

[borisnet]

I think the less we know, the better.

I want to believe :-D Oops wrong forum !

 

[AltecBX]

First of all I can't teamview to my pc since it crashes when downloading torrents over 5gb so that's not a solution to reboot my machine after a crash.

Right now I'm mostly serving 2-5 connections on my main Windows 8.1 machine. Usually 2 remotely and 2 locally 95% of the time. I'm anticipating to still have about 5 concurrent connections 95% of the time. After realizing my real scenario for 2015, I'm anticipating to have more like 10 max users, not 15. Mostly 5-6 users sat nights which are the days I usually see remote users connecting at the same time. So (10) is the maximum amount of users that will have access to the server through Plex but will I never think they will connect all at the same time.

Right now I stream 2 internal streams at 8Mbs and 2-3 remote at 3Mbs. I want to ante up that to 2 internal at still 8Mbs, and 5-6 remote at 3Mbs.
90% of media is 720P while 8% is 1080P and the rest is usually a mix of 480i/P. It will also start growing to 4k contents as soon as I built the new array.

As for ESXI, why would you recommend that for me? Are you saying I should run a virtual machine on my current hardware setup?

I'm getting really tempted to just go and buy a QNAP TS-853 pro and call it a day. I hear it can transcode 5 4K contents at one time without a problem for the Plex users. So I'm thinking it can server 8-10 users at 720P without an issue as well. The only reason why I haven't done that is because the QNAP has a quad core Celeron 2.0GHz with 8GB of DDR3 at 1333 for $about $1,200. I think I can built a much powerful machine with FreeNas with better hardware for the same or less money. This is why I'm asking for advice on users on this forums who has had success or knowledge on a similar built and purpose. SweetAndLow mentioned he had 7 streams at 1080P on his e3 rig without an issue. I'm not that far away.

 

[mjws00]

It sounds like you lean strongly towards plug and play. In that case I wouldn't recommend anything to do with esxi. Possibly not even FreeNAS.

However, if you have nice server hardware it can give you many great options. I'd break out the plex server on a windows vm. uTorrent, sickbeard, couchpotato, vpn... on another. Dedicated lean and mean secure trading station on another. Those things are much nicer under windows/linux. Then you can allocate resources to each as desired. When something needs a reboot nothing is affected, and your primary workstation is never affected. It is kind of a different mindset, but once you get used to firing up a VM in a few seconds, you can have any platform and chunk of software modularized.

Generally we don't advocate FreeNAS as a VM here. That said, I believe a competent adult can execute on jgrecos vt-d strategies with acceptable risk. I mentioned skill sets early on, because many attracted to this kind of powerful software LOVE to get their hands dirty. Many also bring additional IT related skills or passion to the table.

The quick and dirty option is add a maxed out Haswell e3. Use it AND your current workstation for plex when it hits max load. That puts you in the same ballpark as the QNAP with far better hardware. Simple. Nearly plug and play.

 

[bestboy]

I'm a bit skeptical about the QNAP TS-853 Pro. It has a quadcore Celeron and says it can do hardware accelerated transcoding.
Well, but what does that mean and how does that hardware transcoding work? Is there a separate QNAP video streaming service that can make use of the hardware? I might be wrong, but AFAIK Plex does not use hardware acceleration. It does all its transcoding on the CPU.
So if Plex is a must, then I don't see how the Celeron can serve 5 users with software transcoding. If an E3 1230 v3 can handle 7 users, then what is to expect from the wimpy Celeron that does not even have SSE? 2? maybe 3 users? (comparison)

 

[cyberjock]

I'm a bit skeptical about the QNAP TS-853 Pro. It has a quadcore Celeron and says it can do hardware accelerated transcoding.
Well, but what does that mean and how does that hardware transcoding work? Is there a separate QNAP video streaming service that can make use of the hardware? I might be wrong, but AFAIK Plex does not use hardware acceleration. It does all its transcoding on the CPU.
So if Plex is a must, then I don't see how the Celeron can serve 5 users with software transcoding. If an E3 1230 v3 can handle 7 users, then what is to expect from the wimpy Celeron that does not even have SSE? 2? maybe 3 users? (comparison)

I was wondering when someone was going to figure that out. I wasn't even going to respond because it should have been very obvious to anyone that's read up on Plex transcoding that a Celeron has between 0% and 0% chance of transcoding 7 streams. To be honest, I'm not even sure if my E3-1230v2 could transcode 7 1080p streams simultaneously.

 

[bestboy]

I want to ante up that to 2 internal at still 8Mbs, and 5-6 remote at 3Mbs

About those internal streams: Do they really have to be transcoded?
I'm asking, because usually there are 2 reasons for transcoding in the first place:

1. The target devices do not support the original video format. Transcoding is used to provide compatibility.
2. There are bandwidth limitations in the network that prohibit streaming of the original high bitrate content. Transcoding is used to provide throughput.

The external users need transcoding because of #2. Your upstream is limited to 75 MBit.
For internal users the need for transcoding is not so clear. #2 is usually irrelevant and the only reason for transcoding might be #1.

So, if you can serve internal viewers as direct streams, that might change the equation and leave the need for transcodings to 5-6 external viewers. And that is in the ballpark of a single E3 and can be handled with moderate hardware.

 

[AltecBX]

This is what I'm serving now on my current machine: (6 total users usually not at the same time)
- Local (internal) (2) Roku 3 and (1) wireless device at a time, either a Fire HDX or Appe iPad Air.
- Remote (internet) users are (1) Roku 3, 1 internet based (plex.tv), 1 wireless (Phone running plex software)
I observed 4 users at one time connected to plex and my cpu usage fluctuated from 28-40%. Memory usage for the Plex was around 700MB.

This is what I'm anticipating for the following year or so. (10 total max usually not at the same time as well)
- Local (internal) will be (2) Roku 3 and (1) wireless device at a time, either a Fire HDX or Appe iPad Air.
- Remote (internet) users will be (3) Roku 3, 2 internet based (plex.tv), 2 wireless (Laptop & Phone running plex software)