Help sizing upgrade from FreeNAS mini

[EHN Admin]

Long time reader first time poster

Current environment:
FreeNAS mini:
--Memory 8 GB
--Storage:(4x3)12 TB raidz1
--Jails: Plex, NextCloud

Workstation(Linux Mint):
--Memory: 16 GB
-- CPU: 12 core (6 physical with hyperthreading)
--VMs: Windows 10, FreeBSD (w jails SVN, Web, mysql, postgres(s), wildfly(s))

Lots of windows PCs and Android devices on the network
On a 1Gb backbone and 'N' wifi (vLANs in use)

FreeNAS mini is 5 years old and full. Time to move on. I am currently thinking about buying a 2U system (12 bays), 4 cores, 32 GB memory from XI.
My desire is to double maybe triple the space, should keep me going for another 5 years.

My main goals are to...
--stream 2 HD movies at once
--Move all my FreeBSD jails off the workstation onto the server as well
Note: While I have many jails they are all very lightly used, programming a small side opensource project, they're devlopement tools and testing environments

Questions:
1) How do I size throughput of 2 HD movies
1a) Would 5400 RPM still be sufficient?
1b) Will 1 vdev work or do I need multiple?
1c) Is it true that read cache won't help streaming data (such as a movie)?
1d) Would I be in danger of saturating the 1Gb network?
2) Does a rack mountable server really need a rack to be stored in? This would be my first :)
3) Should a go for a 4U machine and leave some open bays just in case?

I don't have a lot of money to spend so I want to size it just right to get what I need (want) but not spend money on useless hardware.
Thanks for all your thoughts and comments in advance.

 

[Chris Moore]

Questions:
1) How do I size throughput of 2 HD movies
1a) Would 5400 RPM still be sufficient?
1b) Will 1 vdev work or do I need multiple?
1c) Is it true that read cache won't help streaming data (such as a movie)?
1d) Would I be in danger of saturating the 1Gb network?
2) Does a rack mountable server really need a rack to be stored in? This would be my first :)
3) Should a go for a 4U machine and leave some open bays just in case?

In no particular order...
I have one pool of 12 drives, laid out in 2 RAID-Z2 vdevs of 6 drives each and that is plenty fast to stream two movies at once, probably more. More vdevs gives you faster access to the data. I did a good bit of research before I picked my layout and part of the reason for my selection is that I could start with six drives, add six more and later add six more with the ultimate goal of 24 drives in the pool.
Streaming is more limited by the processor (depending on the source file) than it is the file access, in my experience. The thing to look for is the PassMark score of the processor.
Here is an example: https://www.cpubenchmark.net/cpu.php?cpu=Intel+Core+i3-550+@+3.20GHz
That CPU could probably only handle one HD stream, but if you were to move to a newer CPU like this one: https://www.cpubenchmark.net/cpu.php?cpu=Intel+Xeon+E3-1240+V2+@+3.40GHz
That could probably handle three streams, depending on the actual resolution and frame-rate.

Rotational speed is more important when you are talking random workloads. If you are primarily doing streaming video, that is sequential and if you have more drives in the pool, the work is distributed. I would not hesitate to use 5400 RPM drives. I like the Seagate drives personally.

1c. Yes, that is true.

1d. My 4k TV is plugged into my 1GB wired network and I don't have any network issues. I can still do other things on the network and be playing a movie.

I have rack mount servers sitting on their sides as if they are tower systems. The only concern I had was that they might tip over. I put furniture on either side of them to prevent it and it has not been a problem for me. I did replace the original fans with quieter fans. I have two now and am working on putting a 24bay unit together now that will give me a third. Some day, when I can talk my wife into it, I may get a rack and mount them in it, but for now, this is working.

I would suggest going with a 24bay (4U) second-hand system from eBay. You can't beat the value for the hardware as long as you find one that is properly equipped. If I were buying, this is the one I would get: https://www.ebay.com/itm/292252408126
I have used eBay to source all my equipment from for the past six years, except for the hard drives. I built my first three NAS systems with used hard drives and the longevity was not what I would have liked. Suffice it to say, I have lots of experience changing hard drives due to failure. My most recent systems are using all new drives and the failure rate has been much, much lower.

PS: That server needs a couple parts to make it complete, but it is a good value.

 

[EHN Admin]

Thanks for the information, I'll definitely pay attention to the cpu PassMark score.
Just remembered that my livining room TV is an old 720p, while the play room one is 1080 connected to the PS4. In the case of plex do you know if the server or the client is downsizing from 1080 to 720?
Fans won't be a problem my freenas mini is in the basement next to the switch, this new machine will take it's place.

Thanks again,

 

[Huib]

regarding passmark, this is a good guideline.
for each 720 P stream you need 1500 passmark score
and for each 1080P stream you need 2000 passmark score.
That's taken from the plex recomendations https://support.plex.tv/hc/en-us/articles/201774043-What-kind-of-CPU-do-I-need-for-my-Server-
Note that if you are going to use virtual machines, bear in mind that the cores you reserve for those will affect the "pass mark score" available for your streaming.
So if you need 2 1080P streams, and you have 4 virtual machines with each one core on a 8 core machine, you just threw away half of your pass mark score available for plex (since you have experiance in this stuff it might be redundant information but I'm trying to be somewhat complete).

Will 1 vdev work or do I need multiple?

It's not my strongest topic, but it's my understanding that vm's will require a lot of random read/writes. That will be handeled better by mirrors.
For streaming only it realy doesn't matter. I don't think you can buy drives slow enough to have a problem with read speed for streaming 2 1080P movies.

In the case of plex do you know if the server or the client is downsizing from 1080 to 720?

In case of plex it's done by the server. The same goes for transcoding if your client doesn't support seperate subtitle and video streams, or if your client doesn't support the codec used in the source file. If the client supports everything in the source file plex will "direct stream" and no transcoding is required.

Would I be in danger of saturating the 1Gb network?

Not by streaming movies. A 1080P stream would be around 10Mbps so FAR away from freaking out a 1 Gb network. Even if you for some reason have insane high quality media.

I can't speak about the rest of your network usage though. If you hammer the server with a lot of vm's or just network clients you could I guess...
Perhaps you can dig into your freenas reporting tab to see how heavily it's used and do the same for your linux machine. That should give you a good idea of how your load is right now.
My router also keeps some statistics on network trafic. Perhaps yours has a similar function?

Hope that helps

 

[Chris Moore]

In the case of plex do you know if the server or the client is downsizing from 1080 to 720?

I have my Plex installed in a Jail on my FreeNAS and it does all the transcode (resize) of the video before it is pushed out to the network. I have watched the CPU usage and I can hear the fan ramp-up if I am in the room with the server, but it is very dependent on the source video as to how much actual utilization the CPU gets. There is no need for a GPU as Plex does not use it for transcode. The built-in video chip on the Supermicro system board handles all the video you need.

 

[Huib]

I have watched the CPU usage and I can hear the fan ramp-up if I am in the room with the server

My server is standing in my living room. The best investment in my system was the huge cpu cooler that is even hardly auditable at full load. It's those little differences in usage that make a one size fits all solution imposible :)

 

[EHN Admin]

This all very helpful, thank you. In my case if Plex needs 2000 pass mark units for each 1080 stream, a total of 8k should do me fine. I can always leave the plex jail with unlimited resources, and limit the resources to the other jails, I probably can also modify the QOS settings on my switch to preference the movie streams. Don't want the kids and wife to squawk about jitter when I'm importing data into a database. Happy wife, Happy life. It'll probably be a few months before I get this all set up, but I'll try to post the configuration so others that may have a multi-purpose box like mine can learn from it.

Thanks again!

 

[Chris Moore]

I can always leave the plex jail with unlimited resources, and limit the resources to the other jails, I probably can also modify the QOS settings on my switch to preference the movie streams. Don't want the kids and wife to squawk about jitter when I'm importing data into a database.

I didn't need to make any customization, I installed the standard Plex plugin, and didn't have to make any network changes. I do have a Cisco switch and all my clients are on the wired LAN but my system runs video smooth even when I am re-silvering the array after replacing a disk. As long as the CPU has the guts and there is enough RAM, you will have no trouble. Just don't skimp to save a buck, that will bite you.

 

[Chris Moore]

The best investment in my system was the huge cpu cooler that is even hardly auditable at full load.

I am using the stock coolers that came with the CPU. I bought better ones, I just have not gone back in to change them out. I hear some whine from the Intel cooler but it is the fault of the cooler. If you are going to have the system where the sound makes a difference, quiet fans are money well spent.