Using FreeNAS only as a Plex server

[Nick2253]

So you are telling me 3 devices direct playing while 2 clients remotely are transcoding a 4k file should have my cpu usage lower then 80%?

I misunderstood. I was thinking you had that with only 3 direct streams at once.

The fact that you have five simultaneous streams, and when you get rid of a single direct stream, your stutter goes away, makes me think the issue is with your NAS. Either the IOPS are limiting you, or your bandwith is limiting you. Just because you are not maxing out your network connection does not mean that you are not maxing out your storage array.

In any case, a 1060 will only help you if the limitation is transcoding. With your CPU only at 80%, it is clear to me that your limitation is not transcoding.

 

[Chris Moore]

Now this I didnt even think of, if the remote clients are using 1080 files it doesnt even stutter when I have way more connections to the server. Just didnt want to have extra files if I didnt have to...

It might be easier to expand the storage of the existing NAS than to totally build a new solution. I was reading that the Synology you have can be expanded with external disks. I am not sure how that works, but it might be easier in the long run than building a whole new system or even than adding that 1060.

 

[Mylex]

I have the space I just am kinda OCD I guess. I do not like unnecessary dupes. I am using almost a third of it.

 

[Mylex]

I misunderstood. I was thinking you had that with only 3 direct streams at once.

The fact that you have five simultaneous streams, and when you get rid of a single direct stream, your stutter goes away, makes me think the issue is with your NAS. Either the IOPS are limiting you, or your bandwith is limiting you. Just because you are not maxing out your network connection does not mean that you are not maxing out your storage array.

In any case, a 1060 will only help you if the limitation is transcoding. With your CPU only at 80%, it is clear to me that your limitation is not transcoding.

NP, The drive in the plex server is a 3dnand 3x2 nvme it gets real world 600MB\s read even though its rated far higher and the stuttering issue is there when the media is internal.

 

[Chris Moore]

I have the space I just am kinda Ocd I guess. I don't like unnecessary dupes. Im using almost a third of it.

I used a utility to scan my NAS not long ago and identified almost a terabyte of duplicate files where I have made backups over the years. I really need to take some time and clean it up.

 

[MatthewSteinhoff]

but I will atleast give the 1060 a try

You're welcome to buy a 1060. They're stupid expensive right now thanks to cryptocurrency miners.

That said, it ain't going to help. Have you read the Plex documents on hardware-accelerated transcoding?

• "On Linux, hardware-accelerated decoding is not supported on NVIDIA GPUs. Intel Quick Sync is required for hardware-accelerated decoding.
• "The video quality may be lower, appearing more blurry or blocky. (Hardware-accelerated video encoders are faster, but lower quality than software encoders.)"

Your Intel i7-8700 should be able to pump out seven transcoded HD streams. Depending on who you believe and what footnotes are applied, Plex can transcode one or two 4K streams using that same processor. (And, even if it couldn't, the next step up would be a $2,000 CPU. And you still couldn't transcode more than a 4K stream or two at a time.) But, really, the Synology NAS seems to be holding you back. Look at the video playback numbers from ServeTheHome. The throughput from the DS916 is less than a single bare drive on read. By half, give or take.

As others have noted, off-hours transcoding to optimize video instead of live transcoding is probably the best solution.

Cheers,
Matt

 

[Mylex]

I was home during my break from work trying to see if I could get a plan to switch over to FreeNAS if there was a shot. With it doing the same thing off my internal SSD, I'm giving up and just running a batch job I found that will optimize the 4k movies to 1080 files they can easily stream. The system as it is actually impresses me with what it can do, I did not think it would give me 2 4k transcodes and a few direct streams at the same time. Until my remote users are willing to pay the difference for synchronous gigabit service then this will have to work, not interested in spending for just this... More storage than I am game.

 

[diskdiddler]

Trying to get better transcode performance(remote streams to family members in other houses) compared to Ubuntu server 16.04. I currently have my setup as an i7-8700 with 32GB of ram running off a small ssd. I have all devices in my house direct playing, most at once here would be three but I have two remote users that are frequently streaming. My library is about 75%-1080/25%-4k and slowly moving towards more 4k. Just wondering the overhead of FreeNas compared to Ubuntu server. If I replace Ubuntu with FreeNas my media will still be fed from my DS916+ with 8GB of ram. I'm just wondering if I would get rid of the slight stutter when all are going at once. My CPU hits about 80% with memory not even 50% used. My internet speeds are 150/20 and they are limited to one 8meg stream each.

I'm actually surprised that only 5 users can max out an 8700 to be honest. I'm not super well versed with Plex but that seems like some surprisingly high overhead.
Do you know where the bottleneck is? 20mbit upload with 8/8 per user should be ok.

Where is the buffering? All users, only the remote users?
H264 or H265?

Is it 1GBit from the Plex 8700 server to the NAS?

 

[diskdiddler]

Next, I would check your assumption that everything is direct playing. If they are all really direct playing, should be easily handled by a RaspberryPi. The fact your CPU gets to 80% tells me that something (or somethings) are being transcoded.

Plex is wonderful & incredibly stupid. The damn thing will transcode for fun, I think it's into hurting itself or something?
It's so frustrating how often it defaults to transcoding.

 

[diskdiddler]

There is the problem. A Synology DS916+ only has 4 drives and the documentation says it provides reading speed of 225 MB/s. That won't saturate the network, but it is certainly a bottleneck.

Definitely 100% not this at all.
He's not copying the files from the NAS at full speed, he's playing them back / streaming them.

I just opened 4 copies of VLC and opened the 4 largest MKV files on my server (22GB, 25GB, 18GB and 16GB) and played the 4 movies back all at once.
My network card was using 9 to 10MB/s

 

[Mylex]

I'm actually surprised that only 5 users can max out an 8700 to be honest. I'm not super well versed with Plex but that seems like some surprisingly high overhead.
Do you know where the bottleneck is? 20mbit upload with 8/8 per user should be ok.

Where is the buffering? All users, only the remote users?
H264 or H265?

Is it 1GBit from the Plex 8700 server to the NAS?

Yeah the bottleneck is having to transcode 40-70k bitrate files... with all that horse power the transcoding h265 whether hardware or software isn't were it needs to be in my scenario. I can easily do 7+ streams of 1080 transcodes. If I had better then 20 upload, lets say even 100 up I could direct stream to them. I dont know if this is a good thing or not but our ISP really has its customers setup like a building network. It should take more then one hop for me to get across 15-20 miles to my brother-in-laws.

 

[Mylex]

Definitely 100% not this at all.
He's not copying the files from the NAS at full speed, he's playing them back / streaming them.

I just opened 4 copies of VLC and opened the 4 largest MKV files on my server (22GB, 25GB, 18GB and 16GB) and played the 4 movies back all at once.
My network card was using 9 to 10MB/s

I do not believe its the NAS as my cheapo SSD shows the same behavior when I tested internal before coming here. That thing shows actual read speed of 600 so if that cannot be removed I was just thinking the CPU either was struggling and the 80% it was displaying was either wrong or causing it to throttle possibly.

 

[IQless]

I used a utility to scan my NAS not long ago and identified almost a terabyte of duplicate files where I have made backups over the years.

Can I ask which utility you used?

 

[diskdiddler]

Can I ask which utility you used?

I'm definitely your man for this.

You want to use a package called "Duplicate Cleaner Pro" to search for binarily identical dupes, unlike 1 tool that I used, it doesn't make mistakes (!!) can also refine with similar filenames or any filename.
It has an audio and image mode too, which will identify rotated images etc.


As for video, it's a paid app and not cheap but DAMN is "Video Comparer" excellent, it's fantastically good code, whoever wrote this knows their stuff. It will index your videos and create some kind of CRC based on the content - then matching videos.

Literally a 1080p and 240p video with identical contents would be identified with it.
It will even support split modes too, so if you have a video which may be contained, within another video (15 minutes cut out of a 2 hour movie, etc)
VERY good, not cheap, I emailed the dev and begged for a discount, totally worth it.


Finally, treesize pro and Ztree (or other tools) to scan the disk for wildly large folders and files etc.

(My file system is a horrific mess, if I was stupid rich I'd throw immense power at it, turn on some kind of advanced compression, de-dupe)

 

[diskdiddler]

I don't believe its the nas as my cheapo ssd shows the same behavior when I tested internal before coming here. That thing shows actual read spead of 600 so if that can't removed it I was just thinking the cpu either was struggling and the 80% it was displaying was either wrong or causing it to throttle possibly.

Strongly recommend, for experimental purposes that you drop the transcode rate for the outsiders to something like max, 2/3mbit.
Before you say 'whoah!!!' I don't know how (I really don't!) but I can watch plex stuff remotely on my ipad from my Australian ADSL2 line, which has a 90KB/s upload (!!?) that's what? .7mbit? The quality is shockingly good, all things considered, I don't know what it transcodes INTO but it's shocking for 90KB/s

That may eliminate some CPU issues.

 

[Mylex]

Strongly recommend, for experimental purposes that you drop the transcode rate for the outsiders to something like max, 2/3mbit.
Before you say 'whoah!!!' I don't know how (I really don't!) but I can watch plex stuff remotely on my ipad from my Australian ADSL2 line, which has a 90KB/s upload (!!?) that's what? .7mbit? The quality is shockingly good, all things considered, I don't know what it transcodes INTO but it's shocking for 90KB/s

That may eliminate some CPU issues.

Defintely CPU related, lowered the transcode quality to 4mps 720 and I cant even start a third stream. CPU shoots right to 100%( 2X4k streams trancoded to 720 and 2 direct play streams). Only solutions are provide more power to transcode, higher upload bandwidth to direct stream the 4k movies or optimize the 4 k library so they just stream to the clients without transcoding 4k files.

 

[Chris Moore]

just running a batch job I found that will optimize the 4k movies to 1080 files

Would you share a link to that script? I might be able to use that myself.

 

[Mylex]

Would you share a link to that script? I might be able to use that myself.

Sure no problem


https://forums.plex.tv/discussion/205393/batch-video-conversion-tool This led me to the next one
https://github.com/dswinton/Batch-H.264-Converter

 

[Chris Moore]

Can I ask which utility you used?

I found it a few years ago and one of the best things about it (for me) is that it has a Linux version also. The bad thing about it is that it is no longer being maintained. It works well, but there won't be any new versions, unless someone picks the project up and runs with it.
Here is a link to it:
https://www.hardcoded.net/dupeguru/
It is the best duplicate scanner I have found because it doesn't just look for name matches, it actually looks at file content regardless and will tell you the percentage of match. That way if you (for example) have two word documents that are 88% the same, you can take a look and decide if you actually need them both or if one is just old and can be deleted. I have a large quantity of complete duplicated files where the only difference is which directory they are in.
Because it is looking at file content and making comparisons, it can take a while, depending on how many files you have to scan. It took more than a day for it to scan my entire NAS. I have not found anything better.

PS. It is open source, so you could always grab the source code and update it yourself...
https://github.com/hsoft/dupeguru

 

[diskdiddler]

Defintely CPU related, lowered the transcode quality to 4mps 720 and I can't even start a third stream. CPU shoots right to 100%( 2X4k streams trancoded to 720 and 2 direct play streams). Only solutions are provide more power to transcode, higher upload bandwidth to direct stream the 4k movies or optimize the 4 k library so they just stream to the clients without transcoding 4k files.

That's a shame, Plex can be very greedy at times.

I just want the lowest possible power CPU, which can transcode 4k hdr x265 for ONE user. (Ie: future proof Plex CPU, but 1 user ever)

Damned if I know what CPU that is