Simultaneous writes over NFS in the same pool.

[DQQ]

I not exactly sure how to phrase the question, so bear with me.

I have a media library that's been cataloged, roughly 15TBs, and I would like to encode them all to .mp4 so I can view them over an HTML5 player instead of transcoding with PLEX or something. I used a budget E3 Xeon in my FreeNAS box so the encoding speed is limited. To speed up the encodes, I'm thinking about making separate NFS shares (i.e. one for movies, another for documentaries, etc), and then using ffmpeg on different workstations inside the house to encode the files in each sub-directory over the network.

Does it matter that I'm writing to the same pool at the same time? My disks are 8x 10TB WD white labels in a single Z2 pool. To clarify, each workstation would be limited to their own NFS share, just going into sub-directories to encode media, so I'm not going to read the same files with different hosts. I'm also curious if doing this will somehow affect the lifespan of my drives beyond normal wear and tear, as in, is there's an alternative method to this that would achieve the same results with less wear?

One other idea I had was to temporarily tear down a beefier machine (dual E5s), install FreeNAS on there, import the ZFS pool, use a jail to encode all the files, and then moving the pool back to my original NAS box; but even in that scenario, the question stands, could I also use additional machines inside my house to speed up the process?

 

[Meyers]

You'll probably be fine considering this is a CPU bound task that will likely limit any sort of network / disk throughput. Obviously you'll hit the max the system can handle depending on how many clients you have encoding files. Since you're using a single RAID-Z2 pool you might not be able to handle that many.

I wouldn't worry about wear. This sounds like a fairly light use case.

 

[Chris Moore]

I use a different program to do that, but the thing I have had best results from is copying the file to be transcoded to the workstation, complete the transcode then copy the new file back to the server and finally delete the original file, if that is the intent. Trying to access the file over the network during transcode slows the transcode process significantly. I use a SSD as my scratch disk on the workstation.

 

[Meyers]

copying the file to be transcoded to the workstation, complete the transcode then copy the new file back to the server

Great point. That's how our production video encoding system that I wrote works too.