Video editing is one of the more demanding NAS workloads these days. The move to 4K and 8K video means that massive files need to be transferred with reliability and speed. LinusTech Media is one of the organizations that has these challenges, and they recently migrated from Windows Server Storage Spaces to TrueNAS 12.0 for its speed and reliability.
As always, Linus Sebastian and Jake Tivy produced a thoroughly entertaining and technically educational video. It’s well worth the 20 minutes to watch, and of this writing has nearly 2 million views already.
Linus Media previously used a Windows Storage Server Spaces server for video editing and TrueNAS as a reliable backup and archive of their data. They identified several key problems with their Windows storage that hindered their business operations:
-
- Insufficient throughput to the drive storage
- Excessive writing of unprotected data to RAM
- “Blue Screen of Death” crashes happening too frequently
- Storage “hanging” under high write loads – disrupting the whole business
- The storage was too slow – making editing more difficult
In looking for a replacement, they were looking for a Linux system with the latest ZFS, Samba, and preferably a UI. While TrueNAS SCALE meets those requirements, it’s still in BETA, so we recommended TrueNAS CORE to them because they were running a production workload that was critical to their business. TrueNAS CORE 12, which has a decade of inherited development and community testing, has been in a release state for over a year. This made it a very mature, excellent choice for a mission critical use-case of this nature.
They built a high-end system with a 32 core AMD EPYC processor, 256 GB of RAM, dual 100Gbe NIC, and Gen4 NVMe drives. Performance was excellent and demonstrated the ZFS file system operating at 14-18GB/s internally within the system, far in excess of what the Windows storage server could achieve.
They then set about testing it with their video editors and ongoing transfers via SMB. When storage performance is adequate, the editors see very little lag and the system can stop on a dime, without “runaway footage”. The TrueNAS system delivered that performance and can likely sustain it for many more editors.
The reliability of TrueNAS under load comes from the magic of OpenZFS which is engineered for great stability and robustness. Unlike Windows Server, OpenZFS limits the amount of DRAM used for “dirty” or unprotected data. Its design also behaves much more consistently under high load and hence supports more critical workloads like video editing and virtualization storage. FreeBSD and TrueNAS have also had extensive field testing (millions of machine years) as a dedicated storage environment which has largely eliminated the system crash issues of the overly complex and general purpose Windows Server environment.
Technical Notes
The video included a few technical recommendations which are worth discussing.
Enable TRIM on SSDs: TRIM probably isn’t required for the performance seen. However, the primary reason for not enabling TRIM by default is that some SSDs behave badly with TRIM enabled. The Kioxia CD6 SSDs used in the video do not have this issue.
Disabling Compression: They disabled compression for storing video files. That’s not a bad decision, but might not have been necessary. OpenZFS is very good at detecting and skipping over incompressible data, so the penalty for leaving the default (compression on) isn’t likely to be noticeable in most cases.
Using ARC only for Metadata: For applications where data is not shared directly between many clients, it’s not a bad strategy. It reduces the amount of CPU used for managing a large ARC with data. If the cache hit rate would otherwise be low, then it is worth considering. For most use-cases, the use of ARC is beneficial.
SMB Multi-Channel: Multi-channel does improve the maximum performance seen by a single client but is rarely needed. It is not enabled by default because there are corner cases, particularly over a WAN, where a file can be corrupted. However, this is unlikely to happen in a LAN workgroup environment. The latest Samba in TrueNAS SCALE now has multi-channel readily available.
Single vs Multiple VDEVs: It was indicated that a single VDEV performed about as well as two VDEVs. This is probably dependent on the testing approach being used. For a single client it is likely to be true, but for many clients, more VDEVs can significantly improve overall performance.
Multiple boot drives: Yes, TrueNAS lets you mirror boot drives internally. For older SATA SSDs and HDDs, we did recommend mirror drives. For newer and high quality M.2 SSD boot drives, we are finding a single boot drive to provide enterprise level system reliability.
RAID-Z1 vs RAID-Z2: They used RAID-Z1 (single parity) for their pool. This is generally fine with good quality SSDs (test or burn-in the SSDs first!) which resilver quickly and fail infrequently. As they indicated, they also have a good backup plan. With HDDs, we recommend RAID-Z2 for reliable redundancy on the local storage pool.
Get Jake to deploy TrueNAS: Linus made a comical jab at some previous issues he had with FreeNAS. In his defense, he was often trying to push the envelope on new features/ hardware and trying to do it while doing a video. Jake sets up his systems methodically in the background, tests them thoroughly, and makes sure they work. As much as we love Linus, new TrueNAS users should aim to be more like Jake :-). Alternatively, give iXsystems a call and we’ll get you a turnkey appliance that removes any guesswork.
The TrueNAS Software
TrueNAS CORE is free, just like the older FreeNAS. TrueNAS 12.0-U6 was released on October 5, 2021 and includes a number of fixes, about ten improvements, and a few platform enhancements. These include:
-
- NFSv4 HA improvements (Enterprise version)
- More extensive SMB regression testing with various SMB options
- SMB shadow copies feature has been fixed (bug in U5)
- OpenZFS dedup performance improvements by defaulting to SHA-512 algorithm
- Better webUI snapshot filtering by name
- UI improvements for disk temperature monitoring and UEFI booting
- Ongoing improvements to UI & middleware performance for large drive counts
- Improved UI for plugin updates/upgrades
- Various M-Series and R-Series webUI improvements
Please check out the updated TrueNAS documentation even if you don’t upgrade today. We’re extremely grateful for all the contributions received thus far and encourage the community to keep the suggestions coming!
TrueNAS SCALE will perform similarly to the version Linus and Jake tests. TrueNAS SCALE is entering the release-candidate stage on Tuesday 26th October. Performance testing of SCALE will be happening over the coming months and business production use of SCALE should start in Q1 2022.
TrueNAS Hardware Platforms
TrueNAS 12.0-U6 is compatible with all of the iXsystems platforms from the TrueNAS Minis, to the power-efficient X-Series, all the way up to the flagship High Availability (HA) M-Series. There is also an R-Series product line that can run CORE, Enterprise, and SCALE editions of TrueNAS. All of these can be updated via the web UI and include graphical enclosure management.
For an HA solution with similar performance to the system built by Linus, look at the TrueNAS M50 or M60. These systems will continue to operate even if one of the controllers were to fail. Combined with Enterprise support, you will have a 24×365 system with five nines of availability (99.999%).
If you have any questions or comments, we’d love to hear them on the community forums, on the TrueNAS subreddit, or in response to this blog. If you need additional information on how TrueNAS can streamline, accelerate, and unify data management for your business, please contact us, or visit the newly redesigned truenas.com and download the TrueNAS version that best fits your needs.
