This is historic content that may contain outdated information. For the newest information on FreeNAS and TrueNAS, please visit TrueNAS.com or read our latest Blogs.
This year’s OpenZFS Developer Summit featured many of the greatest minds in file systems development including Dr. Kirk McKusick of the Berkeley Fast File System, Matthew Ahrens, George Wilson and Mark Maybee of Sun Solaris ZFS, Paweł Jakub Dawidek, Brian Behlendorf and Jörgen Lundman of the FreeBSD, Linux and macOS ZFS porting efforts, plus Allan Jude of the “ZFS Books” and many other contributors to OpenZFS.
Taking place at the same venue as the original 2012 ZFS Day, the mood was one of restrained elation and confidence, rather than the cautious optimism that struggled to penetrate the uncertainty surrounding the then-recent acquisition by Oracle of Sun Microsystems, the developer of ZFS. It would be nearly a year before the OpenZFS project was announced in September of 2013 and the topics were generally ideas of how to improve ZFS such as “Channel Programs” integrated scripting, rather than reports on the status or completion of new projects. Five years in, this handful of interesting ideas have given way to a solid stream of remarkable news.
Without a doubt, the most stunning news came from Jörgen Lundman who not only announced but demonstrated OpenZFS running on Windows 10. At first, it wasn’t clear if his mention of “zpool.exe” was a joke but his live compilation of OpenZFS and use of it to copy files via Windows Explorer proved it to be real. This is, after all, the person who ported OpenZFS to macOS, for which I am eternally grateful given that I am saving this report to it as I type. Jörgen was clear about the fact that OpenZFS for Windows is far from production readiness but I for one consider his work the most promising solution yet for mitigating ransomware on the Windows platform. When I asked Jörgen if OpenZFS encryption was coming to macOS, he responded that it is already in. This comes on the heels of NetBSD developer Chuck Silvers announcing this month that he is importing FreeBSD’s OpenZFS and dtrace into NetBSD. This growth makes OpenZFS the most portable file system after FAT32, supporting all Illumos derivatives, all FreeBSD derivatives, GNU/Linux, macOS, and NetBSD.
The next most talked-about moment at the event was Mark Maybee’s keynote about Oracle ZFS, which is now significantly different from OpenZFS. In the time since the projects diverged in 2010, OpenZFS has replaced over half of the original Sun ZFS code and both projects have consistently added unique and often incompatible features. Mark made it clear that Oracle, which now considers itself a key Cloud provider, “has discussed” improved ZFS integration with GNU/Linux in support of their cloud platform. He did not reveal however what form this better support for ZFS in GNU/Linux will take, be it a binary-only solution, adoption of OpenZFS, or the unlikely but not impossible relicensing of Oracle ZFS in a Linux-compatible manner. True Linux-compatibility would come in the form of relicensing Oracle ZFS to GPLv2 or BSD/MIT/ISC, but should they go to GPLv2, the move would limit Oracle ZFS’ support to GPLv2-licensed operating systems, a limiting factor of Btrfs. Given how many Solaris developers they recently laid off, they just may push the relicensing “easy button” and their going the permissive route just may bring OpenBSD into the fold. Refactoring the two projects would be a non-trivial task but I can safely say that the participants are highly motivated.
Beyond these historic announcements about OpenZFS’ growth to new platforms, there was a steady stream of OpenZFS improvement news at the Developer Summit. Of these reports, Matt Ahrens’ “Proposal for 1,000x better dedup performance” stood out given that deduplication has long been a sensitive topic on ZFS. Because Delphix is not a heavy user of deduplication, this proposal is a perfect opportunity for collaboration the way iXsystems and the FreeBSD Foundation are working with Matt on the vdev expansion project. This work will allow you to, for example, expand a 4 disk RaidZ1 vdev to one with 5 or more disks. Faster scrubbing and rebuilding projects were also presented, plus a pool-wide “checkpoint” that would allow you to roll back all aspects of a zpool when upgrading your OS or performing a significant administrative task. Prakash Surya of Delphix gave a talk on his proposal to double OpenZFS synchronous write speed and gave what one attendee called the “best overview of the ZIL ever”. Isaac Huang from Intel gave an update on their DRAID “distributed spare” system and Allan Jude described his porting of Facebook’s Zstandard compression to OpenZFS, which promises a highly-flexible balance of performance and compression ratios. The slides from all of these talks and reports can be found at OpenZFS.org plus there are photos of the speakers in the iXsystems photo album.
Day Two of the event included project status updates and a hackathon at GitHub headquarters in San Francisco. Not only was the conversation excellent but so was the food. Of the FreeBSD attendees, Allan Jude worked on OpenZFS-native “next boot”, Nick Wolff of iXsystems worked on ztest reproducibility and I worked on a script to generate arbitrary depths and widths of directories and datasets for testing purposes. We all got countless questions answered by the respective experts on various components of OpenZFS and had quite a few laughs thanks to the like-mindedness of the audience. Events like this often serve as reunions for ex-Sun employees and I have never seen them as positive and supportive of one another. It is easy to forget how significant Sun was as an independent Unix innovator and many of my colleagues were devastated by the Oracle acquisition. Fortunately, Oracle has since made its position on Solaris pretty clear, leaving the Illumos and BSD communities as the de facto guardians of BSD Unix and I am proud to see this unprecedented collaboration between these amazing teams.
Michael Dexter, Senior Analyst