If you’ve noticed, we aren’t big on publishing generic IOPS or latency numbers for TrueNAS or FreeNAS storage. Now, we realize if you’re comparing storage solutions by looking at brochures and data sheets, this might be a little frustrating at first, but I assure you we haven’t done this because we’re hiding something. In fact, it’s quite the opposite. We do it because it’s in your best interest.
Benchmarks, by nature, produce generalized numbers, which means they can be misleading at best, and at worst, meaningless. They can be valuable in very specific scenarios but are less useful than generic numbers in the real world. A Technical Marketing person can typically cherry-pick a benchmark result to make their product look good and other vendors look bad. The only way to truly gauge your performance requirements and whether or not a solution is adequate is to test it in your own environment with your own specific workloads. This is why we offer our Storage No-Risk Guarantee (SNRG) program for TrueNAS that allows you to do just that.
Really, there are two types of benchmarks commonly seen in the world of storage. The first type is what we refer to internally as “hero” benchmarks. These are benchmarks like fio with synthetic workloads that are designed to test raw I/O performance, almost always at 4K block size (an IO size seldomly used by modern applications, by the way). Whenever you see “performance over 1 million IOPs!” on a website or piece of marketing, these numbers were almost certainly derived with this type of benchmark. These are the benchmarks the Marketing Team wants.
The second type of benchmark is what we call a “pathological” benchmark, which attempts to approximate “worst case” real world application scenarios by being as resource intensive as possible, therefore testing the limits of storage performance and stability. These are the benchmarks Engineering and QA Teams use, since they tend to help identify design flaws or regressions. SPC-1/SPC-2 are good examples of this type of “pathological” benchmark. Marketing Teams will typically only allow these results to be published if they’ve brought their Formula 1 cars to the race, however. What I mean by this is that they will create massive storage rigs with seven or eight figure price tags, which might produce impressive SPC numbers, but aren’t at all representative of a typical array the average customer could afford.
The simple truth is that storage performance is dependent on many factors. Random I/O is not the same as sequential I/O. Block size matters and must match the application. Furthermore, the same size may not be used by multiple applications in your workload. Storage topology is important. RAID layout must be considered. Cache matters and must be sized appropriately to the workload. If you’ve come to iXsystems looking for hyperbole like “Blazing Performance” or heavily caveated claims like “Up to One Million IOPS!”, then you’re in the wrong place. We are careful to not use the fuzzy math that will ultimately leave you disappointed. What we do, however, is work with you to create the best solution for your needs.
We’ve built storage and servers for thousands of companies. We have clients in every major industry, including education, high-tech, entertainment, manufacturing, finance, government, and healthcare, and the one common thread is that TrueNAS performed significantly better than the existing vendor’s solution.
We’re proud of the fact that we aren’t a VC-funded startup under immense pressure to outpace “burn rate” or exceed a marketing-driven IOPS target. We also aren’t a household name that can simply rely on the reputation of our brand. We are building our reputation on satisfying our customers, and we believe that starts with honest and realistic marketing, especially when it comes to performance.
TrueNAS provides flexible performance and can be configured and tuned in a number of ways to suit most storage applications. If we don’t think our storage or servers are a good fit for you, our engineers will gladly tell you up-front. If you’re wondering if TrueNAS will meet your performance targets, let’s work together to assess your needs and design a right-sized solution to fit them. Contact us today to find out more about our SNRG program so that you can test TrueNAS in your environment and base your performance decisions on real-world results.
iXsystems