Hi, I'm considering a FreeNas 9.3 on a
Backblaze Pod 5.0 (for backup puprposes). What are you thoughts on the new version? Is it playing nicely with FreeBSD?
I am running a similar setup but with Backblaze's old partner,
45 Drives. I am using their
30 drive unit with direct attach setup (no backplanes) and connecting them to 2x LSI 9201 cards. It runs great. Also, I am using only one 750w PSU. The unit is on and sitting next to me and runs relatively quiet. i.e.. very quiet compared to a server room, kind of 'loud' compared to a quiet PC or home theater set-up.
I have also just installed a 45 Drives Q30 at an ultra high-def (2K/4K/6K) film and post-production/editing company:
SuperMicro X10DRL motherboard
Dual Xeon E5-2620 v3 @ 2.4GHz
256GB RAM
2x 125GB SSD boot drives
2 x LSI 9201-16i HBA cards
28 x 4TB WD Re drives (it holds 30, but one drive died and waiting on a replacement)
3 x 1o drive vdevs RAID Z2, AFP shares
3 x dual Intel X540T2BLK 10Gbe NICs (LAGG/LACP all 6 ports into one)
Netgear XS728T 10 Gbe 24-port managed switch (LAGG/LACP to NAS)
Mixture of late 2013 MacPro and iMacs to XS728T via CAT6 and Sonnet Twin 10G Thunderbolt to Ethernet adaptors.
Storinator NICs and Sonnets are set to "mtu 9000"
Running repeated tests over the 10GbE network, Mac clients consistently see 750-800MB/sec writes ans 550-650MB/sec reads. That's good enough for multiple streams of most 4K and even 6K raw codecs
We ran a test earlier in the week where we overlayed six simultaneous(but, all unique) Redcoode 4K 4:1 @24fps streams(127MB/sec) in a single Adobe Premiere session on a late-2013 Mac Pro. The Q30 had no problem, but the $7k Red Rocket-X expansion card could not handle the load.
So, we then ran six Prores 422 HQ 2K streams(~32MB/sec) to three different Macs without any stuttering or artifacts. Finally, we ran several Prores 4444 XQ streams to each of the Macs and still no problem.
These are unlikely loads for this small production company, even though they have gotten used to working with raw codecs.
Is it OK to be used also in a little bit more performance sensitive environment (VFX work - around 30 users) - loaded with lots of RAM 128 or 256GB and stripped SSDs for caching?
It depends on what you specifically want to use the box for - video streams or compositing, etc. Go for the most RAM can can cram into the box, ZFS loves RAM. SSD cache(as L2ARC) does not enhance the performance of FreeNAS with regard to sequential/streaming content. FreeNAS(actually ZFS) is set by default to bypass cache for streaming. From
ZFS Tuning Guide - "
By default the L2ARC does not attempt to cache streaming/sequential workloads, on the assumption that the combined throughput of your pool disks exceeds the throughput of the L2ARC devices, and therefore, this workload is best left for the pool disks to serve".
In fact, adding extra SSD cache may actually degrade performance as the HDD RAID pool bandwidth is far beyond that of SSD drives and the resulting tables needed to cache and flush data. In my Q30 below, the 30 drive pool has total bandwidth of ~2GB/sec on writes and ~1.6GB/sec reads. I think SSD's in RAID-0 are ~1.1GB/sec
PS. In my opinion, stay away from the Rocket 750 cards if you can. The upside of using them is cost savings (about ~$360 savings for the same amount of ports) and saving one PCI slot. The upside of the 3x LSI 9201 are: 1) can always re-sale on ebay, 2) FreeNAS proven bulletproof so headache free, 3) Potentially much faster.
I originally had the Rocket 750 installed in this Q30. But, when we had a SATA cable go bad and a port on the motherboard, I put dual LSI 9201's in the replacement Q30. Performance seems to be better
The Rocket was designed primarily for one function - maximizing the number of drives controlled from a single card. Saves a PCIe slot and simplifies things. And in datacenters, that is perfect. Performance is a distant consideration to density and cost.
I actually wonder if one HBA for each vdev might even be better.....the cost is minor compared to the overall NAS price. I came across a fairly well documented website that made this claim.