BiffBlendon
Dabbler
- Joined
- Jan 6, 2018
- Messages
- 20
Is FreeNAS 11.1 (or ANY version of FN), with the proper hardware (listed below) CAPABLE of saturating a 40GbE fiber NIC channel?
Our motion picture scanners can scan and WRITE 4K / 5K color TIFF or DPX frames at 30 FPS. 30 FPS at 4K 10-bit Color DPX is ~1,500 MB/s (~1.5GB/s -- i.e. NOT *bits* but BYTES). Each 10-bit color 4K DPX frame is ~50 MB/each, time 30 per second. Plus we'll usually have a 2K ProRes 4444HQ file and 96k WAV file output concurrently, so we need SUSTAINED sequential writes to storage of 20Gb/s or about 2GB/s indefinitely.
Our hardware is:
Supermicro Dual E5-2630 v2 CPU
128GB DDR3 ECC SDRAM
LSI dual-channel SAS3 12G HBA
2x Supermicro 45-disk SAS JBOD *each* with 30x 8TB HGST SAS3 12G spinning disks.
EACH 30-Disk JBOD is on its own 12G SAS channel on the HBA
Mellanox 2-port 40GbE QSFP+ Fiber NIC
Client/Scanner PC is an Asus-based unit with matching Mellanox 40GbE NIC and gobs of CPU, GPU and RAM for scanner operation.
Client PC is Windows 10 Enterprise (LTSB)
We have storage that can handle the throughput from the Mellanox CX3 / Scanner PC, but we're getting abysmal performance from our FreeNAS build over SMB.
The only "tuning" we've done is adjust MTU to 9000 on both FN and Win10.
Open-E, and other paid solutions, operate MUCH faster on the same hardware, but we assume this is because we have to dig deep and perform some significant tuning. We're running a SINGLE ~400TB (available) RAIDZ3 volume to maximize number of spindles and available space.
Should we run a different RAIDZ level (or mirror, which is not desirable due to capacity penalty, but doable if necessary)?
What type of volume, file system, network, memory caching, or other tuning should we perform.
We are running the latest 11.1 build downloaded yesterday (Friday, January 5, 2018) and have done nothing in terms of tweaking or running any additional plugins, etc.
We're in Irvine, California, if there are any true FreeNAS gurus looking for a little work on the side.
Thanks!
Our motion picture scanners can scan and WRITE 4K / 5K color TIFF or DPX frames at 30 FPS. 30 FPS at 4K 10-bit Color DPX is ~1,500 MB/s (~1.5GB/s -- i.e. NOT *bits* but BYTES). Each 10-bit color 4K DPX frame is ~50 MB/each, time 30 per second. Plus we'll usually have a 2K ProRes 4444HQ file and 96k WAV file output concurrently, so we need SUSTAINED sequential writes to storage of 20Gb/s or about 2GB/s indefinitely.
Our hardware is:
Supermicro Dual E5-2630 v2 CPU
128GB DDR3 ECC SDRAM
LSI dual-channel SAS3 12G HBA
2x Supermicro 45-disk SAS JBOD *each* with 30x 8TB HGST SAS3 12G spinning disks.
EACH 30-Disk JBOD is on its own 12G SAS channel on the HBA
Mellanox 2-port 40GbE QSFP+ Fiber NIC
Client/Scanner PC is an Asus-based unit with matching Mellanox 40GbE NIC and gobs of CPU, GPU and RAM for scanner operation.
Client PC is Windows 10 Enterprise (LTSB)
We have storage that can handle the throughput from the Mellanox CX3 / Scanner PC, but we're getting abysmal performance from our FreeNAS build over SMB.
The only "tuning" we've done is adjust MTU to 9000 on both FN and Win10.
Open-E, and other paid solutions, operate MUCH faster on the same hardware, but we assume this is because we have to dig deep and perform some significant tuning. We're running a SINGLE ~400TB (available) RAIDZ3 volume to maximize number of spindles and available space.
Should we run a different RAIDZ level (or mirror, which is not desirable due to capacity penalty, but doable if necessary)?
What type of volume, file system, network, memory caching, or other tuning should we perform.
We are running the latest 11.1 build downloaded yesterday (Friday, January 5, 2018) and have done nothing in terms of tweaking or running any additional plugins, etc.
We're in Irvine, California, if there are any true FreeNAS gurus looking for a little work on the side.
Thanks!