I have a slow Read and Write performance of the Truenas server. Below is my hardware setup:
SUPERMICRO 6049P-ECR36H includes board X11DPH-T
64GB of RAM
Model: CSE-847BE1C4-R1K23LPB
SUPERMICRO 6049P-ECR36H includes board X11DPH-T
RAM 64GB
CPU x2Intel Xeon Silver 4110 SR3GH 8-Core 2.1GHz
Dual 10GBase-T LAN ports with Intel X722 + PHY Intel X557
AOC-S3008L-L8E – SAS controller
Local drive: nvme 256GB drive
Vdev1 18x12TB drives in Raid Z2 WD enterprise level drives (STORAGE CAPACITY 175TB)
Vdev2 18x18TB drives in RaidZ2 Exos X18 Seagate (261TB)
Current performance:
Vdev1: R: 170 MB/s W: 307MB/s
Vdev2: R: 160 MB/s W: 280 MB/s
Testing:
I have tested the performance by moving large files from vdev1 and vdev2 pools. I also tested performance moving large files between server and workstation, my workstation is connected with 10G switch. The results were similar.
Resource consumption:
My CPU usage never goes beyond 20% in any of the tests. RAM I always have more then 25GB of RAM available.
What I am looking for:
I need a good balance between drive pool capacity, performance, and reliability. I went with RAIDZ2 because 2 drives out of 18 can fail before I loose data, I feel this is a good level of redundancy for my system. I started with vdev1 pool, and instead of expanding it I created a second pool vdev2 because I didn't want to mix the drive sizes. I tested a single drive by writing large files to it and I was getting about 200MB/s write speeds. Since I have RAID6 of 18 drives my expected performance was that i would get triple that speed. But that is not the case. Also write speed is almost double the read speed doesn't make much sense to me.
Questions:
1. How do I properly bench test the system?
2. Is Z2 for 18 drives a correct setup? Is there a better setup with similar level of redundancy by better performance?
3. What can i do to increase the Read and Write performance? IO is not so critical because this is a storage server with max 3 users that move data back and forth.
4. I believe I have a lot of computer resources that i not being fully utilized (CPU, RAM) is there anything that can be done to have high resource utilization with boosted R/W pool performance?
I am new to TrueNAS and I would be very grateful for any help on troubleshooting and proper setup.
SUPERMICRO 6049P-ECR36H includes board X11DPH-T
64GB of RAM
Model: CSE-847BE1C4-R1K23LPB
SUPERMICRO 6049P-ECR36H includes board X11DPH-T
RAM 64GB
CPU x2Intel Xeon Silver 4110 SR3GH 8-Core 2.1GHz
Dual 10GBase-T LAN ports with Intel X722 + PHY Intel X557
AOC-S3008L-L8E – SAS controller
Local drive: nvme 256GB drive
Vdev1 18x12TB drives in Raid Z2 WD enterprise level drives (STORAGE CAPACITY 175TB)
Vdev2 18x18TB drives in RaidZ2 Exos X18 Seagate (261TB)
Current performance:
Vdev1: R: 170 MB/s W: 307MB/s
Vdev2: R: 160 MB/s W: 280 MB/s
Testing:
I have tested the performance by moving large files from vdev1 and vdev2 pools. I also tested performance moving large files between server and workstation, my workstation is connected with 10G switch. The results were similar.
Resource consumption:
My CPU usage never goes beyond 20% in any of the tests. RAM I always have more then 25GB of RAM available.
What I am looking for:
I need a good balance between drive pool capacity, performance, and reliability. I went with RAIDZ2 because 2 drives out of 18 can fail before I loose data, I feel this is a good level of redundancy for my system. I started with vdev1 pool, and instead of expanding it I created a second pool vdev2 because I didn't want to mix the drive sizes. I tested a single drive by writing large files to it and I was getting about 200MB/s write speeds. Since I have RAID6 of 18 drives my expected performance was that i would get triple that speed. But that is not the case. Also write speed is almost double the read speed doesn't make much sense to me.
Questions:
1. How do I properly bench test the system?
2. Is Z2 for 18 drives a correct setup? Is there a better setup with similar level of redundancy by better performance?
3. What can i do to increase the Read and Write performance? IO is not so critical because this is a storage server with max 3 users that move data back and forth.
4. I believe I have a lot of computer resources that i not being fully utilized (CPU, RAM) is there anything that can be done to have high resource utilization with boosted R/W pool performance?
I am new to TrueNAS and I would be very grateful for any help on troubleshooting and proper setup.