Hello all,
I'm trying to find the problem in my SSD pool for a long time now. I read a lot in the forum and tried recommended optimizations. I also tried to disable sync, compression etc. but nothing changed the massive performance drops I have. I hope that someone can help me.
My system details are in my signature.
Here is my test scenario:
I just copy the game Red Dead Redemption 2 from my workstation over SMB to my pool. On the Ironwolf pool with only HDDs I get consistent write speeds:
Speed is somewhere between 400MB/s up to 600MB/s
Now my SSD pool for comparison:
Again same set of data, Red Dead Redeption 2
Speed drops down to 10MB/s, stays there for a while and peaks up to 200MB/s just to drop again.
Here are my pool stats:
I tried plugging the SSDs into the Raid controller and directly onto the mainboard, but the performance is exactly the same.
Here is the CPU utilization during the copy process
What else can I try? I would really like to use the ssd pool as vm storage for Proxmox, but the performance drops cause a very high IO delay, so I had to move all the volumes back to the proxmox host for now.
I'm trying to find the problem in my SSD pool for a long time now. I read a lot in the forum and tried recommended optimizations. I also tried to disable sync, compression etc. but nothing changed the massive performance drops I have. I hope that someone can help me.
My system details are in my signature.
Here is my test scenario:
I just copy the game Red Dead Redemption 2 from my workstation over SMB to my pool. On the Ironwolf pool with only HDDs I get consistent write speeds:
Speed is somewhere between 400MB/s up to 600MB/s
Now my SSD pool for comparison:
Again same set of data, Red Dead Redeption 2
Speed drops down to 10MB/s, stays there for a while and peaks up to 200MB/s just to drop again.
Here are my pool stats:
Code:
root@truenas[~]# zpool list NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT SSD 2.72T 389G 2.34T - - 3% 13% 1.00x ONLINE /mnt boot-pool 220G 11.8G 208G - - 2% 5% 1.00x ONLINE - ironwolf 54.5T 26.5T 28.1T - - 0% 48% 1.00x ONLINE /mnt root@truenas[~]# zpool iostat -v capacity operations bandwidth pool alloc free read write read write ---------------------------------------- ----- ----- ----- ----- ----- ----- SSD 403G 2.33T 1 97 56.7K 2.79M raidz1-0 403G 2.33T 1 97 56.7K 2.79M b43d33d3-ed83-4225-981b-87fc54005ab1 - - 0 31 19.2K 951K bb441c9d-1620-479b-bf02-ea49e01ba1a2 - - 0 35 19.0K 951K ce6b35db-f921-4f27-ad87-b1869cf6db52 - - 0 31 18.5K 951K ---------------------------------------- ----- ----- ----- ----- ----- ----- ironwolf 26.5T 28.1T 35 30 1.47M 1.45M raidz1-0 26.5T 28.1T 35 30 1.47M 1.45M b4e23e80-118c-4443-9a0e-ba0ff94ec993 - - 7 6 304K 298K 673bf22b-4598-4a6d-bd73-a319fd6cfc69 - - 6 5 297K 298K b3737655-bb3a-48e0-b53a-157d6f0b0f00 - - 6 5 298K 298K 03ff53e2-7639-41da-aaf0-50ae020a1ea0 - - 7 6 300K 298K 5cde391e-b06c-4f1a-ba04-14b6484ff741 - - 7 5 302K 298K ---------------------------------------- ----- ----- ----- ----- ----- -----
I tried plugging the SSDs into the Raid controller and directly onto the mainboard, but the performance is exactly the same.
Here is the CPU utilization during the copy process
What else can I try? I would really like to use the ssd pool as vm storage for Proxmox, but the performance drops cause a very high IO delay, so I had to move all the volumes back to the proxmox host for now.