got tired of waiting and quit - but here's some of the data
@jgreco @Davvo - side note - when I ran it the other day from the truenas shell the 1st time - the --SLOW-- and ++FAST++ on sdh was not there in those results.
Disk Disk Size MB/sec %ofAvg
------- ---------- ------ ------
sda 12000138MB 206 104
sdb 12000138MB 204 103
sdc 12000138MB 204 104
sdd 12000138MB 203 103
sde 12000138MB 196 99
sdf 12000138MB 203 103
sdg 12000138MB 196 99
sdh 12000138MB 165 84 --SLOW--
This next test attempts to read all devices in parallel. This is
primarily a stress test of your disk controller, but may also find
limits in your PCIe bus, SAS expander topology, etc. Ideally, if
all of your disks are of the same type and connected the same way,
then all of your disks should be able to read their contents in
about the same amount of time. Results that are unusually slow or
unusually fast may be tagged as such. It is up to you to decide if
there is something wrong.
Performing initial parallel array read
Sat Jan 28 20:43:51 CST 2023
The disk sda appears to be 12000138 MB.
Disk is reading at about 192 MB/sec
This suggests that this pass may take around 1039 minutes
Serial Parall % of
Disk Disk Size MB/sec MB/sec Serial
------- ---------- ------ ------ ------
sda 12000138MB 206 200 97
sdb 12000138MB 204 202 99
sdc 12000138MB 204 203 99
sdd 12000138MB 203 202 99
sde 12000138MB 196 194 99
sdf 12000138MB 203 202 99
sdg 12000138MB 196 193 99
sdh 12000138MB 165 193 117 ++FAST++
Awaiting completion: initial parallel array read
Sun Jan 29 14:45:32 CST 2023
Completed: initial parallel array read
Disk's average time is 62092 seconds per disk
Disk Bytes Transferred Seconds %ofAvg
------- ----------------- ------- ------
sda 12000138625024 60634 98
sdb 12000138625024 60626 98
sdc 12000138625024 60596 98
sdd 12000138625024 60914 98
sde 12000138625024 64321 104
sdf 12000138625024 60778 98
sdg 12000138625024 63963 103
sdh 12000138625024 64901 105
This next test attempts to read all devices while forcing seeks.
This is primarily a stress test of your hard disks. It does thhis
by running several simultaneous dd sessions on each disk.
Performing initial parallel seek-stress array read
Sun Jan 29 14:45:32 CST 2023
The disk sda appears to be 12000138 MB.
Disk is reading at about 176 MB/sec
This suggests that this pass may take around 1136 minutes
Serial Parall % of
Disk Disk Size MB/sec MB/sec Serial
------- ---------- ------ ------ ------
sda 12000138MB 206 174 85
sdb 12000138MB 204 174 86
sdc 12000138MB 204 175 86
sdd 12000138MB 203 175 86
sde 12000138MB 196 168 86
sdf 12000138MB 203 174 86
sdg 12000138MB 196 168 86
sdh 12000138MB 165 168 102
Awaiting completion: initial parallel seek-stress array read