Camcontrol report 136Mb/s for SSD

[amorphys]

Hi,
I installed the TrueNAS-12.0-U6.1 on a HP Proliant DL380 Gen9 with 3 Smart HBA H240 with 24 SATA SSD attached to it.
Everything work well until i start to make some test benchmarks etc for the pool.
Then i realized that maybe my SSD are not config as best ( using some WD BLUE 1 TB SSD SATA 3 6Gb/s)
Here are my outputs that make me thinking is this normal ?


camcontrol: sending ATA GET_NATIVE_MAX_ADDRESS_EXT via pass_16 with timeout of 30000 msecs
pass0: <WDC WDS100T2B0A-00SM50 415020WD> ACS-4 ATA SATA 3.x device
pass0: 135.168MB/s transfers, Command Queueing Enabled

protocol ACS-4 ATA SATA 3.x
device model WDC WDS100T2B0A-00SM50
firmware revision 415020WD
serial number 211612A002FC
WWN 5001b448bab4c44d
additional product id
cylinders 16383
heads 16
sectors/track 63
sector size logical 512, physical 512, offset 0
LBA supported 268435455 sectors
LBA48 supported 1953525168 sectors
PIO supported PIO4
DMA supported WDMA2 UDMA6
media RPM non-rotating
Zoned-Device Commands no

The speed that i see is not what i expected so i ran other commands
root@truenas[~]# dmesg | grep -i transfers

da5: 135.168MB/s transfers
da21: 135.168MB/s transfers
da16: 135.168MB/s transfers
da19: 135.168MB/s transfers
da20: 135.168MB/s transfers
da1: 135.168MB/s transfers
da8: 135.168MB/s transfers
da11: 135.168MB/s transfers
da17: 135.168MB/s transfers
da18: 135.168MB/s transfers
da6: 135.168MB/s transfers
da2: 135.168MB/s transfers
da3: 135.168MB/s transfers
da12: 135.168MB/s transfers
da13: 135.168MB/s transfers
da7: 135.168MB/s transfers
da23: 135.168MB/s transfers
da22: 135.168MB/s transfers
da10: 135.168MB/s transfers
da0: 135.168MB/s transfers
da15: 135.168MB/s transfers
da9: 135.168MB/s transfers
da4: 135.168MB/s transfers
da14: 135.168MB/s transfers

same story here so i ran this command : smartctl -a /dev/da0

smartctl 7.2 2020-12-30 r5155 [FreeBSD 12.2-RELEASE-p10 amd64] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family: WD Blue / Red / Green SSDs
Device Model: WDC WDS100T2B0A-00SM50
Serial Number: 211612A002FC
LU WWN Device Id: 5 001b44 8bab4c44d
Firmware Version: 415020WD
User Capacity: 1,000,204,886,016 bytes [1.00 TB]
Sector Size: 512 bytes logical/physical
Rotation Rate: Solid State Device
Form Factor: 2.5 inches
TRIM Command: Available, deterministic, zeroed
Device is: In smartctl database [for details use: -P show]
ATA Version is: ACS-4 T13/BSR INCITS 529 revision 5
SATA Version is: SATA 3.3, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is: Thu Dec 9 03:22:23 2021 PST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

so where is the problem why the camcontrol see only 135MB ? should i put some tunable also for SSD please advice
My pool is made from 5 vdev using RAIDZ2 so i was expected some nice write and reads

 

[jgreco]

You need to pull the janky CCISS based RAID controller (not a true HBA) and replace it with an LSI HBA. The CCISS stuff is all ancient, cruddy stuff, and is probably running at SATA 1.5Gbps. Errors are not reported correctly by CCISS and it is going to be a total misadventure.

The following article was written about this sort of thing. Points 2) and 4) are specifically and specially relevant to you.

What's all the noise about HBAs, and why can't I use a RAID controller?

1) An HBA is a Host Bus Adapter. This is a controller that allows SAS and SATA devices to be attached to, and communicate directly with, a server. RAID controllers typically aggregate several disks into a Virtual Disk abstraction of some sort...

www.truenas.com

 

[amorphys]

Manny thanks this is also what i was suspected i will try with a true HBA like LSI

indeed dmseg show that it is used ciss for HP 240

root@truenas[~]# dmesg | grep -i H240
ciss0: <HP Smart Array H240> port 0x3000-0x30ff mem 0x95400000-0x954fffff,0x95500000-0x955003ff at device 0.0 numa-domain 0 on pci5
ciss1: <HP Smart Array H240> port 0x2000-0x20ff mem 0x95200000-0x952fffff,0x95300000-0x953003ff at device 0.0 numa-domain 0 on pci9
ciss2: <HP Smart Array H240> port 0xa000-0xa0ff mem 0xc8100000-0xc81fffff,0xc8200000-0xc82003ff at device 0.0 numa-domain 1 on pci20

 

[jgreco]

This NAS is headed for a disaster of biblical proportions. Fire and brimstone coming down from the skies! Rivers and seas boiling! Forty years of darkness! Earthquakes, volcanoes... The dead rising from the grave! Human sacrifice, dogs and cats living together... MASS HYSTERIA!

Ok ok so not QUITE that bad. But it really is a truly bad controller to use with ZFS because CCISS was designed to try to be a "one size fits all" and ended up being a "one size fits all poorly" misadventure, no matter how amazingly talented the people who worked on this were. If you are using SSD's, your best bet is probably to look for something with the LSI SAS2308 chipset on it.

Please feel free to stop back and report on this as it evolves.