I write in case someone have a similar problem.
I was battling with random
istgt_iscsi_write_pdu_internal: ERROR iscsi_write() failed (errno=32)
istgt_iscsi_task_response: ERROR iscsi_write_pdu() failed
sender: ERROR iscsi_task_response() failed on iqn etc.
under heavy load.
The problem was traced back further to zfs going periodically unresponsive for 20-30 second under heavy load even unrelated to iscsi. a simple heavy dd could knock out the system for half a minute.
I've found a lot of posts with case like this, with or without iscsi, and all the answers were pointing to tweaks and advanced tuning of the ZFS pools. Some tunings helped the system to stay responsive a bit longer, some made it more stable but with huge performance losses.
Out of desperation after checking every single sysctl value and comparing it to another identical system that was running just fine (all identical) I've made the same comparison between the values in the BIOS of the 2 machines. Well, turns out the mobo had a default setting on "mode: IDE" for the SATA controller. On the system with no problems the setting was manually changed to "mode:AHCI" from the get-go, on the one with problems the setting was left on IDE mode.
Changed to AHCI, at boot time an additional "Command Queuing Enabled" appears for every SATA disk in the pool and voilà, no more unresponsive ZFS, no more iscsi error and dropped connection under heavy load. Now the iscsi drives are performing at the maximum speed possibile on a 1GB link.
I doubt that this is the solution to all the similar problem I've seen posted here and there but in case someone has done the same mistake...
I was battling with random
istgt_iscsi_write_pdu_internal: ERROR iscsi_write() failed (errno=32)
istgt_iscsi_task_response: ERROR iscsi_write_pdu() failed
sender: ERROR iscsi_task_response() failed on iqn etc.
under heavy load.
The problem was traced back further to zfs going periodically unresponsive for 20-30 second under heavy load even unrelated to iscsi. a simple heavy dd could knock out the system for half a minute.
I've found a lot of posts with case like this, with or without iscsi, and all the answers were pointing to tweaks and advanced tuning of the ZFS pools. Some tunings helped the system to stay responsive a bit longer, some made it more stable but with huge performance losses.
Out of desperation after checking every single sysctl value and comparing it to another identical system that was running just fine (all identical) I've made the same comparison between the values in the BIOS of the 2 machines. Well, turns out the mobo had a default setting on "mode: IDE" for the SATA controller. On the system with no problems the setting was manually changed to "mode:AHCI" from the get-go, on the one with problems the setting was left on IDE mode.
Changed to AHCI, at boot time an additional "Command Queuing Enabled" appears for every SATA disk in the pool and voilà, no more unresponsive ZFS, no more iscsi error and dropped connection under heavy load. Now the iscsi drives are performing at the maximum speed possibile on a 1GB link.
I doubt that this is the solution to all the similar problem I've seen posted here and there but in case someone has done the same mistake...
