HP Cisco B22 iSCSI Drops to FreeNAS - WARNING: icl_conn_receive_pdu: received data segment length 14342874 is larger than negotiated MaxDataSegmentLen

[zimmy6996]

> WARNING: icl_conn_receive_pdu: received data segment length 14342874 is larger than negotiated MaxDataSegmentLength 131072; dropping connection
> WARNING: 172.16.1.149 (iqn.1990-07.com.emulex:80-c1-6e-76-53-b1): connection error; dropping connection
> WARNING: 172.16.1.149 (iqn.1990-07.com.emulex:80-c1-6e-76-53-b1): connection error; dropping connection
> WARNING: 172.16.1.149 (iqn.1990-07.com.emulex:80-c1-6e-76-53-b1): waiting for CTL to terminate 5 tasks
> WARNING: 172.16.1.149 (iqn.1990-07.com.emulex:80-c1-6e-76-53-b1): tasks terminated

Hey all ... Recently, I added a HP blade center to my network. Previously all of my ESXi hosts were standalone servers. I now added a HP Bladecenter with a few HP Gen8 BL460 blades. Chassis is connected via HP Cisco B22s. The blades were configured using the Emulex iSCSI interfaces native on the blades.

Ever since adding the blades, each night I'll see some random alerts generated on FreeNAS, similar to above. Everything appears to be running fine, but it seems like occasionally a packet hits that makes one of the likes to a blade reset itself. Each server has 2 iSCSI interfaces, so im not seeing any impact to the production VMs that are being hosted. But I'd like to get to the bottom of what this error means, and if there is a way I can correct it.

My switching fabric is Cisco Nexus 5010s. Does this alert come from Jumbo Packets? Does this alert come from something misconfigured on the Emulex interfaces? I can't seem to find anything config wise that would be unique to the fabric interfaces coming from the B22's. Maybe I stop using the hardware Emulex interfaces, and use the software iSCSI interface for ESXi over the Emulex interfaces?

 

[Samuel Tai]

It's not clear from your post where the error is appearing. Is the error flagged on FreeNAS or Emulex?

Basically, one side is sending an iSCSI PDU that's too large (13.7 MiB), where the max PDU size previously negotiated was 128 kiB. This has nothing to do with jumbo frames, TCP MSS, or TCP windows.

 

[zimmy6996]

It's not clear from your post where the error is appearing. Is the error flagged on FreeNAS or Emulex?

Basically, one side is sending an iSCSI PDU that's too large (13.7 MiB), where the max PDU size previously negotiated was 128 kiB. This has nothing to do with jumbo frames, TCP MSS, or TCP windows.

The errors are actually showing up on the FreeNAS side. I get them in the daily security run output from each one of my FreeNAS servers. It is happening on both of them. The question is, is there a tweak on the ESXi host that needs to be set to avoid this? I can't imagine I want to simply ignore the errors.

 

[Samuel Tai]

Configuring Advanced Parameters for iSCSI

You might need to configure additional parameters for iSCSI initiators on your ESXi host. For example, some iSCSI storage systems require ARP (Address Resolution Protocol) redirection to move iSCSI traffic dynamically from one port to another. In this case, you must activate the ARP redirection...

docs.vmware.com

You may need to tweak MaxBurstLength, but I'd open a support call with vmWare to make sure, as this isn't supposed to be a parameter that needs to be tweaked.

 

[zimmy6996]

Configuring Advanced Parameters for iSCSI

You might need to configure additional parameters for iSCSI initiators on your ESXi host. For example, some iSCSI storage systems require ARP (Address Resolution Protocol) redirection to move iSCSI traffic dynamically from one port to another. In this case, you must activate the ARP redirection...

docs.vmware.com

You may need to tweak MaxBurstLength, but I'd open a support call with vmWare to make sure, as this isn't supposed to be a parameter that needs to be tweaked.

I'll have to do some digging, because if I drill in to the host level, and properties of the iSCSI interface, and hit advanced, I can see there is a setting for MaxBurstLength, but those settings are greyed out and locked.

 

[zimmy6996]

Should I consider moving to the software iSCSI over the Emulex option?

 

[Samuel Tai]

Should I consider moving to the software iSCSI over the Emulex option?

Worth a shot. The option may be greyed out as iSCSI is active.

 

[HoneyBadger]

I'll have to do some digging, because if I drill in to the host level, and properties of the iSCSI interface, and hit advanced, I can see there is a setting for MaxBurstLength, but those settings are greyed out and locked.

SSH to your host and use esxcli iscsi adapter param get -A vmhbaN to see what it's at now. (I'm at 262144 for mine.)

What versions of vSphere and FreeNAS are you running?

Should I consider moving to the software iSCSI over the Emulex option?

Worth a shot, but it might mess with your VirtualConnect NIC mappings if you're using them (I assume this is for the 554FLB LOM?)