Chelsio interface being created with txcsum/tso4/tso6 disabled?

Stilez

Guru
Joined
Apr 8, 2016
Messages
529
I'm booting my NAS and I get this:

# ifconfig
cxgbe0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 4096
options=ac00b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6>
ether ****
hwaddr ****
inet xx.xx.xx.xx netmask 0xff000000 broadcast xx.xx.xx.xx
nd6 options=9<PERFORMNUD,IFDISABLED>
media: Ethernet 10Gbase-SR <full-duplex,rxpause,txpause>
status: active
cxgbe1:
flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=ec07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,
LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
ether ****
hwaddr ****
nd6 options=9<PERFORMNUD,IFDISABLED>
media: Ethernet none
status: no carrier
and in /var/log/messages:

Jan 4 03:11:24 SVR cxgbe0: tso4 disabled due to -txcsum.
Jan 4 03:11:24 SVR cxgbe0: tso6 disabled due to -txcsum6.
Jan 4 03:11:24 SVR cxgbe0: enable txcsum first.
Jan 4 03:11:24 SVR bridge0: error setting capabilities on cxgbe0: 35
As far as I know, I've never disabled these checksums, the only option for that IF is the MTU, and cxgbe0 is bridged to NextCloud only - how do I fix this?
 
Last edited:

Chris Moore

Hall of Famer
Joined
May 2, 2015
Messages
10,080
This says that the status is "no carrier", so your interface is not connected to anything. What is the problem?
 

Stilez

Guru
Joined
Apr 8, 2016
Messages
529
This says that the status is "no carrier", so your interface is not connected to anything. What is the problem?
It's a dual port NIC card with the 2nd port disconnected; youve misread the output (easily done). Added newlines to OP for clarity.

Cxgbe0 is active/up, but for reasons unknown has rxcsum enabled, but not txcsum. Cxgbe1 is disconnected/no media, but ifconfig shows it has rxcsum and txcsum both enabled (if it were to be connected).

How do I find out why txcsum (only) ended up disabled on cxgbe0 (only), and how to fix it?
 

Chris Moore

Hall of Famer
Joined
May 2, 2015
Messages
10,080
It's a dual port NIC card with the 2nd port disconnected; youve misread the output (easily done). Added newlines to OP for clarity.
I see now.
Was this a new card or was it purchased used? I have a dual port Chelsio NIC in one of my systems and it doesn't get this kind of error. I wonder if port 0 might be defective. Have you tried switching over to port 1 to see if the fault follows the software configuration?
 

Stilez

Guru
Joined
Apr 8, 2016
Messages
529
Same card/system I've been using for ages. Rebooted my NAS on a new motherboard (supermicro X10SRL-F instead of X10SRi-F with all cards, ram etc moved across), but this shouldn't be able to affect txcsum setting, much less change it for a single port alone. Otherwise no changes whatsoever to OS, config, tunables, network setup, NIC card, NIC port being used, or cable. Havent switched ports, but if I did, I'd want the unused port to work as expected with checksum offload, regardless whether its active or not.

TBH since txcsum is enabled on the other port, it's probably not necessary to physically change which port is in use, wouldn't explain much as the fault already hasn't followed to both ports.
 

Chris Moore

Hall of Famer
Joined
May 2, 2015
Messages
10,080
TBH since txcsum is enabled on the other port, it's probably not necessary to physically change which port is in use, wouldn't explain much as the fault already hasn't followed to both ports.
It depends on the hardware. I have seen network cards where there was a separate logic path for each port and one could fail electrically while the others continued to work. This is why you can have a 48 port network switch with one bad port. It doesn't happen often, but I have seen it happen.
Since you just moved the card from one system board to another, it is possible that something happened in the move. My suggestion would be to pull the card out and blow it off with compressed air. Then put it back together and try again. If port 0 still is not working, try port 1. If the problem follows the software, you might be able to say it is a setting that somehow got changed, but you said there is no software change. That tells me the problem is a hardware fault. This is troubleshooting the hardware. You can't think it is a setting if no settings changed.
 

Stilez

Guru
Joined
Apr 8, 2016
Messages
529
Hmm. It doesn't seem to be hardware after all.

I didn't want to reboot just yet, so I tried re-enabling in CLI. Oddly, it seems that it worked. Which leaves the question, why was it disabled initially on boot, and will it behave after reboot (but we won't know that until it happens)...

Here's the relevant CLI, all output snipped except "OPTIONS" lines:

Code:
# ifconfig cxgbe0
options=ac00b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6>

# ifconfig cxgbe1
options=ec07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>

# ifconfig cxgbe0 rxcsum txcsum tso lro
ifconfig: tso: Resource temporarily unavailable

# ifconfig cxgbe0 -rxcsum -lro

# ifconfig cxgbe0
options=ac01ba<TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6>

# ifconfig cxgbe0 rxcsum lro

# ifconfig cxgbe0
options=ac05bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6>

It added TXCSUM and TSO4, but objected to adding TSO. Strange. Any ideas?
 

desertrider

Dabbler
Joined
Apr 19, 2015
Messages
12
SUMMATION: These TXCSUM, TSO, BRIDGE0 log notices were caused by my jail Resilio (rslsync) on startup – I turned off autostart, of the jail rslsync, then rebooted and now I have full control of the network interface and no more notices (so now I know what is causing this issue).

Does anybody know how to fix this issue / network settings in my rslsync jail (I see Stilez left off with “Any Ideas”? but now I have identified the source of these notices) or do I leave it alone (everything is working)

NOTES: I am getting the same issues as Stilez (all Stilez post exactly as my issues), SPECIFICALLY:

Issue / Log errors on boot:
tso4 disabled due to -txcsum
tso6 disabled due to -txcsum
bridge0: error setting capabilities on cxgbe0: 35

AND when running command:
# ifconfig cxgbe0 rxcsum txcsum tso lro
ifconfig: tso: Resource temporarily unavailable

Now if I STOP (turn off autostart) the rslsync jail these messages go away, and you can turn on / off TXCSUM but then when I restart rslsync the messages return – so rslsync is turning off TXCSUM (-txcsum) and attempting some command on bridge0, thereby triggering these messages.

Also, my Resilio jail is working and my 10GbE seems to be performing OK – so I could ignore all this but just want to see if anybody has “Any Ideas” if I leave it alone or adjust some configuration?
 
Last edited:

jmjordan

Cadet
Joined
Apr 18, 2020
Messages
1
I think this might be an issue with iocage in general not just with rslsync. I have been able to reproduce this problem when I start any jail using vnet networking. Unlike @desertrider I am seeing a significant performance hit after this error occurs. My 10Gbps connection drops to 2Gbps. I am able to fix the issue by disabling vnet networking on all of my running jails and restarting networking.
 

Stilez

Guru
Joined
Apr 8, 2016
Messages
529
Last edited:

Newtofreenas

Dabbler
Joined
Apr 17, 2019
Messages
22
I am so glad this is a new thread and that others are having this problem too. Not that you guys are having problems, but that someone has seen my problems. Been fighting this for 4 days now.

I am able to work 100% with both the new NIC and the on board 1G connected. But if I transfer a file and watch the dashboard, I notice that the Download part is 100% from the 1G and the upload is 100% on the 40G.

System is perfect if both cables are plugged but wont do anything if the 1G is disconnected, however the 1G works fine if the 40G is disconnected.

If I go computer to Freenas with a DAC directly connected then it works amazingly. But freenas will not work when connected to the pfsense router with only the 40G connected.

If i turn off the boot on start for my jail/plugin the tscsum errors do go away.
cxgbe0: tso4 disabled due to -txcsum.
cxgbe0: tso6 disabled due to -txcsum6.

However I still haven't been successful with getting a working freenas box with only the 40G connection plugged in.

I look forward to further follow ups on this problem
 

SweetAndLow

Sweet'NASty
Joined
Nov 6, 2013
Messages
6,421
So this is all normal. Offloading is not supported when using vnet. Been like this forever.

Sorry it took you so long to figure that out.
 

Stilez

Guru
Joined
Apr 8, 2016
Messages
529
So this is all normal. Offloading is not supported when using vnet. Been like this forever.
Sorry it took you so long to figure that out.
So I guess the practical question is this:
  1. Is there a way to run (or configure within the main or jail OS) something like NextCloud plugin, in such a way that it doesn't block offloading on the physical NIC, if there's a shared physical NIC for the NAS and it's plugins?

    (Obviously if there are 2 physical NICs one can dedicate one to the NAS and the other to the plugins/jails. But even then the same question exists, because its still beneficial to have offloading on the jail network connections anyway if achievable)

  2. If not, out of curiosity, what is the technical issue behind it and next best workarounds, if any?
 
Top