SSH connections sometimes dropped/laggy on 11.2-U5

[jotef]

Hi everybody,

after several attempts (and rollbacks) I gave it another shot to update from 11.1-U7 to 11.2-U5 yesterday. My first attempt was with 11.1-U6.3 to 11.2-U1, which did not work "out-of-the-box" as it introduced "slow" networking/lags within the GUI and via SSH and non-working VNET iocage jails. At that time I did not have the time to debug the issues, therefore I simply rolled back to 11.1, where everything works smoothly.

What I could remedy for now with regard to the "slow" network stuff and the non-working jails in 11.2-U5 was following:

• Changing my main FreeNAS interface bge0 from DHCP to a static address as mentioned in this old issue made the unresponsive UI go away
• Changing all my iocage jails from vnet_default_interface=none to auto like mentioned in this issue made the jails startable again

Nevertheless, still there are some erratic issues revolving ssh like e.g. ssh connections either being dropped packet_write_wait: Connection to ... port 22: Broken pipe or that sometimes all typing hangs for some seconds and appears in one flush. In 11.1 there were no such issues with regard to ssh.

Could anyone give me a pointer about how to tackle this or does anyone else experienced those issues and came up with a fix? Any help/pointer would be much appreciated. Thx in advance!

Btw. FreeNas is running on a HP MicroServer Gen8 and here if the output of ifconfig (not all jails running atm) and my tunables in rc.conf:

ifconfig:

Code:

bge0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=c0099<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,VLAN_HWTSO,LINKSTATE>
    ether 00:fd:45:fc:1c:44
    hwaddr 00:fd:45:fc:1c:44
    inet 10.85.10.10 netmask 0xffffff00 broadcast 10.85.10.255
    nd6 options=9<PERFORMNUD,IFDISABLED>
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
bge1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=c019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE>
    ether 00:fd:45:fc:1c:45
    hwaddr 00:fd:45:fc:1c:45
    nd6 options=9<PERFORMNUD,IFDISABLED>
    media: Ethernet autoselect
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
    options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
    inet6 ::1 prefixlen 128
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
    inet 127.0.0.1 netmask 0xff000000
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    groups: lo
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    ether 02:10:12:38:48:00
    nd6 options=9<PERFORMNUD,IFDISABLED>
    groups: bridge
    id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    member: vnet0:10 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 12 priority 128 path cost 2000
    member: vnet0:7 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 11 priority 128 path cost 2000
    member: vnet0:4 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 10 priority 128 path cost 2000
    member: bge0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 1 priority 128 path cost 20000
    member: vlan20 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 7 priority 128 path cost 55
bridge1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    ether 02:10:12:38:48:01
    nd6 options=9<PERFORMNUD,IFDISABLED>
    groups: bridge
    id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    member: vlan30 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 8 priority 128 path cost 55
bridge2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    ether 02:10:12:38:48:02
    nd6 options=9<PERFORMNUD,IFDISABLED>
    groups: bridge
    id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    member: vlan89 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 9 priority 128 path cost 55
vlan20: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=80001<RXCSUM,LINKSTATE>
    ether 00:fd:45:fc:1c:44
    inet 10.85.20.10 netmask 0xffffff00 broadcast 10.85.20.255
    nd6 options=9<PERFORMNUD,IFDISABLED>
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
    vlan: 20 vlanpcp: 0 parent interface: bge0
    groups: vlan
vlan30: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=80001<RXCSUM,LINKSTATE>
    ether 00:fd:45:fc:1c:44
    inet 10.85.30.10 netmask 0xffffff00 broadcast 10.85.30.255
    nd6 options=9<PERFORMNUD,IFDISABLED>
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
    vlan: 30 vlanpcp: 0 parent interface: bge0
    groups: vlan
vlan89: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=80001<RXCSUM,LINKSTATE>
    ether 00:fd:45:fc:1c:44
    inet 10.85.89.10 netmask 0xffffff00 broadcast 10.85.89.255
    nd6 options=9<PERFORMNUD,IFDISABLED>
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
    vlan: 89 vlanpcp: 0 parent interface: bge0
    groups: vlan
vnet0:4: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: test2 as nic: epair0b
    options=8<VLAN_MTU>
    ether 02:ff:60:4b:48:3c
    hwaddr 02:75:d0:00:0a:0a
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    groups: epair
vnet0:7: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: test as nic: epair0b
    options=8<VLAN_MTU>
    ether 02:ff:60:ae:1b:75
    hwaddr 02:75:d0:00:0b:0a
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    groups: epair
vnet0:10: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: acme-11-2 as nic: epair0b
    options=8<VLAN_MTU>
    ether 02:ff:60:f9:9f:ae
    hwaddr 02:75:d0:00:0c:0a
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    groups: epair

tunables:

Code:

cloned_interfaces="bridge0 bridge1 bridge2"
ifconfig_bridge0="addm vlan20 up"
ifconfig_bridge1="addm vlan30 up"
ifconfig_bridge2="addm vlan89 up"

 

[jotef]

Seems I was beeing to optimistic about the jails.
It seems that the first jail that starts triggers that the default interface bge0 is added to the corresponding bridge that is assigned to the jail (e.g. interfaces:vnet0:bridge0) and then receives an IP via DHCP over that interface (which is wrong as they should receive it via their respective VLAN). Furthermore, then all other jails that have assigned other bridges (in my case bridge1 and bridge2 – due to different VLANs – cannot start up properly and the IP4 column says DHCP(Network Issue). Is there some sort of regression with regard to vlans, dhcp in conjunction with jails and net with 11.2? Strange that everything worked smoothly without an issue for months in 11.1...

 

[Newfoundland.Republic]

Hmmm... I was having some similar issues but I am using physical interfaces on FreeNAS (VLANs on my switch as member ports - not tagged)

My tunables are:

Code:

cloned_interfaces="bridge25 bridge30"
ifconfig_bridge25="addm igb2 up"
ifconfig_bridge30="addm igb3 up"
ifconfig_igb2="up"
ifconfig_igb3="up"

I ignored bridge0 altogether and aligned the bridge names with the subnets. (See my post here) I'm not sure if adding the interfaces up later in the tunables makes any difference.

Question/thought: In your jail configuration under Advanced Mode | Networking does the default interface show the VLANs? For the physical interfaces I can select the appropriate interface; do you get the VLAN options?

 

[jotef]

Hmmm... I was having some similar issues but I am using physical interfaces on FreeNAS (VLANs on my switch as member ports - not tagged)

My tunables are:

Code:

cloned_interfaces="bridge25 bridge30"
ifconfig_bridge25="addm igb2 up"
ifconfig_bridge30="addm igb3 up"
ifconfig_igb2="up"
ifconfig_igb3="up"

I ignored bridge0 altogether and aligned the bridge names with the subnets. (See my post here) I'm not sure if adding the interfaces up later in the tunables makes any difference.

Question/thought: In your jail configuration under Advanced Mode | Networking does the default interface show the VLANs? For the physical interfaces I can select the appropriate interface; do you get the VLAN options?

Hey, thanks for your reply. As soon as I'm back home I will try to check this. What's really strange is that my primary interface `bge0` is added to the first bridge of which the jail is a member that starts first. In 11.1 the primary interface was never a member of any bridge. I don't know which component introduces this. I will come back when I have more information. Thx!

Offtopic: How can I edit my own posts. I am not able to find this option.

 

[Newfoundland.Republic]

I have the same thing happening. I think that this is the default action (bridge0 is assigned to the primary interface). I "ignore" that by manually assigning the bridges to the individual physical interfaces and then map them in the jain config.

Offtopic: I use a web browser and on the bottom left of my posts I have the "edit" option. Are you logged in? Are you using an app to view the posts?

 

[jotef]

I have the same thing happening. I think that this is the default action (bridge0 is assigned to the primary interface). I "ignore" that by manually assigning the bridges to the individual physical interfaces and then map them in the jain config.

Offtopic: I use a web browser and on the bottom left of my posts I have the "edit" option. Are you logged in? Are you using an app to view the posts?

You are my hero ;). I just adjusted my tunables according to your pattern (skipped bridge0 altogether) and for all my jails I set the corresponding vlan as default interface. Now my jails come up and are receiving their IP via DHCP from the according VLAN. I now just let it run for some time and hope that nothing blows up ;).

Offtopic: I'm using the latest Firefox but I can only see "Report" on the left side of my own posts...

 

[Newfoundland.Republic]

I'm using Chrome with no issues....