SOLVED Intermittent loss of connection

zey · Nov 4, 2017

Excuse my ignorance.

I don't know what's going on with my networking. Here's some history.

This is a new server I just built. I have ESXi 6.5 Update 1. I have the onboard LSI chip in pass-through to a FreeNAS VM. After I install and configured FreeNAS I decided to also active pass-through on one of the NICs. I removed the virtual NIC and configured the pass-through NIC. Now every now and again I'm having random network issues ex: I'll be watching a movie via Kodi and the movie would randomly stop.

Here's what I'm seeing.

Code:

Nov  4 13:53:56 dobby smbd: dnssd_clientstub ConnectToServer: connect()-> No of tries: 1
Nov  4 21:57:31 dobby smbd: dnssd_clientstub ConnectToServer: connect()-> No of tries: 2
Nov  4 21:57:32 dobby smbd: dnssd_clientstub ConnectToServer: connect()-> No of tries: 3
Nov  4 21:57:33 dobby smbd: dnssd_clientstub ConnectToServer: connect() failed path:/var/run/mdnsd Socket:35 Err:-1 Errno:2 No such file or directory
Nov  4 13:54:09 dobby root: /etc/rc: WARNING: failed precmd routine for minio
Nov  4 13:54:16 dobby bridge0: Ethernet address: 02:9b:ea:41:dd:00
Nov  4 13:54:16 dobby kernel: bridge0: link state changed to UP
Nov  4 13:54:16 dobby kernel: bridge0: link state changed to UP
Nov  4 13:54:16 dobby kernel: igb0: promiscuous mode enabled
Nov  4 13:54:16 dobby epair0a: Ethernet address: 02:ff:e0:00:04:0a
Nov  4 13:54:16 dobby epair0b: Ethernet address: 02:ff:30:00:05:0b
Nov  4 13:54:16 dobby kernel: epair0a: link state changed to UP
Nov  4 13:54:16 dobby kernel: epair0a: link state changed to UP
Nov  4 13:54:16 dobby kernel: epair0b: link state changed to UP
Nov  4 13:54:16 dobby kernel: epair0b: link state changed to UP
Nov  4 13:54:16 dobby kernel: igb0: link state changed to DOWN
Nov  4 13:54:16 dobby kernel: igb0: link state changed to DOWN
Nov  4 13:54:16 dobby kernel: epair0a: promiscuous mode enabled
Nov  4 13:54:21 dobby kernel: igb0: link state changed to UP
Nov  4 13:54:21 dobby kernel: igb0: link state changed to UP
Nov  4 14:14:28 dobby kernel: arp: 10.10.11.11 moved from 02:ff:e0:00:04:0a to 00:25:90:5d:fe:3f on epair0b
Nov  4 19:11:55 dobby kernel: arp: 10.10.11.11 moved from 02:ff:e0:00:04:0a to 00:25:90:5d:fe:3f on epair0b
Nov  4 19:13:16 dobby warden: Building new Jail... Please wait...
Nov  4 19:13:16 dobby warden: zfs clone cargo/.warden-template-standard@clean cargo/UrBackup
Nov  4 19:13:16 dobby warden: Success!
Nov  4 19:13:16 dobby warden: Jail created at /mnt/cargo/UrBackup
Nov  4 19:13:17 dobby epair1a: Ethernet address: 02:ff:e0:00:05:0a
Nov  4 19:13:17 dobby epair1b: Ethernet address: 02:ff:30:00:06:0b
Nov  4 19:13:17 dobby kernel: epair1a: link state changed to UP
Nov  4 19:13:17 dobby kernel: epair1a: link state changed to UP
Nov  4 19:13:17 dobby kernel: epair1b: link state changed to UP
Nov  4 19:13:17 dobby kernel: epair1b: link state changed to UP
Nov  4 19:13:17 dobby kernel: epair1a: promiscuous mode enabled
Nov  4 19:47:47 dobby kernel: epair1a: link state changed to DOWN
Nov  4 19:47:47 dobby kernel: epair1a: link state changed to DOWN
Nov  4 19:47:47 dobby kernel: epair1b: link state changed to DOWN
Nov  4 19:47:47 dobby kernel: epair1b: link state changed to DOWN
Nov  4 19:47:47 dobby kernel: ifa_maintain_loopback_route: deletion failed for interface epair1b: 3
Nov  4 19:47:48 dobby kernel: ifa_maintain_loopback_route: deletion failed for interface lo0: 48
Nov  4 19:47:49 dobby epair1a: Ethernet address: 02:ff:e0:00:05:0a
Nov  4 19:47:49 dobby epair1b: Ethernet address: 02:ff:30:00:06:0b
Nov  4 19:47:49 dobby kernel: epair1a: link state changed to UP
Nov  4 19:47:49 dobby kernel: epair1a: link state changed to UP
Nov  4 19:47:49 dobby kernel: epair1b: link state changed to UP
Nov  4 19:47:49 dobby kernel: epair1b: link state changed to UP
Nov  4 19:47:49 dobby kernel: epair1a: promiscuous mode enabled
Nov  4 19:51:15 dobby kernel: epair1a: link state changed to DOWN
Nov  4 19:51:15 dobby kernel: epair1a: link state changed to DOWN
Nov  4 19:51:15 dobby kernel: epair1b: link state changed to DOWN
Nov  4 19:51:15 dobby kernel: epair1b: link state changed to DOWN
Nov  4 19:51:15 dobby kernel: ifa_maintain_loopback_route: deletion failed for interface epair1b: 3
Nov  4 19:51:16 dobby kernel: ifa_maintain_loopback_route: deletion failed for interface lo0: 48
Nov  4 19:51:17 dobby epair1a: Ethernet address: 02:ff:e0:00:05:0a
Nov  4 19:51:17 dobby epair1b: Ethernet address: 02:ff:30:00:06:0b
Nov  4 19:51:17 dobby kernel: epair1a: link state changed to UP
Nov  4 19:51:17 dobby kernel: epair1a: link state changed to UP
Nov  4 19:51:17 dobby kernel: epair1b: link state changed to UP
Nov  4 19:51:17 dobby kernel: epair1b: link state changed to UP
Nov  4 19:51:17 dobby kernel: epair1a: promiscuous mode enabled
Nov  4 19:55:28 dobby kernel: epair1a: link state changed to DOWN
Nov  4 19:55:28 dobby kernel: epair1a: link state changed to DOWN
Nov  4 19:55:28 dobby kernel: epair1b: link state changed to DOWN
Nov  4 19:55:28 dobby kernel: epair1b: link state changed to DOWN
Nov  4 19:55:28 dobby kernel: ifa_maintain_loopback_route: deletion failed for interface epair1b: 3
Nov  4 19:55:30 dobby kernel: ifa_maintain_loopback_route: deletion failed for interface lo0: 48
Nov  4 19:55:35 dobby epair1a: Ethernet address: 02:ff:e0:00:05:0a
Nov  4 19:55:35 dobby epair1b: Ethernet address: 02:ff:30:00:06:0b
Nov  4 19:55:35 dobby kernel: epair1a: link state changed to UP
Nov  4 19:55:35 dobby kernel: epair1a: link state changed to UP
Nov  4 19:55:35 dobby kernel: epair1b: link state changed to UP
Nov  4 19:55:35 dobby kernel: epair1b: link state changed to UP
Nov  4 19:55:35 dobby kernel: epair1a: promiscuous mode enabled
Nov  4 20:02:16 dobby kernel: Limiting closed port RST response from 980 to 200 packets/sec
Nov  4 20:02:16 dobby kernel: Limiting closed port RST response from 980 to 200 packets/sec
Nov  4 20:02:18 dobby kernel: Limiting closed port RST response from 441 to 200 packets/sec
Nov  4 20:02:18 dobby kernel: Limiting closed port RST response from 441 to 200 packets/sec
Nov  4 20:02:19 dobby kernel: Limiting closed port RST response from 800 to 200 packets/sec
Nov  4 20:02:19 dobby kernel: Limiting closed port RST response from 800 to 200 packets/sec

I believe "kernel: Limiting closed port RST response from 800 to 200 packets/sec" is from me attempting to setup UrBackup jail following the steps provided by UrBackup. Which does not appear to be working.

IceBoosteR · Nov 4, 2017

Short explanation of what is what:
https://blog.plitc.eu/2014/freebsd-10-vimage-virtualized-network-stack-mit-if_bridge-epair/
http://shawndebnath.com/articles/2016/03/27/freebsd-jails-with-vlan-howto.html
If a epair goes up or down, the connection will be dropped for about 1 sec, because new routes are built internally.

zey · Nov 4, 2017

IceBoosteR,

Thanks for the quick reply. However I went to the two links provided. One is in I believe Polish and the other one well they both provide way more information than I believe I am looking for. I just want to stop the intermittent loss of connection.

IceBoosteR · Nov 4, 2017

zey said:
IceBoosteR,

Thanks for the quick reply. However I went to the two links provided. One is in I believe Polish and the other one well they both provide way more information than I believe I am looking for. I just want to stop the intermittent lost of connection.

Haha, alright ;)
So do you have any jails and/or plugins running in your FreeNAS VM?
If not, epairs should not be touched in any way, which is causing the problem.
What is it normally looks like when you start/stop jails:

Code:

epair0a: promiscuous mode enabled
em0: link state changed to UP
epair1a: Ethernet address: 02:ff:e0:00:05:0a
epair1b: Ethernet address: 02:ff:30:00:06:0b
epair1a: link state changed to UP
epair1b: link state changed to UP
epair1a: promiscuous mode enabled
epair2a: Ethernet address: 02:ff:e0:00:06:0a
epair2b: Ethernet address: 02:ff:30:00:07:0b
epair2a: link state changed to UP
epair2b: link state changed to UP
epair2a: promiscuous mode enabled
epair0a: link state changed to DOWN
epair0b: link state changed to DOWN
ifa_maintain_loopback_route: deletion failed for interface epair0b: 3
ifa_maintain_loopback_route: deletion failed for interface lo0: 48
arp: 192.XXX.XXX.XXX moved from 02:ff:e0:00:06:0a to xx:xx:xx:xx:xx:xx

Maybe you can share the output of "ifconfig" and a screenshot from the GUI?

zey · Nov 4, 2017

I do have two jails. The first is SABnzbd which appears to be working fine. The second is a jail I am attempting to setup for UrBackup. I followed their howto but there seems to be missing some steps because the error looks like the port isn't open.

Code:

igb0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500												 
		options=2400b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,RXCSUM_IPV6>								 
		ether 00:25:90:5d:fe:3f																									 
		inet 10.10.11.11 netmask 0xffffff00 broadcast 10.10.11.255																 
		nd6 options=9<PERFORMNUD,IFDISABLED>																						
		media: Ethernet autoselect (1000baseT <full-duplex>)																		
		status: active																											 
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384																   
		options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>																	   
		inet6 ::1 prefixlen 128																									 
		inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2																				 
		inet 127.0.0.1 netmask 0xff000000																						   
		nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>																				   
		groups: lo																												 
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500													   
		ether 02:9b:ea:41:dd:00																									 
		nd6 options=1<PERFORMNUD>																								   
		groups: bridge																											 
		id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15																 
		maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200																	
		root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0																	
		member: epair1a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>															   
				ifmaxaddr 0 port 5 priority 128 path cost 2000																	 
		member: epair0a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>															   
				ifmaxaddr 0 port 4 priority 128 path cost 2000																	 
		member: igb0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>																 
				ifmaxaddr 0 port 1 priority 128 path cost 20000																	 
epair0a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500											   
		options=8<VLAN_MTU>																										 
		ether 02:ff:e0:00:04:0a																									 
		nd6 options=1<PERFORMNUD>																								   
		media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)																		 
		status: active																											 
		groups: epair																											   
epair1a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500											   
		options=8<VLAN_MTU>																										 
		ether 02:ff:e0:00:05:0a																									 
		nd6 options=1<PERFORMNUD>																								   
		media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)																		 
		status: active																											 
		groups: epair

Also I have two SFP ports and plan on adding them to the VM via pass-through. I haven't yet because I'm afraid it will just add issues to the ones I already have.

zey · Nov 8, 2017

After a few days of rocking my head. I decided to remove the pass-through. I have a strong feeling it has something to do with it. I also realized the on-board SFP+ is an Intel x550 so no FreeBSD support as of yet. So I added an other uplink to the virtual switch in ESXi.

Hopefully this will resolve my issues for now.

Reset to factory default kept em0 and added igb0. Set a static IP and everything is working now. Not sure what I did that resolved the issue but it's done!

Important Announcement for the TrueNAS Community.

SOLVED Intermittent loss of connection

zey

Explorer

IceBoosteR

Guru

zey

Explorer

IceBoosteR

Guru

zey

Explorer

zey

Explorer

Similar threads