link state keeps changing between DOWN and UP

[dradovsky]

A few weeks ago I started receiving local security run output messages like this:

Code:

freenas.local kernel log messages:
> em0: link state changed to DOWN
> em0: link state changed to UP
> em0: link state changed to DOWN
> em0: link state changed to UP
> em0: link state changed to DOWN
> em0: link state changed to UP
> em0: link state changed to DOWN
> em0: link state changed to UP
> em0: link state changed to DOWN
> em0: link state changed to UP
> em0: link state changed to DOWN
> em0: link state changed to UP
> em0: link state changed to DOWN
> em0: link state changed to UP
> em0: link state changed to DOWN
> em0: link state changed to UP
> em0: link state changed to DOWN
> em0: link state changed to UP
> em0: link state changed to DOWN
> em0: link state changed to UP
> em0: link state changed to DOWN
> em0: link state changed to UP
> em0: link state changed to DOWN
> em0: link state changed to UP
> em0: link state changed to DOWN
> em0: link state changed to UP

-- End of security output --

I did a google search and came across a FreeNAS forum conversation(https://forums.freenas.org/index.php?threads/security-run-output.18309/) that suggested my NIC or a bad cable was causing this. My NIC was an Intel Pro/1000 GT Desktop Adapter. After first trying a different cable and switching ports on my Netgear GS108v2 network switch to see if that made a difference (it didn't), I replaced the NIC with the same model. After a few days, the problem began again.

One thing I did notice was the security run output first shows about six lines of down and ups, then later shows about 24 lines, then later still shows many more than that. And soon thereafter, I get a message from my Mac's Time Machine backup application that it can't make contact.

I am running FreeNAS-9.2.0-RELEASE-x64 (ab098f4) and have never had a problem since installing it about two years ago.

My platform is Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz with 4 Gigs of ram.

Any suggestions?

Thanks,
Dan

 

[joeschmuck]

Dan,
The problem I see here is that you replaced the NIC once and the problem went away for a while but then came back. In an effort to troubleshoot this problem try these steps...

1) Power off the NAS for ~15 minutes and then power it back on. How long before the problems return?
2) If the problem go away for a while (a day or longer and then return), then Reboot the system. If the problems return quickly go step 4. How long before the problems return?
3) If the problems return in a similar time as the power off then I would suspect a software issue. Hopefully you have an older copy of your configuration file laying around, I wouldn't use one you created since the problems started. Use a new USB flash boot device and install your version of FreeNAS onto the device and restore your old configuration file. See if the problem happens again.
4) If the problem occurs yet again, I'd be looking at possibly your FreeNAS computer power supply or fan failures, heat buildup, etc... It could even be the MB but I doubt it.

Now lets quickly address the last thing, that something on your network, could be the time machine software is causing an issue with the protocol. It can happen but normally because of a bug. Since you are running an old version of FreeNAS, this is possible but I wouldn't jump to it right off the bat, however if you made a change just before this problem started, well just remember to think outside the box.

 

[dradovsky]

Dan,
The problem I see here is that you replaced the NIC once and the problem went away for a while but then came back. In an effort to troubleshoot this problem try these steps...

1) Power off the NAS for ~15 minutes and then power it back on. How long before the problems return?
2) If the problem go away for a while (a day or longer and then return), then Reboot the system. If the problems return quickly go step 4. How long before the problems return?
3) If the problems return in a similar time as the power off then I would suspect a software issue. Hopefully you have an older copy of your configuration file laying around, I wouldn't use one you created since the problems started. Use a new USB flash boot device and install your version of FreeNAS onto the device and restore your old configuration file. See if the problem happens again.
4) If the problem occurs yet again, I'd be looking at possibly your FreeNAS computer power supply or fan failures, heat buildup, etc... It could even be the MB but I doubt it.

Now lets quickly address the last thing, that something on your network, could be the time machine software is causing an issue with the protocol. It can happen but normally because of a bug. Since you are running an old version of FreeNAS, this is possible but I wouldn't jump to it right off the bat, however if you made a change just before this problem started, well just remember to think outside the box.

Thanks Joe, I'll do as you suggest and report back.

 

[Patrick Linley]

Mine started doing this a few weeks ago... sometimes it is just a quick up/down/up, but it's doing it at the same time every day - 3:01am - latest release.

 

[Peter Hood]

Mine started doing this a few weeks ago... sometimes it is just a quick up/down/up, but it's doing it at the same time every day - 3:01am - latest release.

Same here, even exact same time as on Patrick's System. - 03:01am

 

[wblock]

You are responding to a thread that is old, so "latest release" is not clear. The hardware is not described, either. 3AM is when daily system reports are generated. That could still be a hardware problem, a shaky power supply that is having trouble with the sudden demand for power from disks.

For now, please describe the hardware and the exact version number.

 

[Andreas Wideroe]

I seem to have the same symptons on my HP Proliant ML110 g6 running Freenas 11 RC4:

Code:

bge0: link state changed to DOWN												
bge0: link state changed to UP												
bge0: watchdog timeout -- resetting											
bge0: link state changed to DOWN												
bge0: link state changed to UP												
bge0: watchdog timeout -- resetting											
bge0: link state changed to DOWN												
bge0: link state changed to UP

Seems to happen when there is a certain load on the system or when I click some buttons too quick in the gui.

I have a second NIC in the machine Intel 1Gbps adapter em0 which I will try to active tomorrow.

The problem is quite annoying. Anyone figured out a solution or cause yet?

Thanks,
Andreas

 

[Andreas Wideroe]

Changing NIC and cable helps a bit it seems, but it still goes down and seems to be related to load. I know my server is not the fastest one. I only use it for testing purposes, still I find it a bit odd that the network goes down like this.

/Andreas

 

[joeschmuck]

Have you done a direct connection between your FreeNAS computer and your controlling computer and watched to see if the connections continue to go up/down ?

Changing NIC and cable helps a bit it seems, but it still goes down and seems to be related to load.

Specifically what NIC are you using (make/model). Also, what is you wiring connectivity? (what is the ethernet cable connected to?)

You need to try to isolate the problem to hardware by eliminating all the hardware you can. If you are connected to an ethernet switch, replace the switch or connect the ethernet to the a computer directly. It's that type of stuff.

 

[silentflight15]

I'll add that I randomly had this happen this morning as well during a network file transfer. Freenas 11.0-RELEASE using just the MSI motherboard ethernet, but I've never seen this before including during much heavier loads I have to believe.

 

[wblock]

Please start new threads for different hardware. And be specific about exactly which hardware you have, in particular the network card and the switch.