Time Out

[MikeSuede]

Having a weird issue with the clock on my server. I'm set to the correct time zone and after a reset of the clock or a reboot of the system, it shows the correct time but only for a few days. Then slows down anywhere from 50 minutes to 2 and a half hours (it's different each time). I haven't received error messages regarding the clock. Strange, I guess... So, just wanted to know if anyone else ever experienced this problem and if so, how did you deal with it.

Mike

 

[DrKK]

This shouldn't be happening. FreeNAS servers contact ntp servers to keep the time correct. When the clock is screwed up, can you show me the output to this:

Code:

ntpq -np

 

[cyberjock]

And your hardware please...

 

[MikeSuede]

This shouldn't be happening. FreeNAS servers contact ntp servers to keep the time correct. When the clock is screwed up, can you show me the output to this:

Code:

ntpq -np

This is what I get.
[root@freenas ~]# ntpq -np
ntpq: read: Connection refused
[root@freenas ~]#

 

[MikeSuede]

And your hardware please...

My hardware is listed in my signature. I hope it's viewable.

HP Z600 / FreeNAS 9.3-Stable / Intel(R) Xeon(R) CPU E5506 @ 2.13 GHz / 24533MB DDR3 ECC / Western Digital Red 3TB x 6 / ScanDisc Cruzer Fit 16GB x 2

 

[DrKK]

This is what I get.
[root@freenas ~]# ntpq -np
ntpq: read: Connection refused
[root@freenas ~]#

well I think we know what the problem is.

:)

Go into System->General->NTP servers, and screen shot that for me (or, tell me what servers are configured)

Also, drop to the shell, and see if either, both, or none of the following work:

Code:

ping 8.8.8.8

Code:

dig www.google.com

We should have this solved fairly quickly; you almost certainly just have a minor config or network issue.

 

[MikeSuede]

well I think we know what the problem is.

:)

Go into System->General->NTP servers, and screen shot that for me (or, tell me what servers are configured)

Also, drop to the shell, and see if either, both, or none of the following work:

Code:

ping 8.8.8.8

Code:

dig www.google.com

We should have this solved fairly quickly; you almost certainly just have a minor config or network issue.

Here's the screen shot. As for the ping 8.8.8.8, I guess it's working because it's still running. When it finishes, I'll try the second one.
Ok, dig www.google.com works also.

 

[Ericloewe]

NTP issues aside, a ~1hr clock drift in a spawn of a few days screams serious issues with the system's Real Time Clock.

Though it would be weird (RTCs are supposed to draw power from mains when available), a battery nearing the end of its life could be causing this. I recommend replacing the CR3220 in your motherboard with a new one, since it's a simple, cheap troubleshooting step.

 

[MikeSuede]

NTP issues aside, a ~1hr clock drift in a spawn of a few days screams serious issues with the system's Real Time Clock.

Though it would be weird (RTCs are supposed to draw power from mains when available), a battery nearing the end of its life could be causing this. I recommend replacing the CR3220 in your motherboard with a new one, since it's a simple, cheap troubleshooting step.

Thanks, I thought if it was the battery, then it never would have never started out correct. Well, I will pick-up a replacement today just in case.

 

[DrKK]

but you're still saying, you drop to shell, and you can't do this:

Code:

[root@giskard] ~# ntpq -np
  remote  refid  st t when poll reach  delay  offset  jitter
==============================================================================
+4.53.160.75  142.66.101.13  2 u  476 1024  377  29.567  -3.001  0.945
+107.170.224.8  47.187.174.51  2 u  697 1024  377  77.700  -1.070  1.446
*129.6.15.28  .ACTS.  1 u  501 1024  377  69.776  27.468  1.871

 

[MikeSuede]

but you're still saying, you drop to shell, and you can't do this:

Code:

[root@giskard] ~# ntpq -np
  remote  refid  st t when poll reach  delay  offset  jitter
==============================================================================
+4.53.160.75  142.66.101.13  2 u  476 1024  377  29.567  -3.001  0.945
+107.170.224.8  47.187.174.51  2 u  697 1024  377  77.700  -1.070  1.446
*129.6.15.28  .ACTS.  1 u  501 1024  377  69.776  27.468  1.871

No, unfortunately I still get this.

Code:

[root@freenas ~]# ntpq -np                                                     
ntpq: read: Connection refused                                                 
[root@freenas ~]#    

 

[jgreco]

FreeBSD is well known to have problems with the clock on various platforms, which are broken in fun and entertaining ways - side effect of the wonderful world of PC's. The usual fix is to change the timecounter being used.

But first it would be a good idea to debug ntpd and see if it is actually bailing, or not configured correctly in the first place. Can you reboot the system, make sure ntpd is running with the correct time, and then keep a careful eye out for any log messages from ntpd? For bonus points you can check with ntpq every few hours to see if there's a significant offset.

It should be a fixable problem.

 

[jgreco]

Also post the results from

Code:

~# sysctl kern.timecounter.choice
kern.timecounter.choice: TSC-low(1000) ACPI-fast(900) HPET(950) i8254(0) dummy(-1000000)
~# sysctl kern.timecounter.hardware
kern.timecounter.hardware: TSC-low
~#

 

[DrKK]

it's very odd that ntpd is bailing.

What happens when you manually restart it?

Code:

service ntpd restart

In your main system configuration screen, did you set your timezone?

 

[jgreco]

it's very odd that ntpd is bailing.

The heck it is. Have you *WORKED* with PC hardware?!?!

What happens when you manually restart it?

Code:

service ntpd restart

In your main system configuration screen, did you set your timezone?

I'd rather see it after a system reboot, does the same thing plus some other things that are immediately helpful. The timezone setting is irrelevant; it is primarily a session affectation, when the time is retrieved via gettimeofday(2). The system keeps track of time in a timezone-agnostic manner.

 

[MikeSuede]

it's very odd that ntpd is bailing.

What happens when you manually restart it?

Code:

service ntpd restart

In your main system configuration screen, did you set your timezone?

Code:

[root@freenas ~]# service ntpd restart                                        
ntpd not running? (check /var/run/ntpd.pid).                                  
Starting ntpd.                                                                
[root@freenas ~]#   

And yes I did set my timezone during my initial config.

Ok, the manual restart worked and here are the results.

Code:

[root@freenas ~]# ntpq -np                                                     
     remote           refid      st t when poll reach   delay   offset  jitter 
============================================================================== 
*50.116.38.157   138.39.23.13     2 u   27   64   77   31.199   -3.162   3.723 
+50.22.155.163   209.51.161.238   2 u   30   64   77   85.816   -9.368   3.851 
+142.54.181.202  64.113.32.5      2 u    9   64   77   47.427   -3.185  10.733 
[root@freenas ~]#   

 

[MikeSuede]

FreeBSD is well known to have problems with the clock on various platforms, which are broken in fun and entertaining ways - side effect of the wonderful world of PC's. The usual fix is to change the timecounter being used.

But first it would be a good idea to debug ntpd and see if it is actually bailing, or not configured correctly in the first place. Can you reboot the system, make sure ntpd is running with the correct time, and then keep a careful eye out for any log messages from ntpd? For bonus points you can check with ntpq every few hours to see if there's a significant offset.

It should be a fixable problem.

Alright, system rebooted (also changed motherboard battery as per Ericloewe, just in case).
Ntpq is running.

Code:

[root@freenas ~]# ntpq -np                                                     
     remote           refid      st t when poll reach   delay   offset  jitter 
============================================================================== 
+104.131.51.97   199.102.46.70    2 u   45   64  377   16.350    8.705   7.782 
+172.82.134.51   152.2.133.54     2 u   16   64  377   36.852   19.717   4.234 
*67.18.187.111   129.7.1.66       2 u   29   64  377   61.597   21.790   4.894 
[root@freenas ~]#   

And I will check for any log messages regarding ntpq or offsets (I like bonus points... Lol)

 

[MikeSuede]

Also post the results from

Code:

~# sysctl kern.timecounter.choice
kern.timecounter.choice: TSC-low(1000) ACPI-fast(900) HPET(950) i8254(0) dummy(-1000000)
~# sysctl kern.timecounter.hardware
kern.timecounter.hardware: TSC-low
~#

Code:

[root@freenas ~]# sysctl kern.timecounter.choice                               
kern.timecounter.choice: TSC(1000) ACPI-fast(900) i8254(0) dummy(-1000000)     
[root@freenas ~]# sysctl kern.timecounter.hardware                             
kern.timecounter.hardware: TSC                                                 
[root@freenas ~]#      

 

[MikeSuede]

It's been only a few hours and I'm outta wack again (first time it's ever happen this fast). The clock is off about 1 hour and 10 minutes. Ntpq error message showed this:

Code:

Oct 29 02:57:59 freenas ntpd[2190]: time reset -0.131017 s
Oct 29 03:56:28 freenas ntpd[2190]: time correction of 5833 seconds exceeds sanity limit (1000); set clock manually to the correct UTC time.

and of course, it stopped running.

 

[Ericloewe]

It's been only a few hours and I'm outta wack again (first time it's ever happen this fast). The clock is off about 1 hour and 10 minutes. Ntpq error message showed this:

Code:

Oct 29 02:57:59 freenas ntpd[2190]: time reset -0.131017 s
Oct 29 03:56:28 freenas ntpd[2190]: time correction of 5833 seconds exceeds sanity limit (1000); set clock manually to the correct UTC time.

and of course, it stopped running.

Not knowing much about ntpd, it seems to be reacting appropriately to serious issues with the RTC. It sounds like a hardware problem.