Locking up periodically and gaps in report info

[Dusan]

I hate to say me too, but I also have seen gaps in my reporting graphs, I haven't had the time to figure out why though,

Not all reporting gaps are a reason for worry. The FreeNAS GUI/middleware will restart collectd for various reasons causing small gaps. It is normal to see reporting gaps when you: create a dataset, import/unlock a pool, delete a snapshot, ...

 

[StevenA]

Not all reporting gaps are a reason for worry. The FreeNAS GUI/middleware will restart collectd for various reasons causing small gaps. It is normal to see reporting gaps when you: create a dataset, import/unlock a pool, delete a snapshot, ...

BINGO!

Looked at my log and collectd was restarted in the one gap I saw this morning.

I wasn't worried about it yet as it didn't seem to effect anything, been far too busy with other things then figuring out things that were not causing ill effects.

Steven

 

[steve6341]

New motherboard (S5000PSLSAS), different processors (2 x Intel E5430), different USB drive with a clean install of FreeNAS-9.2.0-RELEASE-x64. Same issues.
System time is all over the place, gaps in the graphs, and (most importantly) shares eventually become unresponsive until you either open the web interface or SSH to the system. Does anyone have any other suggestions? Should my Wired Memory always consume nearly all of my ram? There is only about 2GB free, the rest of the 32GB is "Wired." The "Swap Utilization" graph always shows 14GB Free and oGB Used. Should the swap be getting used? I have been reading about using a dedicated SSD drive for the ZIL log. Should I try that? Or disabling sync writes in NFS?
I'm running out of ideas on this... I am in a jam now because I have several projects that are nearly 2 months behind because I am working on getting this storage reliable. (I realize no one here is responsible for that or is really all that concerned) Any suggestions would be greatly appreciated.

 

[DJ9]

Personally, I would have went with a Supermicro motherboard vs some old discontinued Intel motherboard.

Considering Intel is getting out of the motherboard business.

Just saying. ;)

 

[DrKK]

In answer to some of your questions, sir:

1) The "Swap" would not be utilized in most cases. 0 is normal.
2) Most of your RAM *will* be wired by the read cache, that *is* normal. It'll probably grow until it has like all but ~1GB of RAM in its ARC.

I'm not sure what else to tell you about your ongoing problems.

 

[steve6341]

Well, technically they are getting out of the desktop motherboard business; they intend to continue manufacturing server motherboards. The S5000 is a pretty solid server-class motherboard, even if it is discontinued. Good, bad, or otherwise this is the system I am working with. I would have a hard time convincing my superiors to purchased a pile of new equipment to run an open-source system that, so far, I can't prove is all that reliable. That is in no way a criticism of FreeNAS or the people that put all the work into making it. I was impressed with the product enough to choose it early on in this project. I am trying my best to stick with it and make it work. But I digress...

 

[cyberjock]

Well, technically they are getting out of the desktop motherboard business; they intend to continue manufacturing server motherboards.

No, what I read is that Haswell is the end of the road for all motherboard manufacturing. I'm not the least bit surprised because quite a few people have random unexplainable problems with Intel boards that are clearly repeatable on Intel boards, but not on other boards that are similar but my other manufacturers.

You aren't the first to complain about the S5000 here, and you definitely won't be the last. I've been accused more than once of being an Intel shill. The reality is that while I love many of their products, I do not ever endorse their motherboards. They are weird, and they often do unexplained things. Try to find documentation that the S5000 board supports FreeBSD. Strangely, I haven't ever found anything saying it is. So that *could* be part of the problem.

And all that stuff DrKK said is completely true.

A ZIL or L2ARC isn't going to help with your problems.

You might want to examine your BIOS settings and see if they are correct. Maybe look for a BIOS update.

Edit: and a Google search seems to add confusion to this mess. There's a bunch of websites that say "desktop motherboard division" and others that say "motherboard division". /facepalm

 

[steve6341]

One last post (from me) and I will let this thread die gracefully. But just in case it helps someone else: I loaded FreeNAS 8.3.2 and all of my problems appear to have gone away. I tried a variety of other things since my last post: I researched/changed every setting in the BIOS, tried a different brand add-on network card, different power supply, only running with 1 processor installed, several Tunables including disabling ACPI, and chased interrupt storms (because I found something interesting about it in the FreeBSD Handbook). None of that worked. With 8.3.2 I have moved just over 2TB of data back and everything looks good. So if you happen to be using "some old discontinued Intel motherboard" the latest version of FreeNAS may not be for you. Thanks (everyone) for all the feedback.

 

[rm-r]

In summary.... i too tried 8.3.2 and my problems went away - I couldn't import my main pool because it was created on 9.1.1 and wont import as newer zfs version. i did create a small test pool, with shares and all was well.

I also noticed a MASSIVE improvement if I disabled autotune....

I have tried right up to the latest 9.2.1 alpha but still issues

I'm waiting delivery of.....

SUPERMICRO MBD-X9SCA-F-O LGA 1155 Intel C204 ATX Intel Xeon E3 Server Motherboard

Intel E3-1230V2 LGA1155 Processor BX80637E31230V2

Kingston KVR1333D3E9SK2/16G 16GB 1333MHz CL9 DDR3 ValueRAM ECC Memory Kit

fingers crossed this sorts it...... us$600 worth! (Thanks cyberjock)

 

[cyberjock]

Well, since autotune is for 128GB+ systems... duh!?

 

[rm-r]

cyberjock

Well, since autotune is for 128GB+ systems... duh!?

i did RTFM before applying it and it makes no mention of that requirement.. - it mentions "Limited ram" so thought it may help especially as "If you are trying to increase the performance of your FreeNAS® system and suspect that the current hardware may be limiting performance, try enabling autotune."

http://doc.freenas.org/index.php/Settings#Autotune
Autotune

FreeNAS® provides an autotune script which attempts to optimize the system depending upon the hardware which is installed. For example, if a ZFS volume exists on a system with limited RAM, the autotune script will automatically adjust some ZFS sysctl values in an attempt to minimize ZFS memory starvation issues.
The "Enable autotune" checkbox in System → Settings → Advanced is unchecked by default; check it if you would like the autotuner to run at boot time. If you would like the script to run immediately, reboot the system.
If autotuner finds any settings that need adjusting, the changed values will appear in System → Sysctls (for sysctl.conf values) and in System → Tunables (for loader.conf values). If you do not like the changes, you can modify the values that are displayed in the GUI and your changes will override the values that were created by the autotune script. However, if you delete a sysctl or tunable that was created by autotune, it will be recreated at next boot. This is because autotune only creates values that do not already exist.
If you are trying to increase the performance of your FreeNAS® system and suspect that the current hardware may be limiting performance, try enabling autotune.
If you wish to read the script to see which checks are performed, the script is located in /usr/local/bin/autotune.

 

[cyberjock]

Oh.. I totally agree with you rm-r. The manual makes no mention of it. I'll update it right now....

 

[cyberjock]

Updated... this is added

NOTE: Autotune is designed to optimize systems that have >64GB of RAM. Between 64 and 128GB of RAM your results may vary. For systems with 128GB+ of RAM you can expect to see a performance improvement with autotune enabled. Autotune will hurt performance on systems with <64GB of RAM and should not be used.

 

[fracai]

That's good to know. I've definitely seen auto tune stated on the forums as being a good idea for everyone.

 

[cyberjock]

Yeah, I think that info was started by me. It was incorrect. :(

 

[Nate]

Interesting that I also have a realtek NIC. I will go and look at my graphs when I have the chance to see if I have the same odd behavior. I reported something similar in this thread: http://forums.freenas.org/threads/freenas-continually-dropping.17545/#post-94076



Even more interesting is the fact that in that same gap I have this show up in my console message window (from the webgui). It almost perfectly matches up to the same gap timeframe.

Code:

Jan 12 12:36:50 freenas newsyslog[1725]: logfile first created
Jan 12 12:36:50 freenas syslogd: kernel boot file is /boot/kernel/kernel
Jan 12 12:36:50 freenas kernel: Copyright (c) 1992-2013 The FreeBSD Project.
Jan 12 12:36:50 freenas kernel: Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
Jan 12 12:36:50 freenas kernel: The Regents of the University of California. All rights reserved.
Jan 12 12:36:50 freenas kernel: FreeBSD is a registered trademark of The FreeBSD Foundation.
Jan 12 12:36:50 freenas kernel: FreeBSD 9.2-RELEASE #0 r+2315ea3: Fri Dec 20 12:48:50 PST 2013
Jan 12 12:36:50 freenas kernel: root@build.ixsystems.com:/tank/home/jkh/checkout/freenas/os-base/amd64/tank/home/jkh/checkout/freenas/FreeBSD/src/sys/FREENAS.amd64 amd64
Jan 12 12:36:50 freenas kernel: gcc version 4.2.1 20070831 patched [FreeBSD]
Jan 12 12:36:50 freenas kernel: CPU: Intel(R) Core(TM) i3-3240 CPU @ 3.40GHz (3392.37-MHz K8-class CPU)
Jan 12 12:36:50 freenas kernel: Origin = "GenuineIntel"  Id = 0x306a9  Family = 0x6  Model = 0x3a  Stepping = 9
Jan 12 12:36:50 freenas kernel: Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Jan 12 12:36:50 freenas kernel: Features2=0x3d9ae3bf<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,POPCNT,TSCDLT,XSAVE,OSXSAVE,AVX,F16C>
Jan 12 12:36:50 freenas kernel: AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
Jan 12 12:36:50 freenas kernel: AMD Features2=0x1<LAHF>
Jan 12 12:36:50 freenas kernel: Standard Extended Features=0x281<GSFSBASE,SMEP,ENHMOVSB>
Jan 12 12:36:50 freenas kernel: TSC: P-state invariant, performance statistics
Jan 12 12:36:50 freenas kernel: real memory  = 17179869184 (16384 MB)
Jan 12 12:36:50 freenas kernel: avail memory = 16217579520 (15466 MB)
Jan 12 12:36:50 freenas kernel: Event timer "LAPIC" quality 600
Jan 12 12:36:50 freenas kernel: ACPI APIC Table: <ALASKA A M I>
Jan 12 12:36:50 freenas kernel: FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
Jan 12 12:36:50 freenas kernel: FreeBSD/SMP: 1 package(s) x 2 core(s) x 2 SMT threads
Jan 12 12:36:50 freenas kernel: cpu0 (BSP): APIC ID:  0
Jan 12 12:36:50 freenas kernel: cpu1 (AP): APIC ID:  1
Jan 12 12:36:50 freenas kernel: cpu2 (AP): APIC ID:  2
Jan 12 12:36:50 freenas kernel: cpu3 (AP): APIC ID:  3
Jan 12 12:36:50 freenas kernel: WARNING: VIMAGE (virtualized network stack) is a highly experimental feature.
Jan 12 12:36:50 freenas kernel: ACPI Warning: FADT (revision 5) is longer than ACPI 2.0 version, truncating length 268 to 244 (20110527/tbfadt-320)
Jan 12 12:36:50 freenas kernel: ioapic0 <Version 2.0> irqs 0-23 on motherboard
Jan 12 12:36:50 freenas kernel: kbd1 at kbdmux0
Jan 12 12:36:50 freenas kernel: cryptosoft0: <software crypto> on motherboard
Jan 12 12:36:50 freenas kernel: aesni0: No AESNI support.
Jan 12 12:36:50 freenas kernel: padlock0: No ACE support.
Jan 12 12:36:50 freenas kernel: acpi0: <ALASKA A M I> on motherboard
Jan 12 12:36:50 freenas kernel: acpi0: Power Button (fixed)
Jan 12 12:36:50 freenas kernel: acpi0: reservation of 67, 1 (4) failed
Jan 12 12:36:50 freenas kernel: cpu0: <ACPI CPU> on acpi0
Jan 12 12:36:50 freenas kernel: cpu1: <ACPI CPU> on acpi0
Jan 12 12:36:50 freenas kernel: cpu2: <ACPI CPU> on acpi0
Jan 12 12:36:50 freenas kernel: cpu3: <ACPI CPU> on acpi0
Jan 12 12:36:50 freenas kernel: hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Jan 12 12:36:50 freenas kernel: Timecounter "HPET" frequency 14318180 Hz quality 950
Jan 12 12:36:50 freenas kernel: Event timer "HPET" frequency 14318180 Hz quality 550
Jan 12 12:36:50 freenas kernel: Event timer "HPET1" frequency 14318180 Hz quality 440
Jan 12 12:36:50 freenas kernel: Event timer "HPET2" frequency 14318180 Hz quality 440
Jan 12 12:36:50 freenas kernel: Event timer "HPET3" frequency 14318180 Hz quality 440
Jan 12 12:36:50 freenas kernel: Event timer "HPET4" frequency 14318180 Hz quality 440
Jan 12 12:36:50 freenas kernel: atrtc0: <AT realtime clock> port 0x70-0x77 irq 8 on acpi0
Jan 12 12:36:50 freenas kernel: atrtc0: Warning: Couldn't map I/O.
Jan 12 12:36:50 freenas kernel: Event timer "RTC" frequency 32768 Hz quality 0
Jan 12 12:36:50 freenas kernel: attimer0: <AT timer> port 0x40-0x43,0x50-0x53 irq 0 on acpi0
Jan 12 12:36:50 freenas kernel: Timecounter "i8254" frequency 1193182 Hz quality 0
Jan 12 12:36:50 freenas kernel: Event timer "i8254" frequency 1193182 Hz quality 100
Jan 12 12:36:50 freenas kernel: Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
Jan 12 12:36:50 freenas kernel: acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
Jan 12 12:36:50 freenas kernel: pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
Jan 12 12:36:50 freenas kernel: pci0: <ACPI PCI bus> on pcib0
Jan 12 12:36:50 freenas kernel: vgapci0: <VGA-compatible display> port 0xf000-0xf03f mem 0xf7800000-0xf7bfffff,0xe0000000-0xefffffff irq 16 at device 2.0 on pci0
Jan 12 12:36:50 freenas kernel: agp0: <IvyBridge desktop GT1 IG> on vgapci0
Jan 12 12:36:50 freenas kernel: agp0: aperture size is 256M, detected 262140k stolen memory
Jan 12 12:36:50 freenas kernel: pci0: <simple comms> at device 22.0 (no driver attached)
Jan 12 12:36:50 freenas kernel: ehci0: <EHCI (generic) USB 2.0 controller> mem 0xf7c08000-0xf7c083ff irq 16 at device 26.0 on pci0
Jan 12 12:36:50 freenas kernel: usbus0: EHCI version 1.0
Jan 12 12:36:50 freenas kernel: usbus0 on ehci0
Jan 12 12:36:50 freenas kernel: pci0: <multimedia, HDA> at device 27.0 (no driver attached)
Jan 12 12:36:50 freenas kernel: pcib1: <ACPI PCI-PCI bridge> irq 16 at device 28.0 on pci0
Jan 12 12:36:50 freenas kernel: pci1: <ACPI PCI bus> on pcib1
Jan 12 12:36:50 freenas kernel: pcib2: <ACPI PCI-PCI bridge> irq 19 at device 28.3 on pci0
Jan 12 12:36:50 freenas kernel: pci2: <ACPI PCI bus> on pcib2
Jan 12 12:36:50 freenas kernel: re0: <RealTek 8168/8111 B/C/CP/D/DP/E/F/G PCIe Gigabit Ethernet> port 0xe000-0xe0ff mem 0xf0004000-0xf0004fff,0xf0000000-0xf0003fff irq 19 at device 0.0 on pci2
Jan 12 12:36:50 freenas kernel: re0: Using 1 MSI-X message
Jan 12 12:36:50 freenas kernel: re0: Chip rev. 0x2c800000
Jan 12 12:36:50 freenas kernel: re0: MAC rev. 0x00100000
Jan 12 12:36:50 freenas kernel: miibus0: <MII bus> on re0
Jan 12 12:36:50 freenas kernel: rgephy0: <RTL8169S/8110S/8211 1000BASE-T media interface> PHY 1 on miibus0
Jan 12 12:36:50 freenas kernel: rgephy0:  none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow
Jan 12 12:36:50 freenas kernel: re0: Ethernet address: bc:5f:f4:e6:5b:93
Jan 12 12:36:50 freenas kernel: ehci1: <EHCI (generic) USB 2.0 controller> mem 0xf7c07000-0xf7c073ff irq 23 at device 29.0 on pci0
Jan 12 12:36:50 freenas kernel: usbus1: EHCI version 1.0
Jan 12 12:36:50 freenas kernel: usbus1 on ehci1
Jan 12 12:36:50 freenas kernel: isab0: <PCI-ISA bridge> at device 31.0 on pci0
Jan 12 12:36:50 freenas kernel: isa0: <ISA bus> on isab0
Jan 12 12:36:50 freenas kernel: ahci0: <Intel Cougar Point AHCI SATA controller> port 0xf0b0-0xf0b7,0xf0a0-0xf0a3,0xf090-0xf097,0xf080-0xf083,0xf060-0xf07f mem 0xf7c06000-0xf7c067ff irq 19 at device 31.2 on pci0
Jan 12 12:36:50 freenas kernel: ahci0: AHCI v1.30 with 4 3Gbps ports, Port Multiplier not supported
Jan 12 12:36:50 freenas kernel: ahcich0: <AHCI channel> at channel 0 on ahci0
Jan 12 12:36:50 freenas kernel: ahcich1: <AHCI channel> at channel 1 on ahci0
Jan 12 12:36:50 freenas kernel: ahcich4: <AHCI channel> at channel 4 on ahci0
Jan 12 12:36:50 freenas kernel: ahcich5: <AHCI channel> at channel 5 on ahci0
Jan 12 12:36:50 freenas kernel: pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
Jan 12 12:36:50 freenas kernel: acpi_button0: <Power Button> on acpi0
Jan 12 12:36:50 freenas kernel: atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
Jan 12 12:36:50 freenas kernel: atkbd0: <AT Keyboard> irq 1 on atkbdc0
Jan 12 12:36:50 freenas kernel: kbd0 at atkbd0
Jan 12 12:36:50 freenas kernel: atkbd0: [GIANT-LOCKED]
Jan 12 12:36:50 freenas kernel: ichwd0 on isa0
Jan 12 12:36:50 freenas kernel: orm0: <ISA Option ROM> at iomem 0xc0000-0xce7ff on isa0
Jan 12 12:36:50 freenas kernel: sc0: <System console> at flags 0x100 on isa0
Jan 12 12:36:50 freenas kernel: sc0: VGA <16 virtual consoles, flags=0x300>
Jan 12 12:36:50 freenas kernel: vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
Jan 12 12:36:50 freenas kernel: ppc0: cannot reserve I/O port range
Jan 12 12:36:50 freenas kernel: wbwd0: DevID 0xc3 DevRev 0x33, will not attach, please report this.
Jan 12 12:36:50 freenas kernel: coretemp0: <CPU On-Die Thermal Sensors> on cpu0
Jan 12 12:36:50 freenas kernel: est0: <Enhanced SpeedStep Frequency Control> on cpu0
Jan 12 12:36:50 freenas kernel: p4tcc0: <CPU Frequency Thermal Control> on cpu0
Jan 12 12:36:50 freenas kernel: coretemp1: <CPU On-Die Thermal Sensors> on cpu1
Jan 12 12:36:50 freenas kernel: est1: <Enhanced SpeedStep Frequency Control> on cpu1
Jan 12 12:36:50 freenas kernel: p4tcc1: <CPU Frequency Thermal Control> on cpu1
Jan 12 12:36:50 freenas kernel: coretemp2: <CPU On-Die Thermal Sensors> on cpu2
Jan 12 12:36:50 freenas kernel: est2: <Enhanced SpeedStep Frequency Control> on cpu2
Jan 12 12:36:50 freenas kernel: p4tcc2: <CPU Frequency Thermal Control> on cpu2
Jan 12 12:36:50 freenas kernel: coretemp3: <CPU On-Die Thermal Sensors> on cpu3
Jan 12 12:36:50 freenas kernel: est3: <Enhanced SpeedStep Frequency Control> on cpu3
Jan 12 12:36:50 freenas kernel: p4tcc3: <CPU Frequency Thermal Control> on cpu3
Jan 12 12:36:50 freenas kernel: Timecounters tick every 1.000 msec
Jan 12 12:36:50 freenas kernel: ipfw2 (+ipv6) initialized, divert enabled, nat enabled, default to accept, logging disabled
Jan 12 12:36:50 freenas kernel: DUMMYNET 0xfffffe00062bcd40 with IPv6 initialized (100409)
Jan 12 12:36:50 freenas kernel: load_dn_sched dn_sched QFQ loaded
Jan 12 12:36:50 freenas kernel: load_dn_sched dn_sched RR loaded
Jan 12 12:36:50 freenas kernel: load_dn_sched dn_sched WF2Q+ loaded
Jan 12 12:36:50 freenas kernel: load_dn_sched dn_sched FIFO loaded
Jan 12 12:36:50 freenas kernel: load_dn_sched dn_sched PRIO loaded
Jan 12 12:36:50 freenas kernel: usbus0: 480Mbps High Speed USB v2.0
Jan 12 12:36:50 freenas kernel: usbus1: 480Mbps High Speed USB v2.0
Jan 12 12:36:50 freenas kernel: ugen1.1: <Intel> at usbus1
Jan 12 12:36:50 freenas kernel: uhub0: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
Jan 12 12:36:50 freenas kernel: ugen0.1: <Intel> at usbus0
Jan 12 12:36:50 freenas kernel: uhub1: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus0
Jan 12 12:36:50 freenas kernel: ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
Jan 12 12:36:50 freenas kernel: ada0: <WDC WD6400AAKS-65A7B0 01.03B01> ATA-8 SATA 2.x device
Jan 12 12:36:50 freenas kernel: ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
Jan 12 12:36:50 freenas kernel: ada0: Command Queueing enabled
Jan 12 12:36:50 freenas kernel: ada0: 610480MB (1250263728 512 byte sectors: 16H 63S/T 16383C)
Jan 12 12:36:50 freenas kernel: ada0: Previously was known as ad4
Jan 12 12:36:50 freenas kernel: ada1 at ahcich1 bus 0 scbus1 target 0 lun 0
Jan 12 12:36:50 freenas kernel: ada1: <WDC WD6400AAKS-65A7B0 01.03B01> ATA-8 SATA 2.x device
Jan 12 12:36:50 freenas kernel: ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
Jan 12 12:36:50 freenas kernel: ada1: Command Queueing enabled
Jan 12 12:36:50 freenas kernel: ada1: 610480MB (1250263728 512 byte sectors: 16H 63S/T 16383C)
Jan 12 12:36:50 freenas kernel: ada1: Previously was known as ad6
Jan 12 12:36:50 freenas kernel: ada2 at ahcich4 bus 0 scbus2 target 0 lun 0
Jan 12 12:36:50 freenas kernel: ada2: <ST3000DM001-1CH166 CC27> ATA-9 SATA 3.x device
Jan 12 12:36:50 freenas kernel: ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
Jan 12 12:36:50 freenas kernel: ada2: Command Queueing enabled
Jan 12 12:36:50 freenas kernel: ada2: 2861588MB (5860533168 512 byte sectors: 16H 63S/T 16383C)
Jan 12 12:36:50 freenas kernel: ada2: quirks=0x1<4K>
Jan 12 12:36:50 freenas kernel: ada2: Previously was known as ad8
Jan 12 12:36:50 freenas kernel: ada3 at ahcich5 bus 0 scbus3 target 0 lun 0
Jan 12 12:36:50 freenas kernel: ada3: <WDC WD1600AAJS-00B4A0 01.03A01> ATA-8 SATA 2.x device
Jan 12 12:36:50 freenas kernel: ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
Jan 12 12:36:50 freenas kernel: ada3: Command Queueing enabled
Jan 12 12:36:50 freenas kernel: ada3: 152627MB (312581808 512 byte sectors: 16H 63S/T 16383C)
Jan 12 12:36:50 freenas kernel: ada3: Previously was known as ad10
Jan 12 12:36:50 freenas kernel: SMP: AP CPU #1 Launched!
Jan 12 12:36:50 freenas kernel: SMP: AP CPU #3 Launched!
Jan 12 12:36:50 freenas kernel: SMP: AP CPU #2 Launched!
Jan 12 12:36:50 freenas kernel: Timecounter "TSC-low" frequency 1696184090 Hz quality 1000
Jan 12 12:36:50 freenas kernel: Root mount waiting for: usbus1 usbus0
Jan 12 12:36:50 freenas kernel: uhub1: 2 ports with 2 removable, self powered
Jan 12 12:36:50 freenas kernel: uhub0: 2 ports with 2 removable, self powered
Jan 12 12:36:50 freenas kernel: Root mount waiting for: usbus1 usbus0
Jan 12 12:36:50 freenas kernel: ugen1.2: <vendor 0x8087> at usbus1
Jan 12 12:36:50 freenas kernel: uhub2: <vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr 2> on usbus1
Jan 12 12:36:50 freenas kernel: ugen0.2: <vendor 0x8087> at usbus0
Jan 12 12:36:50 freenas kernel: uhub3: <vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr 2> on usbus0
Jan 12 12:36:50 freenas kernel: uhub3: 4 ports with 4 removable, self powered
Jan 12 12:36:50 freenas kernel: uhub2: 6 ports with 6 removable, self powered
Jan 12 12:36:50 freenas kernel: Root mount waiting for: usbus1
Jan 12 12:36:50 freenas kernel: ugen1.3: <vendor 0x13fe> at usbus1
Jan 12 12:36:50 freenas kernel: umass0: <vendor 0x13fe Patriot Memory, class 0/0, rev 2.00/1.10, addr 3> on usbus1
Jan 12 12:36:50 freenas kernel: da0 at umass-sim0 bus 0 scbus5 target 0 lun 0
Jan 12 12:36:50 freenas kernel: da0: < Patriot Memory PMAP> Removable Direct Access SCSI-0 device
Jan 12 12:36:50 freenas kernel: da0: 40.000MB/s transfers
Jan 12 12:36:50 freenas kernel: da0: 7644MB (15654912 512 byte sectors: 255H 63S/T 974C)
Jan 12 12:36:50 freenas kernel: da0: quirks=0x2<NO_6_BYTE>
Jan 12 12:36:50 freenas kernel: Trying to mount root from ufs:/dev/ufs/FreeNASs2a [ro]...
Jan 12 12:36:50 freenas kernel: GEOM_RAID5: Module loaded, version 1.1.20130907.44 (rev 5c6d2a159411)
Jan 12 12:36:50 freenas kernel: ZFS filesystem version: 5
Jan 12 12:36:50 freenas kernel: ZFS storage pool version: features support (5000)
Jan 12 12:36:50 freenas kernel: GEOM_ELI: Device ada3p1.eli created.
Jan 12 12:36:50 freenas kernel: GEOM_ELI: Encryption: AES-XTS 256
Jan 12 12:36:50 freenas kernel: GEOM_ELI:    Crypto: software
Jan 12 12:36:50 freenas kernel: GEOM_ELI: Device ada0p1.eli created.
Jan 12 12:36:50 freenas kernel: GEOM_ELI: Encryption: AES-XTS 256
Jan 12 12:36:50 freenas kernel: GEOM_ELI:    Crypto: software
Jan 12 12:36:50 freenas kernel: GEOM_ELI: Device ada1p1.eli created.
Jan 12 12:36:50 freenas kernel: GEOM_ELI: Encryption: AES-XTS 256
Jan 12 12:36:50 freenas kernel: GEOM_ELI:    Crypto: software
Jan 12 12:36:50 freenas kernel: GEOM_ELI: Device ada2p1.eli created.
Jan 12 12:36:50 freenas kernel: GEOM_ELI: Encryption: AES-XTS 256
Jan 12 12:36:50 freenas kernel: GEOM_ELI:    Crypto: software
Jan 12 12:36:50 freenas root: /etc/rc: WARNING: failed precmd routine for vmware_guestd
Jan 12 12:36:50 freenas ntpd[1941]: ntpd 4.2.4p5-a (1)
Jan 12 12:36:51 freenas kernel: .........+++
Jan 12 12:36:52 freenas smartd[2233]: Device: /dev/ada2, WARNING: A firmware update for this drive may be available,
Jan 12 12:36:52 freenas smartd[2233]: see the following Seagate web pages:
Jan 12 12:36:52 freenas smartd[2233]: http://knowledge.seagate.com/articles/en_US/FAQ/207931en
Jan 12 12:36:52 freenas smartd[2233]: http://knowledge.seagate.com/articles/en_US/FAQ/223651en
Jan 12 12:36:56 freenas avahi-daemon[2759]: WARNING: No NSS support for mDNS detected, consider installing nss-mdns!
Jan 12 12:36:59 freenas ntpd[1942]: time reset -0.571176 s
Jan 12 12:38:58 freenas kernel: bridge0: Ethernet address: 02:fe:4a:c8:9c:00
Jan 12 12:38:58 freenas kernel: re0: promiscuous mode enabled
Jan 12 12:38:58 freenas kernel: bridge0: link state changed to UP
Jan 12 12:38:58 freenas kernel: epair0a: Ethernet address: 02:2b:82:00:07:0a
Jan 12 12:38:58 freenas kernel: epair0b: Ethernet address: 02:2b:82:00:08:0b
Jan 12 12:38:58 freenas kernel: epair0a: link state changed to UP
Jan 12 12:38:58 freenas kernel: epair0b: link state changed to UP
Jan 12 12:38:58 freenas kernel: epair0a: promiscuous mode enabled
Jan 12 12:38:58 freenas kernel: re0: link state changed to DOWN
Jan 12 12:39:02 freenas kernel: re0: link state changed to UP
Jan 12 12:39:06 freenas dhclient: New IP Address (re0): 192.168.1.10
Jan 12 12:39:06 freenas dhclient: New Subnet Mask (re0): 255.255.255.0
Jan 12 12:39:06 freenas dhclient: New Broadcast Address (re0): 192.168.1.255
Jan 12 12:39:06 freenas dhclient: New Routers (re0): 192.168.1.1
Jan 12 12:39:49 freenas manage.py: [py.warnings:193] /usr/local/lib/python2.7/site-packages/django/http/request.py:193: DeprecationWarning: HttpRequest.raw_post_data has been deprecated. Use HttpRequest.body instead.  warnings.warn('HttpRequest.raw_post_data has been deprecated. Use HttpRequest.body instead.', DeprecationWarning)

 

[rm-r]

Code:

Jan 12 12:36:59 freenas ntpd[1942]: time reset -0.571176 s

this is the time jump i was also seeing!

 

[Nate]

Code:

Jan 12 12:36:59 freenas ntpd[1942]: time reset -0.571176 s

this is the time jump i was also seeing!

Well we might be onto something.

I wanted to post this as I stumbled upon it while researching: http://www.xpertnotes.net/freenas-9-1-system-clock-woes/

 

[Dusan]

Even more interesting is the fact that in that same gap I have this show up in my console message window (from the webgui). It almost perfectly matches up to the same gap timeframe.

I don't think your case is related. The syslog you posted contains the entire boot sequence. This means your system crashed and restarted and the gap you see is caused by collectd not running (it starts at the end of the boot process). The other cases reported in this thread are about time drifts & lockups, but I think no one reported a restart.
So, the question you should be asking is not why you see a gaps in the graphs, but why is your system rebooting?

 

[Nate]

I don't think your case is related. The syslog you posted contains the entire boot sequence. This means your system crashed and restarted and the gap you see is caused by collectd not running (it starts at the end of the boot process). The other cases reported in this thread are about time drifts & lockups, but I think no one reported a restart.
So, the question you should be asking is not why you see a gaps in the graphs, but why is your system rebooting?

I wouldn't have any idea. I have nothing on my console or logs to give me a clue. Since last night I went into bios and found that my date and time were off. I changed those around in the bios to match the correct time zone I was in and so far it's been fine. This was one of the first mornings I came on and it wasn't totally locked out. I'm not sure if it's related or not but we will see.