Elliot Dierksen
Guru
- Joined
- Dec 29, 2014
- Messages
- 1,135
I just started getting this message during boot on both my FreeNAS systems (FreeNAS-11.1-U5). Both systems are Cisco UCS C240 M3S. Primary has dual Intel(R) Xeon(R) CPU E5-2660 v2 @ 2.20GHz with 128GB RAM, and the secondary had dual Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz with 64GB of RAM. LSI 9271 in JBOD mode for internal drives (24 2.5" drive bays), LSI 9207-8e, Chelsio T520-CR NIC (storage facing), and onboard Intel 350's (user facing). Both systems boot from an SD card, and the system database is on a mirrored set of 300GB SAS drives. Dmesg and pciconf info below or attached.
I am not normally watching the systems boot, but I think this started at the time when I added the 9207-8e to both systems. The secondary has an HP D2700 external enclosure attached (multipath) to the 9207, but the primary doesn't have anything attached there at the moment. The onboard SATA controller is disabled. I looked at the other forum threads on this issue, but I didn't see anything that matched my symptoms. Any ideas?
Code:
Copyright (c) 1992-2017 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 11.1-STABLE #0 r321665+e0c4ca60dfc(freenas/11.1-stable): Wed May 30 14:18:20 EDT 2018 root@nemesis.tn.ixsystems.com:/freenas-11-releng/freenas/_BE/objs/freenas-11-releng/freenas/_BE/os/sys/FreeNAS.amd64 amd64 FreeBSD clang version 5.0.0 (tags/RELEASE_500/final 312559) (based on LLVM 5.0.0svn) CPU: Intel(R) Xeon(R) CPU E5-2660 v2 @ 2.20GHz (2200.05-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306e4 Family=0x6 Model=0x3e Stepping=4 Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> Features2=0x7fbee3ff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND> AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM> AMD Features2=0x1<LAHF> Structured Extended Features=0x281<FSGSBASE,SMEP,ERMS> XSAVE Features=0x1<XSAVEOPT> VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr TSC: P-state invariant, performance statistics real memory = 139586437120 (133120 MB) avail memory = 133477036032 (127293 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: <Cisco0 CiscoUCS> FreeBSD/SMP: Multiprocessor System Detected: 40 CPUs FreeBSD/SMP: 2 package(s) x 10 core(s) x 2 hardware threads WARNING: VIMAGE (virtualized network stack) is a highly experimental feature. ioapic0 <Version 2.0> irqs 0-23 on motherboard ioapic1 <Version 2.0> irqs 24-47 on motherboard ioapic2 <Version 2.0> irqs 48-71 on motherboard SMP: AP CPU #30 Launched! SMP: AP CPU #31 Launched! SMP: AP CPU #12 Launched! SMP: AP CPU #15 Launched! SMP: AP CPU #11 Launched! SMP: AP CPU #24 Launched! SMP: AP CPU #25 Launched! SMP: AP CPU #20 Launched! SMP: AP CPU #38 Launched! SMP: AP CPU #39 Launched! SMP: AP CPU #6 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #8 Launched! SMP: AP CPU #13 Launched! SMP: AP CPU #32 Launched! SMP: AP CPU #22 Launched! SMP: AP CPU #23 Launched! SMP: AP CPU #18 Launched! SMP: AP CPU #34 Launched! SMP: AP CPU #35 Launched! SMP: AP CPU #17 Launched! SMP: AP CPU #28 Launched! SMP: AP CPU #29 Launched! SMP: AP CPU #19 Launched! SMP: AP CPU #16 Launched! SMP: AP CPU #5 Launched! SMP: AP CPU #14 Launched! SMP: AP CPU #37 Launched! SMP: AP CPU #7 Launched! SMP: AP CPU #10 Launched! SMP: AP CPU #9 Launched! SMP: AP CPU #26 Launched! SMP: AP CPU #4 Launched! SMP: AP CPU #36 Launched! SMP: AP CPU #27 Launched! SMP: AP CPU #3 Launched! SMP: AP CPU #1 Launched! SMP: AP CPU #33 Launched! SMP: AP CPU #21 Launched! Timecounter "TSC-low" frequency 1100023158 Hz quality 1000 random: entropy device external interface kbd1 at kbdmux0 random: registering fast source Intel Secure Key RNG random: fast provider: "Intel Secure Key RNG" nexus0 cryptosoft0: <software crypto> on motherboard aesni0: <AES-CBC,AES-XTS,AES-GCM,AES-ICM> on motherboard padlock0: No ACE support. acpi0: <Cisco0 CiscoUCS> on motherboard acpi0: Power Button (fixed) cpu0: <ACPI CPU> numa-domain 0 on acpi0 cpu1: <ACPI CPU> numa-domain 0 on acpi0 cpu2: <ACPI CPU> numa-domain 0 on acpi0 cpu3: <ACPI CPU> numa-domain 0 on acpi0 cpu4: <ACPI CPU> numa-domain 0 on acpi0 cpu5: <ACPI CPU> numa-domain 0 on acpi0 cpu6: <ACPI CPU> numa-domain 0 on acpi0 cpu7: <ACPI CPU> numa-domain 0 on acpi0 cpu8: <ACPI CPU> numa-domain 0 on acpi0 cpu9: <ACPI CPU> numa-domain 0 on acpi0 cpu10: <ACPI CPU> numa-domain 0 on acpi0 cpu11: <ACPI CPU> numa-domain 0 on acpi0 cpu12: <ACPI CPU> numa-domain 0 on acpi0 cpu13: <ACPI CPU> numa-domain 0 on acpi0 cpu14: <ACPI CPU> numa-domain 0 on acpi0 cpu15: <ACPI CPU> numa-domain 0 on acpi0 cpu16: <ACPI CPU> numa-domain 0 on acpi0 cpu17: <ACPI CPU> numa-domain 0 on acpi0 cpu18: <ACPI CPU> numa-domain 0 on acpi0 cpu19: <ACPI CPU> numa-domain 0 on acpi0 cpu20: <ACPI CPU> numa-domain 1 on acpi0 cpu21: <ACPI CPU> numa-domain 1 on acpi0 cpu22: <ACPI CPU> numa-domain 1 on acpi0 cpu23: <ACPI CPU> numa-domain 1 on acpi0 cpu24: <ACPI CPU> numa-domain 1 on acpi0 cpu25: <ACPI CPU> numa-domain 1 on acpi0 cpu26: <ACPI CPU> numa-domain 1 on acpi0 cpu27: <ACPI CPU> numa-domain 1 on acpi0 cpu28: <ACPI CPU> numa-domain 1 on acpi0 cpu29: <ACPI CPU> numa-domain 1 on acpi0 cpu30: <ACPI CPU> numa-domain 1 on acpi0 cpu31: <ACPI CPU> numa-domain 1 on acpi0 cpu32: <ACPI CPU> numa-domain 1 on acpi0 cpu33: <ACPI CPU> numa-domain 1 on acpi0 cpu34: <ACPI CPU> numa-domain 1 on acpi0 cpu35: <ACPI CPU> numa-domain 1 on acpi0 cpu36: <ACPI CPU> numa-domain 1 on acpi0 cpu37: <ACPI CPU> numa-domain 1 on acpi0 cpu38: <ACPI CPU> numa-domain 1 on acpi0 cpu39: <ACPI CPU> numa-domain 1 on acpi0 attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s Event timer "RTC" frequency 32768 Hz quality 0 hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 Event timer "HPET3" frequency 14318180 Hz quality 340 Event timer "HPET4" frequency 14318180 Hz quality 340 Event timer "HPET5" frequency 14318180 Hz quality 340 Event timer "HPET6" frequency 14318180 Hz quality 340 Event timer "HPET7" frequency 14318180 Hz quality 340 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff numa-domain 0 on acpi0 pcib0: _OSC returned error 0x10 pci0: <ACPI PCI bus> numa-domain 0 on pcib0 pcib1: <ACPI PCI-PCI bridge> irq 26 at device 1.0 numa-domain 0 on pci0 pci1: <ACPI PCI bus> numa-domain 0 on pcib1 pcib2: <ACPI PCI-PCI bridge> irq 26 at device 1.1 numa-domain 0 on pci0 pci2: <ACPI PCI bus> numa-domain 0 on pcib2 igb0: <Intel(R) PRO/1000 Network Connection, Version - 2.5.3-k> port 0x8060-0x807f mem 0xdec00000-0xdecfffff,0xdef0c000-0xdef0ffff irq 27 at device 0.0 numa-domain 0 on pci2 igb0: Using MSIX interrupts with 9 vectors igb0: Ethernet address: e4:c7:22:84:8f:30 igb0: Bound queue 0 to cpu 0 igb0: Bound queue 1 to cpu 1 igb0: Bound queue 2 to cpu 2 igb0: Bound queue 3 to cpu 3 igb0: Bound queue 4 to cpu 4 igb0: Bound queue 5 to cpu 5 igb0: Bound queue 6 to cpu 6 igb0: Bound queue 7 to cpu 7 igb1: <Intel(R) PRO/1000 Network Connection, Version - 2.5.3-k> port 0x8040-0x805f mem 0xdeb00000-0xdebfffff,0xdef08000-0xdef0bfff irq 30 at device 0.1 numa-domain 0 on pci2 igb1: Using MSIX interrupts with 9 vectors igb1: Ethernet address: e4:c7:22:84:8f:31 igb1: Bound queue 0 to cpu 8 igb1: Bound queue 1 to cpu 9 igb1: Bound queue 2 to cpu 10 igb1: Bound queue 3 to cpu 11 igb1: Bound queue 4 to cpu 12 igb1: Bound queue 5 to cpu 13 igb1: Bound queue 6 to cpu 14 igb1: Bound queue 7 to cpu 15 igb2: <Intel(R) PRO/1000 Network Connection, Version - 2.5.3-k> port 0x8020-0x803f mem 0xdea00000-0xdeafffff,0xdef04000-0xdef07fff irq 28 at device 0.2 numa-domain 0 on pci2 igb2: Using MSIX interrupts with 9 vectors igb2: Ethernet address: e4:c7:22:84:8f:32 igb2: Bound queue 0 to cpu 16 igb2: Bound queue 1 to cpu 17 igb2: Bound queue 2 to cpu 18 igb2: Bound queue 3 to cpu 19 igb2: Bound queue 4 to cpu 20 igb2: Bound queue 5 to cpu 21 igb2: Bound queue 6 to cpu 22 igb2: Bound queue 7 to cpu 23 igb3: <Intel(R) PRO/1000 Network Connection, Version - 2.5.3-k> port 0x8000-0x801f mem 0xde900000-0xde9fffff,0xdef00000-0xdef03fff irq 29 at device 0.3 numa-domain 0 on pci2 igb3: Using MSIX interrupts with 9 vectors igb3: Ethernet address: e4:c7:22:84:8f:33 igb3: Bound queue 0 to cpu 24 igb3: Bound queue 1 to cpu 25 igb3: Bound queue 2 to cpu 26 igb3: Bound queue 3 to cpu 27 igb3: Bound queue 4 to cpu 28 igb3: Bound queue 5 to cpu 29 igb3: Bound queue 6 to cpu 30 igb3: Bound queue 7 to cpu 31 pcib3: <ACPI PCI-PCI bridge> irq 32 at device 2.0 numa-domain 0 on pci0 pci3: <ACPI PCI bus> numa-domain 0 on pcib3 mps0: <Avago Technologies (LSI) SAS2308> port 0x7000-0x70ff mem 0xdf140000-0xdf14ffff,0xdf100000-0xdf13ffff irq 32 at device 0.0 numa-domain 0 on pci3 mps0: Firmware: 20.00.07.00, Driver: 21.02.00.00-fbsd mps0: IOCCapabilities: 5a85c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,MSIXIndex,HostDisc> pcib4: <ACPI PCI-PCI bridge> irq 32 at device 2.2 numa-domain 0 on pci0 pci4: <ACPI PCI bus> numa-domain 0 on pcib4 pcib5: <ACPI PCI-PCI bridge> irq 42 at device 3.0 numa-domain 0 on pci0 pci5: <ACPI PCI bus> numa-domain 0 on pcib5 nvme0: <Generic NVMe Device> mem 0xdf210000-0xdf213fff irq 40 at device 0.0 numa-domain 0 on pci5 pcib6: <ACPI PCI-PCI bridge> irq 16 at device 17.0 numa-domain 0 on pci0 pci6: <ACPI PCI bus> numa-domain 0 on pcib6 ehci0: <Intel Patsburg USB 2.0 controller> mem 0xdf301000-0xdf3013ff irq 22 at device 26.0 numa-domain 0 on pci0 usbus0: EHCI version 1.0 usbus0 numa-domain 0 on ehci0 usbus0: 480Mbps High Speed USB v2.0 pcib7: <ACPI PCI-PCI bridge> irq 17 at device 28.0 numa-domain 0 on pci0 pci7: <ACPI PCI bus> numa-domain 0 on pcib7 pcib8: <ACPI PCI-PCI bridge> irq 19 at device 28.7 numa-domain 0 on pci0 pci8: <ACPI PCI bus> numa-domain 0 on pcib8 vgapci0: <VGA-compatible display> mem 0xdd000000-0xddffffff,0xde810000-0xde813fff,0xde000000-0xde7fffff irq 19 at device 0.0 numa-domain 0 on pci8 vgapci0: Boot video device ehci1: <Intel Patsburg USB 2.0 controller> mem 0xdf300000-0xdf3003ff irq 23 at device 29.0 numa-domain 0 on pci0 usbus1: EHCI version 1.0 usbus1 numa-domain 0 on ehci1 usbus1: 480Mbps High Speed USB v2.0 pcib9: <ACPI PCI-PCI bridge> at device 30.0 numa-domain 0 on pci0 pci9: <ACPI PCI bus> numa-domain 0 on pcib9 isab0: <PCI-ISA bridge> at device 31.0 numa-domain 0 on pci0 isa0: <ISA bus> numa-domain 0 on isab0 pcib10: <ACPI Host-PCI bridge> numa-domain 0 on acpi0 pci10: <ACPI PCI bus> numa-domain 0 on pcib10 pci10: <dasp, performance counters> at device 14.1 (no driver attached) pci10: <dasp, performance counters> at device 19.1 (no driver attached) pci10: <dasp, performance counters> at device 19.5 (no driver attached) pcib11: <ACPI Host-PCI bridge> numa-domain 1 on acpi0 pcib11: _OSC returned error 0x10 pci11: <ACPI PCI bus> numa-domain 1 on pcib11 pcib12: <ACPI PCI-PCI bridge> irq 50 at device 1.0 numa-domain 1 on pci11 pci12: <ACPI PCI bus> numa-domain 1 on pcib12 pcib13: <ACPI PCI-PCI bridge> irq 56 at device 2.0 numa-domain 1 on pci11 pci13: <ACPI PCI bus> numa-domain 1 on pcib13 AVAGO MegaRAID SAS FreeBSD mrsas driver version: 06.712.04.00-fbsd mrsas0: <AVAGO Thunderbolt SAS Controller> port 0xf000-0xf0ff mem 0xfbc60000-0xfbc63fff,0xfbc00000-0xfbc3ffff irq 56 at device 0.0 numa-domain 1 on pci13 mrsas0: Using MSI-X with 16 number of vectors mrsas0: FW supports <16> MSIX vector,Online CPU 40 Current MSIX <16> mrsas0: MSI-x interrupts setup success pcib14: <ACPI PCI-PCI bridge> irq 64 at device 3.0 numa-domain 1 on pci11 pci14: <ACPI PCI bus> numa-domain 1 on pcib14 t5nex0: <Chelsio T520-CR> mem 0xfb300000-0xfb37ffff,0xfa000000-0xfaffffff,0xfbb04000-0xfbb05fff irq 64 at device 0.4 numa-domain 1 on pci14 cxl0: <port 0> numa-domain 1 on t5nex0 cxl0: Ethernet address: 00:07:43:2d:14:70 cxl0: 16 txq, 8 rxq (NIC) cxl1: <port 1> numa-domain 1 on t5nex0 cxl1: Ethernet address: 00:07:43:2d:14:78 cxl1: 16 txq, 8 rxq (NIC) t5nex0: PCIe gen3 x8, 2 ports, 18 MSI-X interrupts, 51 eq, 17 iq pci14: <mass storage, SCSI> at device 0.5 (no driver attached) pci14: <serial bus, Fibre Channel> at device 0.6 (no driver attached) pcib15: <ACPI Host-PCI bridge> numa-domain 1 on acpi0 pci15: <ACPI PCI bus> numa-domain 1 on pcib15 pci15: <dasp, performance counters> at device 14.1 (no driver attached) pci15: <dasp, performance counters> at device 19.1 (no driver attached) pci15: <dasp, performance counters> at device 19.5 (no driver attached) acpi_button0: <Power Button> on acpi0 acpi_acad0: <AC Adapter> on acpi0 acpi_syscontainer0: <System Container> on acpi0 acpi_syscontainer1: <System Container> on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 ichwd0: <Intel Patsburg watchdog timer> on isa0 ichwd0: ICH WDT present but disabled in BIOS or hardware device_attach: ichwd0 attach returned 6 ichwd0: <Intel Patsburg watchdog timer> at port 0x430-0x437,0x460-0x47f on isa0 ichwd0: ICH WDT present but disabled in BIOS or hardware device_attach: ichwd0 attach returned 6 orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xc8000-0xc87ff on isa0 sc0: <System console> at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 coretemp0: <CPU On-Die Thermal Sensors> numa-domain 0 on cpu0 est0: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu0 coretemp1: <CPU On-Die Thermal Sensors> numa-domain 0 on cpu1 est1: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu1 coretemp2: <CPU On-Die Thermal Sensors> numa-domain 0 on cpu2 est2: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu2 coretemp3: <CPU On-Die Thermal Sensors> numa-domain 0 on cpu3 est3: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu3 coretemp4: <CPU On-Die Thermal Sensors> numa-domain 0 on cpu4 est4: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu4 coretemp5: <CPU On-Die Thermal Sensors> numa-domain 0 on cpu5 est5: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu5 coretemp6: <CPU On-Die Thermal Sensors> numa-domain 0 on cpu6 est6: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu6 coretemp7: <CPU On-Die Thermal Sensors> numa-domain 0 on cpu7 est7: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu7 coretemp8: <CPU On-Die Thermal Sensors> numa-domain 0 on cpu8 est8: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu8 coretemp9: <CPU On-Die Thermal Sensors> numa-domain 0 on cpu9 est9: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu9 coretemp10: <CPU On-Die Thermal Sensors> numa-domain 0 on cpu10 est10: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu10 coretemp11: <CPU On-Die Thermal Sensors> numa-domain 0 on cpu11 est11: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu11 coretemp12: <CPU On-Die Thermal Sensors> numa-domain 0 on cpu12 est12: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu12 coretemp13: <CPU On-Die Thermal Sensors> numa-domain 0 on cpu13 est13: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu13 coretemp14: <CPU On-Die Thermal Sensors> numa-domain 0 on cpu14 est14: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu14 coretemp15: <CPU On-Die Thermal Sensors> numa-domain 0 on cpu15 est15: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu15 coretemp16: <CPU On-Die Thermal Sensors> numa-domain 0 on cpu16 est16: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu16 coretemp17: <CPU On-Die Thermal Sensors> numa-domain 0 on cpu17 est17: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu17 coretemp18: <CPU On-Die Thermal Sensors> numa-domain 0 on cpu18 est18: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu18 coretemp19: <CPU On-Die Thermal Sensors> numa-domain 0 on cpu19 est19: <Enhanced SpeedStep Frequency Control> numa-domain 0 on cpu19 coretemp20: <CPU On-Die Thermal Sensors> numa-domain 1 on cpu20 est20: <Enhanced SpeedStep Frequency Control> numa-domain 1 on cpu20 coretemp21: <CPU On-Die Thermal Sensors> numa-domain 1 on cpu21 est21: <Enhanced SpeedStep Frequency Control> numa-domain 1 on cpu21 coretemp22: <CPU On-Die Thermal Sensors> numa-domain 1 on cpu22 est22: <Enhanced SpeedStep Frequency Control> numa-domain 1 on cpu22 coretemp23: <CPU On-Die Thermal Sensors> numa-domain 1 on cpu23 est23: <Enhanced SpeedStep Frequency Control> numa-domain 1 on cpu23 coretemp24: <CPU On-Die Thermal Sensors> numa-domain 1 on cpu24 est24: <Enhanced SpeedStep Frequency Control> numa-domain 1 on cpu24 coretemp25: <CPU On-Die Thermal Sensors> numa-domain 1 on cpu25 est25: <Enhanced SpeedStep Frequency Control> numa-domain 1 on cpu25 coretemp26: <CPU On-Die Thermal Sensors> numa-domain 1 on cpu26 est26: <Enhanced SpeedStep Frequency Control> numa-domain 1 on cpu26 coretemp27: <CPU On-Die Thermal Sensors> numa-domain 1 on cpu27 est27: <Enhanced SpeedStep Frequency Control> numa-domain 1 on cpu27 coretemp28: <CPU On-Die Thermal Sensors> numa-domain 1 on cpu28 est28: <Enhanced SpeedStep Frequency Control> numa-domain 1 on cpu28 coretemp29: <CPU On-Die Thermal Sensors> numa-domain 1 on cpu29 est29: <Enhanced SpeedStep Frequency Control> numa-domain 1 on cpu29 coretemp30: <CPU On-Die Thermal Sensors> numa-domain 1 on cpu30 est30: <Enhanced SpeedStep Frequency Control> numa-domain 1 on cpu30 coretemp31: <CPU On-Die Thermal Sensors> numa-domain 1 on cpu31 est31: <Enhanced SpeedStep Frequency Control> numa-domain 1 on cpu31 coretemp32: <CPU On-Die Thermal Sensors> numa-domain 1 on cpu32 est32: <Enhanced SpeedStep Frequency Control> numa-domain 1 on cpu32 coretemp33: <CPU On-Die Thermal Sensors> numa-domain 1 on cpu33 est33: <Enhanced SpeedStep Frequency Control> numa-domain 1 on cpu33 coretemp34: <CPU On-Die Thermal Sensors> numa-domain 1 on cpu34 est34: <Enhanced SpeedStep Frequency Control> numa-domain 1 on cpu34 coretemp35: <CPU On-Die Thermal Sensors> numa-domain 1 on cpu35 est35: <Enhanced SpeedStep Frequency Control> numa-domain 1 on cpu35 coretemp36: <CPU On-Die Thermal Sensors> numa-domain 1 on cpu36 est36: <Enhanced SpeedStep Frequency Control> numa-domain 1 on cpu36 coretemp37: <CPU On-Die Thermal Sensors> numa-domain 1 on cpu37 est37: <Enhanced SpeedStep Frequency Control> numa-domain 1 on cpu37 coretemp38: <CPU On-Die Thermal Sensors> numa-domain 1 on cpu38 est38: <Enhanced SpeedStep Frequency Control> numa-domain 1 on cpu38 coretemp39: <CPU On-Die Thermal Sensors> numa-domain 1 on cpu39 est39: <Enhanced SpeedStep Frequency Control> numa-domain 1 on cpu39 random: unblocking device. ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Timecounters tick every 1.000 msec freenas_sysctl: adding account. freenas_sysctl: adding directoryservice. freenas_sysctl: adding middlewared. freenas_sysctl: adding network. freenas_sysctl: adding services. ipfw2 (+ipv6) initialized, divert enabled, nat enabled, default to accept, logging disabled ugen0.1: <Intel EHCI root HUB> at usbus0 ugen1.1: <Intel EHCI root HUB> at usbus1 uhub0: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus0 uhub1: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1 nvd0: <INTEL SSDPED1D280GA> NVMe namespace nvd0: 267090MB (547002288 512 byte sectors) mrsas0: Disestablish mrsas intr hook uhub0: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered ugen0.2: <vendor 0x8087 product 0x0024> at usbus0 uhub2 numa-domain 0 on uhub0 uhub2: <vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr 2> on usbus0 ugen1.2: <vendor 0x8087 product 0x0024> at usbus1 uhub3 numa-domain 0 on uhub1 uhub3: <vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr 2> on usbus1 uhub2: 6 ports with 6 removable, self powered uhub3: 8 ports with 8 removable, self powered ugen0.3: <Avocent Cisco USB Composite Device-0> at usbus0 ukbd0 numa-domain 0 on uhub2 ukbd0: <Keyboard> on usbus0 kbd0 at ukbd0 ugen1.3: <Avocent Avocent USBIAC> at usbus1 ukbd1 numa-domain 0 on uhub3 ukbd1: <EP1 Interrupt> on usbus1 kbd2 at ukbd1 ugen1.4: <Cypress West Bridge> at usbus1 umass0 numa-domain 0 on uhub3 umass0: <Cypress West Bridge, class 0/0, rev 2.00/0.02, addr 4> on usbus1 umass0: SCSI over Bulk-Only; quirks = 0x4100 umass0:4:0: Attached to scbus4 ugen1.5: <vendor 0x0424 product 0x2512> at usbus1 uhub4 numa-domain 0 on uhub3 uhub4: <vendor 0x0424 product 0x2512, class 9/0, rev 2.00/b.b3, addr 5> on usbus1 uhub4: MTT enabled uhub4: 2 ports with 2 removable, self powered run_interrupt_driven_hooks: still waiting after 60 seconds for xpt_config run_interrupt_driven_hooks: still waiting after 120 seconds for xpt_config mrsas0: Initiating Target RESET because of SCSI IO timeout! mrsas0: Task management NOT SUPPORTED for CAM target:26 mrsas0: target reset FAIL!! mrsas0: Initiaiting OCR because of TM FAILURE! da1 at mrsas0 bus 1 scbus2 target 28 lun 0 da2 at mrsas0 bus 1 scbus2 target 29 lun 0 da0 at mrsas0 bus 1 scbus2 target 27 lun 0 da4 at mrsas0 bus 1 scbus2 target 31 lun 0 da5 at mrsas0 bus 1 scbus2 target 32 lun 0 da3 at mrsas0 bus 1 scbus2 target 30 lun 0 da1: <ATA ST91000640NS CC03> Fixed Direct Access SPC-4 SCSI device da3: <ATA ST91000640NS CC03> Fixed Direct Access SPC-4 SCSI device da3: Serial Number 9XG8SX92 da3: 150.000MB/s transfers da3: 953869MB (1953525168 512 byte sectors) da19 at umass-sim0 bus 0 scbus4 target 0 lun 0 da19: <HV Hypervisor_0 1.01> Removable Direct Access SCSI device da19: Serial Number 012345678901 da19: 40.000MB/s transfers da19: 29696MB (60817408 512 byte sectors) da19: quirks=0x2<NO_6_BYTE> da15 at mrsas0 bus 1 scbus2 target 51 lun 0 da15: <ATA ST91000640NS BK03> Fixed Direct Access SPC-4 SCSI device da15: Serial Number 9XG1A4XV da15: 150.000MB/s transfers da15: 953869MB (1953525168 512 byte sectors) da4: <ATA ST91000640NS CC03> Fixed Direct Access SPC-4 SCSI device da4: Serial Number 9XG7T4CV da4: 150.000MB/s transfersda1: Serial Number 9XG7TGL6 da1: 150.000MB/s transfers da1: 953869MB (1953525168 512 byte sectors)
I am not normally watching the systems boot, but I think this started at the time when I added the 9207-8e to both systems. The secondary has an HP D2700 external enclosure attached (multipath) to the 9207, but the primary doesn't have anything attached there at the moment. The onboard SATA controller is disabled. I looked at the other forum threads on this issue, but I didn't see anything that matched my symptoms. Any ideas?
