Hi all,
Been running into an issue over the last week and am running out of ideas. I was previously using Unraid on trial and found the performance was bad. So I tried setting up a FreeNAS 11.3 instance last weekend but have had problems for a week. If the server was not being accessed, it would take about one hour to crash and start producing similar messages to the one below. This morning nuked everything and started over with a fresh install of 11.2 hoping it was just a version issue. Now the issue comes up within 15 minutes of powering on. When this happens I lose access to the WebUI and all services and the message below appears when running
What is unavailable when it crashes:
Below is the error output from the console when the system crashes. SMART reports do not list any immediate errors but since the messages below are related to the hard drives, I imagine that is going to play a part. I have SSH turned on so I am able to still access that for logs and debugging. I reseated all the hard drives this morning to make sure the connection was solid. I still have my Dell PowerVault MD1200 that I can test with if it seems that the array itself is possibly the issue. I also have iLO setup in case that would be another tool to use in troubleshooting. iLO is how I do all my setup since I run my server completely headless (and I don't have a VGA monitor).
Hardware:
Been running into an issue over the last week and am running out of ideas. I was previously using Unraid on trial and found the performance was bad. So I tried setting up a FreeNAS 11.3 instance last weekend but have had problems for a week. If the server was not being accessed, it would take about one hour to crash and start producing similar messages to the one below. This morning nuked everything and started over with a fresh install of 11.2 hoping it was just a version issue. Now the issue comes up within 15 minutes of powering on. When this happens I lose access to the WebUI and all services and the message below appears when running
dmesg
. I have an online and local backup from when I was running UnRaid so I can nuke and start over if needed.What is unavailable when it crashes:
- Plugins/Jails
- File Sharing
- WebUI
Below is the error output from the console when the system crashes. SMART reports do not list any immediate errors but since the messages below are related to the hard drives, I imagine that is going to play a part. I have SSH turned on so I am able to still access that for logs and debugging. I reseated all the hard drives this morning to make sure the connection was solid. I still have my Dell PowerVault MD1200 that I can test with if it seems that the array itself is possibly the issue. I also have iLO setup in case that would be another tool to use in troubleshooting. iLO is how I do all my setup since I run my server completely headless (and I don't have a VGA monitor).
Hardware:
- HP Proliant DL360 G7
- CPU: Dual Intel Xeon E5620 2.40GHz
- RAM: 12GB 10600R ECC DDR3
- Boot Disk: Adata 16GB USB 3.1 (Connected to internal 2.0 port).
- HBA Card: LSI/Avago 9207-8e (IT Mode)
- SAS Enclosure: Sans Digital EliteSTOR ES316X6+BS (16-Bay Disk Shelf)
- Hard Drives (RAIDz2 w/ Cache):
- 2x WD RED 1TB (2-3 y/o, power on time about 1 year)
- 2x WD Blue 1TB (Brand New)
- 1x WD Green 1TB
- 1x SanDisk 256GB SSD
Code:
Copyright (c) 1992-2018 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 11.2-STABLE #0 r325575+c9231c7d6bd(HEAD): Mon Nov 18 22:46:47 UTC 2019 root@nemesis:/freenas-releng/freenas/_BE/objs/freenas-releng/freenas/_BE/os/sys/FreeNAS.amd64 amd64 FreeBSD clang version 6.0.0 (tags/RELEASE_600/final 326565) (based on LLVM 6.0.0) VT(vga): resolution 640x480 CPU: Intel(R) Xeon(R) CPU E5620 @ 2.40GHz (2400.14-MHz K8-class CPU) Origin="GenuineIntel" Id=0x206c2 Family=0x6 Model=0x2c Stepping=2 Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> Features2=0x29ee3ff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AESNI> AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM> AMD Features2=0x1<LAHF> VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID TSC: P-state invariant, performance statistics real memory = 12884901888 (12288 MB) avail memory = 12416528384 (11841 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: <HP ProLiant> FreeBSD/SMP: Multiprocessor System Detected: 16 CPUs FreeBSD/SMP: 2 package(s) x 4 core(s) x 2 hardware threads WARNING: VIMAGE (virtualized network stack) is a highly experimental feature. Firmware Warning (ACPI): Invalid length for FADT/Pm1aControlBlock: 32, using default 16 (20171214/tbfadt-850) Firmware Warning (ACPI): Invalid length for FADT/Pm2ControlBlock: 32, using default 8 (20171214/tbfadt-850) ioapic1 <Version 2.0> irqs 24-47 on motherboard ioapic0 <Version 2.0> irqs 0-23 on motherboard SMP: AP CPU #1 Launched! SMP: AP CPU #15 Launched! SMP: AP CPU #7 Launched! SMP: AP CPU #11 Launched! SMP: AP CPU #3 Launched! SMP: AP CPU #10 Launched! SMP: AP CPU #6 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #5 Launched! SMP: AP CPU #14 Launched! SMP: AP CPU #12 Launched! SMP: AP CPU #8 Launched! SMP: AP CPU #13 Launched! SMP: AP CPU #9 Launched! SMP: AP CPU #4 Launched! Timecounter "TSC-low" frequency 1200070494 Hz quality 1000 random: entropy device external interface kbd1 at kbdmux0 nexus0 vtvga0: <VT VGA driver> on motherboard cryptosoft0: <software crypto> on motherboard aesni0: <AES-CBC,AES-XTS,AES-GCM,AES-ICM> on motherboard padlock0: No ACE support. acpi0: <HP ProLiant> on motherboard acpi0: Power Button (fixed) cpu0: <ACPI CPU> on acpi0 cpu1: <ACPI CPU> on acpi0 cpu2: <ACPI CPU> on acpi0 cpu3: <ACPI CPU> on acpi0 cpu4: <ACPI CPU> on acpi0 cpu5: <ACPI CPU> on acpi0 cpu6: <ACPI CPU> on acpi0 cpu7: <ACPI CPU> on acpi0 cpu8: <ACPI CPU> on acpi0 cpu9: <ACPI CPU> on acpi0 cpu10: <ACPI CPU> on acpi0 cpu11: <ACPI CPU> on acpi0 cpu12: <ACPI CPU> on acpi0 cpu13: <ACPI CPU> on acpi0 cpu14: <ACPI CPU> on acpi0 cpu15: <ACPI CPU> on acpi0 attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 Event timer "HPET3" frequency 14318180 Hz quality 340 atrtc0: <AT realtime clock> port 0x70-0x71 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s Event timer "RTC" frequency 32768 Hz quality 0 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x908-0x90b on acpi0 pcib0: <ACPI Host-PCI bridge> on acpi0 pci0: <ACPI PCI bus> on pcib0 pcib1: <ACPI PCI-PCI bridge> at device 1.0 on pci0 pci1: <ACPI PCI bus> on pcib1 ciss0: <HP Smart Array P410i> port 0x4000-0x40ff mem 0xfbc00000-0xfbdfffff,0xfbbf0000-0xfbbf0fff irq 28 at device 0.0 on pci1 ciss0: PERFORMANT Transport pcib2: <ACPI PCI-PCI bridge> at device 2.0 on pci0 pci2: <ACPI PCI bus> on pcib2 pcib3: <ACPI PCI-PCI bridge> at device 3.0 on pci0 pci3: <ACPI PCI bus> on pcib3 mps0: <Avago Technologies (LSI) SAS2308> port 0x5000-0x50ff mem 0xfbff0000-0xfbffffff,0xfbf80000-0xfbfbffff irq 24 at device 0.0 on pci3 mps0: Firmware: 20.00.07.00, Driver: 21.02.00.00-fbsd mps0: IOCCapabilities: 5a85c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,MSIXIndex,HostDisc> pcib4: <ACPI PCI-PCI bridge> at device 4.0 on pci0 pci4: <ACPI PCI bus> on pcib4 pcib5: <ACPI PCI-PCI bridge> at device 5.0 on pci0 pci5: <ACPI PCI bus> on pcib5 pcib6: <ACPI PCI-PCI bridge> at device 6.0 on pci0 pci6: <ACPI PCI bus> on pcib6 pcib7: <ACPI PCI-PCI bridge> at device 7.0 on pci0 pci7: <ACPI PCI bus> on pcib7 bce0: <HP NC382i DP Multifunction Gigabit Server Adapter (C0)> mem 0xf6000000-0xf7ffffff irq 30 at device 0.0 on pci7 miibus0: <MII bus> on bce0 brgphy0: <BCM5709 10/100/1000baseT PHY> PHY 1 on miibus0 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow bce0: Using defaults for TSO: 65518/35/2048 bce0: Ethernet address: b4:b5:2f:69:4f:84 bce0: bce0: link state changed to DOWN ASIC (0x57092003); Rev (C0); Bus (PCIe x4, 2.5Gbps); B/C (5.2.3); Bufs (RX:2;TX:2;PG:8); Flags (SPLT|MSI|MFW); MFW (NCSI 2.0.12) Coal (RX:6,6,18,18; TX:20,20,80,80) bce1: <HP NC382i DP Multifunction Gigabit Server Adapter (C0)> mem 0xf4000000-0xf5ffffff irq 37 at device 0.1 on pci7 miibus1: <MII bus> on bce1 brgphy1: <BCM5709 10/100/1000baseT PHY> PHY 1 on miibus1 brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow bce1: Using defaults for TSO: 65518/35/2048 bce1: Ethernet address: b4:b5:2f:69:4f:86 bce1: bce1: link state changed to DOWN ASIC (0x57092003); Rev (C0); Bus (PCIe x4, 2.5Gbps); B/C (5.2.3); Bufs (RX:2;TX:2;PG:8); Flags (SPLT|MSI|MFW); MFW (NCSI 2.0.12) Coal (RX:6,6,18,18; TX:20,20,80,80) pcib8: <ACPI PCI-PCI bridge> at device 8.0 on pci0 pci8: <ACPI PCI bus> on pcib8 bce2: <HP NC382i DP Multifunction Gigabit Server Adapter (C0)> mem 0xf8000000-0xf9ffffff irq 31 at device 0.0 on pci8 miibus2: <MII bus> on bce2 brgphy2: <BCM5709 10/100/1000baseT PHY> PHY 1 on miibus2 brgphy2: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow bce2: Using defaults for TSO: 65518/35/2048 bce2: Ethernet address: b4:b5:2f:69:3f:c8 bce2: bce2: link state changed to DOWN ASIC (0x57092003); Rev (C0); Bus (PCIe x4, 2.5Gbps); B/C (5.2.3); Bufs (RX:2;TX:2;PG:8); Flags (SPLT|MSI|MFW); MFW (NCSI 2.0.12) Coal (RX:6,6,18,18; TX:20,20,80,80) pcib9: <ACPI PCI-PCI bridge> at device 9.0 on pci0 pci9: <ACPI PCI bus> on pcib9 pcib10: <ACPI PCI-PCI bridge> at device 10.0 on pci0 pci10: <ACPI PCI bus> on pcib10 pci0: <base peripheral, interrupt controller> at device 20.0 (no driver attached) pci0: <base peripheral, interrupt controller> at device 20.1 (no driver attached) pci0: <base peripheral, interrupt controller> at device 20.2 (no driver attached) pcib11: <ACPI PCI-PCI bridge> at device 28.0 on pci0 pci11: <ACPI PCI bus> on pcib11 pcib12: <ACPI PCI-PCI bridge> at device 28.4 on pci0 pci12: <ACPI PCI bus> on pcib12 uhci0: <HP iLO Standard Virtual USB controller> port 0x3c00-0x3c1f irq 17 at device 0.4 on pci12 usbus0 on uhci0 usbus0: 12Mbps Full Speed USB v1.0 uhci1: <Intel 82801JI (ICH10) USB controller USB-A> port 0x1000-0x101f irq 20 at device 29.0 on pci0 usbus1 on uhci1 usbus1: 12Mbps Full Speed USB v1.0 uhci2: <Intel 82801JI (ICH10) USB controller USB-B> port 0x1020-0x103f irq 23 at device 29.1 on pci0 usbus2 on uhci2 usbus2: 12Mbps Full Speed USB v1.0 uhci3: <Intel 82801JI (ICH10) USB controller USB-C> port 0x1040-0x105f irq 22 at device 29.2 on pci0 usbus3 on uhci3 usbus3: 12Mbps Full Speed USB v1.0 uhci4: <Intel 82801JI (ICH10) USB controller USB-F> port 0x1060-0x107f irq 23 at device 29.3 on pci0 usbus4 on uhci4 usbus4: 12Mbps Full Speed USB v1.0 ehci0: <Intel 82801JI (ICH10) USB 2.0 controller USB-A> mem 0xf3bf0000-0xf3bf03ff irq 20 at device 29.7 on pci0 usbus5: EHCI version 1.0 usbus5 on ehci0 usbus5: 480Mbps High Speed USB v2.0 pcib13: <ACPI PCI-PCI bridge> at device 30.0 on pci0 pci13: <ACPI PCI bus> on pcib13 vgapci0: <VGA-compatible display> port 0x2000-0x20ff mem 0xe8000000-0xefffffff,0xf3cf0000-0xf3cfffff irq 23 at device 3.0 on pci13 vgapci0: Boot video device isab0: <PCI-ISA bridge> at device 31.0 on pci0 isa0: <ISA bus> on isab0 atapci0: <Intel ICH10 SATA300 controller> at device 31.2 on pci0 ata2: <ATA channel> at channel 0 on atapci0 acpi_tz0: <Thermal Zone> on acpi0 ipmi0: <IPMI System Interface> port 0xca2-0xca3 on acpi0 ipmi0: KCS mode found at io 0xca2 on acpi atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0 atkbd0: <AT Keyboard> irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] acpi_syscontainer0: <System Container> port 0x2e-0x2f,0x620-0x65f,0x680-0x69f,0x600-0x61f,0x660-0x67f,0x300-0x31f on acpi0 uart0: <Non-standard ns8250 class UART with FIFOs> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 ichwd0: <Intel ICH10 watchdog timer> on isa0 ichwd0: ICH WDT present but disabled in BIOS or hardware device_attach: ichwd0 attach returned 6 ichwd0: <Intel ICH10 watchdog timer> at port 0x930-0x937,0x960-0x97f on isa0 ichwd0: ICH WDT present but disabled in BIOS or hardware device_attach: ichwd0 attach returned 6 orm0: <ISA Option ROM> at iomem 0xc0000-0xcafff on isa0 uart1: <Non-standard ns8250 class UART with FIFOs> at port 0x2f8 irq 3 on isa0 coretemp0: <CPU On-Die Thermal Sensors> on cpu0 est0: <Enhanced SpeedStep Frequency Control> on cpu0 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 13 device_attach: est0 attach returned 6 coretemp1: <CPU On-Die Thermal Sensors> on cpu1 est1: <Enhanced SpeedStep Frequency Control> on cpu1 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 13 device_attach: est1 attach returned 6 coretemp2: <CPU On-Die Thermal Sensors> on cpu2 est2: <Enhanced SpeedStep Frequency Control> on cpu2 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 13 device_attach: est2 attach returned 6 coretemp3: <CPU On-Die Thermal Sensors> on cpu3 est3: <Enhanced SpeedStep Frequency Control> on cpu3 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 13 device_attach: est3 attach returned 6 coretemp4: <CPU On-Die Thermal Sensors> on cpu4 est4: <Enhanced SpeedStep Frequency Control> on cpu4 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 13 device_attach: est4 attach returned 6 coretemp5: <CPU On-Die Thermal Sensors> on cpu5 est5: <Enhanced SpeedStep Frequency Control> on cpu5 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 13 device_attach: est5 attach returned 6 coretemp6: <CPU On-Die Thermal Sensors> on cpu6 est6: <Enhanced SpeedStep Frequency Control> on cpu6 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 13 device_attach: est6 attach returned 6 coretemp7: <CPU On-Die Thermal Sensors> on cpu7 est7: <Enhanced SpeedStep Frequency Control> on cpu7 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 13 device_attach: est7 attach returned 6 coretemp8: <CPU On-Die Thermal Sensors> on cpu8 est8: <Enhanced SpeedStep Frequency Control> on cpu8 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 13 device_attach: est8 attach returned 6 coretemp9: <CPU On-Die Thermal Sensors> on cpu9 est9: <Enhanced SpeedStep Frequency Control> on cpu9 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 13 device_attach: est9 attach returned 6 coretemp10: <CPU On-Die Thermal Sensors> on cpu10 est10: <Enhanced SpeedStep Frequency Control> on cpu10 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 13 device_attach: est10 attach returned 6 coretemp11: <CPU On-Die Thermal Sensors> on cpu11 est11: <Enhanced SpeedStep Frequency Control> on cpu11 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 13 device_attach: est11 attach returned 6 coretemp12: <CPU On-Die Thermal Sensors> on cpu12 est12: <Enhanced SpeedStep Frequency Control> on cpu12 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 13 device_attach: est12 attach returned 6 coretemp13: <CPU On-Die Thermal Sensors> on cpu13 est13: <Enhanced SpeedStep Frequency Control> on cpu13 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 13 device_attach: est13 attach returned 6 coretemp14: <CPU On-Die Thermal Sensors> on cpu14 est14: <Enhanced SpeedStep Frequency Control> on cpu14 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 13 device_attach: est14 attach returned 6 coretemp15: <CPU On-Die Thermal Sensors> on cpu15 est15: <Enhanced SpeedStep Frequency Control> on cpu15 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 13 device_attach: est15 attach returned 6 ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Timecounters tick every 1.000 msec freenas_sysctl: adding account. freenas_sysctl: adding directoryservice. freenas_sysctl: adding middlewared. freenas_sysctl: adding network. freenas_sysctl: adding services. ipfw2 (+ipv6) initialized, divert enabled, nat enabled, default to accept, logging disabled ugen3.1: <Intel UHCI root HUB> at usbus3 ugen1.1: <Intel UHCI root HUB> at usbus1 ugen4.1: <Intel UHCI root HUB> at usbus4 ugen2.1: <Intel UHCI root HUB> at usbus2 ugen5.1: <Intel EHCI root HUB> at usbus5 ugen0.1: <HP UHCI root HUB> at usbus0 uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus3 uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1 uhub2: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus4 uhub3: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus5 uhub4: <HP UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0 uhub5: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2 mps0: SAS Address for SATA device = 4e66525204c9e17e mps0: SAS Address for SATA device = 3976383cba9b8d7e ipmi0: IPMI device rev. 1, firmware rev. 1.28, version 2.0 mps0: SAS Address from SATA device = 4e66525204c9e17e mps0: SAS Address from SATA device = 3976383cba9b8d7e mps0: SAS Address for SATA device = dd615e69f4bac689 ipmi0: Number of channels 0 mps0: SAS Address for SATA device = 4e626647f2c7cf95 mps0: SAS Address for SATA device = 6305523a9b71a0a4 ipmi0: Attached watchdog mps0: SAS Address for SATA device = dd616467f3c9b271 mps0: SAS Address for SATA device = 6269520597baa982 mps0: SAS Address from SATA device = dd615e69f4bac689 mps0: SAS Address from SATA device = 4e626647f2c7cf95 mps0: SAS Address from SATA device = 6305523a9b71a0a4 mps0: SAS Address from SATA device = dd616467f3c9b271 mps0: SAS Address from SATA device = 6269520597baa982 uhub0: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered uhub2: 2 ports with 2 removable, self powered uhub4: 2 ports with 2 removable, self powered uhub5: 2 ports with 2 removable, self powered ugen0.2: <HP Virtual Keyboard> at usbus0 ukbd0 on uhub4 ukbd0: <Virtual Keyboard > on usbus0 kbd2 at ukbd0 uhub3: 8 ports with 8 removable, self powered ugen5.2: <Innostor PenDrive> at usbus5 umass0 on uhub3 umass0: <Innostor PenDrive, class 0/0, rev 2.10/0.01, addr 2> on usbus5 umass0: SCSI over Bulk-Only; quirks = 0x8100 umass0:5:0: Attached to scbus5 random: unblocking device. ses0 at mps0 bus 0 scbus2 target 12 lun 0 ses0: <SansDigi SDEXP24 0e03> Fixed Enclosure Services SPC-3 SCSI device ses0: Serial Number ses0: 600.000MB/s transfers ses0: Command Queueing enabled ses0: SCSI-3 ENC Device da7 at umass-sim0 bus 0 scbus5 target 0 lun 0 da0 at mps0 bus 0 scbus2 target 8 lun 0 da7: <Innostor Innostor 1.00> Removable Direct Access SPC-4 SCSI device da7: Serial Number 151876716014230 da7: 40.000MB/s transfers da7: 15074MB (30871552 512 byte sectors) da7: quirks=0x12<NO_6_BYTE,NO_RC16> da6 at mps0 bus 0 scbus2 target 24 lun 0 da0: <ATA WDC WD10EFRX-68F 0A82> Fixed Direct Access SPC-4 SCSI device da0: Serial Number WD-WCC4J4DNXFZA da0: 600.000MB/s transfers da0: Command Queueing enabled da6: <ATA WDC WD10EZEX-08W 1A02> Fixed Direct Access SPC-4 SCSI device da6: Serial Number WD-WCC6Y5VRRSE2 da6: 600.000MB/s transfers da6: Command Queueing enabled da0: 953869MB (1953525168 512 byte sectors) da0: quirks=0x8<4K> da6: 953869MB (1953525168 512 byte sectors) da1 at mps0 bus 0 scbus2 target 10 lun 0 ses0: da0,pass0: Element descriptor: 'Disk-01' da4 at mps0 bus 0 scbus2 target 20 lun 0 da1: <ATA WDC WD10EFRX-68F 0A82> Fixed Direct Access SPC-4 SCSI device da4: <ATA SanDisk SDSSDA24 00RL> Fixed Direct Access SPC-4 SCSI device da4: Serial Number 151554400865 da4: 600.000MB/s transfers da4: Command Queueing enabled ses0: da0,pass0: SAS Device Slot Element: 1 Phys at Slot 1 ses0: phy 0: SATA device ses0: phy 0: parent 500180d0000564ff addr 500180d0000564c0 da1: Serial Number WD-WCC4J0XCFEHX da1: 600.000MB/s transfers da1: Command Queueing enabled da1: 953869MB (1953525168 512 byte sectors) da4: 228936MB (468862128 512 byte sectors) da1: quirks=0x8<4K> da2 at mps0 bus 0 scbus2 target 13 lun 0 da3 at mps0 bus 0 scbus2 target 14 lun 0 da2: <ATA WDC WD10EADS-22M 0A01> Fixed Direct Access SPC-4 SCSI device da2: Serial Number WD-WCAV5H015044 da2: 300.000MB/s transfers da2: Command Queueing enabled da2: 953869MB (1953525168 512 byte sectors) da5 at mps0 bus 0 scbus2 target 22 lun 0 da3: <ATA APPLE HDD ST1000 AQ03> Fixed Direct Access SPC-4 SCSI device da3: Serial Number Z4YER3LS da3: 300.000MB/s transfers da3: Command Queueing enabled da3: 953869MB (1953525168 512 byte sectors) da5: <ATA WDC WD10EZEX-08W 1A02> Fixed Direct Access SPC-4 SCSI device da5: Serial Number WD-WCC6Y5PTSDYJ da5: 600.000MB/s transfers da5: Command Queueing enabled da5: 953869MB (1953525168 512 byte sectors) ses0: da2,pass3: Element descriptor: 'Disk-02' ses0: da2,pass3: SAS Device Slot Element: 1 Phys at Slot 2 ses0: phy 0: SATA device ses0: phy 0: parent 500180d0000564ff addr 500180d0000564c1 ses0: da1,pass1: Element descriptor: 'Disk-05' ses0: da1,pass1: SAS Device Slot Element: 1 Phys at Slot 5 ses0: phy 0: SATA device ses0: phy 0: parent 500180d0000564ff addr 500180d0000564dc ses0: da3,pass4: Element descriptor: 'Disk-06' ses0: da3,pass4: SAS Device Slot Element: 1 Phys at Slot 6 ses0: phy 0: SATA device ses0: phy 0: parent 500180d0000564ff addr 500180d0000564dd ses0: da6,pass7: Element descriptor: 'Disk-10' ses0: da6,pass7: SAS Device Slot Element: 1 Phys at Slot 10 ses0: phy 0: SATA device ses0: phy 0: parent 500180d0000564ff addr 500180d0000564e1 ses0: da4,pass5: Element descriptor: 'Disk-12' ses0: da4,pass5: SAS Device Slot Element: 1 Phys at Slot 12 ses0: phy 0: SATA device ses0: phy 0: parent 500180d0000564ff addr 500180d0000564e3 ses0: da5,pass6: Element descriptor: 'Disk-13' ses0: da5,pass6: SAS Device Slot Element: 1 Phys at Slot 13 ses0: phy 0: SATA device ses0: phy 0: parent 500180d0000564ff addr 500180d0000564d8 Trying to mount root from zfs:freenas-boot/ROOT/default []... GEOM_MIRROR: Device mirror/swap0 launched (2/2). GEOM_MIRROR: Device mirror/swap1 launched (2/2). GEOM_ELI: Device mirror/swap0.eli created. GEOM_ELI: Encryption: AES-XTS 128 GEOM_ELI: Crypto: hardware GEOM_ELI: Device mirror/swap1.eli created. GEOM_ELI: Encryption: AES-XTS 128 GEOM_ELI: Crypto: hardware hwpmc: SOFT/16/64/0x67<INT,USR,SYS,REA,WRI> TSC/1/64/0x20<REA> IAP/4/48/0x3ff<INT,USR,SYS,EDG,THR,REA,WRI,INV,QUA,PRC> IAF/3/48/0x67<INT,USR,SYS,REA,WRI> UCP/8/48/0x3f8<EDG,THR,REA,WRI,INV,QUA,PRC> UCF/1/48/0x60<REA,WRI> bce0: Gigabit link up! bce0: link state changed to UP bce0: Gigabit link up! ums0 on uhub4 ums0: <Virtual Mouse> on usbus0 GEOM_MIRROR: Device mirror/swap2 launched (2/2). GEOM_ELI: Device mirror/swap2.eli created. GEOM_ELI: Encryption: AES-XTS 128 GEOM_ELI: Crypto: hardware mps0: mpssas_prepare_remove: Sending reset for target ID 14 da3 at mps0 bus 0 scbus2 target 14 lun 0 da3: <ATA APPLE HDD ST1000 AQ03> s/n Z4YER3LS detached GEOM_MIRROR: Device swap0: provider da3p1 disconnected. (da3:mps0:0:14:0): Periph destroyed mps0: Unfreezing devq for target ID 14 GEOM_ELI: Device mirror/swap0.eli destroyed. GEOM_MIRROR: Device swap0: provider destroyed. GEOM_MIRROR: Device swap0 destroyed. mps0: mpssas_prepare_remove: Sending reset for target ID 8 mps0: mpssas_prepare_remove: Sending reset for target ID 13 da0 at mps0 bus 0 scbus2 target 8 lun 0 da0: <ATA WDC WD10EFRX-68F 0A82> s/n WD-WCC4J4DNXFZA detached da2 at mps0 bus 0 scbus2 target 13 lun 0 da2: <ATA WDC WD10EADS-22M 0A01> s/n WD-WCAV5H015044 detached GEOM_MIRROR: Device swap1: provider da2p1 disconnected. GEOM_MIRROR: Device swap2: provider da0p1 disconnected. (da2:mps0:0:13:0): Periph destroyed (da0:mps0:0:8:0): Periph destroyed mps0: Unfreezing devq for target ID 8 mps0: Unfreezing devq for target ID 13 mps0: mpssas_prepare_remove: Sending reset for target ID 22 mps0: mpssas_prepare_remove: Sending reset for target ID 10 mps0: mpssas_prepare_remove: Sending reset for target ID 24 da5 at mps0 bus 0 scbus2 target 22 lun 0 da5: <ATA WDC WD10EZEX-08W 1A02> s/n WD-WCC6Y5PTSDYJ detached da1 at mps0 bus 0 scbus2 target 10 lun 0 da1: <ATA WDC WD10EFRX-68F 0A82> s/n WD-WCC4J0XCFEHX detached da6 at mps0 bus 0 scbus2 target 24 lun 0 da6: <ATA WDC WD10EZEX-08W 1A02> s/n WD-WCC6Y5VRRSE2 detached GEOM_MIRROR: Device swap1: provider da1p1 disconnected. GEOM_MIRROR: Device swap2: provider da6p1 disconnected. GEOM_MIRROR: Device swap2: provider destroyed. GEOM_MIRROR: Device swap2 destroyed. GEOM_ELI: Device mirror/swap2.eli destroyed. GEOM_ELI: Detached mirror/swap2.eli on last close. (da1:mps0:0:10:0): Periph destroyed (da6:mps0:0:24:0): Periph destroyed (da5:mps0:0:22:0): Periph destroyed GEOM_MIRROR: Device swap1: provider destroyed. GEOM_MIRROR: Device swap1 destroyed. mps0: Unfreezing devq for target ID 22 mps0: Unfreezing devq for target ID 10 mps0: Unfreezing devq for target ID 24 GEOM_ELI: Device mirror/swap1.eli destroyed. GEOM_ELI: Detached mirror/swap1.eli on last close. mpssas_get_sata_identify: error reading SATA PASSTHRU; iocstatus = 0x47 mpssas_get_sata_identify: error reading SATA PASSTHRU; iocstatus = 0x804b mps0: Sleeping 3 seconds after SATA ID error to wait for spinup mps0: SAS Address for SATA device = dd615e69f4bac689 mps0: SAS Address for SATA device = dd616467f3c9b271 mps0: SAS Address from SATA device = dd615e69f4bac689 mps0: SAS Address from SATA device = dd616467f3c9b271 ses0: da0,pass0: Element descriptor: 'Disk-10' ses0: da0,pass0: SAS Device Slot Element: 1 Phys at Slot 10 ses0: phy 0: SATA device ses0: phy 0: parent 500180d0000564ff addr 500180d0000564e1 ses0: da1,pass1: Element descriptor: 'Disk-13' ses0: da1,pass1: SAS Device Slot Element: 1 Phys at Slot 13 ses0: phy 0: SATA device ses0: phy 0: parent 500180d0000564ff addr 500180d0000564d8 da0 at mps0 bus 0 scbus2 target 24 lun 0 da0: <ATA WDC WD10EZEX-08W 1A02> Fixed Direct Access SPC-4 SCSI device da0: Serial Number WD-WCC6Y5VRRSE2 da0: 600.000MB/s transfers da0: Command Queueing enabled da0: 953869MB (1953525168 512 byte sectors) da1 at mps0 bus 0 scbus2 target 22 lun 0 da1: <ATA WDC WD10EZEX-08W 1A02> Fixed Direct Access SPC-4 SCSI device da1: Serial Number WD-WCC6Y5PTSDYJ da1: 600.000MB/s transfers da1: Command Queueing enabled da1: 953869MB (1953525168 512 byte sectors) mps0: SAS Address for SATA device = 3976383cba9b8d7e mps0: SAS Address from SATA device = 3976383cba9b8d7e ses0: da2,pass3: Element descriptor: 'Disk-02' ses0: da2,pass3: SAS Device Slot Element: 1 Phys at Slot 2 ses0: phy 0: SATA device ses0: phy 0: parent 500180d0000564ff addr 500180d0000564c1 da2 at mps0 bus 0 scbus2 target 13 lun 0 da2: <ATA WDC WD10EADS-22M 0A01> Fixed Direct Access SPC-4 SCSI device da2: Serial Number WD-WCAV5H015044 da2: 300.000MB/s transfers da2: Command Queueing enabled da2: 953869MB (1953525168 512 byte sectors) mps0: SAS Address for SATA device = 4e66525204c9e17e mps0: SAS Address from SATA device = 4e66525204c9e17e mps0: SAS Address for SATA device = 4e626647f2c7cf95 mps0: SAS Address from SATA device = 4e626647f2c7cf95 ses0: da5,pass6: Element descriptor: 'Disk-01' ses0: da5,pass6: SAS Device Slot Element: 1 Phys at Slot 1 ses0: phy 0: SATA device ses0: phy 0: parent 500180d0000564ff addr 500180d0000564c0 ses0: da3,pass4: Element descriptor: 'Disk-05' ses0: da3,pass4: SAS Device Slot Element: 1 Phys at Slot 5 ses0: phy 0: SATA device ses0: phy 0: parent 500180d0000564ff addr 500180d0000564dc da3 at mps0 bus 0 scbus2 target 10 lun 0 da3: <ATA WDC WD10EFRX-68F 0A82> Fixed Direct Access SPC-4 SCSI device da3: Serial Number WD-WCC4J0XCFEHX da3: 600.000MB/s transfers da3: Command Queueing enabled da3: 953869MB (1953525168 512 byte sectors) da3: quirks=0x8<4K> da5 at mps0 bus 0 scbus2 target 8 lun 0 da5: <ATA WDC WD10EFRX-68F 0A82> Fixed Direct Access SPC-4 SCSI device da5: Serial Number WD-WCC4J4DNXFZA da5: 600.000MB/s transfers da5: Command Queueing enabled da5: 953869MB (1953525168 512 byte sectors) da5: quirks=0x8<4K>