k1n6b0b
Cadet
- Joined
- Jan 16, 2018
- Messages
- 3
Hello! First post. This is likely a BSD kernel issue but I'm pretty new to BSD and not sure how to debug this.
Stuff search has pointed me to but not worked:
Thanks!
I've attached dmesg output of a system boot without the disk shelf attached.
and a dmesg after connecting the 2 cables to the shelf (this is after the volume is imported/known)
Stuff search has pointed me to but not worked:
- Disable firewire (I dont have firewire)
- Add the following to /boot/device.hints
hint.ata.0.disabled="1"
hint.ata.1.disabled="1"
This appears to have the affect of not showing the HBA in lspci (so its disabled/masked out?). Either way, not helpful for me!
- HBA: NetApp X2065A-R6 AKA PMC-Sierra PM8003
05:00.0 Serial Attached SCSI controller: PMC-Sierra Inc. Device 8001 (rev 05)
06:00.0 Serial Attached SCSI controller: PMC-Sierra Inc. Device 8001 (rev 05) - Disks: Netapp DS2246 Disk Shelf (basically an external SAS 2U with an expander)
- FYI I am booting from USB in all scenarios
- HP DL380 G7 (newest BIOS)
- ASUS P10S-M uATX - Server LGA 1151 Intel C232 (newest BIOS)
- If I boot the system without anything connected, it boots fine -- but then will not connect to the Disk Shelf **if the volume is not known by FreeNAS**
- If I boot the system with the shelf connected (single or dual cables), after GRUB as we are loading, the lights turn green. Shortly after the system hangs and begins spitting
run_interrupt_driven_hookserrors (forever) - If I boot the system with the shelf connected (single or dual cables) - as above - but then remove all cables when it hangs... no luck.. still hangs forever
- If I boot the system without anything connected, wait for the green lights on the shelf (part way through the kernel loading) -- PULL the cables -- wait until I get past
random: unblocking device--- (looks like the end of scanning devices) -- RECONNECT them . Everything works great! - If I use a different HBA (like a SAS 2003 with a mini SAS - QSFP+ cable) I do not have any issues, so I expect this is hardware driver related to the PMC-Sierra HBA. Something the kernel is trying to do/scan that it obviously doesnt need but it does need to activate the connection
- After I "import volume" (even if that volume isnt all the disks in the shelf) I still *cannot boot* with it attached, however I can plug the cables in after boot, and it will load the disks but does not load the volume:
0 (Error) Error getting available spaceshow in the GUI under View Volumes
Thanks!
I've attached dmesg output of a system boot without the disk shelf attached.
and a dmesg after connecting the 2 cables to the shelf (this is after the volume is imported/known)
root@nas1:~ # dmesg
Copyright (c) 1992-2017 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 11.1-STABLE #0 r321665+d4625dcee3e(freenas/11.1-stable): Wed Dec 13 16:33:42 UTC 2017
root@gauntlet:/freenas-11-releng/freenas/_BE/objs/freenas-11-releng/freenas/_BE/os/sys/FreeNAS.amd64 amd64
FreeBSD clang version 5.0.0 (tags/RELEASE_500/final 312559) (based on LLVM 5.0.0svn)
CPU: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz (4008.20-MHz K8-class CPU)
Origin="GenuineIntel" Id=0x506e3 Family=0x6 Model=0x5e Stepping=3
Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0x7ffafbbf<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,SDBG,FMA,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
AMD Features2=0x121<LAHF,ABM,Prefetch>
Structured Extended Features=0x29c6fbf<FSGSBASE,TSCADJ,SGX,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,NFPUSG,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PROCTRACE>
XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID
TSC: P-state invariant, performance statistics
real memory = 19058917376 (18176 MB)
avail memory = 16489033728 (15725 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <ALASKA A M I >
FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s) x 2 hardware threads
WARNING: VIMAGE (virtualized network stack) is a highly experimental feature.
ioapic0 <Version 2.0> irqs 0-119 on motherboard
SMP: AP CPU #1 Launched!
SMP: AP CPU #2 Launched!
SMP: AP CPU #3 Launched!
SMP: AP CPU #4 Launched!
SMP: AP CPU #5 Launched!
SMP: AP CPU #6 Launched!
SMP: AP CPU #7 Launched!
Timecounter "TSC-low" frequency 2004099208 Hz quality 1000
random: entropy device external interface
kbd1 at kbdmux0
random: registering fast source Intel Secure Key RNG
random: fast provider: "Intel Secure Key RNG"
nexus0
cryptosoft0: <software crypto> on motherboard
aesni0: <AES-CBC,AES-XTS,AES-GCM,AES-ICM> on motherboard
padlock0: No ACE support.
acpi0: <ALASKA A M I > on motherboard
acpi0: Power Button (fixed)
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
cpu2: <ACPI CPU> on acpi0
cpu3: <ACPI CPU> on acpi0
cpu4: <ACPI CPU> on acpi0
cpu5: <ACPI CPU> on acpi0
cpu6: <ACPI CPU> on acpi0
cpu7: <ACPI CPU> on acpi0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 24000000 Hz quality 950
Event timer "HPET" frequency 24000000 Hz quality 550
atrtc0: <AT realtime clock> port 0x70-0x77 irq 8 on acpi0
atrtc0: Warning: Couldn't map I/O.
atrtc0: registered as a time-of-day clock, resolution 1.000000s
Event timer "RTC" frequency 32768 Hz quality 0
attimer0: <AT timer> port 0x40-0x43,0x50-0x53 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1808-0x180b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pcib0: _OSC returned error 0x4
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> irq 16 at device 1.0 on pci0
pci1: <ACPI PCI bus> on pcib1
pcib2: <ACPI PCI-PCI bridge> mem 0xde500000-0xde51ffff irq 16 at device 0.0 on pci1
pci2: <ACPI PCI bus> on pcib2
pcib3: <PCI-PCI bridge> irq 16 at device 4.0 on pci2
pci3: <PCI bus> on pcib3
pcib4: <PCI-PCI bridge> irq 17 at device 5.0 on pci2
pci4: <PCI bus> on pcib4
pmspcv0: <PMC Sierra SPC SAS-SATA Card> mem 0xde430000-0xde43ffff,0xde420000-0xde42ffff,0xde410000-0xde41ffff,0xde400000-0xde40ffff irq 17 at device 0.0 on pci4
pcib5: <PCI-PCI bridge> irq 16 at device 8.0 on pci2
pci5: <PCI bus> on pcib5
pmspcv1: <PMC Sierra SPC SAS-SATA Card> mem 0xde230000-0xde23ffff,0xde220000-0xde22ffff,0xde210000-0xde21ffff,0xde200000-0xde20ffff irq 16 at device 0.0 on pci5
xhci0: <Intel Sunrise Point USB 3.0 controller> mem 0xde800000-0xde80ffff irq 16 at device 20.0 on pci0
xhci0: 32 bytes context size, 64-bit DMA
usbus0: waiting for BIOS to give up control
xhci_interrupt: host controller halted
usbus0 on xhci0
usbus0: 5.0Gbps Super Speed USB v3.0
pci0: <simple comms> at device 22.0 (no driver attached)
ahci0: <Intel Sunrise Point AHCI SATA controller> port 0xf050-0xf057,0xf040-0xf043,0xf020-0xf03f mem 0xde810000-0xde811fff,0xde81e000-0xde81e0ff,0xde81d000-0xde81d7ff irq 16 at device 23.0 on pci0
ahci0: AHCI v1.31 with 6 6Gbps ports, Port Multiplier not supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich1: <AHCI channel> at channel 1 on ahci0
ahcich2: <AHCI channel> at channel 2 on ahci0
ahcich3: <AHCI channel> at channel 3 on ahci0
ahcich4: <AHCI channel> at channel 4 on ahci0
ahcich5: <AHCI channel> at channel 5 on ahci0
pcib6: <ACPI PCI-PCI bridge> at device 28.0 on pci0
pci6: <ACPI PCI bus> on pcib6
pcib7: <ACPI PCI-PCI bridge> at device 0.0 on pci6
pci7: <ACPI PCI bus> on pcib7
vgapci0: <VGA-compatible display> port 0xe000-0xe07f mem 0xdc000000-0xddffffff,0xde000000-0xde01ffff irq 17 at device 0.0 on pci7
vgapci0: Boot video device
pcib8: <ACPI PCI-PCI bridge> at device 28.5 on pci0
pci8: <ACPI PCI bus> on pcib8
igb0: <Intel(R) PRO/1000 Network Connection, Version - 2.5.3-k> port 0xd000-0xd01f mem 0xde700000-0xde77ffff,0xde780000-0xde783fff irq 20 at device 0.0 on pci8
igb0: Using MSIX interrupts with 5 vectors
igb0: Ethernet address: 34:97:f6:00:df:b3
igb0: link state changed to UP
igb0: PHY reset is blocked due to SOL/IDER session.
igb0: Bound queue 0 to cpu 0
igb0: Bound queue 1 to cpu 1
igb0: Bound queue 2 to cpu 2
igb0: Bound queue 3 to cpu 3
pcib9: <ACPI PCI-PCI bridge> at device 28.6 on pci0
pci9: <ACPI PCI bus> on pcib9
igb1: <Intel(R) PRO/1000 Network Connection, Version - 2.5.3-k> port 0xc000-0xc01f mem 0xde600000-0xde67ffff,0xde680000-0xde683fff irq 21 at device 0.0 on pci9
igb1: Using MSIX interrupts with 5 vectors
igb1: Ethernet address: 34:97:f6:00:df:b4
igb1: Bound queue 0 to cpu 4
igb1: Bound queue 1 to cpu 5
igb1: Bound queue 2 to cpu 6
igb1: Bound queue 3 to cpu 7
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
pci0: <memory> at device 31.2 (no driver attached)
acpi_button0: <Sleep Button> on acpi0
acpi_button1: <Power Button> on acpi0
acpi_tz0: <Thermal Zone> on acpi0
acpi_tz1: <Thermal Zone> on acpi0
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
acpi_syscontainer0: <System Container> on acpi0
wbwd0: <Nuvoton NCT6791 (0xc8/0x03) Watchdog Timer> at port 0x2e-0x2f on isa0
orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xc8000-0xc8fff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
coretemp0: <CPU On-Die Thermal Sensors> on cpu0
est0: <Enhanced SpeedStep Frequency Control> on cpu0
coretemp1: <CPU On-Die Thermal Sensors> on cpu1
est1: <Enhanced SpeedStep Frequency Control> on cpu1
coretemp2: <CPU On-Die Thermal Sensors> on cpu2
est2: <Enhanced SpeedStep Frequency Control> on cpu2
coretemp3: <CPU On-Die Thermal Sensors> on cpu3
est3: <Enhanced SpeedStep Frequency Control> on cpu3
coretemp4: <CPU On-Die Thermal Sensors> on cpu4
est4: <Enhanced SpeedStep Frequency Control> on cpu4
coretemp5: <CPU On-Die Thermal Sensors> on cpu5
est5: <Enhanced SpeedStep Frequency Control> on cpu5
coretemp6: <CPU On-Die Thermal Sensors> on cpu6
est6: <Enhanced SpeedStep Frequency Control> on cpu6
coretemp7: <CPU On-Die Thermal Sensors> on cpu7
est7: <Enhanced SpeedStep Frequency Control> on cpu7
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
Timecounters tick every 1.000 msec
freenas_sysctl: adding account.
freenas_sysctl: adding directoryservice.
freenas_sysctl: adding middlewared.
freenas_sysctl: adding network.
freenas_sysctl: adding services.
ipfw2 (+ipv6) initialized, divert enabled, nat enabled, default to accept, logging disabled
nvme cam probe device init
ugen0.1: <0x8086 XHCI root HUB> at usbus0
uhub0: <0x8086 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
uhub0: 22 ports with 22 removable, self powered
ugen0.2: <SanDisk Ultra Fit> at usbus0
umass0 on uhub0
umass0: <SanDisk Ultra Fit, class 0/0, rev 3.00/1.00, addr 1> on usbus0
umass0: SCSI over Bulk-Only; quirks = 0xc100
umass0:9:0: Attached to scbus9
ugen0.3: <American Megatrends Inc. Virtual Hub> at usbus0
uhub1 on uhub0
uhub1: <7-port Hub> on usbus0
uhub1: 5 ports with 5 removable, self powered
ugen0.4: <American Megatrends Inc. Virtual Cdrom Device> at usbus0
umass1 on uhub1
umass1: <Virtual Cdrom> on usbus0
umass1: SCSI over Bulk-Only; quirks = 0x0100
umass1:10:1: Attached to scbus10
ugen0.5: <American Megatrends Inc. Virtual Floppy Device> at usbus0
umass2 on uhub1
umass2: <Virtual Floppy> on usbus0
umass2: SCSI over Bulk-Only; quirks = 0x0100
umass2:11:2: Attached to scbus11
ugen0.6: <American Megatrends Inc. Virtual HardDisk Device> at usbus0
umass3 on uhub1
umass3: <Virtual HardDisk> on usbus0
umass3: SCSI over Bulk-Only; quirks = 0x0100
umass3:12:3: Attached to scbus12
ugen0.7: <American Megatrends Inc. Virtual Keyboard and Mouse> at usbus0
ukbd0 on uhub1
ukbd0: <Keyboard Interface> on usbus0
kbd2 at ukbd0
random: unblocking device.
da0 at umass-sim0 bus 0 scbus9 target 0 lun 0
da0: <SanDisk Ultra Fit 1.00> Removable Direct Access SPC-4 SCSI device
da0: Serial Number 4C530001221114103442
da0: 400.000MB/s transfers
da0: 29328MB (60063744 512 byte sectors)
da0: quirks=0x2<NO_6_BYTE>
da1 at umass-sim2 bus 2 scbus11 target 0 lun 0
da1: <AMI Virtual Floppy0 1.00> Removable Direct Access SCSI device
da1: Serial Number AAAABBBBCCCC2
da1: 40.000MB/s transfers
da1: Attempt to query device size failed: NOT READY, Medium not present
da1: quirks=0x2<NO_6_BYTE>
da2 at umass-sim3 bus 3 scbus12 target 0 lun 0
da2: <AMI Virtual HDisk0 1.00> Removable Direct Access SCSI device
da2: Serial Number AAAABBBBCCCC3
da2: 40.000MB/s transfers
da2: Attempt to query device size failed: NOT READY, Medium not present
da2: quirks=0x2<NO_6_BYTE>
cd0 at umass-sim1 bus 1 scbus10 target 0 lun 0
cd0: <AMI Virtual CDROM0 1.00> Removable CD-ROM SCSI device
cd0: Serial Number AAAABBBBCCCC1
cd0: 40.000MB/s transfers
cd0: Attempt to query device size failed: NOT READY, Medium not present
cd0: quirks=0x10<10_BYTE_ONLY>
Trying to mount root from zfs:freenas-boot/ROOT/default []...
igb1: link state changed to UP
ipmi0: <IPMI System Interface> port 0xca2,0xca3 on acpi0
ipmi0: KCS mode found at io 0xca2 on acpi
ipmi0: IPMI device rev. 1, firmware rev. 2.04, version 2.0
ipmi0: Number of channels 0
ipmi0: Attached watchdog
GEOM_RAID5: Module loaded, version 1.3.20140711.62 (rev f91e28e40bf7)
hwpmc: SOFT/16/64/0x67<INT,USR,SYS,REA,WRI> TSC/1/64/0x20<REA> IAP/4/48/0x3ff<INT,USR,SYS,EDG,THR,REA,WRI,INV,QUA,PRC> IAF/3/48/0x67<INT,USR,SYS,REA,WRI>
ums0 on uhub1
ums0: <Mouse Interface> on usbus0
ums0: 3 buttons and [Z] coordinates ID=0
nfsd: can't register svc name
pid 1711 (syslog-ng), uid 0: exited on signal 6 (core dumped)
da3 at pmspcbsd1 bus 0 scbus1 target 0 lun 0
da3: <NETAPP X423_HCOBE900A10 NA00> Fixed Direct Access SPC-4 SCSI device
da3: Serial Number KPHR5N6F
da3: 600.000MB/s transfers
da3: Command Queueing enabled
da3: 858483MB (1758174768 512 byte sectors)
da4 at pmspcbsd1 bus 0 scbus1 target 1 lun 0
GEOM_MULTIPATH: disk9 created
da4: <NETAPP X423_HCOBE900A10 NA00> Fixed Direct Access SPC-4 SCSI device
da4: Serial Number KPHS4GPF
da4: 600.000MB/s transfers
da4: Command Queueing enabled
da4: 858483MB (1758174768 512 byte sectors)
da5 at pmspcbsd1 bus 0 scbus1 target 2 lun 0
da5: <NETAPP X423_HCOBE900A10 NA00> Fixed Direct Access SPC-4 SCSI device
da5: Serial Number KPHRPK4F
da5: 600.000MB/s transfers
da5: Command Queueing enabled
da5: 858483MB (1758174768 512 byte sectors)
GEOM_MULTIPATH: da3 added to disk9
da6 at pmspcbsd1 bus 0 scbus1 target 3 lun 0
da6: <NETAPP X423_HCOBE900A10 NA00> Fixed Direct Access SPC-4 SCSI device
da6: Serial Number KPHPLW4F
da6: 600.000MB/s transfers
da6: Command Queueing enabled
da6: 858483MB (1758174768 512 byte sectors)
da7 at pmspcbsd1 bus 0 scbus1 target 4 lun 0
da7: <NETAPP X423_HCOBE900A10 NA00> Fixed Direct Access SPC-4 SCSI device
da7: Serial Number KPHNZ5GF
da7: 600.000MB/s transfers
da7: Command Queueing enabled
da7: 858483MB (1758174768 512 byte sectors)
GEOM_MULTIPATH: disk8 created
da8 at pmspcbsd1 bus 0 scbus1 target 5 lun 0
da8: <NETAPP X423_HCOBE900A10 NA00> Fixed Direct Access SPC-4 SCSI device
da8: Serial Number KPHPNHRF
da8: 600.000MB/s transfers
da8: Command Queueing enabled
da8: 858483MB (1758174768 512 byte sectors)
GEOM_MULTIPATH: da4 added to disk8
da9 at pmspcbsd1 bus 0 scbus1 target 6 lun 0
da9: <NETAPP X423_HCOBE900A10 NA00> Fixed Direct Access SPC-4 SCSI device
da9: Serial Number KPHRJ7HF
da9: 600.000MB/s transfers
da9: Command Queueing enabled
da9: 858483MB (1758174768 512 byte sectors)
da10 at pmspcbsd1 bus 0 scbus1 target 7 lun 0
da10: <NETAPP X423_HCOBE900A10 NA00> Fixed Direct Access SPC-4 SCSI device
da10: Serial Number KPHMXTVF
da10: 600.000MB/s transfers
da10: Command Queueing enabled
da10: 858483MB (1758174768 512 byte sectors)
da11 at pmspcbsd1 bus 0 scbus1 target 8 lun 0
da11: <NETAPP X423_HCOBE900A10 NA00> Fixed Direct Access SPC-4 SCSI device
da11: Serial Number KPHPSY6F
da11: 600.000MB/s transfers
da11: Command Queueing enabled
da11: 858483MB (1758174768 512 byte sectors)
da12 at pmspcbsd1 bus 0 scbus1 target 9 lun 0
da12: <HITACHI HUC10909 CLAR900 C430> Fixed Direct Access SPC-4 SCSI device
da12: Serial Number KXGWASVJ
da12: 600.000MB/s transfers
da12: Command Queueing enabled
da12: 858483MB (1758174768 512 byte sectors)
da13 at pmspcbsd1 bus 0 scbus1 target 10 lun 0
da13: <HITACHI HUC10909 CLAR900 C430> Fixed Direct Access SPC-4 SCSI device
da13: Serial Number KXGW597J
da13: 600.000MB/s transfers
da13: Command Queueing enabled
da13: 858483MB (1758174768 512 byte sectors)
da14 at pmspcbsd1 bus 0 scbus1 target 11 lun 0
GEOM_MULTIPATH: disk7 created
da14: <HITACHI HUC10909 CLAR900 C430> Fixed Direct Access SPC-4 SCSI device
da14: Serial Number KXGWAJBJ
da14: 600.000MB/s transfers
da14: Command Queueing enabled
da14: 858483MB (1758174768 512 byte sectors)
da15 at pmspcbsd1 bus 0 scbus1 target 12 lun 0
da15: <HITACHI HUC10909 CLAR900 C430> Fixed Direct Access SPC-4 SCSI device
da15: Serial Number KXGWAYUJ
da15: 600.000MB/s transfers
da15: Command Queueing enabled
da15: 858483MB (1758174768 512 byte sectors)
GEOM_MULTIPATH: da5 added to disk7
da16 at pmspcbsd1 bus 0 scbus1 target 13 lun 0
da16: <HITACHI HUC10909 CLAR900 C430> Fixed Direct Access SPC-4 SCSI device
da16: Serial Number KXGW5MHJ
da16: 600.000MB/s transfers
da16: Command Queueing enabled
da16: 858483MB (1758174768 512 byte sectors)
da17 at pmspcbsd1 bus 0 scbus1 target 14 lun 0
da17: <NETAPP X423_HCOBE900A10 NA00> Fixed Direct Access SPC-4 SCSI device
da17: Serial Number KPHPRZ3F
da17: 600.000MB/s transfers
da17: Command Queueing enabled
da17: 858483MB (1758174768 512 byte sectors)
GEOM_MULTIPATH: disk6 created
GEOM_MULTIPATH: da6 added to disk6
GEOM_MULTIPATH: disk5 created
GEOM_MULTIPATH: da7 added to disk5
GEOM_MULTIPATH: disk4 created
GEOM_MULTIPATH: da8 added to disk4
GEOM_MULTIPATH: disk3 created
GEOM_MULTIPATH: da9 added to disk3
GEOM_MULTIPATH: disk2 created
GEOM_MULTIPATH: da10 added to disk2
GEOM_MULTIPATH: disk1 created
GEOM_MULTIPATH: da11 added to disk1
GEOM_MULTIPATH: disk10-20197929 created
GEOM_MULTIPATH: da12 added to disk10-20197929
GEOM_MULTIPATH: disk11 created
GEOM_MULTIPATH: da13 added to disk11
GEOM_MULTIPATH: disk14 created
GEOM_MULTIPATH: da14 added to disk14
GEOM_MULTIPATH: disk12 created
GEOM_MULTIPATH: da15 added to disk12
GEOM_MULTIPATH: disk13 created
GEOM_MULTIPATH: da16 added to disk13
GEOM_MULTIPATH: disk15 created
GEOM_MULTIPATH: da17 added to disk15
da18 at pmspcbsd1 bus 0 scbus1 target 15 lun 0
da18: <NETAPP X422_TAL13600A10 NA01> Fixed Direct Access SCSI-3 device
da18: Serial Number 35T0A08UFUZ6
da18: 600.000MB/s transfers
da18: Command Queueing enabled
da18: 572325MB (1172123568 512 byte sectors)
GEOM_MULTIPATH: geom disk11/9aedb3f5-f357-11e7-b949-005056a08e13 exists already
GEOM_MULTIPATH: 86f0ea3c-dbbd-11e7-a5bc-80c16e2595c4 will be (temporarily) disk11-12192000
GEOM_MULTIPATH: disk11-12192000 created
GEOM_MULTIPATH: da18 added to disk11-12192000
da19 at pmspcbsd1 bus 0 scbus1 target 16 lun 0
da19: <NETAPP X422_TAL13600A10 NA01> Fixed Direct Access SCSI-3 device
da19: Serial Number 35S0A067FUZ6
da19: 600.000MB/s transfers
da19: Command Queueing enabled
da19: 572325MB (1172123568 512 byte sectors)
GEOM_MULTIPATH: geom disk12/9b02b6a8-f357-11e7-b949-005056a08e13 exists already
GEOM_MULTIPATH: 871172e6-dbbd-11e7-a5bc-80c16e2595c4 will be (temporarily) disk12-19531424
GEOM_MULTIPATH: disk12-19531424 created
GEOM_MULTIPATH: da19 added to disk12-19531424
da20 at pmspcbsd1 bus 0 scbus1 target 17 lun 0
da20: <NETAPP X422_TAL13600A10 NA01> Fixed Direct Access SCSI-3 device
da20: Serial Number 35S0A0LWFUZ6
da20: 600.000MB/s transfers
da20: Command Queueing enabled
da20: 572325MB (1172123568 512 byte sectors)
GEOM_MULTIPATH: geom disk3/4367b530-f33b-11e7-a50c-005056a08e13 exists already
GEOM_MULTIPATH: 85d53ed4-dbbd-11e7-a5bc-80c16e2595c4 will be (temporarily) disk3-30364406
GEOM_MULTIPATH: disk3-30364406 created
GEOM_MULTIPATH: da20 added to disk3-30364406
da21 at pmspcbsd1 bus 0 scbus1 target 18 lun 0
da21: <NETAPP X422_TAL13600A10 NA01> Fixed Direct Access SCSI-3 device
da21: Serial Number 35Q0A0B1FUZ6
da21: 600.000MB/s transfers
da21: Command Queueing enabled
da21: 572325MB (1172123568 512 byte sectors)
GEOM_MULTIPATH: geom disk2/435d1f22-f33b-11e7-a50c-005056a08e13 exists already
GEOM_MULTIPATH: 85ae7fe0-dbbd-11e7-a5bc-80c16e2595c4 will be (temporarily) disk2-138094730
GEOM_MULTIPATH: disk2-138094730 created
GEOM_MULTIPATH: da21 added to disk2-138094730
da22 at pmspcbsd1 bus 0 scbus1 target 19 lun 0
da22: <NETAPP X422_TAL13600A10 NA01> Fixed Direct Access SCSI-3 device
da22: Serial Number 35T0A0EBFUZ6
da22: 600.000MB/s transfers
da22: Command Queueing enabled
da22: 572325MB (1172123568 512 byte sectors)
GEOM_MULTIPATH: disk10 created
GEOM_MULTIPATH: da22 added to disk10
da23 at pmspcbsd1 bus 0 scbus1 target 20 lun 0
da23: <NETAPP X422_TAL13600A10 NA01> Fixed Direct Access SCSI-3 device
da23: Serial Number 35S0A0P3FUZ6
da23: 600.000MB/s transfers
da23: Command Queueing enabled
da23: 572325MB (1172123568 512 byte sectors)
GEOM_MULTIPATH: geom disk7/439453ec-f33b-11e7-a50c-005056a08e13 exists already
GEOM_MULTIPATH: 86640e54-dbbd-11e7-a5bc-80c16e2595c4 will be (temporarily) disk7-172554839
GEOM_MULTIPATH: disk7-172554839 created
GEOM_MULTIPATH: da23 added to disk7-172554839
da24 at pmspcbsd1 bus 0 scbus1 target 21 lun 0
da24: <NETAPP X422_TAL13600A10 NA01> Fixed Direct Access SCSI-3 device
da24: Serial Number 35Q0A009FUZ6
da24: 600.000MB/s transfers
da24: Command Queueing enabled
da24: 572325MB (1172123568 512 byte sectors)
GEOM_MULTIPATH: geom disk8/439e8dc2-f33b-11e7-a50c-005056a08e13 exists already
GEOM_MULTIPATH: 86863dfe-dbbd-11e7-a5bc-80c16e2595c4 will be (temporarily) disk8-175594836
GEOM_MULTIPATH: disk8-175594836 created
GEOM_MULTIPATH: da24 added to disk8-175594836
da25 at pmspcbsd1 bus 0 scbus1 target 22 lun 0
da25: <NETAPP X422_TAL13600A10 NA01> Fixed Direct Access SCSI-3 device
da25: Serial Number 35T0A0A2FUZ6
da25: 600.000MB/s transfers
da25: Command Queueing enabled
da25: 572325MB (1172123568 512 byte sectors)
GEOM_MULTIPATH: geom disk9/43aae705-f33b-11e7-a50c-005056a08e13 exists already
GEOM_MULTIPATH: 86aa32f6-dbbd-11e7-a5bc-80c16e2595c4 will be (temporarily) disk9-150384306
GEOM_MULTIPATH: disk9-150384306 created
GEOM_MULTIPATH: da25 added to disk9-150384306
da26 at pmspcbsd1 bus 0 scbus1 target 23 lun 0
da26: <NETAPP X422_TAL13600A10 NA01> Fixed Direct Access SCSI-3 device
da26: Serial Number 35S0A0OJFUZ6
da26: 600.000MB/s transfers
da26: Command Queueing enabled
da26: 572325MB (1172123568 512 byte sectors)
GEOM_MULTIPATH: geom disk1/43533121-f33b-11e7-a50c-005056a08e13 exists already
GEOM_MULTIPATH: 858b8277-dbbd-11e7-a5bc-80c16e2595c4 will be (temporarily) disk1-135526787
ses0 at pmspcbsd1 bus 0 scbus1 target 24 lun 0
GEOM_MULTIPATH: disk1-135526787 created
ses0: <NETAPP DS224IOM6 0173> Fixed Enclosure Services SPC-3 SCSI device
ses0: Serial Number SHFMS1520000114
ses0: 300.000MB/s transfers
ses0: Command Queueing enabled
ses0: SCSI-3 ENC Device
GEOM_MULTIPATH: da26 added to disk1-135526787
ses0: da3,pass4: SAS Device Slot Element: 1 Phys at Slot 0
ses0: phy 0: SAS device type 1 id 1
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 500a098005cba9ff addr 5000cca01660b38a
ses0: da4,pass5: SAS Device Slot Element: 1 Phys at Slot 1
ses0: phy 0: SAS device type 1 id 1
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 500a098005cba9ff addr 5000cca016627356
ses0: da5,pass6: SAS Device Slot Element: 1 Phys at Slot 2
ses0: phy 0: SAS device type 1 id 1
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 500a098005cba9ff addr 5000cca01661a24e
ses0: da6,pass7: SAS Device Slot Element: 1 Phys at Slot 3
ses0: phy 0: SAS device type 1 id 1
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 500a098005cba9ff addr 5000cca0165fa89e
ses0: da7,pass8: SAS Device Slot Element: 1 Phys at Slot 4
ses0: phy 0: SAS device type 1 id 1
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 500a098005cba9ff addr 5000cca0165e80d2
ses0: da8,pass9: SAS Device Slot Element: 1 Phys at Slot 5
ses0: phy 0: SAS device type 1 id 1
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 500a098005cba9ff addr 5000cca0165fc11e
ses0: da9,pass10: SAS Device Slot Element: 1 Phys at Slot 6
ses0: phy 0: SAS device type 1 id 1
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 500a098005cba9ff addr 5000cca01661528e
ses0: da10,pass11: SAS Device Slot Element: 1 Phys at Slot 7
ses0: phy 0: SAS device type 1 id 1
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 500a098005cba9ff addr 5000cca0165c9ab2
ses0: da11,pass12: SAS Device Slot Element: 1 Phys at Slot 8
ses0: phy 0: SAS device type 1 id 1
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 500a098005cba9ff addr 5000cca0165ff4b2
ses0: da12,pass13: SAS Device Slot Element: 1 Phys at Slot 9
ses0: phy 0: SAS device type 1 id 1
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 500a098005cba9ff addr 5000cca05731b9ca
ses0: da13,pass14: SAS Device Slot Element: 1 Phys at Slot 10
ses0: phy 0: SAS device type 1 id 1
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 500a098005cba9ff addr 5000cca0573167a2
ses0: da14,pass15: SAS Device Slot Element: 1 Phys at Slot 11
ses0: phy 0: SAS device type 1 id 1
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 500a098005cba9ff addr 5000cca05731b62a
ses0: da15,pass16: SAS Device Slot Element: 1 Phys at Slot 12
ses0: phy 0: SAS device type 1 id 1
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 500a098005cba9ff addr 5000cca05731bcae
ses0: da16,pass17: SAS Device Slot Element: 1 Phys at Slot 13
ses0: phy 0: SAS device type 1 id 1
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 500a098005cba9ff addr 5000cca057316c9a
ses0: da17,pass18: SAS Device Slot Element: 1 Phys at Slot 14
ses0: phy 0: SAS device type 1 id 1
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 500a098005cba9ff addr 5000cca0165fe61e
ses0: da18,pass19: SAS Device Slot Element: 1 Phys at Slot 15
ses0: phy 0: SAS device type 1 id 1
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 500a098005cba9ff addr 5000039628319672
ses0: da19,pass20: SAS Device Slot Element: 1 Phys at Slot 16
ses0: phy 0: SAS device type 1 id 1
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 500a098005cba9ff addr 500003962830d9de
ses0: da20,pass21: SAS Device Slot Element: 1 Phys at Slot 17
ses0: phy 0: SAS device type 1 id 1
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 500a098005cba9ff addr 500003962831170a
ses0: da21,pass22: SAS Device Slot Element: 1 Phys at Slot 18
ses0: phy 0: SAS device type 1 id 1
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 500a098005cba9ff addr 50000396282bacee
ses0: da22,pass23: SAS Device Slot Element: 1 Phys at Slot 19
ses0: phy 0: SAS device type 1 id 1
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 500a098005cba9ff addr 500003962831a65e
ses0: da23,pass24: SAS Device Slot Element: 1 Phys at Slot 20
ses0: phy 0: SAS device type 1 id 1
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 500a098005cba9ff addr 500003962831249a
ses0: da24,pass25: SAS Device Slot Element: 1 Phys at Slot 21
ses0: phy 0: SAS device type 1 id 1
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 500a098005cba9ff addr 50000396282b7ce2
ses0: da25,pass26: SAS Device Slot Element: 1 Phys at Slot 22
ses0: phy 0: SAS device type 1 id 1
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 500a098005cba9ff addr 500003962831979e
ses0: da26,pass27: SAS Device Slot Element: 1 Phys at Slot 23
ses0: phy 0: SAS device type 1 id 1
ses0: phy 0: protocols: Initiator( None ) Target( SSP )
ses0: phy 0: parent 500a098005cba9ff addr 50000396283123b2
