ZFS array no longer found

[one800higgins]

Just ran into a problem and I'm having an issue figuring out what caused it and/or how to resolve it. I rebooted my FreeNAS earlier today and when it came back online, one of my arrays is no longer found:

"The volume vol1 (ZFS) state is UNKNOWN"

vol1 was a 2 SSD array, mirrored. FreeNAS is suddenly acting like the disks don't even exist anymore... My other volumn (6 disk array) is running just fine.

I run regular health checks, scrubs, etc and I've had no errors or warnings on these disks. It's only after rebooting that suddenly this issue has come up and I can't seem to figure out what happened or how to resolve it (if I even can).

Anyone have any suggestions / thoughts?

EDIT: More info below...

Version: FreeNAS-9.10-STABLE-201604261518
CPU: Intel(R) Xeon(R) CPU E3-1231 v3 @ 3.40GHz
Mobo: SuperMicro MBD-X10SLH-F-O
RAM: 2x Crucial 16GB KVR16R11D4/16
Hard drives: 2x SanDisk SDSSDX-240G-G25 (plus 6x 3TB WD Reds, but no issue there)

"zpool status" doesn't recognize the missing array (vol1):

Code:

  pool: freenas-boot
state: ONLINE
  scan: scrub repaired 0 in 0h2m with 0 errors on Thu Mar 31 03:47:37 2016
config:

    NAME        STATE     READ WRITE CKSUM
    freenas-boot  ONLINE       0     0     0
      mirror-0  ONLINE       0     0     0
        da0p2   ONLINE       0     0     0
        da1p2   ONLINE       0     0     0

errors: No known data errors

  pool: vol0
state: ONLINE
  scan: scrub repaired 0 in 4h44m with 0 errors on Fri Apr 15 08:44:19 2016
config:

    NAME                                            STATE     READ WRITE CKSUM
    vol0                                            ONLINE       0     0     0
      raidz2-0                                      ONLINE       0     0     0
        gptid/5698b868-3222-11e5-b6bc-0cc47a4ec610  ONLINE       0     0     0
        gptid/57686acd-3222-11e5-b6bc-0cc47a4ec610  ONLINE       0     0     0
        gptid/57d5ccdb-3222-11e5-b6bc-0cc47a4ec610  ONLINE       0     0     0
        gptid/583f4da8-3222-11e5-b6bc-0cc47a4ec610  ONLINE       0     0     0
        gptid/58a2d378-3222-11e5-b6bc-0cc47a4ec610  ONLINE       0     0     0
        gptid/5901c578-3222-11e5-b6bc-0cc47a4ec610  ONLINE       0     0     0

errors: No known data errors

"dmesg" output:

Code:

Copyright (c) 1992-2016 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
    The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 10.3-RELEASE #0 f935af8(freebsd10): Tue Apr 26 08:30:39 PDT 2016
    root@build.ixsystems.com:/tank/home/nightlies/build-freenas9/_BE/objs/tank/home/nightlies/build-freenas9/_BE/trueos/sys/FreeNAS.amd64 amd64
FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512
CPU: Intel(R) Xeon(R) CPU E3-1231 v3 @ 3.40GHz (3400.07-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0x306c3  Family=0x6  Model=0x3c  Stepping=3
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x7ffafbff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
  AMD Features2=0x21<LAHF,ABM>
  Structured Extended Features=0x2fbb<FSGSBASE,TSCADJ,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,NFPUSG>
  XSAVE Features=0x1<XSAVEOPT>
  VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID
  TSC: P-state invariant, performance statistics
real memory  = 34896609280 (33280 MB)
avail memory = 33261326336 (31720 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <SUPERM SMCI--MB>
FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s) x 2 SMT threads
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  1
 cpu2 (AP): APIC ID:  2
 cpu3 (AP): APIC ID:  3
 cpu4 (AP): APIC ID:  4
 cpu5 (AP): APIC ID:  5
 cpu6 (AP): APIC ID:  6
 cpu7 (AP): APIC ID:  7
random: <Software, Yarrow> initialized
WARNING: VIMAGE (virtualized network stack) is a highly experimental feature.
ioapic0 <Version 2.0> irqs 0-23 on motherboard
module_register_init: MOD_LOAD (vesa, 0xffffffff80dce260, 0) error 19
kbd1 at kbdmux0
cryptosoft0: <software crypto> on motherboard
aesni0: <AES-CBC,AES-XTS> on motherboard
padlock0: No ACE support.
acpi0: <SUPERM SMCI--MB> on motherboard
acpi0: Power Button (fixed)
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
cpu2: <ACPI CPU> on acpi0
cpu3: <ACPI CPU> on acpi0
cpu4: <ACPI CPU> on acpi0
cpu5: <ACPI CPU> on acpi0
cpu6: <ACPI CPU> on acpi0
cpu7: <ACPI CPU> on acpi0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 950
Event timer "HPET" frequency 14318180 Hz quality 550
atrtc0: <AT realtime clock> port 0x70-0x77 irq 8 on acpi0
Event timer "RTC" frequency 32768 Hz quality 0
attimer0: <AT timer> port 0x40-0x43,0x50-0x53 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1808-0x180b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
xhci0: <Intel Lynx Point USB 3.0 controller> mem 0xf7300000-0xf730ffff irq 16 at device 20.0 on pci0
xhci0: 32 bytes context size, 64-bit DMA
xhci0: Port routing mask set to 0xffffffff
usbus0 on xhci0
pci0: <simple comms> at device 22.0 (no driver attached)
pci0: <simple comms> at device 22.1 (no driver attached)
ehci0: <Intel Lynx Point USB 2.0 controller USB-B> mem 0xf7314000-0xf73143ff irq 16 at device 26.0 on pci0
usbus1: EHCI version 1.0
usbus1 on ehci0
pcib1: <ACPI PCI-PCI bridge> irq 16 at device 28.0 on pci0
pci1: <ACPI PCI bus> on pcib1
pcib2: <ACPI PCI-PCI bridge> at device 0.0 on pci1
pci2: <ACPI PCI bus> on pcib2
vgapci0: <VGA-compatible display> port 0xe000-0xe07f mem 0xf6000000-0xf6ffffff,0xf7000000-0xf701ffff irq 16 at device 0.0 on pci2
vgapci0: Boot video device
pcib3: <ACPI PCI-PCI bridge> irq 18 at device 28.2 on pci0
pci3: <ACPI PCI bus> on pcib3
igb0: <Intel(R) PRO/1000 Network Connection, Version - 2.5.3-k> port 0xd000-0xd01f mem 0xf7200000-0xf727ffff,0xf7280000-0xf7283fff irq 18 at device 0.0 on pci3
igb0: Using MSIX interrupts with 5 vectors
igb0: Ethernet address: 0c:c4:7a:4e:c6:10
igb0: Bound queue 0 to cpu 0
igb0: Bound queue 1 to cpu 1
igb0: Bound queue 2 to cpu 2
igb0: Bound queue 3 to cpu 3
pcib4: <ACPI PCI-PCI bridge> irq 19 at device 28.3 on pci0
pci4: <ACPI PCI bus> on pcib4
igb1: <Intel(R) PRO/1000 Network Connection, Version - 2.5.3-k> port 0xc000-0xc01f mem 0xf7100000-0xf717ffff,0xf7180000-0xf7183fff irq 19 at device 0.0 on pci4
igb1: Using MSIX interrupts with 5 vectors
igb1: Ethernet address: 0c:c4:7a:4e:c6:11
igb1: Bound queue 0 to cpu 4
igb1: Bound queue 1 to cpu 5
igb1: Bound queue 2 to cpu 6
igb1: Bound queue 3 to cpu 7
pcib5: <ACPI PCI-PCI bridge> irq 16 at device 28.4 on pci0
pci5: <ACPI PCI bus> on pcib5
ehci1: <Intel Lynx Point USB 2.0 controller USB-A> mem 0xf7313000-0xf73133ff irq 23 at device 29.0 on pci0
usbus2: EHCI version 1.0
usbus2 on ehci1
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
ahci0: <Intel Lynx Point AHCI SATA controller> port 0xf050-0xf057,0xf040-0xf043,0xf030-0xf037,0xf020-0xf023,0xf000-0xf01f mem 0xf7312000-0xf73127ff irq 19 at device 31.2 on pci0
ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich1: <AHCI channel> at channel 1 on ahci0
ahcich2: <AHCI channel> at channel 2 on ahci0
ahcich3: <AHCI channel> at channel 3 on ahci0
ahcich4: <AHCI channel> at channel 4 on ahci0
ahcich5: <AHCI channel> at channel 5 on ahci0
acpi_button0: <Sleep Button> on acpi0
acpi_button1: <Power Button> on acpi0
acpi_tz0: <Thermal Zone> on acpi0
acpi_tz1: <Thermal Zone> on acpi0
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
uart2: <16550 or compatible> port 0x3e8-0x3ef irq 7 on acpi0
ichwd0: <Intel Lynx Point watchdog timer> on isa0
ichwd0: ICH WDT present but disabled in BIOS or hardware
device_attach: ichwd0 attach returned 6
wbwd0: <Nuvoton NCT6776 (0xc3/0x33) Watchdog Timer> at port 0x2e-0x2f on isa0
ichwd0: <Intel Lynx Point watchdog timer> at port 0x1830-0x1837,0x1860-0x187f on isa0
ichwd0: ICH WDT present but disabled in BIOS or hardware
device_attach: ichwd0 attach returned 6
orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xc8000-0xc8fff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: CGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3d0-0x3db iomem 0xb8000-0xbffff on isa0
coretemp0: <CPU On-Die Thermal Sensors> on cpu0
est0: <Enhanced SpeedStep Frequency Control> on cpu0
coretemp1: <CPU On-Die Thermal Sensors> on cpu1
est1: <Enhanced SpeedStep Frequency Control> on cpu1
coretemp2: <CPU On-Die Thermal Sensors> on cpu2
est2: <Enhanced SpeedStep Frequency Control> on cpu2
coretemp3: <CPU On-Die Thermal Sensors> on cpu3
est3: <Enhanced SpeedStep Frequency Control> on cpu3
coretemp4: <CPU On-Die Thermal Sensors> on cpu4
est4: <Enhanced SpeedStep Frequency Control> on cpu4
coretemp5: <CPU On-Die Thermal Sensors> on cpu5
est5: <Enhanced SpeedStep Frequency Control> on cpu5
coretemp6: <CPU On-Die Thermal Sensors> on cpu6
est6: <Enhanced SpeedStep Frequency Control> on cpu6
coretemp7: <CPU On-Die Thermal Sensors> on cpu7
est7: <Enhanced SpeedStep Frequency Control> on cpu7
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
Timecounters tick every 1.000 msec
ipfw2 (+ipv6) initialized, divert enabled, nat enabled, default to accept, logging disabled
random: unblocking device.
usbus0: 5.0Gbps Super Speed USB v3.0
usbus1: 480Mbps High Speed USB v2.0
usbus2: 480Mbps High Speed USB v2.0
ugen1.1: <Intel> at usbus1
uhub0: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
ugen0.1: <0x8086> at usbus0
uhub1: <0x8086 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
ugen2.1: <Intel> at usbus2
uhub2: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus2
uhub0: 2 ports with 2 removable, self powered
uhub2: 2 ports with 2 removable, self powered
uhub1: 21 ports with 21 removable, self powered
ugen1.2: <vendor 0x8087> at usbus1
uhub3: <vendor 0x8087 product 0x8008, class 9/0, rev 2.00/0.05, addr 2> on usbus1
ugen2.2: <vendor 0x8087> at usbus2
uhub4: <vendor 0x8087 product 0x8000, class 9/0, rev 2.00/0.05, addr 2> on usbus2
uhub3: 6 ports with 6 removable, self powered
uhub4: 8 ports with 8 removable, self powered
ugen0.2: <vendor 0x0000> at usbus0
uhub5: <vendor 0x0000 product 0x0001, class 9/0, rev 2.00/0.00, addr 1> on usbus0
uhub5: 4 ports with 3 removable, self powered
ugen0.3: <vendor 0x0557> at usbus0
ukbd0: <vendor 0x0557 product 0x2419, class 0/0, rev 1.10/1.00, addr 2> on usbus0
kbd0 at ukbd0
ugen0.4: <SanDisk> at usbus0
umass0: <SanDisk Ultra Fit, class 0/0, rev 3.00/1.00, addr 3> on usbus0
umass0:  SCSI over Bulk-Only; quirks = 0x8100
umass0:7:0:-1: Attached to scbus7
ugen0.5: <SanDisk> at usbus0
umass1: <SanDisk Ultra Fit, class 0/0, rev 3.00/1.00, addr 4> on usbus0
umass1:  SCSI over Bulk-Only; quirks = 0x8100
umass1:8:1:-1: Attached to scbus8
ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
ada0: <WDC WD30EFRX-68EUZN0 80.00A80> ACS-2 ATA SATA 3.x device
ada0: Serial Number WD-WMC4N1351504
ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada0: Command Queueing enabled
ada0: 2861588MB (5860533168 512 byte sectors)
ada0: quirks=0x1<4K>
ada0: Previously was known as ad4
ada1 at ahcich1 bus 0 scbus1 target 0 lun 0
ada1: <WDC WD30EFRX-68EUZN0 80.00A80> ACS-2 ATA SATA 3.x device
ada1: Serial Number WD-WMC4N1351789
ada1: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada1: Command Queueing enabled
ada1: 2861588MB (5860533168 512 byte sectors)
ada1: quirks=0x1<4K>
ada1: Previously was known as ad6
ada2 at ahcich2 bus 0 scbus2 target 0 lun 0
ada2: <WDC WD30EFRX-68AX9N0 80.00A80> ACS-2 ATA SATA 3.x device
ada2: Serial Number WD-WMC1T0928803
ada2: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada2: Command Queueing enabled
ada2: 2861588MB (5860533168 512 byte sectors)
ada2: quirks=0x1<4K>
ada2: Previously was known as ad8
ada3 at ahcich3 bus 0 scbus3 target 0 lun 0
ada3: <WDC WD30EFRX-68AX9N0 80.00A80> ACS-2 ATA SATA 3.x device
ada3: Serial Number WD-WMC1T1969409
ada3: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada3: Command Queueing enabled
ada3: 2861588MB (5860533168 512 byte sectors)
ada3: quirks=0x1<4K>
ada3: Previously was known as ad10
ada4 at ahcich4 bus 0 scbus4 target 0 lun 0
ada4: <WDC WD30EFRX-68EUZN0 82.00A82> ACS-2 ATA SATA 3.x device
ada4: Serial Number WD-WCC4N3VR2FC5
ada4: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada4: Command Queueing enabled
ada4: 2861588MB (5860533168 512 byte sectors)
ada4: quirks=0x1<4K>
ada4: Previously was known as ad12
ada5 at ahcich5 bus 0 scbus5 target 0 lun 0
ada5: <WDC WD30EFRX-68EUZN0 82.00A82> ACS-2 ATA SATA 3.x device
ada5: Serial Number WD-WCC4N1SSDN1L
ada5: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada5: Command Queueing enabled
ada5: 2861588MB (5860533168 512 byte sectors)
ada5: quirks=0x1<4K>
ada5: Previously was known as ad14
da0 at umass-sim0 bus 0 scbus7 target 0 lun 0
da0: <SanDisk Ultra Fit 1.00> Removable Direct Access SPC-4 SCSI device
da0: Serial Number 4C531001440312110550
da0: 400.000MB/s transfers
da0: 14832MB (30375936 512 byte sectors)
da0: quirks=0x2<NO_6_BYTE>
da1 at umass-sim1 bus 1 scbus8 target 0 lun 0
da1: <SanDisk Ultra Fit 1.00> Removable Direct Access SPC-4 SCSI device
da1: Serial Number 4C530001050313115060
da1: 400.000MB/s transfers
da1: 14832MB (30375936 512 byte sectors)
da1: quirks=0x2<NO_6_BYTE>
SMP: AP CPU #1 Launched!
SMP: AP CPU #3 Launched!
SMP: AP CPU #6 Launched!
SMP: AP CPU #2 Launched!
SMP: AP CPU #4 Launched!
SMP: AP CPU #5 Launched!
SMP: AP CPU #7 Launched!
Timecounter "TSC-low" frequency 1700036926 Hz quality 1000
Trying to mount root from zfs:freenas-boot/ROOT/FreeNAS-0a90dd7369b81ac71925bc9d69cb57e0 []...
GEOM_RAID5: Module loaded, version 1.3.20140711.62 (rev f91e28e40bf7)
igb0: link state changed to UP
ipmi0: <IPMI System Interface> port 0xca2,0xca3 on acpi0
ipmi0: KCS mode found at io 0xca2 on acpi
ipmi0: IPMI device rev. 1, firmware rev. 1.42, version 2.0
ipmi0: Number of channels 2
ipmi0: Attached watchdog
ichwd0: <Intel Lynx Point watchdog timer> at port 0x1830-0x1837,0x1860-0x187f on isa0
ichwd0: ICH WDT present but disabled in BIOS or hardware
device_attach: ichwd0 attach returned 6
hwpmc: SOFT/16/64/0x67<INT,USR,SYS,REA,WRI> TSC/1/64/0x20<REA> IAP/4/48/0x3ff<INT,USR,SYS,EDG,THR,REA,WRI,INV,QUA,PRC> IAF/3/48/0x67<INT,USR,SYS,REA,WRI> UCP/8/48/0x3f8<EDG,THR,REA,WRI,INV,QUA,PRC> UCF/1/48/0x60<REA,WRI>
igb0: link state changed to DOWN
igb0: link state changed to UP
ums0: <vendor 0x0557 product 0x2419, class 0/0, rev 1.10/1.00, addr 2> on usbus0
ums0: 3 buttons and [Z] coordinates ID=0
vboxdrv: fAsync=0 offMin=0x35e offMax=0xf1e
GEOM_ELI: Device ada0p1.eli created.
GEOM_ELI: Encryption: AES-XTS 128
GEOM_ELI:     Crypto: hardware
GEOM_ELI: Device ada1p1.eli created.
GEOM_ELI: Encryption: AES-XTS 128
GEOM_ELI:     Crypto: hardware
GEOM_ELI: Device ada2p1.eli created.
GEOM_ELI: Encryption: AES-XTS 128
GEOM_ELI:     Crypto: hardware
GEOM_ELI: Device ada3p1.eli created.
GEOM_ELI: Encryption: AES-XTS 128
GEOM_ELI:     Crypto: hardware
GEOM_ELI: Device ada4p1.eli created.
GEOM_ELI: Encryption: AES-XTS 128
GEOM_ELI:     Crypto: hardware
GEOM_ELI: Device ada5p1.eli created.
GEOM_ELI: Encryption: AES-XTS 128
GEOM_ELI:     Crypto: hardware
nfsd: can't register svc name

"smartctl --scan" doesn't list the disks:

Code:

/dev/ada0 -d atacam # /dev/ada0, ATA device
/dev/ada1 -d atacam # /dev/ada1, ATA device
/dev/ada2 -d atacam # /dev/ada2, ATA device
/dev/ada3 -d atacam # /dev/ada3, ATA device
/dev/ada4 -d atacam # /dev/ada4, ATA device
/dev/ada5 -d atacam # /dev/ada5, ATA device

 

[joeschmuck]

It sure would help if you posted your system specs and version of FreeNAS you are running, per our forum rules. Also post the output of "zpool status" and any other messages you have. Additionally why did you reboot FreeNAS? Did you move the computer, open the case, move a wire? Also include the connectivity of the drives (what controller/ports they are connected to).

 

[one800higgins]

@joeschmuck Sorry, I updated the original post with more info. I rebooted because the server was having trouble fetching updates from FreeNAS. I haven't been on my server in a few weeks due to being out of town, so when I saw it couldn't fetch, I was definitely too worn out to dig through the logs and see what happened while I was out of town.

The reboot fixed the network issue, but when it came back online, the 2 SSD drives were no longer recognized.

 

[DrKK]

let us see

Code:

camcontrol devlist

please.

 

[joeschmuck]

Yes, I see 8 drives posted (6 for your pool and 2 for the USB boot devices), none of the SSDs are listed.

So, how are the SSDs physically connected to the machine, meaning the SATA ports and Power Connections? I suspect that either they don't have power (most likely) and maybe if you have a modular power supply one of the connections may have come loose, the SATA connections both were removed (not likely unless someone got into the case), or they are both dead (possible, I had a drive that died after so many hours of power on time and a firmware change fixed it) but unlikely.

EDIT: And it could be a bad power supply too. You will have to let us know just a bit more.

 

[one800higgins]

"camcontrol devlist"

Code:

<WDC WD30EFRX-68EUZN0 80.00A80>    at scbus0 target 0 lun 0 (ada0,pass0)
<WDC WD30EFRX-68EUZN0 80.00A80>    at scbus1 target 0 lun 0 (ada1,pass1)
<WDC WD30EFRX-68AX9N0 80.00A80>    at scbus2 target 0 lun 0 (ada2,pass2)
<WDC WD30EFRX-68AX9N0 80.00A80>    at scbus3 target 0 lun 0 (ada3,pass3)
<WDC WD30EFRX-68EUZN0 82.00A82>    at scbus4 target 0 lun 0 (ada4,pass4)
<WDC WD30EFRX-68EUZN0 82.00A82>    at scbus5 target 0 lun 0 (ada5,pass5)
<SanDisk Ultra Fit 1.00>           at scbus7 target 0 lun 0 (da0,pass6)
<SanDisk Ultra Fit 1.00>           at scbus8 target 0 lun 0 (da1,pass7)

Nada :/

Yes, I see 8 drives posted (6 for your pool and 2 for the USB boot devices), none of the SSDs are listed.

So, how are the SSDs physically connected to the machine, meaning the SATA ports and Power Connections? I suspect that either they don't have power (most likely) and maybe if you have a modular power supply one of the connections may have come loose, the SATA connections both were removed (not likely unless someone got into the case), or they are both dead (possible, I had a drive that died after so many hours of power on time and a firmware change fixed it) but unlikely.

EDIT: And it could be a bad power supply too. You will have to let us know just a bit more.

They are SATA connections and I know no one's been in there to touch anything. That said, I think you may be on to something with the modular power supply... IIRC, those 2 drives are one a single connection on the PSU. Perhaps there's an issue with the cable or that specific input on the PSU. I'll have to test that out in the morning since it's pretty late now.

Thanks for the suggestion! I hadn't considered a PSU issue.

 

[DrKK]

Thanks for the suggestion! I hadn't considered a PSU issue.

Yeah, that's actually very astute of him to suggest that. What are the odds BOTH of those SSD's would shit the bed at the same moment; pretty low. Makes much more sense that they are not detected because they are not powered.

 

[zambanini]

some ssd do not say hello, when they wear out. what was your workload?

put one of them in another pc and see if it will be recognized within the bios.

in general, take care that your ssds do not have the same amount of data written to them (historical, sorry for my bad english), before you add them to a pool. so you have a chance that not all of them die at the same time.

 

[joeschmuck]

The SSDs are the Pro versions with a 10 year warranty (if I looked up the part number correctly) so I would expect then to last for a long time. There is also no firmware update for them, not that it means anything yet. When I had my Crucial M4 SSD crap out on me, I was probably one of the first hundred to find the flaw because my system runs 24/7 and after hitting a certain number of run hours the SSD would stop working. Cycling power to it would make it work for 1 hour and then it would stop again. A new firmware solved the problem. I'm not saying the OP has a firmware issue but these things do happen. I'm still on the mindset of it being power related since that makes more sense to me at this time.

 

[one800higgins]

I honestly haven't opened the server up in about a year so I completely forgot that the 2 SSDs were on a SATA controller (non-raid) card because the MOBO only has 6 SATA slots. I suspect this is the likely culprit. Just pulled the drives and card out so I can begin some testing and went ahead and ordered a replacement card so I could get it in time for same day delivery in case that is the root issue.

Thank you Amazon same-day shipping for making computer repairs quick again... Don't need to wait a week for a Newegg order to be processed and delivered!

Edit: Nope, wasn't the controller. The PCI slot bit it. Back up and running though!

 

[joeschmuck]

Good to hear you figured it out.