importing pool reboots freenas

[reidara]

I have a zfs pool which i raidz1
One drive dont report anymore to bios and is UNAVAIL in zpool import result.
--------------------------------------------------------------------------------
pool: supervolume
id: 3261155225326581703
state: DEGRADED
status: One or more devices are missing from the system.
action: The pool can be imported despite missing or damaged devices. The
fault tolerance of the pool may be compromised if imported.
see: http://illumos.org/msg/ZFS-8000-2Q
config:

supervolume DEGRADED
raidz1-0 DEGRADED
gptid/f1ba6158-e8ae-11e2-b0f3-60a44c640002 ONLINE
gptid/f23bc149-e8ae-11e2-b0f3-60a44c640002 ONLINE
gptid/f2bd7dd9-e8ae-11e2-b0f3-60a44c640002 ONLINE
6732861267941806277 UNAVAIL cannot open
gptid/f3c0f709-e8ae-11e2-b0f3-60a44c640002 ONLINE
cache
gptid/1fb1da24-1c5c-11e3-97bb-60a44c640002
--------------------------------------------------------------------------------

When I do 'zpool import -F supervolume' it starts filling the screen up with a lot of text fast and.... reboots

Guess im safe with Z1 ?
Any clues ?

PS! I have tried booting with a new fresh 9.2.1.7 usb and imported to that with the same result.

 

[Ericloewe]

DO NOT USE -f UNLESS TOLD TO DO SO BY SOMEONE WHO KNOWS WHAT THEY'RE DOING!

With the somewhat dramatic warning out of the way, please use CODE tags to paste the output of zpool status. The formatting here is extremely important.

That said, the pool *should* be working, so something else is going on.

What's your hardware and FreeNAS version?

 

[cyberjock]

Also include in the CODE tags the output of "zpool import" and "dmesg".

 

[reidara]

Code:

Copyright (c) 1992-2013 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 9.2-RELEASE-p9 #0 r262572+cc525fe: Thu Jul  3 14:22:45 PDT 2014
    root@build3.ixsystems.com:/fusion/jenkins/9.2.1.6-RELEASE/os-base/amd64/fusion/jenkins/9.2.1.6-RELEASE/FreeBSD/src/sys/FREENAS.amd64 amd64
gcc version 4.2.1 20070831 patched [FreeBSD]
CPU: Intel(R) Core(TM) i5-4570S CPU @ 2.90GHz (2893.36-MHz K8-class CPU)
  Origin = "GenuineIntel"  Id = 0x306c3  Family = 0x6  Model = 0x3c  Stepping = 3
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x7ffafbff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,<b11>,FMA,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
  AMD Features2=0x21<LAHF,ABM>
  Standard Extended Features=0x2fbb<GSFSBASE,TSCADJ,SMEP,ENHMOVSB,INVPCID>
  TSC: P-state invariant, performance statistics
real memory  = 8589934592 (8192 MB)
avail memory = 7910817792 (7544 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <ALASKA A M I>
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s)
cpu0 (BSP): APIC ID:  0
cpu1 (AP): APIC ID:  2
cpu2 (AP): APIC ID:  4
cpu3 (AP): APIC ID:  6
WARNING: VIMAGE (virtualized network stack) is a highly experimental feature.
ACPI Warning: FADT (revision 5) is longer than ACPI 2.0 version, truncating length 268 to 244 (20110527/tbfadt-320)
ioapic0 <Version 2.0> irqs 0-23 on motherboard
kbd1 at kbdmux0
cryptosoft0: <software crypto> on motherboard
aesni0: <AES-CBC,AES-XTS> on motherboard
padlock0: No ACE support.
acpi0: <ALASKA A M I> on motherboard
ACPI Error: [RAMB] Namespace lookup failure, AE_NOT_FOUND (20110527/psargs-392)
ACPI Exception: AE_NOT_FOUND, Could not execute arguments for [RAMW] (Region) (20110527/nsinit-380)
acpi0: Power Button (fixed)
acpi0: reservation of 67, 1 (4) failed
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
cpu2: <ACPI CPU> on acpi0
cpu3: <ACPI CPU> on acpi0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 950
Event timer "HPET" frequency 14318180 Hz quality 550
Event timer "HPET1" frequency 14318180 Hz quality 440
Event timer "HPET2" frequency 14318180 Hz quality 440
Event timer "HPET3" frequency 14318180 Hz quality 440
Event timer "HPET4" frequency 14318180 Hz quality 440
atrtc0: <AT realtime clock> port 0x70-0x77 irq 8 on acpi0
atrtc0: Warning: Couldn't map I/O.
Event timer "RTC" frequency 32768 Hz quality 0
attimer0: <AT timer> port 0x40-0x43,0x50-0x53 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1808-0x180b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> irq 16 at device 1.0 on pci0
pci1: <ACPI PCI bus> on pcib1
vgapci0: <VGA-compatible display> port 0xf000-0xf03f mem 0xf7800000-0xf7bfffff,0xe0000000-0xefffffff irq 16 at device 2.0 on pci0
pci0: <multimedia, HDA> at device 3.0 (no driver attached)
pci0: <serial bus, USB> at device 20.0 (no driver attached)
pci0: <simple comms> at device 22.0 (no driver attached)
em0: <Intel(R) PRO/1000 Network Connection 7.3.8> port 0xf080-0xf09f mem 0xf7c00000-0xf7c1ffff,0xf7c3d000-0xf7c3dfff irq 20 at device 25.0 on pci0
em0: Using an MSI interrupt
em0: Ethernet address: 60:a4:4c:64:00:02
ehci0: <EHCI (generic) USB 2.0 controller> mem 0xf7c3c000-0xf7c3c3ff irq 16 at device 26.0 on pci0
usbus0: EHCI version 1.0
usbus0 on ehci0
pci0: <multimedia, HDA> at device 27.0 (no driver attached)
ehci1: <EHCI (generic) USB 2.0 controller> mem 0xf7c3b000-0xf7c3b3ff irq 23 at device 29.0 on pci0
usbus1: EHCI version 1.0
usbus1 on ehci1
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
ahci0: <Intel Lynx Point AHCI SATA controller> port 0xf0d0-0xf0d7,0xf0c0-0xf0c3,0xf0b0-0xf0b7,0xf0a0-0xf0a3,0xf060-0xf07f mem 0xf7c3a000-0xf7c3a7ff irq 19 at device 31.2 on pci0
ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich1: <AHCI channel> at channel 1 on ahci0
ahcich2: <AHCI channel> at channel 2 on ahci0
ahcich3: <AHCI channel> at channel 3 on ahci0
ahcich4: <AHCI channel> at channel 4 on ahci0
ahcich5: <AHCI channel> at channel 5 on ahci0
pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
acpi_button0: <Power Button> on acpi0
acpi_tz0: <Thermal Zone> on acpi0
acpi_tz1: <Thermal Zone> on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
wbwd0: DevID 0xc8 DevRev 0x03, will not attach, please report this.
coretemp0: <CPU On-Die Thermal Sensors> on cpu0
est0: <Enhanced SpeedStep Frequency Control> on cpu0
p4tcc0: <CPU Frequency Thermal Control> on cpu0
coretemp1: <CPU On-Die Thermal Sensors> on cpu1
est1: <Enhanced SpeedStep Frequency Control> on cpu1
p4tcc1: <CPU Frequency Thermal Control> on cpu1
coretemp2: <CPU On-Die Thermal Sensors> on cpu2
est2: <Enhanced SpeedStep Frequency Control> on cpu2
p4tcc2: <CPU Frequency Thermal Control> on cpu2
coretemp3: <CPU On-Die Thermal Sensors> on cpu3
est3: <Enhanced SpeedStep Frequency Control> on cpu3
p4tcc3: <CPU Frequency Thermal Control> on cpu3
Timecounters tick every 1.000 msec
ipfw2 (+ipv6) initialized, divert enabled, nat enabled, default to accept, logging disabled
DUMMYNET 0xfffffe000313ac80 with IPv6 initialized (100409)
load_dn_sched dn_sched RR loaded
load_dn_sched dn_sched WF2Q+ loaded
load_dn_sched dn_sched FIFO loaded
load_dn_sched dn_sched PRIO loaded
load_dn_sched dn_sched QFQ loaded
usbus0: 480Mbps High Speed USB v2.0
usbus1: 480Mbps High Speed USB v2.0
ugen0.1: <Intel> at usbus0
uhub0: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus0
ugen1.1: <Intel> at usbus1
uhub1: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
ada0: <ST3000DM001-1CH166 CC24> ATA-8 SATA 3.x device
ada0: Serial Number W1F1Y108
ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada0: Command Queueing enabled
ada0: 2861588MB (5860533168 512 byte sectors: 16H 63S/T 16383C)
ada0: quirks=0x1<4K>
ada0: Previously was known as ad4
ada1 at ahcich1 bus 0 scbus1 target 0 lun 0
ada1: <ST3000DM001-1CH166 CC24> ATA-8 SATA 3.x device
ada1: Serial Number Z1F222PJ
ada1: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada1: Command Queueing enabled
ada1: 2861588MB (5860533168 512 byte sectors: 16H 63S/T 16383C)
ada1: quirks=0x1<4K>
ada1: Previously was known as ad6
ada2 at ahcich2 bus 0 scbus2 target 0 lun 0
ada2: <ST3000DM001-1CH166 CC44> ATA-8 SATA 3.x device
ada2: Serial Number Z1F2PSFZ
ada2: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada2: Command Queueing enabled
ada2: 2861588MB (5860533168 512 byte sectors: 16H 63S/T 16383C)
ada2: quirks=0x1<4K>
ada2: Previously was known as ad8
ada3 at ahcich3 bus 0 scbus3 target 0 lun 0
ada3: <WDC WD30EZRX-00DC0B0 80.00A80> ATA-9 SATA 3.x device
ada3: Serial Number WD-WMC1T4032608
ada3: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada3: Command Queueing enabled
ada3: 2861588MB (5860533168 512 byte sectors: 16H 63S/T 16383C)
ada3: quirks=0x1<4K>
ada3: Previously was known as ad10
ada4 at ahcich4 bus 0 scbus4 target 0 lun 0
ada4: <ST3000DM001-1CH166 CC24> ATA-8 SATA 3.x device
ada4: Serial Number W1F1Z7K6
ada4: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada4: Command Queueing enabled
ada4: 2861588MB (5860533168 512 byte sectors: 16H 63S/T 16383C)
ada4: quirks=0x1<4K>
ada4: Previously was known as ad12
ada5 at ahcich5 bus 0 scbus5 target 0 lun 0
ada5: <INTENSO 5.0.4> ATA-8 SATA 3.x device
ada5: Serial Number 130608901433
ada5: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada5: Command Queueing enabled
ada5: 114473MB (234441648 512 byte sectors: 16H 63S/T 16383C)
ada5: Previously was known as ad14
SMP: AP CPU #3 Launched!
SMP: AP CPU #2 Launched!
SMP: AP CPU #1 Launched!
Timecounter "TSC-low" frequency 1446682196 Hz quality 1000
uhub1: 2 ports with 2 removable, self powered
uhub0: 2 ports with 2 removable, self powered
Root mount waiting for: usbus1 usbus0
ugen0.2: <vendor 0x8087> at usbus0
uhub2: <vendor 0x8087 product 0x8008, class 9/0, rev 2.00/0.04, addr 2> on usbus0
ugen1.2: <vendor 0x8087> at usbus1
uhub3: <vendor 0x8087 product 0x8000, class 9/0, rev 2.00/0.04, addr 2> on usbus1
uhub2: 6 ports with 6 removable, self powered
uhub3: 8 ports with 8 removable, self powered
Root mount waiting for: usbus1
ugen1.3: <SanDisk> at usbus1
umass0: <SanDisk Ultra, class 0/0, rev 2.00/1.03, addr 3> on usbus1
umass0:  SCSI over Bulk-Only; quirks = 0x4100
umass0:7:0:-1: Attached to scbus7
Trying to mount root from ufs:/dev/ufs/FreeNASs1a [ro]...
mountroot: waiting for device /dev/ufs/FreeNASs1a ...
da0 at umass-sim0 bus 0 scbus7 target 0 lun 0
da0: <SanDisk Ultra 1.20> Removable Direct Access SCSI-5 device
da0: Serial Number 200443203102C94188A3
da0: 40.000MB/s transfers
da0: 7633MB (15633408 512 byte sectors: 255H 63S/T 973C)
da0: quirks=0x2<NO_6_BYTE>
GEOM_RAID5: Module loaded, version 1.1.20130907.44 (rev 5c6d2a159411)
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
GEOM_ELI: Device ada0p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:     Crypto: hardware
GEOM_ELI: Device ada1p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:     Crypto: hardware
GEOM_ELI: Device ada2p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:     Crypto: hardware
GEOM_ELI: Device ada4p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:     Crypto: hardware
Solaris: WARNING: can't open objset for supervolume/.system/syslog
Solaris: WARNING: can't open objset for supervolume/.system/syslog
Solaris: WARNING: can't open objset for supervolume/.system/syslog
Solaris: WARNING: can't open objset for supervolume/.system/syslog
Solaris: WARNING: can't open objset for supervolume/.system/syslog
Solaris: WARNING: can't open objset for supervolume/.system/syslog
Solaris: WARNING: can't open objset for supervolume/.system/syslog
Solaris: WARNING: can't open objset for supervolume/.system/syslog
Solaris: WARNING: can't open objset for supervolume/.system/syslog
vboxdrv: fAsync=0 offMin=0x366 offMax=0xcad
Solaris: WARNING: can't open objset for supervolume/.system/syslog
pid 3994 (zpool), uid 0: exited on signal 6
pid 3998 (zpool), uid 0: exited on signal 6
pid 4003 (zpool), uid 0: exited on signal 6
Waiting (max 60 seconds) for system process `vnlru' to stop...done
Waiting (max 60 seconds) for system process `bufdaemon' to stop...done
Waiting (max 60 seconds) for system process `syncer' to stop...
Syncing disks, vnodes remaining...0 0 done
All buffers synced.
Copyright (c) 1992-2013 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 9.2-RELEASE-p9 #0 r262572+cc525fe: Thu Jul  3 14:22:45 PDT 2014
    root@build3.ixsystems.com:/fusion/jenkins/9.2.1.6-RELEASE/os-base/amd64/fusion/jenkins/9.2.1.6-RELEASE/FreeBSD/src/sys/FREENAS.amd64 amd64
gcc version 4.2.1 20070831 patched [FreeBSD]
CPU: Intel(R) Core(TM) i5-4570S CPU @ 2.90GHz (2893.36-MHz K8-class CPU)
  Origin = "GenuineIntel"  Id = 0x306c3  Family = 0x6  Model = 0x3c  Stepping = 3
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x7ffafbff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,<b11>,FMA,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
  AMD Features2=0x21<LAHF,ABM>
  Standard Extended Features=0x2fbb<GSFSBASE,TSCADJ,SMEP,ENHMOVSB,INVPCID>
  TSC: P-state invariant, performance statistics
real memory  = 8589934592 (8192 MB)
avail memory = 7910817792 (7544 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <ALASKA A M I>
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s)
cpu0 (BSP): APIC ID:  0
cpu1 (AP): APIC ID:  2
cpu2 (AP): APIC ID:  4
cpu3 (AP): APIC ID:  6
WARNING: VIMAGE (virtualized network stack) is a highly experimental feature.
ACPI Warning: FADT (revision 5) is longer than ACPI 2.0 version, truncating length 268 to 244 (20110527/tbfadt-320)
ioapic0 <Version 2.0> irqs 0-23 on motherboard
kbd1 at kbdmux0
cryptosoft0: <software crypto> on motherboard
aesni0: <AES-CBC,AES-XTS> on motherboard
padlock0: No ACE support.
acpi0: <ALASKA A M I> on motherboard
ACPI Error: [RAMB] Namespace lookup failure, AE_NOT_FOUND (20110527/psargs-392)
ACPI Exception: AE_NOT_FOUND, Could not execute arguments for [RAMW] (Region) (20110527/nsinit-380)
acpi0: Power Button (fixed)
acpi0: reservation of 67, 1 (4) failed
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
cpu2: <ACPI CPU> on acpi0
cpu3: <ACPI CPU> on acpi0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 950
Event timer "HPET" frequency 14318180 Hz quality 550
Event timer "HPET1" frequency 14318180 Hz quality 440
Event timer "HPET2" frequency 14318180 Hz quality 440
Event timer "HPET3" frequency 14318180 Hz quality 440
Event timer "HPET4" frequency 14318180 Hz quality 440
atrtc0: <AT realtime clock> port 0x70-0x77 irq 8 on acpi0
atrtc0: Warning: Couldn't map I/O.
Event timer "RTC" frequency 32768 Hz quality 0
attimer0: <AT timer> port 0x40-0x43,0x50-0x53 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1808-0x180b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> irq 16 at device 1.0 on pci0
pci1: <ACPI PCI bus> on pcib1
vgapci0: <VGA-compatible display> port 0xf000-0xf03f mem 0xf7800000-0xf7bfffff,0xe0000000-0xefffffff irq 16 at device 2.0 on pci0
pci0: <multimedia, HDA> at device 3.0 (no driver attached)
pci0: <serial bus, USB> at device 20.0 (no driver attached)
pci0: <simple comms> at device 22.0 (no driver attached)
em0: <Intel(R) PRO/1000 Network Connection 7.3.8> port 0xf080-0xf09f mem 0xf7c00000-0xf7c1ffff,0xf7c3d000-0xf7c3dfff irq 20 at device 25.0 on pci0
em0: Using an MSI interrupt
em0: Ethernet address: 60:a4:4c:64:00:02
ehci0: <EHCI (generic) USB 2.0 controller> mem 0xf7c3c000-0xf7c3c3ff irq 16 at device 26.0 on pci0
usbus0: EHCI version 1.0
usbus0 on ehci0
pci0: <multimedia, HDA> at device 27.0 (no driver attached)
ehci1: <EHCI (generic) USB 2.0 controller> mem 0xf7c3b000-0xf7c3b3ff irq 23 at device 29.0 on pci0
usbus1: EHCI version 1.0
usbus1 on ehci1
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
ahci0: <Intel Lynx Point AHCI SATA controller> port 0xf0d0-0xf0d7,0xf0c0-0xf0c3,0xf0b0-0xf0b7,0xf0a0-0xf0a3,0xf060-0xf07f mem 0xf7c3a000-0xf7c3a7ff irq 19 at device 31.2 on pci0
ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich1: <AHCI channel> at channel 1 on ahci0
ahcich2: <AHCI channel> at channel 2 on ahci0
ahcich3: <AHCI channel> at channel 3 on ahci0
ahcich4: <AHCI channel> at channel 4 on ahci0
ahcich5: <AHCI channel> at channel 5 on ahci0
pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
acpi_button0: <Power Button> on acpi0
acpi_tz0: <Thermal Zone> on acpi0
acpi_tz1: <Thermal Zone> on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
wbwd0: DevID 0xc8 DevRev 0x03, will not attach, please report this.
coretemp0: <CPU On-Die Thermal Sensors> on cpu0
est0: <Enhanced SpeedStep Frequency Control> on cpu0
p4tcc0: <CPU Frequency Thermal Control> on cpu0
coretemp1: <CPU On-Die Thermal Sensors> on cpu1
est1: <Enhanced SpeedStep Frequency Control> on cpu1
p4tcc1: <CPU Frequency Thermal Control> on cpu1
coretemp2: <CPU On-Die Thermal Sensors> on cpu2
est2: <Enhanced SpeedStep Frequency Control> on cpu2
p4tcc2: <CPU Frequency Thermal Control> on cpu2
coretemp3: <CPU On-Die Thermal Sensors> on cpu3
est3: <Enhanced SpeedStep Frequency Control> on cpu3
p4tcc3: <CPU Frequency Thermal Control> on cpu3
Timecounters tick every 1.000 msec
ipfw2 (+ipv6) initialized, divert enabled, nat enabled, default to accept, logging disabled
DUMMYNET 0xfffffe000313ac80 with IPv6 initialized (100409)
load_dn_sched dn_sched RR loaded
load_dn_sched dn_sched WF2Q+ loaded
load_dn_sched dn_sched FIFO loaded
load_dn_sched dn_sched PRIO loaded
load_dn_sched dn_sched QFQ loaded
usbus0: 480Mbps High Speed USB v2.0
usbus1: 480Mbps High Speed USB v2.0
ugen0.1: <Intel> at usbus0
uhub0: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus0
ugen1.1: <Intel> at usbus1
uhub1: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
ada0: <ST3000DM001-1CH166 CC24> ATA-8 SATA 3.x device
ada0: Serial Number W1F1Y108
ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada0: Command Queueing enabled
ada0: 2861588MB (5860533168 512 byte sectors: 16H 63S/T 16383C)
ada0: quirks=0x1<4K>
ada0: Previously was known as ad4
ada1 at ahcich1 bus 0 scbus1 target 0 lun 0
ada1: <ST3000DM001-1CH166 CC24> ATA-8 SATA 3.x device
ada1: Serial Number Z1F222PJ
ada1: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada1: Command Queueing enabled
ada1: 2861588MB (5860533168 512 byte sectors: 16H 63S/T 16383C)
ada1: quirks=0x1<4K>
ada1: Previously was known as ad6
ada2 at ahcich2 bus 0 scbus2 target 0 lun 0
ada2: <ST3000DM001-1CH166 CC44> ATA-8 SATA 3.x device
ada2: Serial Number Z1F2PSFZ
ada2: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada2: Command Queueing enabled
ada2: 2861588MB (5860533168 512 byte sectors: 16H 63S/T 16383C)
ada2: quirks=0x1<4K>
ada2: Previously was known as ad8
ada3 at ahcich3 bus 0 scbus3 target 0 lun 0
ada3: <WDC WD30EZRX-00DC0B0 80.00A80> ATA-9 SATA 3.x device
ada3: Serial Number WD-WMC1T4032608
ada3: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada3: Command Queueing enabled
ada3: 2861588MB (5860533168 512 byte sectors: 16H 63S/T 16383C)
ada3: quirks=0x1<4K>
ada3: Previously was known as ad10
ada4 at ahcich4 bus 0 scbus4 target 0 lun 0
ada4: <ST3000DM001-1CH166 CC24> ATA-8 SATA 3.x device
ada4: Serial Number W1F1Z7K6
ada4: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada4: Command Queueing enabled
ada4: 2861588MB (5860533168 512 byte sectors: 16H 63S/T 16383C)
ada4: quirks=0x1<4K>
ada4: Previously was known as ad12
ada5 at ahcich5 bus 0 scbus5 target 0 lun 0
ada5: <INTENSO 5.0.4> ATA-8 SATA 3.x device
ada5: Serial Number 130608901433
ada5: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada5: Command Queueing enabled
ada5: 114473MB (234441648 512 byte sectors: 16H 63S/T 16383C)
ada5: Previously was known as ad14
SMP: AP CPU #3 Launched!
SMP: AP CPU #2 Launched!
SMP: AP CPU #1 Launched!
Timecounter "TSC-low" frequency 1446682248 Hz quality 1000
uhub1: 2 ports with 2 removable, self powered
uhub0: 2 ports with 2 removable, self powered
Root mount waiting for: usbus1 usbus0
ugen1.2: <vendor 0x8087> at usbus1
uhub2: <vendor 0x8087 product 0x8000, class 9/0, rev 2.00/0.04, addr 2> on usbus1
ugen0.2: <vendor 0x8087> at usbus0
uhub3: <vendor 0x8087 product 0x8008, class 9/0, rev 2.00/0.04, addr 2> on usbus0
uhub3: 6 ports with 6 removable, self powered
uhub2: 8 ports with 8 removable, self powered
Root mount waiting for: usbus1
ugen1.3: <SanDisk> at usbus1
umass0: <SanDisk Ultra, class 0/0, rev 2.00/1.03, addr 3> on usbus1
umass0:  SCSI over Bulk-Only; quirks = 0x4100
umass0:7:0:-1: Attached to scbus7
Trying to mount root from ufs:/dev/ufs/FreeNASs1a [ro]...
mountroot: waiting for device /dev/ufs/FreeNASs1a ...
da0 at umass-sim0 bus 0 scbus7 target 0 lun 0
da0: <SanDisk Ultra 1.20> Removable Direct Access SCSI-5 device
da0: Serial Number 200443203102C94188A3
da0: 40.000MB/s transfers
da0: 7633MB (15633408 512 byte sectors: 255H 63S/T 973C)
da0: quirks=0x2<NO_6_BYTE>
GEOM_RAID5: Module loaded, version 1.1.20130907.44 (rev 5c6d2a159411)
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
GEOM_ELI: Device ada0p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:     Crypto: hardware
GEOM_ELI: Device ada1p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:     Crypto: hardware
GEOM_ELI: Device ada2p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:     Crypto: hardware
GEOM_ELI: Device ada4p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:     Crypto: hardware
Solaris: WARNING: can't open objset for supervolume/.system/syslog
Solaris: WARNING: can't open objset for supervolume/.system/syslog
Solaris: WARNING: can't open objset for supervolume/.system/syslog
Solaris: WARNING: can't open objset for supervolume/.system/syslog
Solaris: WARNING: can't open objset for supervolume/.system/syslog
Solaris: WARNING: can't open objset for supervolume/.system/syslog
Solaris: WARNING: can't open objset for supervolume/.system/syslog
Solaris: WARNING: can't open objset for supervolume/.system/syslog
Solaris: WARNING: can't open objset for supervolume/.system/syslog
vboxdrv: fAsync=0 offMin=0x34f offMax=0xc93

Code:

 zpool import
   pool: supervolume
     id: 3261155225326581703
  state: DEGRADED
 status: One or more devices are missing from the system.
 action: The pool can be imported despite missing or damaged devices.  The
        fault tolerance of the pool may be compromised if imported.
   see: http://illumos.org/msg/ZFS-8000-2Q
 config:

        supervolume                                     DEGRADED
          raidz1-0                                      DEGRADED
            gptid/f1ba6158-e8ae-11e2-b0f3-60a44c640002  ONLINE
            gptid/f23bc149-e8ae-11e2-b0f3-60a44c640002  ONLINE
            gptid/f2bd7dd9-e8ae-11e2-b0f3-60a44c640002  ONLINE
            6732861267941806277                         UNAVAIL  cannot open
            gptid/f3c0f709-e8ae-11e2-b0f3-60a44c640002  ONLINE
        cache
          gptid/1fb1da24-1c5c-11e3-97bb-60a44c640002

No I did not use import -f option.

What I have did is to take out the disk that no longer was showing up in BIOS and inserted a new one (thats not used yet)

gpart show

Code:

gpart show
=>        34  5860533101  ada0  GPT  (2.7T)
          34          94        - free -  (47k)
         128     4194304     1  freebsd-swap  (2.0G)
     4194432  5856338696     2  freebsd-zfs  (2.7T)
  5860533128           7        - free -  (3.5k)

=>        34  5860533101  ada1  GPT  (2.7T)
          34          94        - free -  (47k)
         128     4194304     1  freebsd-swap  (2.0G)
     4194432  5856338696     2  freebsd-zfs  (2.7T)
  5860533128           7        - free -  (3.5k)

=>        34  5860533101  ada2  GPT  (2.7T)
          34          94        - free -  (47k)
         128     4194304     1  freebsd-swap  (2.0G)
     4194432  5856338696     2  freebsd-zfs  (2.7T)
  5860533128           7        - free -  (3.5k)

=>        34  5860533101  ada4  GPT  (2.7T)
          34          94        - free -  (47k)
         128     4194304     1  freebsd-swap  (2.0G)
     4194432  5856338696     2  freebsd-zfs  (2.7T)
  5860533128           7        - free -  (3.5k)

=>       34  234441581  ada5  GPT  (111G)
         34         94        - free -  (47k)
        128  234441487     1  freebsd-zfs  (111G)

=>      63  15633345  da0  MBR  (7.5G)
        63   3590433    1  freebsd  [active]  (1.7G)
   3590496        63       - free -  (31k)
   3590559   3590433    2  freebsd  (1.7G)
   7180992      3024    3  freebsd  (1.5M)
   7184016     41328    4  freebsd  (20M)
   7225344   8408064       - free -  (4.0G)

=>      0  3590433  da0s1  BSD  (1.7G)
        0       16         - free -  (8.0k)
       16  1930241      1  !0  (942M)
  1930257  1660176         - free -  (810M)

=>      0  3590433  da0s2  BSD  (1.7G)
        0       16         - free -  (8.0k)
       16  1930241      1  !0  (942M)
  1930257  1660176         - free -  (810M)

 

[reidara]

So the zpool is contained by RAID-Z1 and has a ZIL Log Writer

Tha Raid-Z1 is 5 x 3TB disks .

One disk is not showing up in bios and is removed. another new is inserted

freenas is booting when i trigger smartctl command to one of the remaining disks /dev/ada2
Seems like this could be a cause. So I really do not hope I have had 2 failed disks......

here is smartctl of alle the disks (without the new one and the ssd)

/dev/ada0

Code:

smartctl -q noserial -a /dev/ada0
smartctl 6.2 2013-07-26 r3841 [FreeBSD 9.2-RELEASE-p9 amd64] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Seagate Barracuda 7200.14 (AF)
Device Model:     ST3000DM001-1CH166
Firmware Version: CC24
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ATA8-ACS T13/1699-D revision 4
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Sat Aug 16 07:57:45 2014 CEST

==> WARNING: A firmware update for this drive may be available,
see the following Seagate web pages:
http://knowledge.seagate.com/articles/en_US/FAQ/207931en
http://knowledge.seagate.com/articles/en_US/FAQ/223651en

SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (  592) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   1) minutes.
Extended self-test routine
recommended polling time:        ( 344) minutes.
Conveyance self-test routine
recommended polling time:        (   2) minutes.
SCT capabilities:              (0x3085) SCT Status supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   116   099   006    Pre-fail  Always       -       110417240
  3 Spin_Up_Time            0x0003   094   094   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   100   100   020    Old_age   Always       -       75
  5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000f   084   060   030    Pre-fail  Always       -       4543744940
  9 Power_On_Hours          0x0032   090   090   000    Old_age   Always       -       9509
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   020    Old_age   Always       -       72
183 Runtime_Bad_Block       0x0032   100   100   000    Old_age   Always       -       0
184 End-to-End_Error        0x0032   100   100   099    Old_age   Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
188 Command_Timeout         0x0032   100   100   000    Old_age   Always       -       0 0 0
189 High_Fly_Writes         0x003a   100   100   000    Old_age   Always       -       0
190 Airflow_Temperature_Cel 0x0022   061   050   045    Old_age   Always       -       39 (Min/Max 39/42)
191 G-Sense_Error_Rate      0x0032   100   100   000    Old_age   Always       -       0
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       65
193 Load_Cycle_Count        0x0032   100   100   000    Old_age   Always       -       407
194 Temperature_Celsius     0x0022   039   050   000    Old_age   Always       -       39 (0 23 0 0 0)
197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
240 Head_Flying_Hours       0x0000   100   253   000    Old_age   Offline      -       9482h+51m+02.143s
241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       40238058608
242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       95451660511

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
No self-tests have been logged.  [To run self-tests, use: smartctl -t]


SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

[root@nasse] ~# smartctl -q noserial -a /dev/ada1
smartctl 6.2 2013-07-26 r3841 [FreeBSD 9.2-RELEASE-p9 amd64] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Seagate Barracuda 7200.14 (AF)
Device Model:     ST3000DM001-1CH166
Firmware Version: CC24
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ATA8-ACS T13/1699-D revision 4
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Sat Aug 16 07:58:03 2014 CEST

==> WARNING: A firmware update for this drive may be available,
see the following Seagate web pages:
http://knowledge.seagate.com/articles/en_US/FAQ/207931en
http://knowledge.seagate.com/articles/en_US/FAQ/223651en

SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (  575) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   1) minutes.
Extended self-test routine
recommended polling time:        ( 335) minutes.
Conveyance self-test routine
recommended polling time:        (   2) minutes.
SCT capabilities:              (0x3085) SCT Status supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   117   099   006    Pre-fail  Always       -       140302936
  3 Spin_Up_Time            0x0003   094   094   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   100   100   020    Old_age   Always       -       75
  5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000f   077   060   030    Pre-fail  Always       -       21722465645
  9 Power_On_Hours          0x0032   090   090   000    Old_age   Always       -       9511
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   020    Old_age   Always       -       72
183 Runtime_Bad_Block       0x0032   100   100   000    Old_age   Always       -       0
184 End-to-End_Error        0x0032   100   100   099    Old_age   Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
188 Command_Timeout         0x0032   100   100   000    Old_age   Always       -       0 0 0
189 High_Fly_Writes         0x003a   096   096   000    Old_age   Always       -       4
190 Airflow_Temperature_Cel 0x0022   062   051   045    Old_age   Always       -       38 (Min/Max 38/41)
191 G-Sense_Error_Rate      0x0032   100   100   000    Old_age   Always       -       0
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       65
193 Load_Cycle_Count        0x0032   100   100   000    Old_age   Always       -       401
194 Temperature_Celsius     0x0022   038   049   000    Old_age   Always       -       38 (0 23 0 0 0)
197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
240 Head_Flying_Hours       0x0000   100   253   000    Old_age   Offline      -       9485h+13m+53.927s
241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       40236827860
242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       95307119057

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
No self-tests have been logged.  [To run self-tests, use: smartctl -t]


SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

/dev/ada1

Code:

smartctl -q noserial -a /dev/ada1
smartctl 6.2 2013-07-26 r3841 [FreeBSD 9.2-RELEASE-p9 amd64] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Seagate Barracuda 7200.14 (AF)
Device Model:     ST3000DM001-1CH166
Firmware Version: CC24
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ATA8-ACS T13/1699-D revision 4
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Sat Aug 16 08:15:58 2014 CEST

==> WARNING: A firmware update for this drive may be available,
see the following Seagate web pages:
http://knowledge.seagate.com/articles/en_US/FAQ/207931en
http://knowledge.seagate.com/articles/en_US/FAQ/223651en

SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (  575) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   1) minutes.
Extended self-test routine
recommended polling time:        ( 335) minutes.
Conveyance self-test routine
recommended polling time:        (   2) minutes.
SCT capabilities:              (0x3085) SCT Status supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   117   099   006    Pre-fail  Always       -       140302936
  3 Spin_Up_Time            0x0003   094   094   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   100   100   020    Old_age   Always       -       75
  5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000f   077   060   030    Pre-fail  Always       -       21722465645
  9 Power_On_Hours          0x0032   090   090   000    Old_age   Always       -       9511
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   020    Old_age   Always       -       72
183 Runtime_Bad_Block       0x0032   100   100   000    Old_age   Always       -       0
184 End-to-End_Error        0x0032   100   100   099    Old_age   Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
188 Command_Timeout         0x0032   100   100   000    Old_age   Always       -       0 0 0
189 High_Fly_Writes         0x003a   096   096   000    Old_age   Always       -       4
190 Airflow_Temperature_Cel 0x0022   062   051   045    Old_age   Always       -       38 (Min/Max 38/41)
191 G-Sense_Error_Rate      0x0032   100   100   000    Old_age   Always       -       0
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       65
193 Load_Cycle_Count        0x0032   100   100   000    Old_age   Always       -       401
194 Temperature_Celsius     0x0022   038   049   000    Old_age   Always       -       38 (0 23 0 0 0)
197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
240 Head_Flying_Hours       0x0000   100   253   000    Old_age   Offline      -       9485h+31m+48.571s
241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       40236827860
242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       95307119057

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
No self-tests have been logged.  [To run self-tests, use: smartctl -t]


SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

/dev/ada4



/dev/ada2

Code:

smartctl -q noserial -a /dev/ada2
smartctl 6.2 2013-07-26 r3841 [FreeBSD 9.2-RELEASE-p9 amd64] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Seagate Barracuda 7200.14 (AF)
Device Model:     ST3000DM001-1CH166
Firmware Version: CC44
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ATA8-ACS T13/1699-D revision 4
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Sat Aug 16 07:58:14 2014 CEST

==> WARNING: A firmware update for this drive may be available,
see the following Seagate web pages:
http://knowledge.seagate.com/articles/en_US/FAQ/207931en
http://knowledge.seagate.com/articles/en_US/FAQ/223651en

SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (  584) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   1) minutes.
Extended self-test routine
recommended polling time:        ( 341) minutes.
Conveyance self-test routine
recommended polling time:        (   2) minutes.
SCT capabilities:              (0x3085) SCT Status supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   115   098   006    Pre-fail  Always       -       85763984
  3 Spin_Up_Time            0x0003   091   091   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   100   100   020    Old_age   Always       -       49
  5 Reallocated_Sector_Ct   0x0033   099   099   010    Pre-fail  Always       -       520
  7 Seek_Error_Rate         0x000f   077   060   030    Pre-fail  Always       -       21721212378
  9 Power_On_Hours          0x0032   090   090   000    Old_age   Always       -       9477
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   020    Old_age   Always       -       47
183 Runtime_Bad_Block       0x0032   100   100   000    Old_age   Always       -       0
184 End-to-End_Error        0x0032   100   100   099    Old_age   Always       -       0
187 Reported_Uncorrect      0x0032   015   015   000    Old_age   Always       -       85
188 Command_Timeout         0x0032   100   098   000    Old_age   Always       -       3 3 4
189 High_Fly_Writes         0x003a   097   097   000    Old_age   Always       -       3
190 Airflow_Temperature_Cel 0x0022   060   048   045    Old_age   Always       -       40 (Min/Max 40/44)
191 G-Sense_Error_Rate      0x0032   100   100   000    Old_age   Always       -       0
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       41
193 Load_Cycle_Count        0x0032   100   100   000    Old_age   Always       -       119
194 Temperature_Celsius     0x0022   040   052   000    Old_age   Always       -       40 (0 23 0 0 0)
197 Current_Pending_Sector  0x0012   096   090   000    Old_age   Always       -       688
198 Offline_Uncorrectable   0x0010   096   090   000    Old_age   Offline      -       688
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
240 Head_Flying_Hours       0x0000   100   253   000    Old_age   Offline      -       9474h+49m+22.613s
241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       40236995876
242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       95327972231

SMART Error Log Version: 1
ATA Error Count: 85 (device log contains only the most recent five errors)
        CR = Command Register [HEX]
        FR = Features Register [HEX]
        SC = Sector Count Register [HEX]
        SN = Sector Number Register [HEX]
        CL = Cylinder Low Register [HEX]
        CH = Cylinder High Register [HEX]
        DH = Device/Head Register [HEX]
        DC = Device Command Register [HEX]
        ER = Error register [HEX]
        ST = Status register [HEX]
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.

Error 85 occurred at disk power-on lifetime: 9445 hours (393 days + 13 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 00 ff ff ff 0f  Error: UNC at LBA = 0x0fffffff = 268435455

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  60 00 40 ff ff ff 4f 00  49d+10:36:51.876  READ FPDMA QUEUED
  60 00 40 ff ff ff 4f 00  49d+10:36:51.876  READ FPDMA QUEUED
  60 00 40 ff ff ff 4f 00  49d+10:36:51.875  READ FPDMA QUEUED
  60 00 00 ff ff ff 4f 00  49d+10:36:51.875  READ FPDMA QUEUED
  2f 00 01 10 00 00 00 00  49d+10:36:51.827  READ LOG EXT

Error 84 occurred at disk power-on lifetime: 9445 hours (393 days + 13 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 00 ff ff ff 0f  Error: UNC at LBA = 0x0fffffff = 268435455

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  60 00 40 ff ff ff 4f 00  49d+10:36:48.988  READ FPDMA QUEUED
  60 00 40 ff ff ff 4f 00  49d+10:36:48.988  READ FPDMA QUEUED
  60 00 40 ff ff ff 4f 00  49d+10:36:48.988  READ FPDMA QUEUED
  60 00 00 ff ff ff 4f 00  49d+10:36:48.986  READ FPDMA QUEUED
  2f 00 01 10 00 00 00 00  49d+10:36:48.951  READ LOG EXT

Error 83 occurred at disk power-on lifetime: 9445 hours (393 days + 13 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 00 ff ff ff 0f  Error: UNC at LBA = 0x0fffffff = 268435455

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  60 00 40 ff ff ff 4f 00  49d+10:36:46.102  READ FPDMA QUEUED
  60 00 40 ff ff ff 4f 00  49d+10:36:46.101  READ FPDMA QUEUED
  60 00 40 ff ff ff 4f 00  49d+10:36:46.101  READ FPDMA QUEUED
  60 00 00 ff ff ff 4f 00  49d+10:36:46.101  READ FPDMA QUEUED
  2f 00 01 10 00 00 00 00  49d+10:36:46.034  READ LOG EXT

Error 82 occurred at disk power-on lifetime: 9445 hours (393 days + 13 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 00 ff ff ff 0f  Error: WP at LBA = 0x0fffffff = 268435455

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  61 00 30 ff ff ff 4f 00  49d+10:36:43.178  WRITE FPDMA QUEUED
  61 00 10 ff ff ff 4f 00  49d+10:36:43.178  WRITE FPDMA QUEUED
  60 00 40 ff ff ff 4f 00  49d+10:36:43.178  READ FPDMA QUEUED
  60 00 40 ff ff ff 4f 00  49d+10:36:43.177  READ FPDMA QUEUED
  60 00 40 ff ff ff 4f 00  49d+10:36:43.177  READ FPDMA QUEUED

Error 81 occurred at disk power-on lifetime: 9445 hours (393 days + 13 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 00 ff ff ff 0f  Error: UNC at LBA = 0x0fffffff = 268435455

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  60 00 00 ff ff ff 4f 00  49d+10:36:39.595  READ FPDMA QUEUED
  60 00 00 ff ff ff 4f 00  49d+10:36:39.595  READ FPDMA QUEUED
  60 00 00 ff ff ff 4f 00  49d+10:36:39.595  READ FPDMA QUEUED
  60 00 c0 ff ff ff 4f 00  49d+10:36:39.594  READ FPDMA QUEUED
  60 00 00 ff ff ff 4f 00  49d+10:36:39.594  READ FPDMA QUEUED

SMART Self-test log structure revision number 1
No self-tests have been logged.  [To run self-tests, use: smartctl -t]


SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

 

[reidara]

I shut down the freenas host and plugged out the new hard disk and replaced the one that was not showing in bios.
suddenly it was recognized again in bios and the zpool volume got online

but it seems like freenas started to resilver to that disk....

now the status is like this

Code:

root@nasse] ~# zpool status
  pool: supervolume
 state: DEGRADED
status: One or more devices has experienced an error resulting in data
        corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
        entire pool from backup.
   see: http://illumos.org/msg/ZFS-8000-8A
  scan: resilvered 37.8G in 3h24m with 2309573 errors on Sat Aug 16 11:49:27 2014
config:

        NAME                                            STATE     READ WRITE CKSUM
        supervolume                                     DEGRADED     0     0 2.22M
          raidz1-0                                      DEGRADED     0     0 4.47M
            gptid/f1ba6158-e8ae-11e2-b0f3-60a44c640002  ONLINE       0     0     0
            gptid/f23bc149-e8ae-11e2-b0f3-60a44c640002  ONLINE       0     0     0
            11368300137973521949                        REMOVED      0     0     0  was /dev/gptid/f2bd7dd9-e8ae-11e2-b0f3-60a44c640002
            gptid/f33d2a9f-e8ae-11e2-b0f3-60a44c640002  ONLINE       0     0     0
            gptid/f3c0f709-e8ae-11e2-b0f3-60a44c640002  ONLINE       0     0 7.38K
        cache
          gptid/1fb1da24-1c5c-11e3-97bb-60a44c640002    ONLINE       0     0     0

errors: 2309931 data errors, use '-v' for a list

 

[joeschmuck]

I hope you have already backed up any data you deem important.

It looks like all your drives are actually recognized, although since you seem to be making changes without waiting for a reply, well I have no idea what configuration you are now in and you likely made it worse. It doesn't look like you followed the FreeNAS User Manual on how to replace a hard drive. You can't just swap drives out and it magically starts resilvering a new drive into the pool. The process goes like this: Offline the failed drive via GUI, Shutdown System, Replace Drive, Turn on system, Replace drive in pool via GUI, Verify resilvering is in progress.

Not that it matters for this particular problem but why do you have a ZIL on this system?

 

[reidara]

Sure , the most important are backed up.
I have not run any commands at all besides zpool import and zpool import -F supervolume.

The resilvering process has started all by itselves at some time... The status is this now on the pool

Code:

zpool status
  pool: supervolume
 state: DEGRADED
status: One or more devices has experienced an error resulting in data
        corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
        entire pool from backup.
   see: http://illumos.org/msg/ZFS-8000-8A
  scan: resilvered 426M in 1h29m with 2308614 errors on Sat Aug 16 14:51:54 2014
config:

        NAME                                            STATE     READ WRITE CKSUM
        supervolume                                     DEGRADED     0     0 1.05M
          raidz1-0                                      DEGRADED     0     0 2.11M
            gptid/f1ba6158-e8ae-11e2-b0f3-60a44c640002  ONLINE       0     0     0
            gptid/f23bc149-e8ae-11e2-b0f3-60a44c640002  ONLINE       0     0     0
            11368300137973521949                        UNAVAIL      0     0     0  was /dev/gptid/f2bd7dd9-e8ae-11e2-b0f3-60a44c640002
            gptid/f33d2a9f-e8ae-11e2-b0f3-60a44c640002  ONLINE       0     0     0
            gptid/f3c0f709-e8ae-11e2-b0f3-60a44c640002  ONLINE       0     0     0
        cache
          gptid/1fb1da24-1c5c-11e3-97bb-60a44c640002    ONLINE       0     0     0

errors: 2308646 data errors, use '-v' for a list

I havent started any replacing command or in GUI yet, but I have detached the drive that made a lot of noise ( smart wise) and inserted a new drive at same size.

Im going to follow the guide "Replacing a Failed Drive" in a couple om minutes I guess ;-)

 

[gpsguy]

Ditto what joeschmuck has said.

You have a number of issues, including "errors: 2308646 data errors, use '-v' for a list"

zpool status -v will show you the affected files

In 9000+ hours of use, no SMART tests have been run on any of the drives.

No self-tests have been logged. [To run self-tests, use: smartctl -t]

A common schedule, might be to run a short test weekly, and a long test twice a month. And, a scrub once or twice a month.

And, you have stuff going on with /dev/ada2

1. Error 85 occurred at disk power-on lifetime: 9445 hours (393 days + 13 hours)
2. When the command that caused the error occurred, the device was active or idle.
3. 
   
4. After command completion occurred, registers were:
5. ER ST SC SN CL CH DH
6. -- -- -- -- -- -- --
7. 40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455

One of your messages said "No I did not use import -f option.", yet later you said "I have not run any commands at all besides zpool import and zpool import -F supervolume."

Slow down ...

 

[reidara]

Ditto what joeschmuck has said.

You have a number of issues, including "errors: 2308646 data errors, use '-v' for a list"

zpool status -v will show you the affected files

In 9000+ hours of use, no SMART tests have been run on any of the drives.

No self-tests have been logged. [To run self-tests, use: smartctl -t]

A common schedule, might be to run a short test weekly, and a long test twice a month. And, a scrub once or twice a month.

And, you have stuff going on with /dev/ada2

1. Error 85 occurred at disk power-on lifetime: 9445 hours (393 days + 13 hours)
2. When the command that caused the error occurred, the device was active or idle.
3. 
   
4. After command completion occurred, registers were:
5. ER ST SC SN CL CH DH
6. -- -- -- -- -- -- --
7. 40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455

One of your messages said "No I did not use import -f option.", yet later you said "I have not run any commands at all besides zpool import and zpool import -F supervolume."

Slow down ...

I user wrong case in the post :
"No I did not use import -f option.", yet later you said "I have not run any commands at all besides zpool import and zpool import -F supervolume."
should be:
"No I did not use import -F option.", yet later you said "I have not run any commands at all besides zpool import and zpool import -F supervolume."
Thanks for notifying .


Yeah... too bad about the smartd test. have to do that.

The /dev/ada2 is removed from the computer physical. It was the one with most smart errors on console.
So Im really ready to start replacing this disk in GUI with anotherone I have attached.

 

[reidara]

resilvering is in progress.

smart tests will be set up when resilvering is done.

Code:

        NAME                                              STATE     READ WRITE CKSUM
        supervolume                                       DEGRADED     0     0 1.06M
          raidz1-0                                        DEGRADED     0     0 2.13M
            gptid/f1ba6158-e8ae-11e2-b0f3-60a44c640002    ONLINE       0     0     0
            gptid/f23bc149-e8ae-11e2-b0f3-60a44c640002    ONLINE       0     0     0
            replacing-2                                   UNAVAIL      0     0     0
              11368300137973521949                        UNAVAIL      0     0     0  was /dev/gptid/f2bd7dd9-e8ae-11e2-b0f3-60a44c640002
              gptid/2acef0ff-2570-11e4-ba4f-60a44c640002  ONLINE       0     0     0  (resilvering)
            gptid/f33d2a9f-e8ae-11e2-b0f3-60a44c640002    ONLINE       0     0     0
            gptid/f3c0f709-e8ae-11e2-b0f3-60a44c640002    ONLINE       0     0    29  (resilvering)
        cache
          gptid/1fb1da24-1c5c-11e3-97bb-60a44c640002      ONLINE       0     0     0

 

[joeschmuck]

I'm pushing this because I feel it will be helpful to you and anyone else who doesn't want to read the user manual when it's time to replace a drive. This is only for the current version 9.2.x of FreeNAS. I printed this out and hung it on my server. Eventually (likely next year) I will have to use this. I will of course upgrade my copy of the procedure if I upgrade my FreeNAS version if there is a needed change. Why did I come up with this? Because in the work environment we create recovery plans and those don't only redirect a person to the user manual, they also contain a customized procedure where needed and I felt this is one of those problems you see crop up all the time, someone having issues with replacing a hard drive. Unfortunately your problem was made larger because you didn't have SMART testing running so you waited until you had a much harder failure to find out.