Failed to geli detach error message while using auto import

[Jents]

Hi,


I have been using freenas for the last couple of months without any problem. I am using a 4 drive zpool ( 3TB each ). But I am afraid I have painted myself into a corner now. It did it in two stages ( all using the GUI )


Stage 1: It all began after I changed the permissions on my cifs share from windows to unix and back while trying to resolve some permission settings. This caused a problem with samba; it refused to start. Since I was still able to ssh into freenas and browse through my data, I was not too worried.


Stage 2: To fix this samba problem I searched on this forum and found one thing to try: detach the volume and auto-import it back again.

Now I wish I had not done that. The auto import function can not find my volume ( the disks are still visible ). typing “zpool list” also does not show any volumes


What I find strange is this error message when I try to auto import the volume:

Code:

Sep 7 14:50:03 jentsfreenas ntpd[2074]: time reset +119.257491 s
Sep 7 14:50:58 jentsfreenas manage.py: [middleware.notifier:1362] Failed to geli detach gptid/972a4c2f-dfa2-11e3-ab04-000c2952c1c1: geli: No such device: gptid/972a4c2f-dfa2-11e3-ab04-000c2952c1c1.
Sep 7 14:50:58 jentsfreenas manage.py: [middleware.notifier:1362] Failed to geli detach gptid/97fbc19c-dfa2-11e3-ab04-000c2952c1c1: geli: No such device: gptid/97fbc19c-dfa2-11e3-ab04-000c2952c1c1.
Sep 7 14:50:58 jentsfreenas manage.py: [middleware.notifier:1362] Failed to geli detach gptid/95816a9f-dfa2-11e3-ab04-000c2952c1c1: geli: No such device: gptid/95816a9f-dfa2-11e3-ab04-000c2952c1c1.
Sep 7 14:50:58 jentsfreenas manage.py: [middleware.notifier:1362] Failed to geli detach gptid/96587e32-dfa2-11e3-ab04-000c2952c1c1: geli: No such device: gptid/96587e32-dfa2-11e3-ab04-000c2952c1c1.

Failed to Detach? Did I somehow fail to unmount it completely and is this why the auto import is not seeing my volume ? Also, I am pretty sure I did not enable encryption. So why is geli then complaining ?


Thanks for any reply!


Jents

 

[dlavigne]

Which version of FreeNAS? Paste the output of zpool status.

 

[Jents]

Hi

Thanks for the reply!

I am using FreeNAS-9.2.1.6-RELEASE-x64 (ddd1e39)

Regarding the zpool status:
# zpool status
no pools available

Jents

 

[Jents]

No-one can help me ?
Do I need to add more information ?

 

[Jents]

Still no one ?

 

[dlavigne]

Well, you don't have a pool which is not good :-(

 

[Jents]

yes, I noticed I do not have a pool
I there any chance this is recoverable ? Do you have any idea about the Failed to Detach geli message ?
When I create a brand new freenas installation and try to auto import the drives, I get the exact same error message...

 

[mjws00]

Your thread keeps catching my eye. Unfortunately I'm not sure we'll get your data as it is pretty tough to determine exactly where it went sideways. But why d0n't we start with the basics.

List your hardware, as per the forum rules.
Give us the output of 'dmesg'
Give us the output of 'zpool import'

I've hunted high and low for you pertaining to your geli error, without any significant progress. It kinda looks like you needed to export your pool and you ended up destroying it? If we're lucky it is a glitch in the gui, and zfs itself will help us. She's a bit unforgiving of error if a mistake was made.

If encryption is involved I'm of little use; as I avoid it like the plague. But if we get all the information possible up maybe we can catch more eyes.

Not sure if you have a backup or the value of your data... The fastest and easiest is to simply rebuild the pool and restore. The reality is most of the time if there is any possible recovery zfs makes it pretty easy and painless. If it "hurts" there is a pretty good chance you are already toast.

p.s Make sure and use code tags so we can read this stuff.

 

[danb35]

Is/was your pool encrypted? And what, exactly, did you do to detach the volume? Did you check the "mark disks as new" box?

 

[Jents]

Hi,


Sorry for replying so late. I've been ill...

I am running freenas under esxi with 4 3TB WD Red drives in passthrough mode and 8GB of RAM assigned to it. Motherboard: ASRock B85M Pro4 CPU: Intel Haswell Core i5 4440S 2.80GHz 6MB Box

( I know, no ECC memory. I only learned of that requirement while looking for a solution for my present problem. I did however run memtest86 for 4 days to see whether memory problems could have caused this. It did not find any memory problems )


This is the output from dsmeg

Code:

Last login: Thu Sep 25 08:50:15 2014 from 192.167.1.10

FreeBSD 9.2-RELEASE-p9 (FREENAS.amd64) #0 r262572+cc525fe: Thu Jul 3 14:22:45 PDT 2014


FreeNAS (c) 2009-2014, The FreeNAS Development Team

All rights reserved.

FreeNAS is released under the modified BSD license.


For more information, documentation, help or support, go here:

http://freenas.org

Welcome to FreeNAS van Jents

Could not chdir to home directory /mnt/Main/jents: No such file or directory

[jents@jentsfreenas] /% dmesg

Copyright (c) 1992-2013 The FreeBSD Project.

Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994

The Regents of the University of California. All rights reserved.

FreeBSD is a registered trademark of The FreeBSD Foundation.

FreeBSD 9.2-RELEASE-p9 #0 r262572+cc525fe: Thu Jul 3 14:22:45 PDT 2014

root@build3.ixsystems.com:/fusion/jenkins/9.2.1.6-RELEASE/os-base/amd64/fusion/jenkins/9.2.1.6-RELEASE/FreeBSD/src/sys/FREENAS.amd64 amd64

gcc version 4.2.1 20070831 patched [FreeBSD]

CPU: Intel(R) Core(TM) i5-4440S CPU @ 2.80GHz (2799.19-MHz K8-class CPU)

Origin = "GenuineIntel" Id = 0x306c3 Family = 0x6 Model = 0x3c Stepping = 3

Features=0xfa3fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,DTS,MMX,FXSR,SSE,SSE2,SS>

Features2=0x9ed83203<SSE3,PCLMULQDQ,SSSE3,FMA,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,HV>

AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>

AMD Features2=0x1<LAHF>

TSC: P-state invariant

real memory = 8589934592 (8192 MB)

avail memory = 8227758080 (7846 MB)

Event timer "LAPIC" quality 600

ACPI APIC Table: <PTLTD APIC >

WARNING: VIMAGE (virtualized network stack) is a highly experimental feature.

MADT: Forcing active-low polarity and level trigger for SCI

ioapic0 <Version 1.1> irqs 0-23 on motherboard

kbd1 at kbdmux0

cryptosoft0: <software crypto> on motherboard

aesni0: <AES-CBC,AES-XTS> on motherboard

padlock0: No ACE support.

acpi0: <INTEL 440BX> on motherboard

acpi0: Power Button (fixed)

Timecounter "HPET" frequency 14318180 Hz quality 950

cpu0: <ACPI CPU> on acpi0

attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0

Timecounter "i8254" frequency 1193182 Hz quality 0

Event timer "i8254" frequency 1193182 Hz quality 100

atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0

Event timer "RTC" frequency 32768 Hz quality 0

Timecounter "ACPI-fast" frequency 3579545 Hz quality 900

acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0

pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0

pci0: <ACPI PCI bus> on pcib0

pcib1: <ACPI PCI-PCI bridge> at device 1.0 on pci0

pci1: <ACPI PCI bus> on pcib1

isab0: <PCI-ISA bridge> at device 7.0 on pci0

isa0: <ISA bus> on isab0

atapci0: <Intel PIIX4 UDMA33 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1060-0x106f at device 7.1 on pci0

ata0: <ATA channel> at channel 0 on atapci0

ata1: <ATA channel> at channel 1 on atapci0

pci0: <bridge> at device 7.3 (no driver attached)

pci0: <base peripheral> at device 7.7 (no driver attached)

vgapci0: <VGA-compatible display> port 0x1070-0x107f mem 0xec000000-0xefffffff,0xfe000000-0xfe7fffff irq 16 at device 15.0 on pci0

mpt0: <LSILogic 1030 Ultra4 Adapter> port 0x1400-0x14ff mem 0xfeba0000-0xfebbffff,0xfebc0000-0xfebdffff irq 17 at device 16.0 on pci0

mpt0: MPI Version=1.2.0.0

pcib2: <ACPI PCI-PCI bridge> at device 17.0 on pci0

pci2: <ACPI PCI bus> on pcib2

em0: <Intel(R) PRO/1000 Legacy Network Connection 1.0.6> port 0x2000-0x203f mem 0xfd5c0000-0xfd5dffff,0xfdff0000-0xfdffffff irq 18 at device 0.0 on pci2

em0: Ethernet address: 00:0c:29:52:c1:c1

pcib3: <ACPI PCI-PCI bridge> at device 21.0 on pci0

pci3: <ACPI PCI bus> on pcib3

mps0: <LSI SAS2008> port 0x4000-0x40ff mem 0xfd4fc000-0xfd4fffff,0xfd480000-0xfd4bffff irq 18 at device 0.0 on pci3

mps0: Firmware: 15.00.00.00, Driver: 16.00.00.00-fbsd

mps0: IOCCapabilities: 1285c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,HostDisc>

pcib4: <ACPI PCI-PCI bridge> at device 21.1 on pci0

pci4: <ACPI PCI bus> on pcib4

pcib5: <ACPI PCI-PCI bridge> at device 21.2 on pci0

pci5: <ACPI PCI bus> on pcib5

pcib6: <ACPI PCI-PCI bridge> at device 21.3 on pci0

pci6: <ACPI PCI bus> on pcib6

pcib7: <ACPI PCI-PCI bridge> at device 21.4 on pci0

pci7: <ACPI PCI bus> on pcib7

pcib8: <ACPI PCI-PCI bridge> at device 21.5 on pci0

pci8: <ACPI PCI bus> on pcib8

pcib9: <ACPI PCI-PCI bridge> at device 21.6 on pci0

pci9: <ACPI PCI bus> on pcib9

pcib10: <ACPI PCI-PCI bridge> at device 21.7 on pci0

pci10: <ACPI PCI bus> on pcib10

pcib11: <ACPI PCI-PCI bridge> at device 22.0 on pci0

pci11: <ACPI PCI bus> on pcib11

pcib12: <ACPI PCI-PCI bridge> at device 22.1 on pci0

pci12: <ACPI PCI bus> on pcib12

pcib13: <ACPI PCI-PCI bridge> at device 22.2 on pci0

pci13: <ACPI PCI bus> on pcib13

pcib14: <ACPI PCI-PCI bridge> at device 22.3 on pci0

pci14: <ACPI PCI bus> on pcib14

pcib15: <ACPI PCI-PCI bridge> at device 22.4 on pci0

pci15: <ACPI PCI bus> on pcib15

pcib16: <ACPI PCI-PCI bridge> at device 22.5 on pci0

pci16: <ACPI PCI bus> on pcib16

pcib17: <ACPI PCI-PCI bridge> at device 22.6 on pci0

pci17: <ACPI PCI bus> on pcib17

pcib18: <ACPI PCI-PCI bridge> at device 22.7 on pci0

pci18: <ACPI PCI bus> on pcib18

pcib19: <ACPI PCI-PCI bridge> at device 23.0 on pci0

pci19: <ACPI PCI bus> on pcib19

pcib20: <ACPI PCI-PCI bridge> at device 23.1 on pci0

pci20: <ACPI PCI bus> on pcib20

pcib21: <ACPI PCI-PCI bridge> at device 23.2 on pci0

pci21: <ACPI PCI bus> on pcib21

pcib22: <ACPI PCI-PCI bridge> at device 23.3 on pci0

pci22: <ACPI PCI bus> on pcib22

pcib23: <ACPI PCI-PCI bridge> at device 23.4 on pci0

pci23: <ACPI PCI bus> on pcib23

pcib24: <ACPI PCI-PCI bridge> at device 23.5 on pci0

pci24: <ACPI PCI bus> on pcib24

pcib25: <ACPI PCI-PCI bridge> at device 23.6 on pci0

pci25: <ACPI PCI bus> on pcib25

pcib26: <ACPI PCI-PCI bridge> at device 23.7 on pci0

pci26: <ACPI PCI bus> on pcib26

pcib27: <ACPI PCI-PCI bridge> at device 24.0 on pci0

pci27: <ACPI PCI bus> on pcib27

pcib28: <ACPI PCI-PCI bridge> at device 24.1 on pci0

pci28: <ACPI PCI bus> on pcib28

pcib29: <ACPI PCI-PCI bridge> at device 24.2 on pci0

pci29: <ACPI PCI bus> on pcib29

pcib30: <ACPI PCI-PCI bridge> at device 24.3 on pci0

pci30: <ACPI PCI bus> on pcib30

pcib31: <ACPI PCI-PCI bridge> at device 24.4 on pci0

pci31: <ACPI PCI bus> on pcib31

pcib32: <ACPI PCI-PCI bridge> at device 24.5 on pci0

pci32: <ACPI PCI bus> on pcib32

pcib33: <ACPI PCI-PCI bridge> at device 24.6 on pci0

pci33: <ACPI PCI bus> on pcib33

pcib34: <ACPI PCI-PCI bridge> at device 24.7 on pci0

pci34: <ACPI PCI bus> on pcib34

acpi_acad0: <AC Adapter> on acpi0

atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0

atkbd0: <AT Keyboard> irq 1 on atkbdc0

kbd0 at atkbd0

atkbd0: [GIANT-LOCKED]

psm0: <PS/2 Mouse> irq 12 on atkbdc0

psm0: [GIANT-LOCKED]

psm0: model IntelliMouse, device ID 3

uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0

uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0

fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0

fd0: <1440-KB 3.5" drive> on fdc0 drive 0

orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xca000-0xcafff,0xdc000-0xdffff,0xe0000-0xe7fff on isa0

sc0: <System console> at flags 0x100 on isa0

sc0: VGA <16 virtual consoles, flags=0x300>

vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0

wbwd0: HEFRAS and EFER do not align: EFER 0x2e DevID 0x0f DevRev 0x00 CR26 0x4f

acpi_throttle0: <ACPI CPU Throttling> on cpu0

coretemp0: <CPU On-Die Thermal Sensors> on cpu0

coretemp0: Tj(target) value 0 does not seem right.

Timecounters tick every 10.000 msec

ipfw2 (+ipv6) initialized, divert enabled, nat enabled, default to accept, logging disabled

DUMMYNET 0xfffffe00051e1dc0 with IPv6 initialized (100409)

load_dn_sched dn_sched RR loaded

load_dn_sched dn_sched WF2Q+ loaded

load_dn_sched dn_sched FIFO loaded

load_dn_sched dn_sched PRIO loaded

load_dn_sched dn_sched QFQ loaded

da0 at mpt0 bus 0 scbus2 target 0 lun 0

da0: <VMware Virtual disk 1.0> Fixed Direct Access SCSI-2 device

da0: 320.000MB/s transfers (160.000MHz, offset 127, 16bit)

da0: Command Queueing enabled

da0: 8192MB (16777216 512 byte sectors: 255H 63S/T 1044C)

cd0 at ata1 bus 0 scbus1 target 0 lun 0

cd0: <NECVMWar VMware IDE CDR10 1.00> Removable CD-ROM SCSI-0 device

cd0: Serial Number 10000000000000000001

cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes)

cd0: Attempt to query device size failed: NOT READY, Medium not present

da1 at mps0 bus 0 scbus3 target 0 lun 0

da1: <ATA WDC WD30EFRX-68E 0A80> Fixed Direct Access SCSI-6 device

da1: Serial Number WD-WMC4N2318917

da1: 600.000MB/s transfers

da1: Command Queueing enabled

da1: 2861588MB (5860533168 512 byte sectors: 255H 63S/T 364801C)

da1: quirks=0x8<4K>

da2 at mps0 bus 0 scbus3 target 1 lun 0

da2: <ATA WDC WD30EFRX-68E 0A80> Fixed Direct Access SCSI-6 device

da2: Serial Number WD-WMC4N2322882

da2: 600.000MB/s transfers

da2: Command Queueing enabled

da2: 2861588MB (5860533168 512 byte sectors: 255H 63S/T 364801C)

da2: quirks=0x8<4K>

da3 at mps0 bus 0 scbus3 target 2 lun 0

da3: <ATA WDC WD30EFRX-68E 0A80> Fixed Direct Access SCSI-6 device

da3: Serial Number WD-WMC4N2321890

da3: 600.000MB/s transfers

da3: Command Queueing enabled

da3: 2861588MB (5860533168 512 byte sectors: 255H 63S/T 364801C)

da3: quirks=0x8<4K>

da4 at mps0 bus 0 scbus3 target 3 lun 0

da4: <ATA WDC WD30EFRX-68E 0A80> Fixed Direct Access SCSI-6 device

da4: Serial Number WD-WMC4N2334941

da4: 600.000MB/s transfers

da4: Command Queueing enabled

da4: 2861588MB (5860533168 512 byte sectors: 255H 63S/T 364801C)

da4: quirks=0x8<4K>

Timecounter "TSC-low" frequency 1399594000 Hz quality 1000

Trying to mount root from ufs:/dev/ufs/FreeNASs2a [ro]...

GEOM_RAID5: Module loaded, version 1.1.20130907.44 (rev 5c6d2a159411)

GEOM_ELI: Device da1p1.eli created.

GEOM_ELI: Encryption: AES-XTS 256

GEOM_ELI: Crypto: hardware

GEOM_ELI: Device da2p1.eli created.

GEOM_ELI: Encryption: AES-XTS 256

GEOM_ELI: Crypto: hardware

GEOM_ELI: Device da3p1.eli created.

GEOM_ELI: Encryption: AES-XTS 256

GEOM_ELI: Crypto: hardware

GEOM_ELI: Device da4p1.eli created.

GEOM_ELI: Encryption: AES-XTS 256

GEOM_ELI: Crypto: hardware

ZFS filesystem version: 5

ZFS storage pool version: features support (5000)

VMware memory control driver initialized

As for zpool import, this returns just nothing.


I do have a backup of almost all of the data on my old NAS. But before I start fresh ( with new ECC DRAM hardware ), I would really like to know what went wrong.

 

[dlavigne]

Post the output of zpool history.

 

[Jents]

[jents@jentsfreenas] /% zpool history
no pools available


@danb35: I do not think I selected encryption when I made the pool. And when I exported the pool, I am 100% certain that I did NOT check mark disks as new

 

[Jents]

eh...Help !

 

[cyberjock]

Sorry, I watched until you started talking about your choice of hardware. non-ECC RAM, FreeNAS in a VM, etc. You shouldn't be surprised with the outcome and I don't think you have a chance in hell of seeing your data again.

We keep telling people not to use non-ECC RAM because it can single handedly destroy a pool if things go wrong.

We keep telling people not to use FreeNAS in a VM because it can single handedly destroy a pool if things go wrong.

We keep telling people not to use desktop hardware because it can cause problems that can result in losing the pool.

You did all three. What *should* we do when you've ignored almost every major "do not do this" rule we have for making sure your data is safe?

As we tell people in the "do not virtualize" thread, don't expect us to work hard on recovering your data. We are only interested in spending time trying to help people that help themselves first. Your choices sound like you ignored all of our major warnings...

I'd strongly recommend you go to backups and use those to restore your data to a *new* system that has *proper* hardware and *isn't* virtualized. It sounds like you are doing this, so good on you from learning from your mistake and for making backups. :D

The experienced users have zero sympathy for people that don't listen to the warnings in the first place. Sorry. Most of the time we don't know what went wrong, but we see patterns of what does and doesn't work and that's where we give most of our advice.

 

[Jents]

I saw the ECC-NON-ECC thread only after I bought the hardware. I am willing to spend the money on an ECC enabled system, but this does not look like it was caused by bad memory.
I ran memtest86 for days after I lost my pool. It ran totally fine.

Also: just before I detached the pool I rebooted my system. After the reboot, the pool was still there.
I admit it is possible some random neutron from outer space just managed to flip a bit that brought my pool down in the five minutes between me booting up my server and detaching the pool
But you will have to agree, that is extremely unlikely.

 

[cyberjock]

Oh yeah, statistically that's extremely unlikely. The problem is that the other "likely" causes are basically not always understood and not easy to identify after the fact. We just know from watching dozens of people lose their data from given scenarios not to do "those scenarios".

Sorry if I sounded harsh above. I wasn't trying to be, but apparently I shouldn't have been posting at 5AM.