Hello everyone!
Having an issue that I have worked all day on with no luck. Hoping I didn't screw up and lose my data.
Hardware
Dell Server
freenas 11.2 (recently upgraded from version 9?)
2x 6 core xeon
72Gb ddr3 ecc ram
11x 3tb disks
Freenas installed to a 60Gb ssd
Pool has been fine for about a year but had minimal use. Last week I decided to install the plex plugin and transfer about 8tb of data to my Raidz3 array. I setup the plugin, transferred the data then went to bed. I woke up and went to login to freenas and the ui wouldnt load. Went downstairs and the server was locked up, As a last resort I pressed the power button on the server and came back an hour later. Nothing. I was forced to do a hard restart
Upon booting up I noticed my pool of 11 disks was now showing as locked and only showed 7 disks. The disk still showed in the system and the drives are fine however they showed as unused, while the others showed as members of my MainStorage pool. The pools tab showed the pool was locked and I got an error in freenas about insufficient replicas.
I looked around on the forum for a while and found the following post which seemed promising but nothing he did worked for me.
www.ixsystems.com
The next thing I noticed was the log showed the following error multiple times per second and was not stopping. Each time with a different vdev_guid
ZFS: vdev state changed, pool_guid=4753035133703495120 vdev_guid=15343288276930064957
I went back to the pools tab and it would not load. I then power cycled the system a few times via the UI and it didnt change, A minute or two after startup the spam would start and even before then the pools tab would not reload. Thankfully I had downloaded the keys for the pool earlier. Then I saved the config.
I then reinstalled freenas and went with 11.3, hoping it had solved a bug or something that had caused this.
I can import the config and it gets me back to the point where the pool shows up but with only the 7/11 drives. It shows it's in a locked state.
I then exported the pool and tried to import it as an existing pool with the recovery key but that doesnt work either. if I select all 11 disks it shows an error that it couldnt mount the ones that got kicked from the pool. If I only select the 7 it drops me to the next page to select a pool and there is nothing in the drop down.
I'm a bit of a noob to freenas and am wondering what is going on. Did I make the whole thing worse by reinstalling freenas, did I actually have 5 drives take a crap at once, or is there a simple way to import those disks back into the pool and be good to go?
Having an issue that I have worked all day on with no luck. Hoping I didn't screw up and lose my data.
Hardware
Dell Server
freenas 11.2 (recently upgraded from version 9?)
2x 6 core xeon
72Gb ddr3 ecc ram
11x 3tb disks
Freenas installed to a 60Gb ssd
Pool has been fine for about a year but had minimal use. Last week I decided to install the plex plugin and transfer about 8tb of data to my Raidz3 array. I setup the plugin, transferred the data then went to bed. I woke up and went to login to freenas and the ui wouldnt load. Went downstairs and the server was locked up, As a last resort I pressed the power button on the server and came back an hour later. Nothing. I was forced to do a hard restart
Upon booting up I noticed my pool of 11 disks was now showing as locked and only showed 7 disks. The disk still showed in the system and the drives are fine however they showed as unused, while the others showed as members of my MainStorage pool. The pools tab showed the pool was locked and I got an error in freenas about insufficient replicas.
I looked around on the forum for a while and found the following post which seemed promising but nothing he did worked for me.
SOLVED - Pool Locked / Pool state is UNKNOWN after power failure
Currently using 11.2-U5. I am having issues with one of my pools named "Volume2". There was a power failure and the system had shut down inadvertently. Upon restart, the first pool (Volume1) works fine but only one disk on the on the second pool (Volume2) is detecting on the Dashboard which is...
www.ixsystems.com
The next thing I noticed was the log showed the following error multiple times per second and was not stopping. Each time with a different vdev_guid
ZFS: vdev state changed, pool_guid=4753035133703495120 vdev_guid=15343288276930064957
I went back to the pools tab and it would not load. I then power cycled the system a few times via the UI and it didnt change, A minute or two after startup the spam would start and even before then the pools tab would not reload. Thankfully I had downloaded the keys for the pool earlier. Then I saved the config.
I then reinstalled freenas and went with 11.3, hoping it had solved a bug or something that had caused this.
I can import the config and it gets me back to the point where the pool shows up but with only the 7/11 drives. It shows it's in a locked state.
I then exported the pool and tried to import it as an existing pool with the recovery key but that doesnt work either. if I select all 11 disks it shows an error that it couldnt mount the ones that got kicked from the pool. If I only select the 7 it drops me to the next page to select a pool and there is nothing in the drop down.
I'm a bit of a noob to freenas and am wondering what is going on. Did I make the whole thing worse by reinstalling freenas, did I actually have 5 drives take a crap at once, or is there a simple way to import those disks back into the pool and be good to go?
