Freenas Crashed- Pool Gone? Help?

[topsub]

I have a 30TB Freenas System. 10x 3TB hard drives. All day i was streaming from it. I went away from the computer came back later and my whole freenas system was down. I restarted the system and this is when trouble happen.

After it booted we get this.


1 Drive completely failed so i replace it and 2 other seem to be on the brink of gone.


Now if i run 'zpool status' i get 'no pools available'. I have been googling trying to find some answers on how to recover my Raid-Z1.

I am kinda lost now as i can't find any other suggestions of what to do next. Every command / suggestion i see it says run some command about your pool but i always get 'no pools available'


Thanks for any help!

 

[SweetAndLow]

When you replaced the first drive did you let it resilver? It looks like you have 2 other drives dying or are dead. RaidZ only supports a single drive failing and you had multiple fail. I think you might be out of luck and your pool is dead.

 

[topsub]

When i rebooted i noticed a drive failed so i put a new one in there but freenas never came up correctly ( as it said .'error getting space' aka first pic)

I wasn't sure if those errors are drives are DYING or if they are DEAD.

 

[cyberjock]

What is the output of "zpool import"? Please post it in pastebin as the forums will mangle the output.

 

[topsub]

What is the output of "zpool import"? Please post it in pastebin as the forums will mangle the output.

Here is my output:
http://pastebin.com/vtRB5ca2

 

[cyberjock]

Yep. Game over. Too many disks are "UNAVAIL".

You also didn't follow the manual for disk replacement or the 3rd UNAVAIL wouldn't exist.

In any case, you are missing 2 disks in a RAIDZ1. You can't mount the pool unless you can get one of your broken disks to work.

 

[topsub]

You also didn't follow the manual for disk replacement or the 3rd UNAVAIL wouldn't exist.

hrmm i done the same thing i have done when replace other disk, pull it, put new one in, then it would resilver it automatically.

did i lost multiply drives at once which is why it failed maybe?

EDIT: Also note when freenas frooze i heard the 1 failed disk and pulled it, Freenas wouldn't boot up with only 9 drives, so thats why i wonder if more then one failed at a time.

 

[cyberjock]

I can't tell you when disks failed, just that they are currently failed. You'd have to do your own research and figure out what happened and when for yourself.

As for the disk replacement, there is a step at the end of the disk replacement that you didn't follow. If you did then the extra UNAVAIL entry would be gone. So I know for 100% certainty you didn't follow the entire instructions for disk replacement. Happens to many people which is why I know better. ;)

 

[joeschmuck]

When you repair your system and recreate your new pool, I highly recommend that you do a RAIDZ2 setup and also setup your email and a daily SMART short test and weekly SMART long test for all your drives. If you had been getting emails then you would have known when a drive was failing, typically before it's fatal, and that would have given you enough time to replace it properly.

As Cyberjock stated, your pool is gone unless you can get one of the first other failed drives to work. You need 9 of the 10 drives.