Powerloss cannot access share... please help!

[FreeNoobs]

My freeNAS 9.10.1 server just had a powerloss (my whole house lost electricity).
Now there is an alert when i log into the web gui saying: CRITICAL: Oct. 4, 2016, 2:50 p.m. - The volume mediaserver (ZFS) state is UNKNOWN:


My setup was Intel i7 2600k, 8GB ram, 8x3 TB sata hdds connected to a 8port raidcard. I was running RAID Z1 (setup done through the wizard).

When clicking on "Storage" and Volumes it says:
Name: Mediaserver
Used: 0 (Error)
Available: Error getting available space
Compression: -
Compression ratio: -
Status: UNKNOWN

When i select view disk i can see all my 8 HDDS (see screenshot)

The folder i used to dump files is called Dump and can be seen on the screenshot, i can access it through my windows PC but it is empty.....

Please help.. There is like 7 years worth of family pictures, downloads, work related files, etc. Am i screwed?

 

[danb35]

From the command line, what's the output of zpool status in code tags? And what's the "raid card" and motherboard you're using?

For future reference, it appears there are a number of ways your setup could be improved for better data security:

• Backup backup backup. If the data is important, it should be backed up, and it really should be backed up offsite.
• More RAM. 8 GB is the minimum, but with 20+ TB of capacity you should consider 16 GB a minimum.
• ECC RAM. Your CPU doesn't support it, and odds are your motherboard doesn't either.
• RAIDZ2. RAIDZ1 isn't really recommended for drives over about 1 TB.
• A UPS that your server can talk to, to shut itself down cleanly when power is lost.

The second and third points are unlikely to be directly relevant to your immediate situation, but nonetheless relevant to your data security. The first and last points are obviously directly relevant; the fourth is uncertain.

 

[Ericloewe]

From the command line, what's the output of zpool status in code tags? And what's the "raid card" and motherboard you're using?

And zpool import while we're at it.

 

[FreeNoobs]

Than you guys for looking into this. The raidcard is: Areca ARC-1220 PCIe x8 3Gb/s 8-Port SATA RAID Controller
Motherboard is: https://www.asus.com/no/Motherboards/P8Z68V_PRO/

Code:

[root@freenas ~]# zpool status												
  pool: freenas-boot															
state: ONLINE																
  scan: scrub repaired 0 in 0h0m with 0 errors on Mon Sep 12 03:45:18 2016	
config:																		
																				
		NAME		STATE	 READ WRITE CKSUM								
		freenas-boot  ONLINE	   0	 0	 0								
		  da8p2	 ONLINE	   0	 0	 0								
																				
errors: No known data errors													
[root@freenas ~]#

[root@freenas ~]# zpool import												
   pool: mediaserver															
	 id: 9834568278484911857													
  state: FAULTED																
status: The pool metadata is corrupted.										
action: The pool cannot be imported due to damaged devices or data.			
		The pool may be active on another system, but can be imported using	
		the '-f' flag.														
   see: http://illumos.org/msg/ZFS-8000-72									
config:																		
																				
		mediaserver									 FAULTED  corrupted data
		  raidz1-0									  ONLINE				
			gptid/1cb9f768-6008-11e6-986a-14dae9139ea1  ONLINE				
			gptid/1d5ea9af-6008-11e6-986a-14dae9139ea1  ONLINE				
			gptid/1dfd07eb-6008-11e6-986a-14dae9139ea1  ONLINE				
			gptid/1eeff3b7-6008-11e6-986a-14dae9139ea1  ONLINE				
		  raidz1-1									  ONLINE				
			gptid/1f96f365-6008-11e6-986a-14dae9139ea1  ONLINE				
			gptid/202b264e-6008-11e6-986a-14dae9139ea1  ONLINE				
			gptid/20d58049-6008-11e6-986a-14dae9139ea1  ONLINE				
			gptid/21652650-6008-11e6-986a-14dae9139ea1  ONLINE				
[root@freenas ~]#

Please let me know if you need more info. Thanks for the advice btw, Iv'e never come to this before. Used to have my files on external USB SATA drives.. One started making funny noises, so I installed FreeNAS on an old computer and figured I would learn how to replace a faulty disk when the time came in FreeNAS. My understanding was that one disk could crash and files were still intact with my current setup?

 

[Ericloewe]

Areca ARC-1220 PCIe x8 3Gb/s 8-Port SATA RAID Controller

Not good at all. Might be the cause of your problem and others you don't know about.

Now, it may be possible to rollback the transactions that were happening when the power was cut, which might fix the corruption.

Try zpool import mediaserver -n -F

 

[danb35]

Try zpool import mediaserver -n -F

To clarify, this command will test whether the pool can be imported this way; it won't actually import it. It's "safe", in that it won't change anything on the pool.

Areca ARC-1220 PCIe x8 3Gb/s 8-Port SATA RAID Controller

Agree with @Ericloewe that this isn't a good choice for FreeNAS. You should seriously consider replacing it with a proper SAS HBA like the LSI 9211-8i or one of its many variants (Dell H200, IBM M1015, etc.).

My understanding was that one disk could crash and files was still intact with my current setup?

Yes, but... The problem with RAID5 (and RAIDZ1) is that once that one disk crashes, you have no redundancy. If you then have a read error on one of the other disks, you have no way to repair that. If that read error is in the middle of critical metadata, you could lose your whole pool. The disk specs would suggest that an unrecoverable read error would be a near-certainty, but it appears the disks are considerably better than spec, because this doesn't actually seem to happen very often.

 

[FreeNoobs]

Thanks guys, i will look into the RAID card mentioned, I know the hdds is crap. They are ripped from the same type of external USB3 drives i had. Got a newer motherboard and 32GB memory to spare. Thanks for imput on raid5/raidz1. Will choose another solution in the future.. The command did not work btw.. I'm to scared to ruin this, so I'll just paste the reply here.

Code:

[root@freenas ~]# zpool import mediaserver -n -F								
too many arguments															
usage:																		
		import [-d dir] [-D]													
		import [-d dir | -c cachefile] [-F [-n]] <pool | id>					
		import [-o mntopts] [-o property=value] ...							
			[-d dir | -c cachefile] [-D] [-f] [-m] [-N] [-R root] [-F [-n]] -a
		import [-o mntopts] [-o property=value] ...							
			[-d dir | -c cachefile] [-D] [-f] [-m] [-N] [-R root] [-F [-n]]	
			<pool | id> [newpool]

 

[danb35]

For the import command, I've used it with the arguments in a different order: zpool import -F -n mediaserver. Try it that way and see if it works.

 

[Ericloewe]

For the import command, I've used it with the arguments in a different order: zpool import -F -n mediaserver. Try it that way and see if it works.

D'oh, got them the other way around.

 

[FreeNoobs]

Code:

root@freenas ~]# zpool import -F -n mediaserver								
Would be able to return mediaserver to its state as of Mon Oct  3 15:00:01 2016.
Would discard approximately 45 seconds of transactions.

That's a good thing right ? :)

 

[danb35]

That sounds like a very good thing. Try zpool import -F mediaserver.

 

[FreeNoobs]

Code:

[root@freenas ~]# zpool import -F mediaserver
[root@freenas ~]#

No messages, but i heard the server hum in excitement :) Any other steps?

 

[danb35]

That's good; ordinarily a pool import will not give any other output. Do zpool status again, and use code tags (the "code" button right above the text box) for the output--they make it much easier to read.

 

[FreeNoobs]

Code:

[root@freenas ~]# zpool status																									 
  pool: freenas-boot																												
state: ONLINE																													 
  scan: scrub repaired 0 in 0h0m with 0 errors on Mon Sep 12 03:45:18 2016														 
config:																															
																																	
		NAME		STATE	 READ WRITE CKSUM																					 
		freenas-boot  ONLINE	   0	 0	 0																					
		  da8p2	 ONLINE	   0	 0	 0																					 
																																	
errors: No known data errors																										
																																	
  pool: mediaserver																												
state: ONLINE																													 
  scan: scrub repaired 0 in 1h26m with 0 errors on Sun Sep 18 01:26:11 2016														
config:																															
																																	
		NAME											STATE	 READ WRITE CKSUM												 
		mediaserver									 ONLINE	   0	 0	 0												 
		  raidz1-0									  ONLINE	   0	 0	 0												 
			gptid/1cb9f768-6008-11e6-986a-14dae9139ea1  ONLINE	   0	 0	 0												 
			gptid/1d5ea9af-6008-11e6-986a-14dae9139ea1  ONLINE	   0	 0	 0												 
			gptid/1dfd07eb-6008-11e6-986a-14dae9139ea1  ONLINE	   0	 0	 0												 
			gptid/1eeff3b7-6008-11e6-986a-14dae9139ea1  ONLINE	   0	 0	 0												 
		  raidz1-1									  ONLINE	   0	 0	 0												 
			gptid/1f96f365-6008-11e6-986a-14dae9139ea1  ONLINE	   0	 0	 0												 
			gptid/202b264e-6008-11e6-986a-14dae9139ea1  ONLINE	   0	 0	 0												 
			gptid/20d58049-6008-11e6-986a-14dae9139ea1  ONLINE	   0	 0	 0												 
			gptid/21652650-6008-11e6-986a-14dae9139ea1  ONLINE	   0	 0	 0												 
																																	
errors: No known data errors																										
[root@freenas ~]#

 

[danb35]

Great. Your pool is online and showing no errors. However, the GUI won't see it since you imported it "behind the GUI's back". You're going to want to scrub the pool, and since your last scrub only took an hour and a half, I'd do it first: zpool scrub mediaserver. You can monitor its progress with zpool status. Once the scrub is finished, export the pool: zpool export mediaserver. Then, using the web GUI, go to Storage -> Import Volume and import the pool.

And then get a UPS pronto. @Ericloewe isn't exactly local to you, but probably has a better idea of what's available there than I do. Perhaps @Dice can help out as well.

 

[FreeNoobs]

Guys, thanks alot for your help :) Dont know where id even begin to gather all my familys data again.. Im going to copy some to my gaming rig and some to my work computer so i have some redundancy if this should happen before i can get sorted with some proper hardware. Regarding the UPS, can u actually configure it to ask the freenas to shutdown? Would love a pointer on where such a magical item can be obtained :)

 

[Ericloewe]

can u actually configure it to ask the freenas to shutdown?

Of course. Information is in the manual.

And then get a UPS pronto. @Ericloewe isn't exactly local to you, but probably has a better idea of what's available there than I do.

APC's stuff is very similar, with slightly different output power/capacity combinations. Back-UPS Pro is okay (there's even a review on Tom's, I expect other models to be similar), Smart-UPS is better.

Cyberpower is harder to find in Europe, and their models seem a bit shoddier than APC's equivalents, judging by Tom's reviews of UPSes.

 

[FreeNoobs]

daamn... still not able to import volume. Did exactly what was written :\ See screenshots. The first one is when i skip decrypting second is when i choose to decrypt.. either way not finding any volumes.

 

[Ericloewe]

It's clearly not encrypted, because it was working just fine in the CLI.

Did you export the pool? If so, try rebooting the server and see what happens.

 

[FreeNoobs]

i did after checking that the scrub was done, trying to reboot now