ZF Pool "Imports" but no Volumes

[Beoren]

Hi Guys,

Long story short is that a power outage killed my USB that I did not clone or backup the config file - I now know to do this!

Still no biggy i'll create a new USB of the same build and import away right!?

well,... Good news is that it does see my old pool "Harley" Bad news is when I go to import it nothing happens, it reboots as if it succeed but I still see no volume in the GUI when it comes back up. I have done this in Shell and the GUI way!

Smart shows all 4 Drives as fine and I ran Mem tests on my 3 4GB sticks. One stick did have some issues so now the rig is down to 8GB.

Any Idea why this happens? I would assume that the data is still sitting on the drives just out of reach! Goal is just to get the data back!

Thank you in advance!

Information:
Build FreeNAS-9.10.2-U5 (561f0d7a1)
Platform AMD Phenom(tm) II X6 1100T Processor
Memory 7904MB

zpool import

Code:

[root@freenas ~]# zpool import												 
   pool: Harley																 
	 id: 8513816246511454350													
  state: ONLINE																 
status: The pool was last accessed by another system.						 
action: The pool can be imported using its name or numeric identifier and	 
		the '-f' flag.														 
   see: http://illumos.org/msg/ZFS-8000-EY									 
config:																		
																				
		Harley										  ONLINE				 
		  raidz1-0									  ONLINE				 
			gptid/9830f349-5372-11e5-9f53-f46d044ea401  ONLINE				 
			gptid/99605f9e-5372-11e5-9f53-f46d044ea401  ONLINE				 
			gptid/99fd8f73-5372-11e5-9f53-f46d044ea401  ONLINE				 
			gptid/9a96a502-5372-11e5-9f53-f46d044ea401  ONLINE

zpool status

Code:

[root@freenas ~]# zpool status												 
  pool: freenas-boot															
state: ONLINE																 
  scan: none requested														 
config:																		 
																				
		NAME		STATE	 READ WRITE CKSUM								 
		freenas-boot  ONLINE	   0	 0	 0								
		  da0p2	 ONLINE	   0	 0	 0								 
																				
errors: No known data errors

Thanks again!

 

[SweetAndLow]

System reboots when you try to import the pool? Was this in the command line or GUI? Try zpool import -f Harley on the cli.

Sent from my Nexus 5X using Tapatalk

 

[Beoren]

Yup reboots, than about 5 seconds of lines flying by on the actual server screen that is to fast to get anything from than it reboots.

zpool import Harley always comes back with this:

Code:

zpool import  Harley										
cannot import 'Harley': pool may be in use from other system, it was last access
ed by freenas.local (hostid: 0x7e5004d7) on Mon Jun 19 23:06:15 2017			
use '-f' to import anyway

Which is referring to the stick that died as the other system.

And with the -f it does as stated above. Lines than reboot. When it comes back up I head to the GUI and nothing under volumes still blank.



I don't know how to see a log about if it failed before the reboot.

Thanks!

 

[danb35]

Try zpool import -F Harley on the cli.

The -F flag is destructive; only use it as a last resort.

 

[SweetAndLow]

The -F flag is destructive; only use it as a last resort.

Ahhh shoot I know, phone auto correct capitalized it on me! I typed a lowercase f. I'll edit the post.

Usually when you have a pool that causes a reboot that means the pool is corrupt, either bad memory or multiple disk failure and metadata corruption. One more question though, did you use dedupe?

We can also try importing the pool in read only mode. I think it's zpool import -f -o readonly=yes Harley. I need to double check that syntax.

Sent from my Nexus 5X using Tapatalk

 

[danb35]

I believe the syntax would be zpool import -f -o readonly=yes Harley.

Edit: If that doesn't work, you could also try zpool import -f -F -n Harley. Yes, I just warned above that the -F flag is destructive. But in combination with the -n flag, it will just test whether it could import the pool, it won't actually do it.

 

[Beoren]

first off, thanks for taking the time with me to try and solve this!

zpool import -f -o readonly=yes Harley Kicks back that the argument is on / off so I did
zpool import -f -o readonly=on Harley and no errors and no reboot.

I checked the Volumes and still nothing there but a zpool status -v shows:

Code:

[root@freenas ~]# zpool status -v																								   
  pool: Harley																													 
state: ONLINE																													 
status: One or more devices has experienced an error resulting in data															 
		corruption.  Applications may be affected.																				 
action: Restore the file in question if possible.  Otherwise restore the															
		entire pool from backup.																									
   see: http://illumos.org/msg/ZFS-8000-8A																						 
  scan: scrub repaired 152K in 20h56m with 1 errors on Sun Jun 18 17:56:28 2017													 
config:																															 
																																	
		NAME											STATE	 READ WRITE CKSUM												 
		Harley										  ONLINE	   0	 0	 0												 
		  raidz1-0									  ONLINE	   0	 0	 1												 
			gptid/9830f349-5372-11e5-9f53-f46d044ea401  ONLINE	   0	 0	 0												 
			gptid/99605f9e-5372-11e5-9f53-f46d044ea401  ONLINE	   0	 0	 0												 
			gptid/99fd8f73-5372-11e5-9f53-f46d044ea401  ONLINE	   0	 0	 0												 
			gptid/9a96a502-5372-11e5-9f53-f46d044ea401  ONLINE	   0	 0	 0												 
																																	
errors: Permanent errors have been detected in the following files:																 
																																	
		/Harley/jails/transmission_1/usr/ports/sysutils/parafly/Makefile															
																																	
  pool: freenas-boot																												
state: ONLINE																													 
  scan: none requested																											 
config:																															 
																																	
		NAME		STATE	 READ WRITE CKSUM																					 
		freenas-boot  ONLINE	   0	 0	 0																					
		  da0p2	 ONLINE	   0	 0	 0																					 
																																	
errors: No known data errors   

How can I repair that, or can I?

Cheers,

 

[SweetAndLow]

Your pool imported! back up that data fast before it's all gone. You will have to rebuild this pool most likely, this is what happens when you don't use ecc memory and have a bad stick. You can also try removing the file that it says is bad but i suspect there are going to be much more corrupt files.

 

[danb35]

You must have missed his post immediately above where the pool did import. Granted, it imported read-only, but it imported nonetheless. Now it's time to get important (or, ideally, all) data backed up so that he can destroy and rebuild the pool. Preferably on a system with ECC RAM.

Edit: I'm positive @SweetAndLow's post (just above mine) said something different when I replied to it, otherwise my reply doesn't make much sense.

 

[Beoren]

Yes it shows as imported as read only but my volumes list is still empty in the GUI. My share location still shows as down. Is another way I am to access the data in this degraded state?

 

[Beoren]

UPDATE-

Ok so using CLI I can get to the data but my CLI skills are pretty rusty.

I would assume plugging in a USB disk formatted NTFS to backup the important stuff using copy commands will work? / best way?

I cant thank you guys enough!

 

[SweetAndLow]

UPDATE-

Ok so using CLI I can get to the data but my CLI skills are pretty rusty.

I would assume plugging in a USB disk formatted NTFS to backup the important stuff using copy commands will work? / best way?

I cant thank you guys enough!

Freenas can't really write to ntfs file systems easily. Might be easier to do zfs on that drive or copy the data off using scp to a Windows computer with that drive attached.

Sent from my Nexus 5X using Tapatalk

 

[danb35]

As @SweetAndLow said, it'd be better to format that USB disk as ZFS (i.e., create a new pool on that disk), and then copy the data--a mass rsync would do the job if you had room for everything, or you could be more selective if necessary.

But we've seen a few cases here recently where pools refused to import, acting like yours, and it's a little disconcerting. Corrupted metadata preventing the pool from importing is one thing, but having it panic the system really shouldn't happen. Makes me wonder if there's a way for the devs to take a look at those disks to see further what's going on. @cyberjock?

 

[cyberjock]

Sorry I didn't respond sooner. I'm out on travel and such.

More than likely there is corruption, but the checksum is matching, so the "check" ZFS does shows no corruption. Generally at this stage it's all about copying off whatever data you can and not let that happen again.

THe proper zpool import command to use at the CLI is:

# zpool import -f -R /mnt <zpool name>

@Beoren Did you get your data off?

 

[Beoren]

I was able to get my data off so thank you all again for helping me out.

When rebuilding the box I have run into issues copying the data back to the new volume. When doing large transfers it would completely freeze. ( No access to the GUI and the transfer stops), I believe this is due to me having 16TB of space and only 8GB ram due to me removing the failed memory stick . I would have to force a reboot and start the process over again.

I Have bought more ram bringing me up to 16GB.

Cheers,

 

[SweetAndLow]

I was able to get my data off so thank you all again for helping me out.

When rebuilding the box I have run into issues copying the data back to the new volume. When doing large transfers it would completely freeze. ( No access to the GUI and the transfer stops), I believe this is due to me having 16TB of space and only 8GB ram due to me removing the failed memory stick . I would have to force a reboot and start the process over again.

I Have bought more ram bringing me up to 16GB.

Cheers,

I would think hardware failure before not enough RAM. 8GB is the minimum so I would think it should work, especially when writing data. Did you test your system?

Sent from my Nexus 5X using Tapatalk