Unable to mount ZFS pool - no space left on pool.

[Sasa Petrovic]

Hello guys,
I have one problem and still I don't know the way how to solve it.
Few days ago my flash with FreeNAS died on my NAS. After reinstall on new flash drive I am not able to import ZFS pool. The problem is with no free space left on pool and it can not create mount points.
How to solve this issue and import volume back to the system? How to delete some files from that pool? Do you have any suggestions?
So, auto import and import from CLI isn't working - could not create mount points...
Thanks in advance.

 

[m0nkey_]

Please provide details as per the forum rules.

Do you remember what level of utilization you had prior it to failing?

 

[Sasa Petrovic]

Sorry for not providing details per forum rules.
Level of utilization was 96-97% I think...
But when I checked in CLI free space was 0 bytes :(

My configuration is: HP Storage works server
16GB of ECC RAM memory, 12x2TB HDD, disks are configured like JBOD than ZFS data pool.
Build FreeNAS-9.10-STABLE-201606270534 (dd17351)
Platform Intel(R) Xeon(R) CPU E5-2407 v2 @ 2.40GHz

 

[Sasa Petrovic]

Anybody? :(

 

[jgreco]

You'll probably need to manually import the pool while in single-user mode, find some large-ish non-critical file to get rid of, and then truncate it, hopefully giving you a tiny bit of space to work with.

You'll then want to go and delete enough stuff that you get down below 90% utilization, preferably even below 80%.

https://www.illumos.org/issues/412

 

[Sasa Petrovic]

Ok...After I tried to import pool from single user mode - I am still not able to mount pool. I am getting the same error message: Failed to create mountpoint. I am attaching picture to see:
Hmm...but now...it seems that I have 491GB free so...I am not full till the end :)
I have tried a lot of things but same thing - can't mount it. Blah...
I mean while I started SCRUB and I have to wait 6h more to finish...We will see after that if there is some change or progress. If u have any other suggestion, please let me know.
Thanks in advance.

 

[jgreco]

You appear to be missing the alternate mountpoint.

Try:

# zpool import -o cachefile=none -R /mnt Bunar

 

[Sasa Petrovic]

Still no luck...I will try now to install PCBSD and to import it there...
I am still getting message = failed to create mountpoint.

 

[Sasa Petrovic]

At the end I managed to mount it on PCBSD but...what was the solution:
At first moment I had same problem - can't create mountpoint. I saw that Bunar is mounted on system and pool /Apple in Bunar couldn't be mounted (same error).
I did a dismount of /Bunar and mount /Bunar/Apple and wholaaa :) It works, data can be accessible.
Now, I did some copying of files witch I needed urgently and now I can't delete anything from /Bunar/Apple .
When I try "rm -r" it's giving me: No space left on device.
I tried also with "echo" but still no luck. File becomes 1byte but still can't delete anything. I did echo for sure something about 15GB.
Also tried with "truncate -s 0" - Same story.
In meanwhile I truncate lot more ...most likely something about 70GB more.
Any suggestion now how to delete something from this pool?
I also google it a lot but no luck with anything.

 

[Sasa Petrovic]

Yup...truncate worked! I just needed to truncate more and more :) And at the end it worked to delete normally!
Now...Back to freenas install...
THanks to all!