SOLVED ZFS pool full after installing Zoneminder, and trying to recover from it.

[Parish Mills]

My storage on Truenas Scale is at 100% using 14.4TB (avail space now at 91.21 MiB), and it happened after I installed the Zoneminder security camera application. Before the Zoneminder install, my storage was at 70% full. My system uses 1 ZFS pool, and I had setup a shared folder (security-cameras) under my Media dataset to store the video.

ZfsPool1
ix-applications
Media
tv-shows
movies
security-cameras
proxmox
scripts

NOTE: I do realize now I probably should have created a new dataset for the security cameras, rather than creating a new folder under my Media dataset.

So far, I have done the following trying to remedy this:

1. uninstalled the Zoneminder application
2. i went to the Truenas shell, and am attempting to delete the files and directories related to Zoneminder. I've been monitoring the deletion of the files and it's taking forever. It's been running now for nearly 14 hours, and so far it hasn't even finished up deleting 2 days worth of images from Zoneminder. It still has 3 days of captured images to delete.

Even with the removal of nearly 2 days worth of Zoneminder images, my storage still shows to be 100% full. Is there an easier way to get past this? Can I restore an older snapshot (pre-Zoneminder) of my Media dataset?

 

[Parish Mills]

Correction on the pool and data sets:

ZfsPool1
ix-applications
Media
tv-shows
movies
security-cameras
proxmox
scripts

 

[Parish Mills]

Sorry, let me try this one last time...

ZfsPool1

• ix-applications
• Media
  • tv-shows
  • movies
  • security-cameras
• proxmox
• scripts

 

[Davvo]

So... I assume media is your dataset, and you are sharing it with SMB or something. In that dataset you have your three folders, right?

Are you using console? Which command to delete?

 

[Parish Mills]

Hi Davvo. Yes, Media is my dataset and I created the "security-cameras" folder within it and set it up as a samba share.

In Scale, I'm using the system shell. I tried a few different ways to "delete" the files and directories, and I believe the last try was with rm -r -v events (which is a folder inside of security-cameras). The system is still running, deleting 1 .jpg file after another.

The first events took place on 12/23/22, so I was wondering if I couldn't just rollback to a Media dataset snapshot from the 22nd.

 

[Davvo]

The first events took place on 12/23/22, so I was wondering if I couldn't just rollback to a Media dataset snapshot from the 22nd.

Yes, this should be your only alternative beside nuking the dataset. Beware that it's an unrecoverable, destructive action.

 

[Parish Mills]

Thank you Davvo. I'm rolling the snapshot back now and waiting to see the end result. It's taking some time, but I'm seeing my available space move in the right direction, and used space is going down. Will report back with the final outcome.

 

[Davvo]

In the future make sure to not go over 80-85% of used space since from there performance drops fast.

 

[Parish Mills]

After performing the snapshot rollback, I can say it was a success. Thank you Davvo!