all space gone in ZFS

[Spearfoot]

yes. did not want to show the directory names, that's all. du -sh /mnt/RAIDz2/Share/* was run.

Yes... very odd.

But you have a problem in that you're somehow using /mnt/RAIDz2/Share as a share, even though it isn't a ZFS dataset. If you're running snapshots on it, that might explain where the 'missing' space is, but it doesn't show up when you run zfs list -o space because, again, it's not a dataset.

How to make it into a dataset? Hmmm... Perhaps something like this:

• Back up the contents of /mnt/RAIDz2/Share
• Delete /mnt/RAIDz2/Share
• Re-create /mnt/RAIDz2/Share as a dataset
  
• Restore the contents from backup
  
• Re-create the appropriate Windows (SMB) share

...or something like that. I don't know of any way to add an existing directory as a dataset; perhaps someone more knowledgable can enlighten us.

 

[asw2012]

This would be easier to explore if you install WinSCP or a similar tool.

I DL'd and installed this, I am tinkering around through FTP in the root to see if I can find the item(s) taking up the space.

 

[Spearfoot]

I DL'd and installed this, I am tinkering around through FTP in the root to see if I can find the item(s) taking up the space.

Outstanding!
But you still need to address your non-standard share setup, or this problem will keep recurring.

 

[asw2012]

this only finds 9.25TB, as the other inquiries also found. I just don't understand - where's it hiding?

I also calculated all of the other directories above the /RAIDz2 and only found 2,5GB or so.

 

[SweetAndLow]

Yes... very odd.

But you have a problem in that you're somehow using /mnt/RAIDz2/Share as a share, even though it isn't a ZFS dataset. If you're running snapshots on it, that might explain where the 'missing' space is, but it doesn't show up when you run zfs list -o space because, again, it's not a dataset.

How to make it into a dataset? Hmmm... Perhaps something like this:

• Back up the contents of /mnt/RAIDz2/Share
• Delete /mnt/RAIDz2/Share
• Re-create /mnt/RAIDz2/Share as a dataset
  
• Restore the contents from backup
  
• Re-create the appropriate Windows (SMB) share

...or something like that. I don't know of any way to add an existing directory as a dataset; perhaps someone more knowledgable can enlighten us.

There is zero reason to make it a dataset, what are you thinking here?

Datasets and shares have zero relationship and using a dataset over a directory makes zero difference.

Sent from my Nexus 5X using Tapatalk

 

[danb35]

But you have a problem in that you're somehow using /mnt/RAIDz2/Share as a share,

The "somehow" is easy enough; FreeNAS doesn't enforce any requirement that shared directories be datasets. If you use the wizard to create a share, it will be a dataset, and that's the general practice we recommend, but there's no baked-in requirement to do so.

I don't know of any way to add an existing directory as a dataset

To my knowledge, there's no way to convert a directory to a dataset. It used to be possible to create a dataset with the same name as an existing directory (with very confusing results, as the data in the directory will be inaccessible and invisible as long as the dataset is mounted), but FreeNAS has added "seat belts" to prevent this now.

But you still need to address your non-standard share setup, or this problem will keep recurring.

I don't see that the share setup has anything to do with this. It makes the problem a little harder to track down, as none of the ZFS tools really know or care about directories, but I don't think it's at all related to the problem itself.

 

[diedrichg]

I do have snapshots configured.



View attachment 19171
I see a max of 9.2TB +/- a few GB



After this ordeal, I am going to research and implement this.



It's set to only backup the windows hard drive.

They DO have snapshots enabled. And if the massive media files are being added then those are being snapped and adding to the data loss.

 

[Spearfoot]

There is zero reason to make it a dataset, what are you thinking here?

I'm thinking that it is "Best Practice" to base shares on datasets, as @danb35 pointed out above. I have always taken this approach.

Datasets and shares have zero relationship and using a dataset over a directory makes zero difference.

Zero relationship? Perhaps. But note that the OP filled up his disk and had trouble diagnosing the problem -- because he didn't use a dataset as the basis for the share where all of the space was used.

 

[danb35]

They DO have snapshots enabled. And if the massive media files are being added then those are being snapped and adding to the data loss.

But du doesn't account for snapshots, does it?

 

[SweetAndLow]

I'm thinking that it is "Best Practice" to base shares on datasets, as @danb35 pointed out above. I have always taken this approach.
Zero relationship? Perhaps. But note that the OP filled up his disk and had trouble diagnosing the problem -- because he didn't use a dataset as the basis for the share where all of the space was used.

It's not really needed to use datasets and that would not have made this problem any different. It's exactly the same problem.

Snapshots also don't impact anything when adding data. Only removing or modifying stuff will cause snapshots to change.

Sent from my Nexus 5X using Tapatalk

 

[Spearfoot]

But du doesn't account for snapshots, does it?

No, it doesn't. Or at least not on my systems....

Which makes it all the more puzzling that the OP found 15TB used in Share when he ran du -sh /mnt/RAIDz2/*, but only ~9TB when he ran du -sh /mnt/RAIDz2/Share/*. Where is the missing 6TB? Probably eaten up by snapshots... He's running Acronis backups to /mnt/RAIDz2/Share/Backups, and I know from my own experience that snapshotting backups very quickly consumes a lot of space.

 

[asw2012]

No, it doesn't. Or at least not on my systems....

Which makes it all the more puzzling that the OP found 15TB used in Share when he ran du -sh /mnt/RAIDz2/*, but only ~9TB when he ran du -sh /mnt/RAIDz2/Share/*. Where is the missing 6TB? Probably eaten up by snapshots... He's running Acronis backups to /mnt/RAIDz2/Share/Backups, and I know from my own experience that snapshotting backups very quickly consumes a lot of space.

I did run into this problem a few years ago where the snapshot cycle was way out of line. I've fixed this since then with a frequency of 1 day, being deleted after 9.

Here's a list of my current snapshots, am I missing something obvious here?

 

[diedrichg]

Here's a list of my current snapshots, am I missing something obvious here?

Could you show your snapshot tasks, please.

I'd then do a full external backup of your important information and then start deleting all the snapshots that are directly associated with the media files. As long as your critical data is not included in the media files snapshots then there would be little worry to delete every media file location snapshot.

 

[asw2012]

Could you show your snapshot tasks, please.

I'd then do a full external backup of your important information and then start deleting all the snapshots that are directly associated with the media files. As long as your critical data is not included in the media files snapshots then there would be little worry to delete every media file location snapshot.

is this what you are asking for?

 

[diedrichg]

This screen: Storage>Periodic Snapshot Tasks

 

[asw2012]

 

[diedrichg]

Hmm. A lifetime of 9 days. Has it been >9 days since you started noticing the storage space go missing?

 

[diedrichg]

With such a short lifespan, which indicates to me that the data isn't that critical, that you could do an external backup of the critical data and then destroy all the snapshots in your system.

Disclaimer: As long as you know the critical data is correct, current and complete.

 

[asw2012]

With such a short lifespan, which indicates to me that the data isn't that critical, that you could do an external backup of the critical data and then destroy all the snapshots in your system.

I'll backup the critical data. But when you put it that way, it seems like all of the data will get destroyed. But I think you're saying just be overly protective of the data, JUST INCASE and all of my data won't get erased / destroyed when I delete all of the snapshots.....

 

[diedrichg]

tl;dr
If you destroy a snapshot, anything that was deleted or overwritten in the past will not be recoverable. If you destroy all your snapshots, you will be stuck with your data as it currently stands - no recovering old files.

I'll backup the critical data. But when you put it that way, it seems like all of the data will get destroyed. But I think you're saying just be overly protective of the data, JUST INCASE and all of my data won't get erased / destroyed when I delete all of the snapshots.....

I mean that, I've personally had situations where I thought my data was complete but Resilio Sync actually got rid of some files and when I went back to look for them (obviously not there) I resorted to the snapshots only to find that I didn't have snapshots going back far enough to recover the data. I then had to resort to my external backup to hope that the data was there.