SOLVED Am I shooting my self in the foot with this snapshotting scheme (head spinning)?

[guermantes]

After moving to 11.3 my legacy snapshotting scheme malfunctioned, so I cleaned house and am now setting up snapshots again. To decrease the amount of snapshots (some of my datasets change seldom) I was thinking of unticking "Allow taking of empty snapshots". But conceptually I can't wrap my head around the long-term consequences of this and fear I may be shooting myself in the foot.

The overall idea is to have daily snaps at 1AM that live for 1 month. And then monthly snaps on the 3rd every month at midnight that live for a 1year. The monthly runs at midnight in order to take precedence over the daily.

In this scenario, the first monthly snap would occur 3 April. But here is what I don't understand: if I modify files on 1 April, the daily snap for 2 April would make sure my old versions are saved in the snap for 1 April. But if then no changes are made during 2 April, then the monthly snap would not occur, would it? After that the daily snaps would continue to pick up changes, but come May, if there are no changes on 2 May then the monthly snapshot would not fire for May either. And this is when all the daily snaps from April begin to get killed off and suddenly I am not able to go back in time to April. And if this scenario repeats itself in June I won't be able to go back to May either.

a) Am I understanding this correctly?
Epilogue: yes. not taking the empty snapshots means there will be no restore points created.

b) If yes, is the solution to allow empty snapshots to be taken for the monthly snaps, because those at first empty snaps will be "filled" as daily snaps are being killed off?
Epilogue: saying that they would eventually be "filled" is not correct. The snapshot is a point in time representation of what my filesystem looked like and will not change, some thwarted incremental logic nestled itself into my brain when I wrote that... :) The solution, however, is the correct one I believe: take empty snapshots if you want to be able to go back in time to the point.

 

[diedrichg]

The snapshots are taken when you have them scheduled whether there are file changes or not.

 

[guermantes]

The snapshots are taken when you have them scheduled whether there are file changes or not.

The help bubble for Allow taking empty snapshots says: "Creates dataset snapshots when there are no changes.". Does that not mean that if this option is unticked and there are no changes, that no snapshot will be taken even though it is time according to schedule?

 

[diedrichg]

My apologies. I forgot they added that setting. I set it to checked as the docs are not helpful with describing what happens if it's unchecked. An empty snapshot does not take up drive space so you will be fine just leaving it checked. I'm curious though if it's unchecked and files are not changed whether or not a snapshot is skipped.

 

[kingtj]

My apologies. I forgot they added that setting. I set it to checked as the docs are not helpful with describing what happens if it's unchecked. An empty snapshot does not take up drive space so you will be fine just leaving it checked. I'm curious though if it's unchecked and files are not changed whether it not a snapshot is skipped.

Interesting.... Yeah, I saw that option on mine and decided NOT to have it take snapshots if nothing had changed. I was just thinking of it like i do our backup systems in our office, where it saves a lot of bandwidth and some CPU usage running that way by default with our workstation to server backups via CrashPlan Pro-e. In that situation, when you go to do a restore? You select the date you wish to restore from, and the software gives you a directory tree of everything as it believes it looked at on that day. If it skipped 5 weeks of backups while the user had the laptop powered off and didn't change anything? That's fine. The restore "calendar" will just be greyed out until you try to select a date that's at last 5 weeks back. But you know you're getting the latest backup when you select the latest restore point it offers you -- assuming you verify there wasn't some kind of other error happening (which would be in the logs for that workstation's backups).

Now -- you all have me questioning if I need to change that setting on FreeNAS!

 

[guermantes]

Now I have run a few test cycles and it's easier to see that I was in fact shooting myself in the foot. The scenario described above would indeed lead to me being without 1-year retention snapshots when the monthly would start dying. As soon as I started watching the process as it happened it became quite clear: if the snapshot does not fire there is nothing to go back to. There is no ZFS magic that can create that snapshot after the fact based on an understanding of my mixed monthly and yearly retention scheme.

I will set my 1-year retention snapshots to allow empty snapshots, so as to be sure to have restore points.

 

[vicmarto]

I think that would be more logical to compare the new snapshot with the one before in the same "Periodic Snapshot Tasks". But reading this, seems this is not the case with 10.3-U1.

Maybe documentation need more clarification.

 

[guermantes]

I think that would be more logical to compare the new snapshot with the one before in the same "Periodic Snapshot Tasks". But reading this, seems this is not the case with 10.3-U1.

I wonder if what you want to achieve can be done without changing how ZFS actually handles snapshots and monitoring of datasets. But I defer to the gurus before I speculate any further. I kind of like how it is now "all about the dataset"; if the dataset changes. Period. Not if the dataset changes in relation to a particular snapshot. That seems to complicate matters.

 

[vicmarto]

I wonder if what you want to achieve can be done without changing how ZFS actually handles snapshots and monitoring of datasets. But I defer to the gurus before I speculate any further.

Yeah, I know, maybe it's not possible. Let's see what the gurus says.

I kind of like how it is now "all about the dataset"; if the dataset changes. Period. Not if the dataset changes in relation to a particular snapshot. That seems to complicate matters.

Well, in my opinion that would be the logical, and the expected behave. But, maybe it's impossible.

 

[Vladimir Vinogradenko]

b) If yes, is the solution to allow empty snapshots to be taken for the monthly snaps, because those at first empty snaps will be "filled" as daily snaps are being killed off?

It should actually work this way, and it does not due to bug which will be fixed in 11.3-U2: https://jira.ixsystems.com/browse/NAS-105346

It will not work though if you set up your daily jobs with 1 month retention period to run at midnight but your monthly jobs with 1 year retention period to run at e.g. 1 AM. In that case monthly job will always see dataset as unchanged (since last snapshot taken an hour ago, assuming NAS is not used at this time of the day) and will never run. We'll document this. So, general rule: when disabling taking empty snapshots, set up tasks with longer retention period to be running at the same time or earlier than tasks with shorter retention period. Or:do not uncheck "allow empty snapshots" if you are running tasks with longer retention period shortly after tasks with shorter retention period.