Dataset snapshot and initial replication seems to be repeating.

[technopop]

Trying to get a better sense since it seems that after a successful initial replication, it's going through it again and I'm watching my free disk space disappear.

I have two machines (library archive and backup) with 150TB of capacity each with GbE connecting them. There is 100TB of data on the library archive server that I want to replicate to the other.

I have set my snapshots to once per week with a 2 week retention and my zfs replication to delete stale snapshots.

It takes about 15 days for that data to replicate to the other server.

The initial replication happens and I can browse through the files. But I'm guessing the replicated snapshot serve as some sort of index and since snapshot lifetime is 2 weeks and stale snapshots are set to delete, the next scheduled replication won't know so it goes through the 15 day process at full network bandwidth again.

1. Is that correct?
2. As it's doing a second 100TB initial replication with only 50TB left, I assume it's going to run out of space and cause problems?
3. So I should wipe the backup dataset, set my snapshot retention to something longer to keep the backup server from having all replicated snapshots deleted and it wanting to start again?

 

[Ericloewe]

Yeah, if your snapshot expires on the remote end before a new one arrives, you have to start over (or use a bookmark). Easy solution: set retention to 20 days.

 

[technopop]

Thank you.

 

[Ericloewe]

I missed the part where it seems to be taking up more disk space. I'm not quite sure what happens in this scenario, since it's rather unusual to only keep snapshots for a few weeks (so you basically never see all snapshots expire), but apparently you'll want to delete the old datasets that were left over and rename stuff as needed, according to your needs.