jlw52761
Explorer
- Joined
- Jan 6, 2020
- Messages
- 87
I'm looking at a DR strategy where I have two TrueNAS SCALE devices at two locations. The primary location serves a dataset as a NFS share, and I am snapping that every 5 minutes. I have a Docker Swarm that uses the NFS for persistent storage, and all that works great today.
The DR strategy I am looking at is to replicate the snapshot from the primary location to the TrueNAS SCALE device as the backup location, which I am currently doing, and I also have nodes in the Docker swarm at that site in a DRAIN state. In that state, they don't take any container load. Now both sites have a leader node in the Swarm, so no worries there, but what I am wanting to do, is that when detecting a failure of the primary site, set the nodes at the backup site to AVAILABLE via a script, meaning they will start any services/containers that aren't running, according to the rules of the service, and connect to the NFS share at the backup location, since that's where they are at.
All this is easy, but, what I am noticing is that at the backup site, if I look at the mount for the replicated snapshot, the data is only from the oldest snapshot, not the latest, which is what I was hoping for. My desire is, that when the new snapshot comes in every 5 minutes, that the mount reflects the new data. I know that when the primary site goes down I will have to disable the replication job and manually snapshot the dataset in the backup location and manually replicate back, that's fine, but was hoping that the "pointer" would automatically move forward on the dataset in the backup site as the new snapshot came in.
If that's not the behavior, fine, I just need to add extra logic to the script to first "restore" to the latest snapshot, I'm mainly wanting to know if what I am observing is normal behavior or if what I desire is the normal and what I'm observing is due to some setting somewhere I flipped without realizing.
My second question is, if I did create a file in the backup location for any reason, and then "restored" the replicated snapshot from the primary location, is that file gone because it didn't exist in the snapshot, or is that file still there, but the data in the snapshot is merged with the dataset?
The DR strategy I am looking at is to replicate the snapshot from the primary location to the TrueNAS SCALE device as the backup location, which I am currently doing, and I also have nodes in the Docker swarm at that site in a DRAIN state. In that state, they don't take any container load. Now both sites have a leader node in the Swarm, so no worries there, but what I am wanting to do, is that when detecting a failure of the primary site, set the nodes at the backup site to AVAILABLE via a script, meaning they will start any services/containers that aren't running, according to the rules of the service, and connect to the NFS share at the backup location, since that's where they are at.
All this is easy, but, what I am noticing is that at the backup site, if I look at the mount for the replicated snapshot, the data is only from the oldest snapshot, not the latest, which is what I was hoping for. My desire is, that when the new snapshot comes in every 5 minutes, that the mount reflects the new data. I know that when the primary site goes down I will have to disable the replication job and manually snapshot the dataset in the backup location and manually replicate back, that's fine, but was hoping that the "pointer" would automatically move forward on the dataset in the backup site as the new snapshot came in.
If that's not the behavior, fine, I just need to add extra logic to the script to first "restore" to the latest snapshot, I'm mainly wanting to know if what I am observing is normal behavior or if what I desire is the normal and what I'm observing is due to some setting somewhere I flipped without realizing.
My second question is, if I did create a file in the backup location for any reason, and then "restored" the replicated snapshot from the primary location, is that file gone because it didn't exist in the snapshot, or is that file still there, but the data in the snapshot is merged with the dataset?