TrueNAS 13.U1.1 - VMware Snap Shot - Guidance and Usage Please.

[Murphy1138]

Hello all, I would like some advice and operational info on using VMWare-Snapshots in the event of Ransomware Rollback correctly.

I have 3x Dell R740 running VMware ESXi, 6.7.0, 19195723 connected over iSCSI via 2x paths to the data store on the TrueNAS 13.U1.1 server with SYNC=ALWAYS Enabled, a Dell R740XD2 with 24 disks.

• 14 x SSD
• 10 x HDD

I have 2x Pools.
TANK01 - is made up of:

• 4x Data vdevs, with 2x disks each in mirrors - Total 8x SSDs
• 1x ZIL vdev with 2x disk mirror. - 2x SSDs
• 1x spare vdev with 2x hot standby disks. - 2x SSDs

TANK02

• 4x data vdevs with 2x disk mirrors - total 8 x HDDs
• 1x ZIL vdev with 2x disk mirror - 2x SSDs
• 1x spare vdev with 2x hot standby disks - 2xHDDs

In vmware snapshots, I have this setup to auth with the vSphere. I have my TANK02/zvol-hdd-lun0 and then I picked the VMS datastore in the drop-down that corresponded to the TANK02/zvol-hdd-lun0. That passes through and all seems good.

Then I set up the snapshot as either a manual task or automatic.

My understanding is that that will tell vSphere that a snap of the VM is needed, the host will snap it, TrueNAS will then snapshot its storage locally, then tell vSphere to remove the snap of the VM and then I can see my snapshots on TrueNAS.

Is that correct?

Next, what is the correct way of reverting the snapshots, obviously TrueNAS does not snap the working memory.
My goal is in the event of a ransomware outbreak and guest servers end up encrypted.

• Power off the VMs on the vSphere host/ESXI Servers.
• Roll back the snap shot on TrueNAS
• Rescan vSphere Storage
• Power up VMs, they boot and will be back to a pre-ransomware state.

Is that the correct process, if not what is?

Many thanks for your time and effort in helping.
Murph.

PS I also have a very robust backup setup in the event of ransomware, this is an extra step.

 

[Murphy1138]

bump

 

[sretalla]

My understanding is that that will tell vSphere that a snap of the VM is needed, the host will snap it, TrueNAS will then snapshot its storage locally, then tell vSphere to remove the snap of the VM and then I can see my snapshots on TrueNAS.

Is that correct?

Yes

Next, what is the correct way of reverting the snapshots, obviously TrueNAS does not snap the working memory.
My goal is in the event of a ransomware outbreak and guest servers end up encrypted.

• Power off the VMs on the vSphere host/ESXI Servers.
• Roll back the snap shot on TrueNAS
• Rescan vSphere Storage
• Power up VMs, they boot and will be back to a pre-ransomware state.

That should do it.

 

[Mauricio Silveira]

Yes


That should do it.

Yes, but...

Next, what is the correct way of reverting the snapshots, obviously TrueNAS does not snap the working memory.
My goal is in the event of a ransomware outbreak and guest servers end up encrypted.

• Power off the VMs on the vSphere host/ESXI Servers.
• Roll back the snap shot on TrueNAS
• Rescan vSphere Storage
• Power up VMs, they boot and will be back to a pre-ransomware state.

I think you should add 2 additional steps. TrueNAS will create a VM snapshot prior to the ZFS snapshot. This VM snapshot should be the most secure (integrity-wise) point in time, since there might be some write(s) between VM snapshot and ZFS snapshot.

1. Power off the VMs on the vSphere host/ESXI Servers.
2. Roll back the snap shot on TrueNAS
3. Rescan vSphere Storage
4. Restore VMs to the snapshot created by TrueNAS ( restore to latest )
5. Remove last snapshot
6. Power up VMs, they boot and will be back to a pre-ransomware state.

According to this source code: https://github.com/truenas/middleware/blob/master/src/middlewared/middlewared/plugins/vmware.py#L438
TrueNAS waits for VM snapshot creation before ZFS snapshot. This validates steps 4 and 5 above. PS: If I'm looking at the right code.

Please, feel free to correct my assumptions if I'm wrong! :)