We're running 2 freenas servers; a primary server that's offering services, and a secondary that has all services disabled (except ssh and SMART). Primary is setup to replicate /mnt/storage to /mnt/storage on secondary. In this configuration, should primary suffer a catastrophic failure, we can restore the primary config on the secondary server, reboot, and resume operation. This worked great up through Freenas 9.2.0
Enter FreeNAS 9.2.1.1
Now there's this mostly undocumented .system volume which lives in /mnt/storage/.system. This is used for samba4 and "leveraged for other purposes"; currently logs and "core files".
The Dilemma
If we upgrade both servers to 9.2.1.1, anything that primary writes to /mnt/storage/.system will be replicated to secondary, replacing whatever secondary may have written to that same path. This means any logs secondary writes will get clobbered every replication cycle, likewise any samba4 or "core files" (though since samba is disabled on secondary, this might not matter. I have no idea what a "core file" is.)
So what's recommended? I believe we have 3 options:
Enter FreeNAS 9.2.1.1
Now there's this mostly undocumented .system volume which lives in /mnt/storage/.system. This is used for samba4 and "leveraged for other purposes"; currently logs and "core files".
The Dilemma
If we upgrade both servers to 9.2.1.1, anything that primary writes to /mnt/storage/.system will be replicated to secondary, replacing whatever secondary may have written to that same path. This means any logs secondary writes will get clobbered every replication cycle, likewise any samba4 or "core files" (though since samba is disabled on secondary, this might not matter. I have no idea what a "core file" is.)
So what's recommended? I believe we have 3 options:
- Create a second pool on secondary just to hold the .system volume. We only have room for 6 disks and they're all already part of the "storage" pool. I'm sure we'll have to delete and recreate that pool, but is it even possible to have 2 pools on a disk? Can this be done with the GUI, or do we need to create partitions on the disks and create the pools in the partitions? How big should the system pool be?
- Leave it as is. secondary won't have logs; maybe syslog can help here, but otherwise no harm. I suppose we could point the /var/log symlink to a ramdisk or similar. What are the core files? Are these a concern? Or is it OK if primary's core files are replacing secondary's every 15 minutes?
- Replicate to a subvolume. By that I mean replicate from /mnt/storage on primary to /mnt/storage/replicated_data on secondary. Now there's no file-clobbering, but it seems like it would also take longer to turn secondary into a replacement for primary should a catastrophic failure occur.