Keeping a fixed size recent files cache on a separate system

[parth]

Hi All,

I'm setting up a new FreeNAS setup at work that needs to accomodate upto ~60TB over the next year. I'm following the commodity hardware guidelines to source hardware and parts. I have no experience working with FreeNAS other that a test system I used at home to store some media and e-books. I was quite happy with the set up and the features that FreeNAS comes with. However, I have a very specific requirement at the office and I was wondering the best way to implement this was.

Basically, other than using a RAIDZ configuration and a backup of the entire data, we want another level of redundancy to make the system highly available. In the case of catastrophic failure of the system where we will need to restore the data from the backup, we want another system to tide us over while the data is being restored. We could solve this by having a duplicate FreeNAS setup but since we have a large amount of data and this requirement is only expected to grow more, we do not want to incur the additional cost of another complete system (we are a small startup after all). All we want is separate fixed-size system (FreeNAS or not) that keeps the latest ~1TB of created files and that keeps removing older files as newer ones come in. Is there a simple way to configure this in FreeNAS? Should I write scripts to rsync new files and remove older ones? I'm open to suggestions on the best way to go about doing this. Thanks for your help.

 

[sretalla]

True High-Availability only comes with TrueNAS (the bigger, paid, enterprise supported brother of FreeNAS).

If your use of that term was only intending to indicate "pseudo-HA", then FreeNAS can do some of that, principally using Replication jobs.

Based on your wish, you won't be using that either, since replication is all or nothing.

What you will need to set up is an rsync copy of files (optionally from a snapshot rather than the real source to avoid file locks) which filters to the last x days/GB of data... you may need to manage that by timestamp rather than size and just balance it as you see how much data it means.

Here's a pointer for some of that:
https://serverfault.com/questions/538767/how-to-rsync-files-folders-from-a-specific-date-forward

 

[parth]

Thanks for the suggestion. I was thinking along those lines but doing the rsync off a snapshot rather than the real source is something I had not thought of.