A script to backup configuration file when changed

Status
Not open for further replies.

Caesar

Contributor
Joined
Feb 22, 2013
Messages
114
I saw cyberjock's post for backing up the config file every night but my config does not change often enough to back it up daily so I thought to my self that it would be nice if freenas had a feature to backup the config to a location of my choice when I made a change. So I wrote this python script to do just that.

Basically this script will copy freenas-v1.db file to a folder of your choice and then will not make another copy unless a change is found in the freenas-v1.db file.


This is the first time I have ever created a python script so be nice if I did not create the most efficient code ever. Also I have done very limited testing of this script so there maybe a bug or two. Just let me know and I will do what I can to fix them.

[Update]
I found a bug in the script so I have moved this script to github so I can make changes and don't have to keep updating this post.

https://github.com/ebright/Freenas-bk-cfg
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
Re: Backup config only if changed!

OOO. I like the idea! Good job. I haven't run it, but I'll definitely have to test it out soon!

The only reason I didn't try to filter out the duplicate copies with my script is because the database is only 200k, so who cares if I have 1000 copies(which would be 3 years worth!)
 

ProtoSD

MVP
Joined
Jul 1, 2011
Messages
3,348
Re: Backup config only if changed!

It might be a good idea to set some # of copies to keep, you might discover too late that you're backing up something you got wrong.
 

Caesar

Contributor
Joined
Feb 22, 2013
Messages
114
Re: Backup config only if changed!

OOO. I like the idea! Good job. I haven't run it, but I'll definitely have to test it out soon!

The only reason I didn't try to filter out the duplicate copies with my script is because the database is only 200k, so who cares if I have 1000 copies(which would be 3 years worth!)

I wasn't worried about the size of the files. I felt like I might be looking for a needle in a haystack after a few months of backups.

ProtoSD said:
It might be a good idea to set some # of copies to keep, you might discover too late that you're backing up something you got wrong.

There is no clean up to this script. if the config changes it renames the current backup file and makes a new copy. so you will have a copy of every config that you have made since running the script.
 

HolyK

Ninja Turtle
Moderator
Joined
May 26, 2011
Messages
653
I like the script, it's working just fine! Anyway i found a little bug. If the script is at the same place as the backups, it will rename the script :D

10/27/2013 02:30:00.948 Searching for previous backup in /mnt/redmirror/misc/cfgbackup/
10/27/2013 02:30:00.948 Found file /mnt/redmirror/misc/cfgbackup/backup.py
10/27/2013 02:30:00.949 Configuration changed. Archiving previous backup
10/27/2013 02:30:00.949 /mnt/redmirror/misc/cfgbackup/backup.py >> /mnt/redmirror/misc/cfgbackup/archive.py

Once i created additional subdir, its working flawlessly!
10/27/2013 02:37:00.410 Searching for previous backup in /mnt/redmirror/misc/cfgbackup/configs/
10/27/2013 02:37:00.410 No previous backup found. Create new backup: /mnt/redmirror/misc/cfgbackup/configs/backup_2013_10_27_02_37.db
10/27/2013 02:41:00.928 Searching for previous backup in /mnt/redmirror/misc/cfgbackup/configs/
10/27/2013 02:41:00.928 Found file /mnt/redmirror/misc/cfgbackup/configs/backup_2013_10_27_02_37.db
10/27/2013 02:41:00.929 Configuration changed. Archiving previous backup
10/27/2013 02:41:00.929 /mnt/redmirror/misc/cfgbackup/configs/backup_2013_10_27_02_37.db >> /mnt/redmirror/misc/cfgbackup/configs/archive_2013_10_27_02_37.db
10/27/2013 02:41:00.929 Creating backup /mnt/redmirror/misc/cfgbackup/configs/backup_2013_10_27_02_41.db

Code:
-rwxr-xr-- 1 root wheel 2748 Oct 27 02:34 backup.py*
drwxrwxr-x 2 root wheel   4 Oct 27 02:41 configs/
-rw-r--r-- 1 root wheel 2601 Oct 27 02:41 freenas-backup.log
 

Caesar

Contributor
Joined
Feb 22, 2013
Messages
114
Thanks for the feedback. I didn't expect anyone to put the script in the same folder as the backup files. Another work around would be to rename the backup.py script to something like AutoBackup.py
 

nojohnny101

Wizard
Joined
Dec 3, 2015
Messages
1,478
sorry to bring up an old post but I was attempting to run this script and I get an error:

Code:
Apr  6 14:33:34 axio cronjob:   File "scripts/backup.py", line 44
Apr  6 14:33:34 axio cronjob:     logging.debug('Automatic Backup Script for FreeNAS Configuration
Apr  6 14:33:34 axio cronjob:                                                                     ^
Apr  6 14:33:34 axio cronjob: SyntaxError: EOL while scanning string literal


has anyone used this script recently and got it working?
 

hervon

Patron
Joined
Apr 23, 2012
Messages
353
Just tried it. Worked fine. Wrote a backup file in the tmp folder. As a matter of fact. I like this script and continue to use it. Adjusted it to write the backup config file on my pool. Thanks Caesar!
 

nojohnny101

Wizard
Joined
Dec 3, 2015
Messages
1,478
@hervon ok thanks, I took a second look after your confirmation and it seems I had a simple path error. got it working now, sweet script!
 

Scharbag

Guru
Joined
Feb 1, 2012
Messages
620
My python skills are weak, weak, weak... But I did figure out the following (borrowing heavily on other peoples scripts):
Code:
# THIS REMOVES OLD FILES FIRST
import time
now = time.time()
old = now - 90 * 24 * 60 * 60

for f in os.listdir(bk_path):
	path = os.path.join(bk_path, f)
	if os.path.isfile(path):
		stat = os.stat(path)
		if stat.st_ctime < old:
			print "removing: ", path
			os.remove(path) # uncomment when you are sure :)
# END OF OLD FILE DELETE CODE


I run this at the beginning of your script to delete files older than 90 days in the directory where the configs are backed up. It is pretty low brow but I have OCD and it helps me keep the directory a little cleaner.

Thanks again for a great script.

Cheers,
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
@Scharbag (and everyone else)

One thing to consider is the backups can be bad if the config file is somehow corrupted. I've had several people that had backup scripts deleting after X days, and they would end up with every backup being bad because they deleted them after a set timeframe. So I, personally, am not a proponent of deleting backups of the config file because of the very small size (which I mentioned above in this thread). When you don't have a way to prove that the backups are good your options diminish. ;)

For some people with very complicated environments or environments where you want to be able to recover from a failed boot device quickly, having a good config file backup is a necessity.

Just something to think about...
 

Scharbag

Guru
Joined
Feb 1, 2012
Messages
620
@cyberjock I hear you sir. I forgot to mention that my backup strategy will keep the files for at least 2 year as I snapshot my backup pool )nightly for 2 weeks, then every 2 weeks after that) before I rsync changes to it. So I will have 90 days of backups available without any work in 2 locations and if they are all bad, I will have at least 2 years of backups available through snapshots. Another point to consider is that if you need to restore a backup older than 90 days, it may be best to rebuild as my system can change a lot in 90 days.

And I agree, the size of the backups is such that I should just not bother with the delete. But, as I said, I have OCD... :)

Cheers,
 

Stux

MVP
Joined
Jun 2, 2016
Messages
4,367
@cyberjock I hear you sir. I forgot to mention that my backup strategy will keep the files for at least 2 year as I snapshot my backup pool )nightly for 2 weeks, then every 2 weeks after that) before I rsync changes to it. So I will have 90 days of backups available without any work in 2 locations and if they are all bad, I will have at least 2 years of backups available through snapshots. Another point to consider is that if you need to restore a backup older than 90 days, it may be best to rebuild as my system can change a lot in 90 days.

And I agree, the size of the backups is such that I should just not bother with the delete. But, as I said, I have OCD... :)

Cheers,

They gzip really well too. About 50KB last I checked.
 

enoch85

Dabbler
Joined
Nov 30, 2016
Messages
33
In FreeNAS Corral you have to change this:
Code:
if matched is True:
		 read1 = file(db_file).read().split('\n')
		 read2 = file(bk_file).read().split('\n')

to this:
Code:
if matched is True:
		with open(db_file, "rb") as f:
			read1 = f.read()
		with open(bk_file, "rb") as f:
			read2 = f.read()

But I'm still wondering which exact file to do a backup of? In /data/ I find
Code:
-rw-r-----   1 www	  www		 786432 Feb 20 07:44 factory-v1.db
-rw-r-----   1 root	 www		 342016 Feb 20 07:54 freenas-v1.db.bak
-rw-r-----   1 root	 operator	351232 Mar 26 20:17 freenas-v1.db.done

Which one should I add as db_file? Or maybe non of them?
 

liteswap

Dabbler
Joined
Jun 7, 2011
Messages
37
With the update to FreeNAS 11, the version of python changes from 2.x to 3.x, which has broken this script. Despite not being a python expert, I've managed to get the file comparison section working as follows:
Code:
# insert this at head of script:
import filecmp
...
# file comparison section:
if matched is True:
#  this section updated to work in python 3 by Manek Dubash 2017
#  read1 = file(db_file).read().split('\n')
#  read2 = file(bk_file).read().split('\n')
#  if read1 == read2:
if filecmp.cmp(db_file,bk_file):


Although it seems to work fine on my servers, I'd be happy to hear from anyone who knows more about python than I do (not difficult) as to why this might not be the right solution.
 
Status
Not open for further replies.
Top