Register for the iXsystems Community to get an ad-free experience and exclusive discounts in our eBay Store.
Resource icon

A script to run rclone on the FreeNAS server to backup NAS data to Backblaze B2 cloud storage. 1.0

A script to run rclone on the FreeNAS server to backup NAS data to Backblaze B2 cloud storage. Can easily be adapted for other cloud providers.

Script was adapted from the one posted in the FreeNAS community forums by Martin Aspeli: https://www.ixsystems.com/community...2-backup-solution-instead-of-crashplan.58423/

Why? Freenas (v11.1 onwards) now supports rclone nativetly and also has GUI entries for Cloudsync, however there is no option to use additional parameters within the GUI, thus I have adapted a script to be run instead via a cron job.

Note: The rclone config file saved via the CLI is different to the one used in the FreeNAS GUI, so its safe to update this configuration file via rclone config without affecting anything you have configured in the FreeNAS GUI.

There are two scripts, one called rclonebackup.sh, which has been created by myself, and one called email_attachments.sh (obtained online, however unsure where from now).

Step 1
First off you need to configure rclone for Backblaze B2 as per: https://rclone.org/b2/, and if encrypting your data with rclone you then need to configure as per: https://rclone.org/crypt/.

Step 2
Download the two shell scripts from my github site.

Step 3
Save the two scripts within a dataset on your FreeNAS server.

Step 4
You now need to edit the rclonebackup.sh script, to update as per your specific requirements:

Code:
src=/mnt/tank
dest=secret:/
email=your@email.address
log_file=/tmp/rclonelog.txt
log_level=NOTICE
min_age=15m
transfers=16

Basically the default settings above will:

src:
backup all data within your pool (/mnt/tank in my case).
dest:
store them in the root of your remote:bucket on Backblaze B2 (secret:/ in my case).
email:
your email address, for backup results and log file.
log_file:
the name and location of log file (no real need to change this).
log_level:
for your first run I recommend this is set to NOTICE, as this will only report errors, otherwise the log file will get very large. Once you have completed your first backup, you can change this it INFO, which will let you know what files were backed-up or deleted.
min_age:
set a minimal age of the file before it is backed, handy for ignoring files currently in use.
transfer:
the amount of simultaneous transfer, Backblaze recommends a lot, so I have set at 16, but you can tweak this depending on your broadband connection.

You then want to tweak the command line which is executed:

Code:
rclone sync \
    --transfers ${transfers} \
    --fast-list \
    --copy-links \
    --min-age ${min_age} \
    --log-level ${log_level} \
    --log-file ${log_file} \
    --b2-hard-delete \
    --exclude "Thumbs.db" \
    --exclude "desktop.ini" \
    --exclude "AlbumArt*" \
    --exclude ".recycle/**" \
    --exclude ".windows" \
    ${src} ${dest}


--fast-list:
Use recursive list if available, uses more memory but fewer Backblaze B2 Class C transactions (i.e. cheaper). Not supported by all cloud providers though.
--copy-links:
Follow symlinks and copy the pointed to item. (delete this line if not wanted)
--b2-hard-delete:
Permanently delete files on remote removal, otherwise hide files. This means no version control of files on Backblaze (saves space & thus costs). This is a Backblaze B2 specific command. (delete this line if not wanted)
--exclude:
These lines will exclude certain files and folders from being backed-up. (delete this if not wanted, or edit, or add additional lines as per your needs)

Finally, at the end of the script you need to edit the location of the email_attachments.sh script:

Code:
/mnt/tank/Sysadmin/scripts/email_attachments.sh ${email} ${email} "${subject}" "${body}" /tmp/rclonelog.txt


In my case, this is at /mnt/tank/Sysadmin/scripts/.

Step 5
You now want to create a new cron job task via the FreeNAS GUI, remembering the command you want to run is the full path to the rclonebackup.sh script, for example:

cronjob.png


I recommend that for the first time you run this, that you do not enable the task. This is because the first run could take some time, many days if you have a few TB to backup. You do not want the server starting a new task while the old one is still running, as this will only confuse the backup process and slow down your server.

So, for your first time, just select the task and select Run Now, i.e.

runnow.png


Note: If the FreeNAS server reboots at any time during this first run, then you will need to restart the process by selecting Run Now again. This will re-scan the files and continue from where it last got to.

Step 6
Once the backup task has completed you should receive an email, along with a log file (of any errors encountered).

You can now edit the cron job and tick the Enabled box, so that the task will run automatically at the times you want. You may also want to edit the rclonebackup.sh script to change the log-level to INFO at this stage.

I hope this helps?

Yours
Jonathan
Thanks: Borre
Author
Hazimil
Downloads
41
First release
Last update
Rating
0.00 star(s) 0 ratings

More resources from Hazimil

Top