Martin Aspeli
Dabbler
- Joined
- Jan 13, 2016
- Messages
- 29
Many of us used Crashplan Home to back up a FreeNAS volume to the cloud. That service is discontinuing, and to be honest, Crashplan was always fairly painful to set up. We now have better alternatives, and here's one of them:
1) Create a jail.
2) Mount a dataset on /mnt/Backup or similar that contains the data you want to back up.
3) Install postfix via ports and configure it as an MTA. I used Gmail's SMTP server. Only needed if you want email alerts of successful/failed backups.
4) Sign up to Backblaze B2 and create a "bucket" for your backups. I think the bucket name as to be globally unique on B2 (waa?).
5) Download rclone from https://rclone.org/downloads/. You likely want the FreeBSD AMD64 binary. Copy it into your jail (or download it via curl in your jail itself). Copy the rclone binary to /usr/local/bin if you prefer to avoid typing the path each time.
6) Configure rclone to talk to your B2 bucket. See https://rclone.org/b2.
7) Run the initial backup. I did about 350Gb in 4 days. You likely want to use something like "screen" to run this in a terminal you can later detatch from and come back to. I used the following command to do this:
You should obviously adjust the backup path and bucket name according to your setup. If you have a large data set, running this command can take a very long time. If you want to get more information about what it is doing, add the option "--log-level INFO".
8) Create a script to run nightly backups. Here's mine:
The if/else/fi bit at the end is sending an email telling me if the backup succeeded or failed (the absence of an email overnight means the backup didn't run). For that to work, you have to have postfix or another MTA set up in the jail. Google how to do this if you don't know already. You could also just ignore this part and check the logs periodically, or find some other alerting system.
The other thing to note is that I've chosen to use the --fast-list option (saves money and time, but uses more memory), --copy-links (follows symlinks in the backup directory) and --b2-hard-delete (by default, rclone only hides files you delete locally, which means they still cost you money; with this option they are gone from the remote backup. That may or may not be what you want!). The --min-age flag is used to let rclone ignore files that have been very recently modified, e.g. they are partially downloaded or transferred to the NAS. I write all the logs to /var/log/rclone.log (for which ideally you'd set up log rotation) and log at INFO level (reasonably chatty if you have a lot of changes, so you may want to dial it down to NOTICE level to avoid filling up your disk with log files).
9) Create a cron job to run this script nightly, e.g.
... and then install with
Adjust the path and timings (5am every night in this case) as required.
That's it. It looks like it'll cost me maybe $20/yr to back up 350gb of photos, which is pretty good, and there's no weird Java/ssh-tunnel/config file mangling like there was with Crashplan.
- Backblaze B2 as cloud storage. It's faster, cheaper and easier to manage.
- rclone in a cron job to sync files nightly.
1) Create a jail.
2) Mount a dataset on /mnt/Backup or similar that contains the data you want to back up.
3) Install postfix via ports and configure it as an MTA. I used Gmail's SMTP server. Only needed if you want email alerts of successful/failed backups.
4) Sign up to Backblaze B2 and create a "bucket" for your backups. I think the bucket name as to be globally unique on B2 (waa?).
5) Download rclone from https://rclone.org/downloads/. You likely want the FreeBSD AMD64 binary. Copy it into your jail (or download it via curl in your jail itself). Copy the rclone binary to /usr/local/bin if you prefer to avoid typing the path each time.
6) Configure rclone to talk to your B2 bucket. See https://rclone.org/b2.
7) Run the initial backup. I did about 350Gb in 4 days. You likely want to use something like "screen" to run this in a terminal you can later detatch from and come back to. I used the following command to do this:
Code:
$ rclone sync --fast-list --min-age 15m --transfers 16 --copy-links /mnt/Backup b2:my-bucket-name
You should obviously adjust the backup path and bucket name according to your setup. If you have a large data set, running this command can take a very long time. If you want to get more information about what it is doing, add the option "--log-level INFO".
8) Create a script to run nightly backups. Here's mine:
Code:
#!/usr/local/bin/bash
src=/mnt/Backup
dest=b2:my-bucket-name
email=my.email@example.org
log_file=/var/log/rclone.log
log_level=INFO
min_age=15m
transfers=16
/usr/local/bin/rclone sync \
--copy-links \
--b2-hard-delete \
--fast-list \
--min-age ${min_age} \
--transfers ${transfers} \
--log-level ${log_level} \
--log-file ${log_file} \
${src} ${dest} $@
rc=$?
if [[ $rc != 0 ]]; then
echo "An error occurred. Please check the logs." | /usr/bin/mail -s "rclone backup error" ${email}
else
echo "Backup succeeded" | /usr/bin/mail -s "rclone backup success" ${email}
fi
The if/else/fi bit at the end is sending an email telling me if the backup succeeded or failed (the absence of an email overnight means the backup didn't run). For that to work, you have to have postfix or another MTA set up in the jail. Google how to do this if you don't know already. You could also just ignore this part and check the logs periodically, or find some other alerting system.
The other thing to note is that I've chosen to use the --fast-list option (saves money and time, but uses more memory), --copy-links (follows symlinks in the backup directory) and --b2-hard-delete (by default, rclone only hides files you delete locally, which means they still cost you money; with this option they are gone from the remote backup. That may or may not be what you want!). The --min-age flag is used to let rclone ignore files that have been very recently modified, e.g. they are partially downloaded or transferred to the NAS. I write all the logs to /var/log/rclone.log (for which ideally you'd set up log rotation) and log at INFO level (reasonably chatty if you have a lot of changes, so you may want to dial it down to NOTICE level to avoid filling up your disk with log files).
9) Create a cron job to run this script nightly, e.g.
Code:
$ crontab -e
... and then install with
Code:
0 5 * * * /root/rclone/rclone-cron.daily >/dev/null 2>&1
Adjust the path and timings (5am every night in this case) as required.
That's it. It looks like it'll cost me maybe $20/yr to back up 350gb of photos, which is pretty good, and there's no weird Java/ssh-tunnel/config file mangling like there was with Crashplan.
