Backing Up to External USB from FreeNAS

[vafk]

@Adrian
Many thanks for the high level view :) which I really appreciate because it gives me lot of impulses which I am currently thinking of. It is probably the longest virus I have (=Windows) and yes the less "feartures" a NAS has to the, the better for the security of my data.

I admit that I overlooked the fact where iXsystems make their profit and now understand that the proposals from private users like me may sound like a joke to them. Perhaps I was mislead by the many gimmicks FreeNAS brought to me over time (jails, VM's, services etc.) which a normal NAS would normally not have. Then I started comparing FreeNAS with systems like QNAP which have this suggested backup function of copying data to NTFS-USB-Drive and thought: hey why the architects cannot put such a simple task to it? Now I understand.

I will have a closer look at TrueNAS and other BSD-based OS because with the time I started to enjoy BSD and although I am infected by the virus (see above) and have very little UNIX knowledge, BSD became my preferred OS. Now my little son started enjoying programming and I wanted to lead him to a virus-free-OS and therefore suggested BSD. Also with the quiet hope that one day he can administer the Home-NAS when I get retired :)

All I have to watch is that I don't mess up my zpools with whatever change I do. I already set up two separate N54L which run FreeNAS to replicate the source pools every week. It's not a true backup but in case the main hardware would brake I have the data next door.

Because I have a few 3TB USB-Drives from earlier days I thought it therefore would be great to have the data also backed up there. It was only my brains that wanted to accomplish the task directly from FreeNAS and it should not be a problem to do it from a separate machine (even if it is done through Linux or Windows from a VM within FreeNAS).

I also learned from your explanation how I can explain it to a new user because I was put up by the very short comments a la "the system is not to designed for" or "you should not to so". You really put it straight forward so a programmer and a businessman understands it. Thank you!

 

[Adrian]

As far as I know, you cannot really get a look at TrueNAS in operation without visiting a TrueNAS machine, or buying a licence, which you probably won't be able to get. iXsystems carefully select and test the hardware for compatibility.

However, take a look at https://www.youtube.com/watch?v=H1OmL90qFBU

Yes, set your children early on the right computing path!

 

[sirrus]

Whether or not this is a good idea/solution remains to be seen as it has not been thoroughly tested, and to be clear I'm not recommending that others do as I have done. What we all do with our own systems is up to each of us.

That said...
I was able to mount an exfat formatted external USB HD in read/write mode outside of a Jail or VM on FreeNAS 11.3-U1. It took me longer to puzzle out a solution to this than I'd care to admit (more than a Saturday morning, less than a full day).

Here's what I did (I mention why below for the curious):

• Create a jail
  (this if for building the code not for actually doing the mounting)
• Add a mount point at /usr/ports on both the host and the jail
  (just type in "/usr/ports" in the field. The UI accepted it for me even though only /mnt was available to choose)
• Start the jail and open the shell/prompt
• Run the following commands inside of the jail:
  • portsnap auto
  • pkg install portmaster
  • pkg install fusefs-libs fusefs-ntfs
    (this is perhaps not needed but I doubt it hurts)
  • pkg install gcc autoconf automake
  • cd /usr/ports/sysutils/fusefs-exfat
  • make
    (there will be several prompts for licenses and options. The defaults are generally fine.)
• Go back to the host and open its shell/prompt
• set the OSVERSION environment variable
  (for me running 11.3-RELEASE-p6 I used "export OSVERSION=1103506")
• cd /usr/ports/sysutils/fusefs-exfat
  (since this was a mount point all of the newly compiled packages will be available here on both the host and the Jail)
• make install
• kldload fuse.ko
  (if you ever see the error "fuse: failed to open fuse device: No such file or directory" you should try this command)
• mount.exfat /dev/<your ex fat partition> /<your mount path>
• Be sure to issue an "umount /<your mount path>" prior to pulling the USB drive so that it can disconnect cleanly

Once again I don't know how well this will hold up over time. Except for the kldload command, it does seem to persist across reboots. Your mileage may vary.

My use case is exactly what this thread is about. I'd like the ability to backup certain files on my FreeNAS to a disk (to be kept off-site) which will be readable basically by any other OS. The readability part is key. One of the more common household disasters that could necessitate a needing an off-site backup is a fire. Morbid as it is, I could die in a fire, and I don't want my family locked out of our records/pictures/etc.

The basic problem with using a Jail is that we cannot mount fuse filesystems there (yet "The jail must have FreeBSD 12.0 or newer installed." which I'm pretty sure isn't possible on FreeNAS 11), and the basic problem with a VM is that there is no clear facility to pass just one USB device through to it (one can, it seems, pass an entire PCI device through, but that is more than I want). One could pass the entire /dev/<disk> (reference) block device to the VM, but you'd have to be very careful to ensure the VM isn't actually running when you disconnect/connect said USB device. Trying to compile/install the port on the FreeNAS system directly doesn't work. In any event, I'd just prefer to do it natively on the base system. It seems cleaner.

 

[deergod]

B) 4TB of stuff that would be really hard to replace but possible with time and effort (rips of my home-video tapes, old digitized photos, the seemingly bottomless pool of 'unsorted' digital photographs from the last 10 years that are also still on SD/CF cards somewhere)
C) 1TB of stuff that is truly irreplaceable (documents, the 'family album' digital photographs, code projects etc).

This pretty much nails down the questions I have about this USB topic. It all comes down to link speed. USB is faster than gigabit. My alternative has been to retain a copy on a local Windows PC with a large single HD, or (actual current solution) has been using rsync with Synology Active Backup for Business (such a bad name) to pull a mirror copy of the B & C data daily from FreeNAS /mnt. Works a treat. I still need to rationalize my "true" C and "true" B data, but until I get a 10GBe link, I'm limited to the gigabit network speed. Also, trying to get a friend to swap me some space on his NAS to securely store the "C" files. Hate paying for that offsite storage. It's encrypted, so it doesn't even have to be a friend. Anyone doing this?

Thanks for all the ideas and thoughts on this thread.

 

[f00b4r00]

Just chiming in on this as I've built a small script to manage a particular zfs replication scheme of mine. It so happens that I need/want to replicate some datasets to an offsite FreeNAS device. The intertube between them is slow, and sometimes the amount of data to exchange is quite high. In those circumstances, a pigeon carrier (myself with a USB external drive) then becomes many times faster that going over the net, and that's what I end up doing every now and then.

In a nutshell what I do is I dump an incremental snapshot (based on the remote machine most recent snapshot) of each dataset to a file (zfs send -R -I) stored directly onto a suitable external USB HDD, then I plug said HDD to the other device and zfs receive the file. Being able to natively rw mount exfat would help, since 1) the 'carry HDD' is not necessarily used for this sole purpose (so the more commonplace the fs the better), and 2) exFAT supports large files, which FAT32 doesn't, so right now I have to play an extra 'split -b 4294967295' dance to get this to work with FAT32. Note that at no point in this process do I touch the underlying data, all I care about are zfs incremental streams.

Anyway, just illustrating another perfectly valid reason to want to mount some common place filesystem on a FreeNAS machine to dump stuff onto :)

 

[gwaitsi]

Just chiming in on this as I've built a small script to manage a particular zfs replication scheme of mine. It so happens that I need/want to replicate some datasets to an offsite FreeNAS device. The intertube between them is slow, and sometimes the amount of data to exchange is quite high. In those circumstances, a pigeon carrier (myself with a USB external drive) then becomes many times faster that going over the net, and that's what I end up doing every now and then.

In a nutshell what I do is I dump an incremental snapshot (based on the remote machine most recent snapshot) of each dataset to a file (zfs send -R -I) stored directly onto a suitable external USB HDD, then I plug said HDD to the other device and zfs receive the file. Being able to natively rw mount exfat would help, since 1) the 'carry HDD' is not necessarily used for this sole purpose (so the more commonplace the fs the better), and 2) exFAT supports large files, which FAT32 doesn't, so right now I have to play an extra 'split -b 4294967295' dance to get this to work with FAT32. Note that at no point in this process do I touch the underlying data, all I care about are zfs incremental streams.

Anyway, just illustrating another perfectly valid reason to want to mount some common place filesystem on a FreeNAS machine to dump stuff onto :)

is it possible to the usb drive as a zfs drive and replicate the pools as a once off job i.e. via a script.

In summary, scripts:
- mounts the drive
- replicates the pools
- dismounts the drive

alternatively, it is worth to virtualise freebas inside a proxmox system and then backup the pools via ubutu openzfs

 

[sretalla]

is it possible to the usb drive as a zfs drive and replicate the pools as a once off job i.e. via a script.

Read this resource:

How to: Backup to local disks

How to: Backups to local disks Overview: This subject comes up often enough that I thought I would start a post of how I do it. Specifically I back up my TrueNAS to a hot-inserted, locally attached disk, then when done, hot-remove and take...

www.truenas.com

alternatively, it is worth to virtualise freebas inside a proxmox system and then backup the pools via ubutu openzfs

No.