Backup to ESATA

[Herman Eggink]

All,

If I were to create a new volume called Backup and assign a new disk connected through esata, I could periodically backup existing volumes (rsync, crashplan), detach the volume in order to store off-site (and import for incremental backups).

Prerequisite is that I would have to be able to import the volume on a different (new) freenas box in case my house burns down. Can that be done?

Thanks,

H

 

[Ericloewe]

Yeah, but a mirrored setup is recommended, for obvious reasons.

You do need to do most of the work manually, which is why few people go down this route.

 

[Herman Eggink]

Hi Eric,

My NAS is has mirrored disks for sure but I need to have a contingency in case my house burns down or we get robbed. I know quite a bit about Linux both FreeBSD and ZFS are still fairly new to me. Manual stuff is OK as long as it can be done.

I just changed by backup (crashplan) to use the esata zfs disk and noticed it's twice as fast as an NTFS drive (which I read is not recommended at all). I will give it a try to see if I can mount it under Ubuntu and can access the backup just in case.

Thanks for the reply!

H

 

[Ericloewe]

Mounting it on OSes other than FreeNAS/FreeBSD may not work, since OpenZFS is a rather fluid thing. FreeNAS will definitely work, though.

 

[alexg]

It is unfortunate that FreeNAS team decided to ditch UFS. I'm currently running weekly rsync to eSATA connected UFS drive and just detach it for offsite storage. UFS file system has wider support on Linux. I didn't want to have another server to rsync remotely.

 

[Ericloewe]

It is unfortunate that FreeNAS team decided to ditch UFS. I'm currently running weekly rsync to eSATA connected UFS drive and just detach it for offsite storage. UFS file system has wider support on Linux. I didn't want to have another server to rsync remotely.

Why do you need UFS fit your scenario? If it comes to that, you can boot FreeNAS on a different machine to rescue your data.

 

[alexg]

Good point. I didn't consider that I could keep a copy of FreeNAS on USB stick in offsite storage. I guess I was just thinking if house burns down it will be easier to access Windows computer. I'll need to rethink my strategy. Thanks

 

[Herman Eggink]

In my case it's not specific to UFS but more ZFS. Whilst I understand I can always attach the volume on a new freenas system, I can ONLY attach it there with no option to access the files on either windows or Linux (tried Ubuntu and it did not like the volume). Imagine my stuff getting stolen or the house burned down. Being able to access in case of emergency requires staging a new NAS again limiting flexibility. That's all.

 

[SirMaster]

You can import FreeNAS 9.3 pools in Linux if you install the HEAD version of ZoL. ZoL has working support for every available feature flag right now (except multi_vdev_crash_dump but normal users have no reason to activate this flag). The Linux guys have been very committed to keeping up with development of OpenZFS. The latest embedded_data feature flag actually got added to Linux before FreeBSD.

*edit* I meant before FreeNAS, not before FreeBSD. FreeNAS got embedded_data in FreeNAS 9.3-BETA on Nov 10 2014, whereas Linux got it back at the end of July/beginning of August

 

[Herman Eggink]

I tried ZoL but could not get it to work. Do you have a link to this HEAD version? Many thanks!

 

[jgreco]

It is unfortunate that FreeNAS team decided to ditch UFS. I'm currently running weekly rsync to eSATA connected UFS drive and just detach it for offsite storage. UFS file system has wider support on Linux. I didn't want to have another server to rsync remotely.

FreeNAS didn't "ditch" UFS. It's still there. It just isn't supported as a filesystem on which to serve files, since it made for GUI complexity. You can still manually mount and manipulate UFS from the CLI.

 

[SirMaster]

I tried ZoL but could not get it to work. Do you have a link to this HEAD version? Many thanks!

I'm not sure for Ubuntu, but for Debian it's easy.
http://zfsonlinux.org/debian.html

"If you wish to use the Debian GNU/Linux ZoL snapshot packages (pre-releases), you will need to uncomment the line(s) related to the 'dailies' repository (at the end of the /etc/apt/sources.list.d/zfsonlinux.list file) by removing the hash mark at the beginning of the line."

Code:

root@debian:~# zpool status
  pool: tank
state: ONLINE
  scan: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        tank        ONLINE       0     0     0
          sdb       ONLINE       0     0     0

errors: No known data errors
root@debian:~# zpool get all
NAME  PROPERTY                    VALUE                       SOURCE
tank  size                        31.8G                       -
tank  capacity                    0%                          -
tank  altroot                     -                           default
tank  health                      ONLINE                      -
tank  guid                        7114201213380460051         default
tank  version                     -                           default
tank  bootfs                      -                           default
tank  delegation                  on                          default
tank  autoreplace                 off                         default
tank  cachefile                   -                           default
tank  failmode                    wait                        default
tank  listsnapshots               off                         default
tank  autoexpand                  off                         default
tank  dedupditto                  0                           default
tank  dedupratio                  1.00x                       -
tank  free                        31.7G                       -
tank  allocated                   56.5K                       -
tank  readonly                    off                         -
tank  ashift                      0                           default
tank  comment                     -                           default
tank  expandsize                  -                           -
tank  freeing                     0                           default
tank  fragmentation               0%                          -
tank  leaked                      0                           default
tank  feature@async_destroy       enabled                     local
tank  feature@empty_bpobj         enabled                     local
tank  feature@lz4_compress        active                      local
tank  feature@spacemap_histogram  active                      local
tank  feature@enabled_txg         active                      local
tank  feature@hole_birth          active                      local
tank  feature@extensible_dataset  enabled                     local
tank  feature@embedded_data       active                      local
tank  feature@bookmarks           enabled                     local
root@debian:~# uname -a
Linux debian 3.2.0-4-amd64 #1 SMP Debian 3.2.65-1+deb7u1 x86_64 GNU/Linux

As you can see, all the features are there.

 

[Herman Eggink]

Excellent. Thanks. Will give it a go. Ubuntu never saw the disk in the first place so hope this helps.

 

[SirMaster]

Eventually Ubuntu will have packages for the latest code. Maybe they already do. Otherwise for HEAD you build that from the git repository. ZFS On Linux 0.6.4 is going to be the "release" version that will have all this code in a few months or so.

 

[cyberjock]

You can import FreeNAS 9.3 pools in Linux if you install the HEAD version of ZoL. ZoL has working support for every available feature flag right now. The Linux guys have been very committed to keeping up with development of OpenZFS. The latest embedded_data feature flag actually got added to Linux before FreeBSD.

I'm not sure for Ubuntu, but for Debian it's easy.
http://zfsonlinux.org/debian.html

"If you wish to use the Debian GNU/Linux ZoL snapshot packages (pre-releases), you will need to uncomment the line(s) related to the 'dailies' repository (at the end of the /etc/apt/sources.list.d/zfsonlinux.list file) by removing the hash mark at the beginning of the line."

Code:

root@debian:~# zpool get all
NAME  PROPERTY                    VALUE                       SOURCE
....
tank  feature@async_destroy       enabled                     local
tank  feature@empty_bpobj         enabled                     local
tank  feature@lz4_compress        active                      local
tank  feature@spacemap_histogram  active                      local
tank  feature@enabled_txg         active                      local
tank  feature@hole_birth          active                      local
tank  feature@extensible_dataset  enabled                     local
tank  feature@embedded_data       active                      local
tank  feature@bookmarks           enabled                     local
root@debian:~# uname -a
Linux debian 3.2.0-4-amd64 #1 SMP Debian 3.2.65-1+deb7u1 x86_64 GNU/Linux

As you can see, all the features are there.

Umm.. that's not all of them...

Code:

# zpool get all tank
NAME  PROPERTY  VALUE  SOURCE
....
tank  feature@async_destroy  enabled  local
tank  feature@empty_bpobj  active  local
tank  feature@lz4_compress  active  local
tank  feature@multi_vdev_crash_dump  disabled  local
tank  feature@spacemap_histogram  disabled  local
tank  feature@enabled_txg  disabled  local
tank  feature@hole_birth  disabled  local
tank  feature@extensible_dataset  disabled  local
tank  feature@embedded_data  disabled  local
tank  feature@bookmarks  disabled  local
tank  feature@filesystem_limits  disabled  local
tank  feature@large_blocks  disabled  local

I could use large_blocks, filesystem_limits, and multi_vdev_crash_dump in my FreeNAS machine right now. AFAIK none of those are supported on Linux at all yet. Not even in HEAD.

Several people have made claims that something was in the linux zfs implementation before FreeBSD, but I have not seen any evidence of that yet. In fact, in the past several people have said that they were very upset when they tried to take their pool to linux and linux didn't support all of the flags. So I'm not sure where these rumors are coming from, but I'm calling them rumors unless some can throw some hard facts around. ;) There have been no confirmed cases of someone taking any then current ZFS feature flags version from a FreeNAS box and putting it on a then current version of linux and have it work (except as read-only for 9.2.0 I believe).

 

[SirMaster]

Sorry I meant to say FreeNAS, not FreeBSD. I meant that *all* the flags in the latest FreeNAS are in ZFS on Linux thus the OP shouldn't have any problems using his FreeNAS pools on Linux.

 

[cyberjock]

Sorry I meant to say FreeNAS, not FreeBSD. I meant that *all* the flags in the latest FreeNAS are in ZFS on Linux thus the OP shouldn't have any problems using his FreeNAS pools on Linux.

Didn't I just disprove that above? I showed your output from Debian and my output from FreeNAS. FreeNAS has more than Debian. Therefore a pool from FreeNAS may not mount in Linux.

I'm also confused on your comment that you meant to say FreeNAS and not FreeBSD. FreeNAS has the same feature flags as FreeBSD. So you could use them interchangeably and still be the same sentence. ;)

 

[SirMaster]

Well, FreeBSD has different feature flag compatibility at different times and it depends if you are talking about RELEASE vs CURRENT vs STABLE as they can have different flag compatibility at different times in their development since ZFS is integrated into the kernel. I know FreeNAS likes to incorporate the newest ZFS code possible too when they do a release since that's the primary purpose of FreeNAS.

Before I posted though I created a pool with data on FreeNAS 9.3 and made sure that it imported and worked read/write on my Linux test box and indeed it did. Those other flags are not activated by default then or else it would complain of the incompatible features.

For example, large_blocks is something set at pool creation time and the FreeNAS GUI doesn't allow for setting this (so it gets set to default) which is still compatible with all the other implementations without the flag.

Also I'm not aware of creating filesystem_limits on FreeNAS GUI or enabling multi_vdev_crash_dump (not sure why a FreeNAS user would ever use this as it's for debugging pools) which are the only other flags not in Linux at the moment so you shouldn't be able to create or get a pool that's incompatible with Linux from FreeNAS unless you use the CLI which is not recommended usage.

So yes I overlooked the actual absolute flag support between the operating systems, but I was sure to test the actual default pool support which for practicality in the case of the OP is what he seemed to be concerned with.


Also FWIW large_blocks and filesystem_limits are already ported and you can merge and build it yourself if you really want/need the support now. The pulls from Illumos for the latest features have been speedy from what I have been seeing and don't lag over BSD pulls by more than a couple days or week or two at the most.

https://github.com/zfsonlinux/zfs/pull/2865

All of the flags you listed from your FreeNAS output will be available in the ZoL release version 0.6.4 which is the next stable release version pretty soon.

 

[cyberjock]

Well, FreeBSD has different feature flag compatibility at different times and it depends if you are talking about RELEASE vs CURRENT vs STABLE as they can have different flag compatibility at different times in their development since ZFS is integrated into the kernel. I know FreeNAS likes to incorporate the newest ZFS code possible too when they do a release since that's the primary purpose of FreeNAS.

Before I posted though I created a pool with data on FreeNAS 9.3 and made sure that it imported and worked read/write on my Linux test box and indeed it did. Those other flags are not activated by default then or else it would complain of the incompatible features.

For example, large_blocks is something set at pool creation time and the FreeNAS GUI doesn't allow for setting this (so it gets set to default) which is still compatible with all the other implementations without the flag.

Also I'm not aware of creating filesystem_limits on FreeNAS GUI or enabling multi_vdev_crash_dump (not sure why a FreeNAS user would ever use this as it's for debugging pools) which are the only other flags not in Linux at the moment so you shouldn't be able to create or get a pool that's incompatible with Linux from FreeNAS unless you use the CLI which is not recommended usage.

So yes I overlooked the actual absolute flag support between the operating systems, but I was sure to test the actual default pool support which for practicality in the case of the OP is what he seemed to be concerned with.


Also FWIW large_blocks and filesystem_limits are already ported and you can merge and build it yourself if you really want/need the support now. The pulls from Illumos for the latest features have been speedy from what I have been seeing and don't lag over BSD pulls by more than a couple days or week or two at the most.

https://github.com/zfsonlinux/zfs/pull/2865

All of the flags you listed from your FreeNAS output will be available in the ZoL release version 0.6.4 which is the next stable release version pretty soon.

No, 9.3 creates pools with all feature flags support enabled/active. FreeNAS has always defaulted to creating a pool with a given version of FreeNAS with all flags enabled/active. This has been a recurring complaint for linux users (and is why I'm so familiar with FreeNAS' ZFS setups...)

I just upgraded my zpool on my main box. Prior it was v500 and I had manually enabled just 3 flags...

Code:

[root@zuul] ~# zpool get all tank
NAME  PROPERTY  VALUE  SOURCE
...
tank  fragmentation  -  -
tank  feature@async_destroy  enabled  local
tank  feature@empty_bpobj  active  local
tank  feature@lz4_compress  active  local
tank  feature@multi_vdev_crash_dump  disabled  local
tank  feature@spacemap_histogram  disabled  local
tank  feature@enabled_txg  disabled  local
tank  feature@hole_birth  disabled  local
tank  feature@extensible_dataset  disabled  local
tank  feature@embedded_data  disabled  local
tank  feature@bookmarks  disabled  local
tank  feature@filesystem_limits  disabled  local
tank  feature@large_blocks  disabled  local

Note that only async_destroy, empty_bpobj, and lz4_compress were active.

The WebGUI gives you a nasty warning in the WebGUI to warn you to upgrade. So I did the upgrade from the WebGUI and this is what I have now...

Code:

zpool get all tank
NAME  PROPERTY  VALUE  SOURCE
...
tank  fragmentation  5%  -
tank  leaked  0  default
tank  feature@async_destroy  enabled  local
tank  feature@empty_bpobj  active  local
tank  feature@lz4_compress  active  local
tank  feature@multi_vdev_crash_dump  enabled  local
tank  feature@spacemap_histogram  active  local
tank  feature@enabled_txg  active  local
tank  feature@hole_birth  active  local
tank  feature@extensible_dataset  enabled  local
tank  feature@embedded_data  active  local
tank  feature@bookmarks  enabled  local
tank  feature@filesystem_limits  enabled  local
tank  feature@large_blocks  enabled  local

So yeah.. FreeNAS defaults to using all feature flags supported and warns you if they aren't all enabled/active. Do you have a pool that is not giving this warning in the FreeNAS WebGUI but also isn't using all flags? That would be bug-ticket worthy.

Also.. look at the fragmentation. It used to not be indicated.. but now it is. Wee!

So I'm back to my original statement that ZFS in linux does not support all the feature flags in ZFS for FreeNAS 9.3. Therefore the pool cannot be mountable in linux in a writeable situation (and may not be mountable read-only). Some of the flags, if enabled/active, will still let you mount the pool read-only.

In the past this has always been the case. When FreeNAS 9.2 came out it supported flags that linux didn't support at that time. The same is true for FreeNAS 9.1, FreeNAS 8.3.x and even 8.2.x.

I am confused because you said that you did it, but I have clear evidence that it shouldn't have. Just from my output of an updated pool on FreeNAS and the flags you listed in your output.

Just for the record, I'm not trying to argue that you are wrong (I think you are but I'm not trying to argue it). I'm trying to get evidence that this can work aside from just someone saying it will because right now my evidence is that it is impossible to work. I'm just too lazy to make a pool on FreeNAS 9.3 and then try to use it in Debian. Not to mention I've gotten my butt handed to me with every attempt to get ZFS to work as a boot device on Debian. :p

 

[SirMaster]

Yes FreeNAS GUI will "enable" all available features for the user when you create or upgrade a pool, but that's not what matters here. If you look at your pool after you upgraded you will see that the 1 flag that is not yet ported to Linux (multi_vdev_crash_dump) as well as the other 2 other flags (filesystem_limits, large_blocks) that are not available from any prebuilt package (but can be built manually since they are ported and complete) are simply "enabled". This is very different from "active" which is when they are actually being used and have started to affect data in your pool.

It appears you don't understand the difference, so allow me to enlighten you.

Feature properties can have one of three values:
○ disabled - this feature will not be used, no on-disk format changes will be made (backwards compatible).
○ enabled - this feature will be used, no on-disk format changes have been made yet, but the software may make them at any time (still backwards compatible).
○ active - this feature has made backwards incompatible on-disk format changes to the pool.

If you also know about how these 3 particular flags (multi_vdev_crash_dump, filesystem_limits, large_blocks) operate and are used then you will also know that they won't become active on their own through normal usage of your zpool, and certainly not through the FreeNAS GUI available as of 9.3. Since you guys always say "never to use the command line" I figured it was fair to say that the pool will remain in a fully supported state. That is at least until the next update of FreeNAS which could obviously change things, but I was just talking about the current version because nobody can really predict the future about what is going to happen between compatibility.

Just for the record, I'm not trying to argue that you are wrong (I think you are but I'm not trying to argue it). I'm trying to get evidence that this can work aside from just someone saying it will because right now my evidence is that it is impossible to work. I'm just too lazy to make a pool on FreeNAS 9.3 and then try to use it in Debian.

I am confused because you said that you did it, but I have clear evidence that it shouldn't have.

You might think you have "clear evidence" but you don't. That's fine you aren't arguing, but I don't know why you would think I was wrong when I clearly stated that I created a pool in FreeNAS 9.3 and imported it into Debian and made sure it fully worked. What possible motivation could I have to lie? The only other way is if you think I made some mistake in my "test", but honestly it seemed like a pretty trivial test to someone like me. I know I hold my peers to a higher regard than that.