UDF support for backups

[no_connection]

With UFS leaving the building it would be nice to have a practical, recoverable, portable and hotpluggable file system for taking backups to or other uses.

Yes it has limitations but the only other format as wildly supported is FAT32 but as that has a bad file size limit it's not an option.
UDF had traditionally been used for CD and DVD but is capable of much more, and as it is supported on almost every OS by default, it is suitable for backup as you can recover it from almost every OS. And it has tool in case the FS gets damaged by time.

As a side note, you could have a USB or eSATA dock next to the nas, plugging in a drive and have it shared automatically through CIFS by ether a config file on the disk or manually in GUI. Leaving you with a practical way to use single disks with hot plug.

ZFS would not be an option for the same abuse.

Not exactly an enterprise request but I can see me using it for home use.
Any merit to this idea?

 

[cyberjock]

Yes and no.

First, we are limited to file systems that are supported in FreeBSD.

Second, we are limited to file systems that offer sufficient compatibility to actually be useful. For example, doing ZFS snapshots to a non-ZFS file system is virtually a no-go.

Third, we'd have to all agree on what that file system is.

The problem is that FreeNAS is targeted for very large systems (think 20TB+ with 100TB+ as not being abnormal). So what file systems are going to be compatible with that range of size? Being that you are also guaranteed to need multiple disks as soon as you want something bigger than 6TB you are instantly looking at ZFS-RAID or hardware-RAID options exclusively. Since requiring hardware-RAID is not the most economical I just gave a natural logical progression for your "alternative" options from everything that possibly exists to just ZFS. See the problem more clearly now?

Unfortunately when you start talking about alternative options none offer the necessary power to be useful to most users. Sure there are options for very limited situations but in terms of something that is all-encompassing your only "good" options are ZFS and ZFS. :/

Once you start scaling up to the scale of doing ZFS it's very hard to scale back down to something "less than ZFS". For most people the best options for backups if you aren't willing to do a second box (or at least a second pool) for ZFS snapshot and replication is pretty much whatever homebrew option you want to do. Be it xcopy, rsync, or whatever.

So yeah, if you can provide a recommendation that actually fits our needs and wants then I'm sure the developers would be willing to add it. The problem is that every file system I'm familiar with has major drawbacks or otherwise isn't going to fit into a category that is "useful for most users".

 

[no_connection]

UDF is already supported by BSD/FreeBSD and already implemented according to #307
Being targeted for very large systems does not mean home users aren't using it.
Transmission and Plex isn't exactly large business targeted features.

http://en.wikipedia.org/wiki/Universal_Disk_Format
http://en.wikipedia.org/wiki/Comparison_of_file_systems

16EB file size means the FS handles at least that disk size, so it would take HDDs quite a while to catch up.
The intent is to use single disks with the ability to pull and push disks in all day without the risk of loosing a pool or destroying the running system.
Something ZFS can't do, and as I recall you also advised against pulling and mounting ZFS pools on a running system.

The biggest features I see needed is file size and file name.

Yes there will always be issues or problems when you want to use less feature rich FS. But the admin/operator should already be aware of such limitation and select the intended use accordingly. Hardware RAID would probably not be the intended.
ZFS snapshot have nothing to do with it as it makes no sense dumping that to a non ZFS FS.
I want an option to NOT use a hard to recover solution for backup, even if it involves just dumping files onto a harddrive. Yes permissions might be left out or files, but as long as you can share it with CIFS you *should* be able to dump it to UDF.

It can absolutely be done through external machine, but it would be nice to have the feature built in.

A config file on the disk could instruct FN to copy the contents of some folders (that you know will fit) onto the disk automatically. You get an email when it starts and an email when it is done.
You could also have the config mount the drive as a CIFS share allowing you to dump data to it through network or access your achieved movie collection that is too big to fit the NAS.
I can see me doing both.
And if there could be an option to generate SHA256 checksum of files or folders it would be golden.

From what I can see ZFS can't keep up with the above intended use, and it can definitely not be plugged in to a windows machine reading files immediately.
UFS can't ether for that matter.

 

[Arwen]

I like the idea better than FAT-32 or UFS, (which is going away).
This would be good, as I can read / write on Linux without any
ZFS export / import issues. (ZFS on Linux would be problematical
as it should support the same pool feature set FreeNAS to useful.)

 

[no_connection]

Made feature request #5539

 

[Arwen]

no_connection,

Thanks. I was wondering what I would use for local
backups and transfers. UDF would help quite bit.

 

[cyberjock]

Looks like the developers have spoken (again)... not to be fixed on ticket 5539.

 

[no_connection]

I know, the request was sound, unfortunately not easily implementable long term.
Even if they add UDF to GUI there is no guarantee it would work months or years from now which I guess is the reason for not pursuing it.
Adding features would not be the problem in this instance, maintaining them would. I guess you can't blame the developers for wanting less things to worry about.

Unfortunately ZFS is not a replacement in this case as it won't work on windows, or mac, or linux. I even doubt FreeBSD would be plug and play with volumes from FreeNAS.
Your data is as locked in as any other proprietary format on a NAS. Only external means of backup is left.

 

[cyberjock]

No, you can *definitely* use it in FreeBSD and linux as I've done importing on both of those. Your limitation is feature flags and such and that's something that you have control of. If you don't keep "upgrading" your pool then the other OSes will catch up to your pool.

This is not a "bug" or "feature" with FreeNAS. This is nothing more than FreeNAS' development cycle and development resources allowing FreeNAS to jump ahead of other OSes.

 

[no_connection]

Well, I saw no such flags when I created my pool, and I saw no way to choose ZFS version ether. A compatibility checkbox perhaps?
Can I format a USB drive with ZFS from FreeNAS and expect it to work on most Linux and FreeBSD systems, and it being plug and play meaning no mucking about in CLI?
If no then I would consider it not portable enough.
It has nothing to do with development cycle, if there is no widespread support for ZFS now then I need something else for backup and transferring of files.
I was hoping to have it inside FreeNAS, even the developers tried UDF but it simply was not doable or the effort would be too high.

It is nothing against FreeNAS or ZFS as I happily let it harbor my files and serving them across the network. I would not stick around otherwise.

 

[cyberjock]

Freenas uses the latest supported on freenas. There is no way to create pools with features disabled or use old versions. If you want to do that you have to use a version of free as that only does the version and flags that you want .