Bug: Importing Ext2/3 or 4 doesn't work

[gwaitsi]

I am trying to move my data from an OMV install.

I tried directly with the original data drive (ext4), but it failed with an error.
I created a transfer drive with ext3 and it too failed with the same error.
I then created a transfer drive with ext2 and it has failed as well.

I keep getting
Error: Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/middlewared/job.py", line 349, in run
await self.future
File "/usr/local/lib/python3.7/site-packages/middlewared/job.py", line 385, in __run_body
rv = await self.method(*([self] + args))
File "/usr/local/lib/python3.7/site-packages/middlewared/schema.py", line 961, in nf
return await f(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/middlewared/plugins/pool.py", line 2027, in import_disk
async with MountFsContextManager(self.middleware, device, src, fs_type, fs_options, ["ro"]):
File "/usr/local/lib/python3.7/site-packages/middlewared/plugins/pool.py", line 283, in __aenter__
await mount(self.device, self.path, *self.args, **self.kwargs)
File "/usr/local/lib/python3.7/site-packages/middlewared/plugins/pool.py", line 142, in mount
output[1].decode("utf-8"),
ValueError: Mount failed (exit code 1):
mount: /dev/ada2p1: Invalid argument

 

[gwaitsi]

well, it is a bit hard for it to work when basic mount commands don't work.

mount -t ext2fs /dev/ada2p1 /mnt/data_volume
mount: -t ext2fs /dev/ada2p1: Invalid Arguement

from the console i see
Aug 23 18:03:58 nas ext2fs: ada2: wrong magic number 0 (expected 0xef53)
Aug 23 18:04:44 nas WARNING: mount of ada2p1 denied due to unsupported optional features:
Aug 23 18:04:44 nas 64bit

Am i missing something, or was freenas designed to not be plug and go

 

[pschatz100]

Just curious... have you tried using the GUI to import the disks?

Here are some comments from the documentation. Maybe there is a clue here:

The Storage ➞ Import Disk screen, shown in Figure 10.6.1, is used to import disks that are formatted with UFS (BSD Unix), FAT(MSDOS) or NTFS (Windows), or EXT2 (Linux) filesystems. This is a designed to be used as a one-time import, copying the data from that disk into a dataset on the FreeNAS® system. Only one disk can be imported at a time.

Note:
Imports of EXT3 or EXT4 filesystems are possible in some cases, although neither is fully supported. EXT3 journaling is not supported, so those filesystems must have an external fsck utility, like the one provided by E2fsprogs utilities, run on them before import. EXT4 filesystems with extended attributes or inodes greater than 128 bytes are not supported. EXT4 filesystems with EXT3 journaling must have an fsck run on them before import, as described above.

 

[gwaitsi]

Just curious... have you tried using the GUI to import the disks?

Here are some comments from the documentation. Maybe there is a clue here:

yes, that is where the error is generated. the mounts are what i tried after the gui import failed

 

[KrisBee]

You were asked to provide the output of these shell commands in your other thread on this topic, but never did.

Code:

gpart show /dev/ada2
geom PART list ada2

You will see from my post that importing a ext2 disk worked without error in my case: https://www.ixsystems.com/community/threads/moving-from-omv-to-freenas-disaster.87022/post-603321

 

[gwaitsi]

You were asked to provide the output of these shell commands in your other thread on this topic, but never did.

Code:

gpart show /dev/ada2
geom PART list ada2

You will see from my post that importing a ext2 disk worked without error in my case: https://www.ixsystems.com/community/threads/moving-from-omv-to-freenas-disaster.87022/post-603321

i already lost my main data disk with the constant moving backwards and forward. i am using the remaining OMV disks on a linux machine to transfer via NTFS. when all that is finished and the data is confirmed i will do the remaining data from the backup drive.

proves the value of the OMV USB Backup plugin....

 

[KrisBee]

It's a pity you did not ask for advice about how to migrate from OMV to FreeNAS before attempting to use ext4/3 which FreeNAS/FreeBSD does not handle, a fact that is clearly stated in the FreeNAS guide in section 10.6.

If you have enough disks to create a zfs pool with datasets on FreeNAS and mount your OMV ext4 disks on a linux machine, then you could simply load your data onto the zfs pool via a network share.

I don't believe there is a bug in the "import disk" function, perhaps you might change the title of this thread.

 

[gwaitsi]

i used gparted on linux mint to create the ext2 disk and it gave the same errors as the ext4 and ext3 that were both from OMV (debian).
In hindsight, you are probably right. should have gone for the network copy, but i figured importing from disk would have been quicker..

 

[KrisBee]

The "import disk function" dumps the imported data into a single dataset on zfs pool. So there's no chance to selectively load data on the zfs pool into different datasets on your zfs pool and organise your data as you load it.

 

[gwaitsi]

The "import disk function" dumps the imported data into a single dataset on zfs pool. So there's no chance to selectively load data on the zfs pool into different datasets on your zfs pool and organise your data as you load it.

not sure i get the relevance here. my data is/was structured in a way it can be straight imported - which is what i am doing, only with an NTFS partition (which is causing a problem because i have filenames that are not supported on windows)

 

[KrisBee]

So have you created your "pool" in FreeNAS and have you created a dataset in that pool to hold the imported data? All your imported data will be in this single dataset hopefully reflecting the file/dir structure on the disk you import from. But depending what you want to do with your data, and how you want to access it, you could now be faced with re-organising it again into different datasets. Have you used zfs before? If not, you need to give some thought on how to make best use of datasets within your pool ( at least view FreeNAS guide section 10.2.10 ).

One obvious snag with the "import data" into one dataset function is that you have to choose if the dataset is of the "generic" or "smb" share type which reflects the intention to share the sata via NFS or SMB. One of the dataset properties - case sensitivy - set at this time is immutable.

 

[gwaitsi]

So have you created your "pool" in FreeNAS and have you created a dataset in that pool to hold the imported data? All your imported data will be in this single dataset hopefully reflecting the file/dir structure on the disk you import from. But depending what you want to do with your data, and how you want to access it, you could now be faced with re-organising it again into different datasets. Have you used zfs before? If not, you need to give some thought on how to make best use of datasets within your pool ( at least view FreeNAS guide section 10.2.10 ).

One obvious snag with the "import data" into one dataset function is that you have to choose if the dataset is of the "generic" or "smb" share type which reflects the intention to share the sata via NFS or SMB. One of the dataset properties - case sensitivy - set at this time is immutable.

your description is correct.
I have basically three folders under the data_volume

data_disk (pool)
home (data set) - nfs & windows clients
media (data set) - nfs & cifs & windows clients
photos (data set) - nfs & cifs & windows clients
surveillance_disk (pool)
onvif (dataset)

I loaded the data in the correct folders so it would be imported into each

 

[KrisBee]

Am I correct in thinking you have only two HDDs in your FreeNAS box and have created two separate pools, each with one HHD? If so, then you have no redundancy and this is not an effective use of zfs.

 

[gwaitsi]

Am I correct in thinking you have only two HDDs in your FreeNAS box and have created two separate pools, each with one HHD? If so, then you have no redundancy and this is not an effective use of zfs.

Correct. in OMV, i use a USB3 ext and make a backup with ext HDDs.
i had a bad experience with raid on OMV. Now i just use USB backups.

They have a very nice USB Backup plugin which is great with an easy gui interface for setting up rync jobs as soon as the USB is plugged in.

 

[ornias]

@gwaitsi Why are you creating a new topic, while your previous one clearly stated that people where able to import but your disk was sending the wrong superblock (not ext2)?

It seems you are trying to cheat the forum support here and I do not like such an attitude.

 

[KrisBee]

By "raid in OMV", I take it you mean just standard linux mdadm ( no LVM on top or mergerfs/snapraid combo) on a basic two HDD mirror, but then opted to just rsync one ( or more?) HDD to another another USB3 external HDD rather have real-time updates via RAID.

If you've turned to FreeNAS in order to use zfs advanced features ( a combo of filesystem and logical volume manager, copy on write features, etc.) and for data integrity, then with just HDDs your pool ought consist of a single mirror vdev. @gwaitsi I think you should be reading more about zfs in order to make best use of FreeNAS. Maybe this as a start: https://www.ixsystems.com/community/resources/introduction-to-zfs.111/

 

[gwaitsi]

By "raid in OMV", I take it you mean just standard linux mdadm ( no LVM on top or mergerfs/snapraid combo) on a basic two HDD mirror, but then opted to just rsync one ( or more?) HDD to another another USB3 external HDD rather have real-time updates via RAID.

If you've turned to FreeNAS in order to use zfs advanced features ( a combo of filesystem and logical volume manager, copy on write features, etc.) and for data integrity, then with just HDDs your pool ought consist of a single mirror vdev. @gwaitsi I think you should be reading more about zfs in order to make best use of FreeNAS. Maybe this as a start: https://www.ixsystems.com/community/resources/introduction-to-zfs.111/

yeh, i had mdadm mirroring and ended up loosing everything without a backup. Still have recovered files to sort through and that was years ago.
Have been using USB Backup with rsync jobs since, and it hasn't failed me. Happy for experience & advice, but i definitely want to retain offsite backup in fire safe. So being able to backup on USB is a must.

I basically have 4 drive slots;
1 x 4TB for the security cameras (so i am not concerned about backup for that disc

1 x 8TB for
- movie / music collection which takes most of the storage
- family photos
- sho documents

the photos and docs are the most important to me.
i will take a look at the article, but what are your suggestions for a small HO setup.

 

[ornias]

@gwaitsi As told to you before in your previous thread, which you seem to have abandoned, you are trying to use FreeNAS/TrueNAS in the same way as you previously used OMV. Thats often not how FreeNAS/TrueNAS is supposed to be used.

- Use ZFS
- If you want to use backups, use ZFs replication
- Don't use single disks, just use a mirror, RaidZ2 or RaidZ3

FreeNAS/TrueNAS is not designed around backups to an usb drive. You might be able to get it to work, but if you want solid backups just use replication and drop forcing Ext2 or external drives onto the system. You want solid backups and there are no guarantees your rsync strategy is going to work out in the end.

 

[gwaitsi]

@gwaitsi As told to you before in your previous thread, which you seem to have abandoned, you are trying to use FreeNAS/TrueNAS in the same way as you previously used OMV. Thats often not how FreeNAS/TrueNAS is supposed to be used.

- Use ZFS
- If you want to use backups, use ZFs replication
- Don't use single disks, just use a mirror, RaidZ2 or RaidZ3

FreeNAS/TrueNAS is not designed around backups to an usb drive. You might be able to get it to work, but if you want solid backups just use replication and drop forcing Ext2 or external drives onto the system. You want solid backups and there are no guarantees your rsync strategy is going to work out in the end.

the ext2 was an attempt to get the data from the OMV disks to freenas disks.
It failed as i indicated, despite being formatted on a completely different system as OMV.
In any case, i formatted an NTFS drive and have passed the data through that to freenas.

i need a backup to a device that can put be put in a firesafe.
That means either a device from the removable bay, or a USB3 device.

I don't need to mirror the drive, i need to backup the data sets, and i don't have the cash to lash out on a string of same sized drives

 

[jgreco]

Thread closed; please do not create what are effectively duplicate topics. Please continue this discussion at

https://www.ixsystems.com/community/threads/moving-from-omv-to-freenas-disaster.87022/