USB external harddrive failure

[stsostso]

Hi,

I want to backup data from my freenas to external harddisks. I use one external WD 2 TB harddisk and one WD 3 TB harddisk. The 2TB disk works just fine, with the 3 TB disk I am having problems. When copying data onto the drive using rsync (shell mode), the systems stalls after a few minutes and the volume with the external harddisk's status is "UNKNOWN". I have rebooted the system and recreated the volume but the same problem happened again. Here is the report I get when clicking on "Edit ZFS Options" on the broken volume:

Environment:

Software Version: FreeNAS-8.3.1-BETA3-x64 (r13264M)
Request Method: GET
Request URL: http://192.168.0.36/storage/volume/zfs-edit/7/


Traceback:
File "/usr/local/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
105. response = middleware_method(request, callback, callback_args, callback_kwargs)
File "/usr/local/www/freenasUI/../freenasUI/freeadmin/middleware.py" in process_view
166. return login_required(view_func)(request, *view_args, **view_kwargs)
File "/usr/local/lib/python2.7/site-packages/django/contrib/auth/decorators.py" in _wrapped_view
20. return view_func(request, *args, **kwargs)
File "/usr/local/www/freenasUI/../freenasUI/storage/views.py" in zfsvolume_edit
513. volume_form = forms.ZFSVolume_EditForm(mp=mp)
File "/usr/local/www/freenasUI/../freenasUI/storage/forms.py" in __init__
1094. self.fields['volume_compression'].initial = data['compression']

Exception Type: KeyError at /storage/volume/zfs-edit/7/
Exception Value: 'compression'

Does anybody have any idea what this means or how to fix my problem? Is the harddisk broken? How would I find out if it were?

Thanks for the help

Stephan

 

[cyberjock]

It does sound like you have a bad drive. The problem with USB(and one of the reasons why it is highly recommended that you don't use USB drives except to copy data to the server) is that if the drive has any problems, its typical for the drive to "disconnect" from USB and possibly "reconnect" later. There is no way to prevent this, but when the drive disconnects you lose any data that is in RAM but not written to the drive. If you read, lots of people have tried USB, and lots thought they had good trustworthy backups until they tried to use it. Then, oops, their backups wouldn't work.

 

[stsostso]

That does not sound very encouraging. Do I also lose data if I use rsync? I thought if I used rsync twice, it would check if there is any data missing and then recopy that data.

Is there a certain way to check if the harddisk is broken?

 

[cyberjock]

Do I also lose data if I use rsync? I thought if I used rsync twice, it would check if there is any data missing and then recopy that data.

If you are copying data to the drive, the rsync task will likely abort with an error that the device is no longer available. If you are trying to rsync data from the drive, the task will likely do the same thing. In either case, you will have to come up with a manual method to recover your data from the drive. Considering the question you are asking, you may find that you will have significant difficulty trying to get your data off of a failing USB disk(if you can at all).

Is there a certain way to check if the harddisk is broken?

You can attempt to run SMART tests on the drives(both short and long). Note that these are NOT 100% ways to verify that a drive is good. Even if both drives are good I question the logic in continuing to use USB drives since a single bad sector can spell total havoc for your data. Being that you are asking these questions you are fairly new to FreeNAS/FreeBSD, so recovery for you would be extremely difficult(if even possible) since you aren't familiar with any methods for recovery nor how to use them correctly. Additionally, because you are likely new to FreeNAS/FreeBSD if your USB drives are good right now but start failing in the future you may not even be able to recognize it until you try to recover your data and figure out that you can't. This is why anyone mentioning using USB drives in the forums is given the answer of "stop using USB.. it likely won't end well for you".

My advice would be to give up trying to use USB drives with FreeNAS and use either a separate machine for backups or use internal drives in your FreeNAS machine for backups. Each option has pluses and minuses and I won't go into discussing them here.

 

[stsostso]

I want to avoid having to buy all the parts for another FreeNas System. Would an external hardisk attached via eSata solve the problem about the data being lost during the backup?

 

[cyberjock]

Not sure. Nobody really uses external drives because of the stuff I've mentioned above plus the fact that people tend to accidentally unplug the power supply and/or power cords. Internal drives are typically protected from accidental unplugging, but external drives aren't. External drives in any form just aren't recommended.

It's really about how much your data is worth. If you don't value your data enough to be "worth" building a proper system and choose to use USB or whatever, that's your choice. Lots of people thought they could handle it and lost their data.

 

[stsostso]

Thank you far all the information. But I'm still not sure what to do.

The data is very valuable (tons of pictures). That's why I also want to store it on an external drive that can be unplugged from the system and stored in a fireproof safe. Since the data does not change very often, the idea was to perform a backup every few weeks and synchronize the data from the FreeNas and the external drive using rsync.

Inside the FreeNas I have installed 3 drives and I use RAID-Z1. To my understanding, the data is still safe should one of the drives fail.

My question is, if the connection via eSATA would be safe enough for that purpose. I will make sure that the drive will not be unplugged during the backup process. Does eSATA have the same problems as USB that you mentioned earlier where the USB drive just disconnects from the system during the backup process?

 

[cyberjock]

RAIDZ1 will only protect you from a single disk failure. If you have bad sectors or read errors on any other disk when your one drive fails, you will lose data. This has happened frequently to users on the forum, which is why I recommend nothing less than RAIDZ2. There's even an article from 2008-ish titled something like "RAID5 will die in 2009" because of the error rates of hard drives.

 

[stsostso]

OK. Thank you. I will change it to RAIDZ2.

Do you have any more information about the eSATA vs USB problem? I could not find any information about this. But I find it hard to believe that I am the only on who wants to backup his data on a drive that can be stored in a safe location away from the server.

 

[cyberjock]

OK. Thank you. I will change it to RAIDZ2.

Do you have any more information about the eSATA vs USB problem? I could not find any information about this. But I find it hard to believe that I am the only on who wants to backup his data on a drive that can be stored in a safe location away from the server.

Yes, but I've explained it every week for the last 2 months that I'm really not interested in explaining it again. It's a dangerous risk(one I'd never take nor recommend). It's your data, and if you think its worth the risk then go for it. At the end of the day I won't miss any sleep if you lose your data.. you will.