Fr33nasnoobi3
Dabbler
- Joined
- Mar 11, 2017
- Messages
- 19
I don't even know where to begin with this FreeNAS issue that I'm having so I guess I'll start from the very beginning and hope the puzzle pieces fall into place. So a few days ago I got an alert that one of the volumes (named data2 unencrypted and no valuable data on) had a drive "disconnected" and was running in a degraded state. I thought that was very odd because this things been running flawlessly for several years aside from the 3 hard drives that have failed over time, alerts of errors on the drive> replaced drive> re-silvered everything ok again.
So I just treated this as a failed hard drive and shutdown the system, replaced the drive and brought it back up to resilver now NONE of the drives in that volume show up at all... keep in mind this is 6x4tb drives in raidz2! So I'm thinking to myself that the little 5.25 to 5x3.5" hard drive adapter (silverstone FS305) for some reason took a dump and I proceeded to remove 6 of the drives from my locked (data1) important files volume that this whole time I had not tried to unlock by the way) Just to see if my theory was right about the silverstone drive adapter dying. So I removed the drives from the data1 volume and sat them aside plugged the data2 drives into the other enclosures booted up and it saw all the drives even the one it thought became disconnected. So problem solved right? WRONG ...
I shutdown replaced the drives I took out in a new enclosure a few days later when the new enclosure came in. Booted up data2 had 2 disconnected drives this time and data1 failed to unlock :( So at this point I figured something is wrong with the install. Pulled boot drives out set them aside got a few new usb drives reloaded freenas yada yada couldn't import data1 or data2... *panic setting in* Now I only know enough about unix/linux/bsd commandline stuff to be considered mildly retarded but know enough to not do anything to bone headed to jeopardize my data. So things I Have tried... removing the enclosures completely to eliminate that possibility, swapped HBA controller cards, swapped sas to sata cables, swapped motherboards (to an identical backup motherboard because redundancy right?), swapped ram, swapped power supplies, removed hba controller cards entirely and used on-board sata connectors, swapped to new sata cables....
So now as it sits HBA cards have been removed I don't have any data i care about on data2 drives so they have been removed. I'm running on the new motherboard directly sata connected to mobo, with original boot disks and this is the error I get when I try to unlock data1
keep in mind that just yesterday all the drives on a new installation came up all online on both raidz2-0 and raidz2-1 and I get random drives that become unavailable not the same ones all the time! :/
and only 2 drives in the raidz2-0 where unavailable yesterday on the old installation and as I understand 2 should be the max I could lose without losing data...
Please any help would be greatly appreciated and if there is a way of mounting and unlocking just the raidz2-1 WITHOUT possibly compromising the data in the volume how do you do that?
So I just treated this as a failed hard drive and shutdown the system, replaced the drive and brought it back up to resilver now NONE of the drives in that volume show up at all... keep in mind this is 6x4tb drives in raidz2! So I'm thinking to myself that the little 5.25 to 5x3.5" hard drive adapter (silverstone FS305) for some reason took a dump and I proceeded to remove 6 of the drives from my locked (data1) important files volume that this whole time I had not tried to unlock by the way) Just to see if my theory was right about the silverstone drive adapter dying. So I removed the drives from the data1 volume and sat them aside plugged the data2 drives into the other enclosures booted up and it saw all the drives even the one it thought became disconnected. So problem solved right? WRONG ...
I shutdown replaced the drives I took out in a new enclosure a few days later when the new enclosure came in. Booted up data2 had 2 disconnected drives this time and data1 failed to unlock :( So at this point I figured something is wrong with the install. Pulled boot drives out set them aside got a few new usb drives reloaded freenas yada yada couldn't import data1 or data2... *panic setting in* Now I only know enough about unix/linux/bsd commandline stuff to be considered mildly retarded but know enough to not do anything to bone headed to jeopardize my data. So things I Have tried... removing the enclosures completely to eliminate that possibility, swapped HBA controller cards, swapped sas to sata cables, swapped motherboards (to an identical backup motherboard because redundancy right?), swapped ram, swapped power supplies, removed hba controller cards entirely and used on-board sata connectors, swapped to new sata cables....
So now as it sits HBA cards have been removed I don't have any data i care about on data2 drives so they have been removed. I'm running on the new motherboard directly sata connected to mobo, with original boot disks and this is the error I get when I try to unlock data1
Code:
Environment: Software Version: FreeNAS-11.1-U4 (89e3d93bc) Request Method: POST Request URL: http://10.10.10.103/storage/volume/1/unlock/?X-Progress-ID=a69edec6-d03c-4e0f-86c7-d2a8e103ff30 Traceback: File "/usr/local/lib/python3.6/site-packages/django/core/handlers/exception.py" in inner 42. response = get_response(request) File "/usr/local/lib/python3.6/site-packages/django/core/handlers/base.py" in _legacy_get_response 249. response = self._get_response(request) File "/usr/local/lib/python3.6/site-packages/django/core/handlers/base.py" in _get_response 178. response = middleware_method(request, callback, callback_args, callback_kwargs) File "./freenasUI/freeadmin/middleware.py" in process_view 162. return login_required(view_func)(request, *view_args, **view_kwargs) File "/usr/local/lib/python3.6/site-packages/django/contrib/auth/decorators.py" in _wrapped_view 23. return view_func(request, *args, **kwargs) File "./freenasUI/storage/views.py" in volume_unlock 1033. form.done(volume=volume) File "./freenasUI/storage/forms.py" in done 2822. raise MiddlewareError(msg) Exception Type: MiddlewareError at /storage/volume/1/unlock/ Exception Value: [MiddlewareError: Volume could not be imported: 1 devices failed to decrypt]
Code:
[root@freenas ~]# zpool status pool: freenas-boot state: ONLINE scan: scrub repaired 0 in 0 days 00:05:23 with 0 errors on Fri Apr 27 03:50:27 2018 config: NAME STATE READ WRITE CKSUM freenas-boot ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 da1p2 ONLINE 0 0 0 da0p2 ONLINE 0 0 0 errors: No known data errors
Code:
[root@freenas ~]# zpool import pool: data1 id: 13473567347727306844 state: UNAVAIL status: The pool was last accessed by another system. action: The pool cannot be imported due to damaged devices or data. see: http://illumos.org/msg/ZFS-8000-EY config: data1 UNAVAIL insufficient replicas raidz2-0 UNAVAIL insufficient replicas 13880681796121711656 UNAVAIL cannot open gptid/d859e08c-be99-11e7-960c-d050992ecffc.eli ONLINE 4943386155760196097 UNAVAIL cannot open gptid/d39fc645-c01b-11e7-960c-d050992ecffc.eli ONLINE 6756959280491151806 UNAVAIL cannot open 5062403531773914712 UNAVAIL cannot open raidz2-1 ONLINE gptid/6d5a535f-e2aa-11e6-81a3-d050992ecffc.eli ONLINE gptid/6e92dd63-e2aa-11e6-81a3-d050992ecffc.eli ONLINE gptid/6fc79c28-e2aa-11e6-81a3-d050992ecffc.eli ONLINE gptid/7103a362-e2aa-11e6-81a3-d050992ecffc.eli ONLINE gptid/723c9664-e2aa-11e6-81a3-d050992ecffc.eli ONLINE gptid/737f80ad-e2aa-11e6-81a3-d050992ecffc.eli ONLINE
keep in mind that just yesterday all the drives on a new installation came up all online on both raidz2-0 and raidz2-1 and I get random drives that become unavailable not the same ones all the time! :/
and only 2 drives in the raidz2-0 where unavailable yesterday on the old installation and as I understand 2 should be the max I could lose without losing data...
Please any help would be greatly appreciated and if there is a way of mounting and unlocking just the raidz2-1 WITHOUT possibly compromising the data in the volume how do you do that?
Last edited by a moderator: