Demogorg0N
Cadet
- Joined
- Apr 19, 2014
- Messages
- 2
hi everyone,
i'm having some major problems on my freenas box.
let me explain what has happened:
this part is on 11.1-U6
due to the recent changes with SickRage, i was left with a download jail that didn't really work,
but since i wanted to move away from the built in plugins & go to a standard jail and run my stuff in there, no problem there.
I made a new standard jail & started installing all my stuff, but ened up with an error of a lib that was only in 11.2
i looked in updates & sure enough, there was a 11.2-stable train.
now, without really looking at what the update would apply, i just hit install...
after the update, the 11.2-beta3 launched, everything looked OK, datastore was available to my ESXi machine, ISCSI volume was active to my MDT server, my SMB shares worked.
the plex jail worked.
basically everything worked flawlessly.
so i did what i always do after a successfull upgrade, i upgraded the pools. Again that worked flawlessly and everything worked.
I then created a new jail with the 11.2-release build, but since it was pretty late already, i started the install & went to sleep.
Now the next day, i woke up and noticed my HUE light didn't work, strange.
noticed the WiFi was down, ok, that explains the light, but why is it down?
since i had to leave for work, i thought well, i'll check it from the office over the openvpn i have running on my pfsense.
tried that, didn't work either...
had to wait till i got home to further check this.
came home, started my workstation, no internet.
hmm, pfsense isn't running, that explains everything.
logged into my ESXi & was greeted with VM's that looked like they didn't have access to their datastore...
browsed the datstore, no data, only the .vSphere-HA folder from when i was playing around with vCenter & a random dump file of a vm from june 15th.
so i tried to login to my freenas box, nothing, an nginx error on the webinterface & timeouts on SSH.
what's going on here...
logged into the IPMI of the server & launched a console.
errors like "INIT can't exec getty"
also completly frozen, could do anything.
timestamps from around 4AM the night before...
reset the server, i could no longer boot from it's 11.2-beta3 boot environment (this was the time i noticed the beta...)
i changed to my 11.6-U6 to see if that booted.
it did, but all the pools were practically empty...
really strange, but i did the pool upgrade, so figured let's reinstall 11.2-beta3 & start again.
so i did, it started again, but again almost all my pools are empty...
they still have their zvol's in them, but most of them have a usage of 100KiB...
but there are a few exceptions:
my Data pool still has it's Datastore zvol (around 9GiB in use, used for ISO's and such) (also all it's recent snapshots are there)
also the ISCSI zvol is still ok, 508GiB in use
my PLEX jail also OK, 35GiB in use.
the datastore pool has 170GiB in use, but these are only snapshots.
and older ones at that (latest one is 4 months old)
the media volume is completly empty (this used to hold around 40TiB of films & series)
and my temp volume is also completly empty (not that bad, only held the downloads till they were processed)
TL:DR: upgraded to 11.2-beta3, crashed, rebooted, most pools are empty...
now my question is, does anybody has any idea what has caused this & if there's a change i could get my data back?
i know i should've had backups, but i could do without it, it's just a pain to collect again, since i've been collecting for over 10 years.
thank in advance,
Thomas
i'm having some major problems on my freenas box.
let me explain what has happened:
this part is on 11.1-U6
due to the recent changes with SickRage, i was left with a download jail that didn't really work,
but since i wanted to move away from the built in plugins & go to a standard jail and run my stuff in there, no problem there.
I made a new standard jail & started installing all my stuff, but ened up with an error of a lib that was only in 11.2
i looked in updates & sure enough, there was a 11.2-stable train.
now, without really looking at what the update would apply, i just hit install...
after the update, the 11.2-beta3 launched, everything looked OK, datastore was available to my ESXi machine, ISCSI volume was active to my MDT server, my SMB shares worked.
the plex jail worked.
basically everything worked flawlessly.
so i did what i always do after a successfull upgrade, i upgraded the pools. Again that worked flawlessly and everything worked.
I then created a new jail with the 11.2-release build, but since it was pretty late already, i started the install & went to sleep.
Now the next day, i woke up and noticed my HUE light didn't work, strange.
noticed the WiFi was down, ok, that explains the light, but why is it down?
since i had to leave for work, i thought well, i'll check it from the office over the openvpn i have running on my pfsense.
tried that, didn't work either...
had to wait till i got home to further check this.
came home, started my workstation, no internet.
hmm, pfsense isn't running, that explains everything.
logged into my ESXi & was greeted with VM's that looked like they didn't have access to their datastore...
browsed the datstore, no data, only the .vSphere-HA folder from when i was playing around with vCenter & a random dump file of a vm from june 15th.
so i tried to login to my freenas box, nothing, an nginx error on the webinterface & timeouts on SSH.
what's going on here...
logged into the IPMI of the server & launched a console.
errors like "INIT can't exec getty"
also completly frozen, could do anything.
timestamps from around 4AM the night before...
reset the server, i could no longer boot from it's 11.2-beta3 boot environment (this was the time i noticed the beta...)
i changed to my 11.6-U6 to see if that booted.
it did, but all the pools were practically empty...
really strange, but i did the pool upgrade, so figured let's reinstall 11.2-beta3 & start again.
so i did, it started again, but again almost all my pools are empty...
they still have their zvol's in them, but most of them have a usage of 100KiB...
but there are a few exceptions:
my Data pool still has it's Datastore zvol (around 9GiB in use, used for ISO's and such) (also all it's recent snapshots are there)
also the ISCSI zvol is still ok, 508GiB in use
my PLEX jail also OK, 35GiB in use.
the datastore pool has 170GiB in use, but these are only snapshots.
and older ones at that (latest one is 4 months old)
the media volume is completly empty (this used to hold around 40TiB of films & series)
and my temp volume is also completly empty (not that bad, only held the downloads till they were processed)
TL:DR: upgraded to 11.2-beta3, crashed, rebooted, most pools are empty...
now my question is, does anybody has any idea what has caused this & if there's a change i could get my data back?
i know i should've had backups, but i could do without it, it's just a pain to collect again, since i've been collecting for over 10 years.
thank in advance,
Thomas
