FreeNAS settings/zpool etc lost after failed firmware upgrade in Barracuda drives

[derio]

Hello everybody, I am a unix noob but somehow, following documentation and tutorials, managed to get a FreeNAS server up and running.

I wanted to upgrade two of my 3T Barracuda drives (ST3000DM001-1CH166) to the new version (used the guide over there). I downloaded the firmware (Barracuda-ALL-GRCC4H.iso) but during the upgrade itself I got the message that the drive model was wrong.. It printed that the firmware was not downloaded to the devices and that I should reboot.

As I rebooted from my FreeNAS usb I got a bunch of errors like
"django.db.utils.DatabaseError: no such column: system_advanced.adv_firmwarevc"
and it seems that all the settings are gone. The nginx server also does not start.

If I open a shell and type
"zpool status"

I get "no pools available"

With "zpool import"
I get
pool: dNAS
id: ....
state: UNAVAIL
status: The pool is formatted using an incompatible version.
action: The pool cannot be imported. Access the pool on a system running newer
software, or recreate the pool from backup.
see: http://www.sun.com/msg/ZFS-8000-A5
config:
dNAS UNAVAIL newer version
raidz1-0 ONLINE
gptid/1...... ONLINE
gptid/2...... ONLINE
gptid/3...... ONLINE

In the /data folder there is a freenas-v1.db but was last modified half an hour ago and the freenas-v1.db.bak is 3 years old..

Can something be done to restore the settings (and the pool) without risk of losing data?

Thanks in advance

Edit: Some extra info: My version is FreeNAS-9.2.0-RELEASE-x64 and I seem to have a month-old db backup but that was before I replaced one faulty drive and resilvered. Could I use this?

 

[m0nkey_]

The fact it says "Access the pool on a system running newer software" suggests you created the pool on a more up to date version of FreeNAS and ran 'zpool upgrade'. Suggest you try and install at least FreeNAS 9.3 or 9.10 and import your pool.

 

[derio]

Hi m0nkey_, thanks for the reply. Thing is, I only ever had this one FreeNAS installation. I originally created the pool on FreeNAS 8.3 I think, then after a while upgraded to 9.2 and then this happened. How could I have switched back to 8.3 without realizing it?

 

[Ericloewe]

Hi m0nkey_, thanks for the reply. Thing is, I only ever had this one FreeNAS installation. I originally created the pool on FreeNAS 8.3 I think, then after a while upgraded to 9.2 and then this happened. How could I have switched back to 8.3 without realizing it?

I can think of a few (weird) scenarios, but I'd definitely try a newer version to see if the pool is mountable.

 

[derio]

So, I installed FreeNAS 9.10 on a new stick, imported the zpool and then upgraded it (via the shell). No errors came up.

However I cannot see the pool in the GUI, only the disks. It finds no volume when I select "Import volume". Should I try to import the individual disks? Is there something to pay special attention to?





Here is the output of some commands that seem to be requested in similar questions in the forum:

camcontrol devlist

Code:

<ST3000DM001-1CH166 CC24>          at scbus3 target 0 lun 0 (ada0,pass0)
<ST3000DM001-1CH166 CC24>          at scbus4 target 0 lun 0 (ada1,pass1)
<ST3000DM001-9YN166 CC4H>          at scbus5 target 0 lun 0 (ada2,pass2)
<TSSTcorp CDDVDW SH-S203D SB00>    at scbus6 target 0 lun 0 (pass3,cd0)
<LaCie iamaKey 0.00>               at scbus8 target 0 lun 0 (da0,pass4)

gpart show

Code:

=>        34  5860533101  ada0  GPT  (2.7T)
          34          94        - free -  (47K)
         128     4194304     1  freebsd-swap  (2.0G)
     4194432  5856338696     2  freebsd-zfs  (2.7T)
  5860533128           7        - free -  (3.5K)

=>        34  5860533101  ada1  GPT  (2.7T)
          34          94        - free -  (47K)
         128     4194304     1  freebsd-swap  (2.0G)
     4194432  5856338696     2  freebsd-zfs  (2.7T)
  5860533128           7        - free -  (3.5K)

=>        34  5860533101  ada2  GPT  (2.7T)
          34          94        - free -  (47K)
         128     4194304     1  freebsd-swap  (2.0G)
     4194432  5856338696     2  freebsd-zfs  (2.7T)
  5860533128           7        - free -  (3.5K)

=>      34  15794109  da0  GPT  (7.5G)
        34      1024    1  bios-boot  (512K)
      1058         6       - free -  (3.0K)
      1064  15793072    2  freebsd-zfs  (7.5G)
  15794136         7       - free -  (3.5K)

glabel status

Code:

                                      Name  Status  Components
gptid/555680e8-6f05-11e2-a223-00173188bc07     N/A  ada0p1
gptid/556cbc05-6f05-11e2-a223-00173188bc07     N/A  ada0p2
gptid/55c5fb38-6f05-11e2-a223-00173188bc07     N/A  ada1p1
gptid/55d28e6f-6f05-11e2-a223-00173188bc07     N/A  ada1p2
gptid/f2043045-f77f-11e5-b90b-00173188bc07     N/A  ada2p1
gptid/f21e00cb-f77f-11e5-b90b-00173188bc07     N/A  ada2p2
gptid/91d5596e-0956-11e6-bb75-00173188bc07     N/A  da0p1

zpool history

Code:

History for 'dNAS':

[...]

2016-03-11.03:00:11 zpool scrub dNAS
2016-03-21.03:00:10 zpool scrub dNAS
2016-03-26.06:03:04 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 951189869556734306
2016-03-26.06:03:04 zpool set cachefile=/data/zfs/zpool.cache dNAS
2016-03-26.09:27:54 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 951189869556734306
2016-03-26.09:27:54 zpool set cachefile=/data/zfs/zpool.cache dNAS
2016-03-26.10:32:08 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 951189869556734306
2016-03-26.10:32:08 zpool set cachefile=/data/zfs/zpool.cache dNAS
2016-03-31.22:12:31 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 951189869556734306
2016-03-31.22:12:31 zpool set cachefile=/data/zfs/zpool.cache dNAS
2016-03-31.22:31:23 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 951189869556734306
2016-03-31.22:31:23 zpool set cachefile=/data/zfs/zpool.cache dNAS
2016-03-31.22:34:34 zpool replace dNAS 148634962024130135 gptid/f21e00cb-f77f-11e5-b90b-00173188bc07
2016-03-31.22:36:04 zpool detach dNAS 148634962024130135
2016-04-04.20:10:36 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 951189869556734306
2016-04-04.20:10:36 zpool set cachefile=/data/zfs/zpool.cache dNAS
2016-04-06.03:00:10 zpool scrub dNAS
2016-04-16.03:00:10 zpool scrub dNAS
2016-04-23.15:47:04 zpool import -f dNAS
2016-04-23.16:01:34 zpool upgrade dNAS

History for 'freenas-boot':
2016-04-23.15:23:41 zpool create -f -o cachefile=/tmp/zpool.cache -o version=28 -O mountpoint=none -O atime=off -O canmount=off freenas-boot da0p2
2016-04-23.15:23:46 zpool set feature@async_destroy=enabled freenas-boot
2016-04-23.15:23:48 zpool set feature@empty_bpobj=enabled freenas-boot
2016-04-23.15:23:49 zpool set feature@lz4_compress=enabled freenas-boot
2016-04-23.15:23:50 zfs set compress=lz4 freenas-boot
2016-04-23.15:23:54 zfs create -o canmount=off freenas-boot/ROOT
2016-04-23.15:23:56 zfs create -o mountpoint=legacy freenas-boot/ROOT/default
2016-04-23.15:24:03 zfs create -o mountpoint=legacy freenas-boot/grub
2016-04-23.15:27:21 zpool set bootfs=freenas-boot/ROOT/default freenas-boot
2016-04-23.15:27:21 zpool set cachefile=/boot/zfs/rpool.cache freenas-boot
2016-04-23.15:30:29 zfs set beadm:nickname=default freenas-boot/ROOT/default
2016-04-23.15:34:29 zfs snapshot -r freenas-boot/ROOT/default@2016-04-23-06:34:21
2016-04-23.15:34:36 zfs clone -o canmount=off -o mountpoint=legacy freenas-boot/ROOT/default@2016-04-23-06:34:21 freenas-boot/ROOT/Initial-Install
2016-04-23.15:34:42 zfs set beadm:nickname=Initial-Install freenas-boot/ROOT/Initial-Install

I would appreciate any help.

 

[Ericloewe]

So, I installed FreeNAS 9.10 on a new stick, imported the zpool and then upgraded it (via the shell). No errors came up.

However I cannot see the pool in the GUI, only the disks. It finds no volume when I select "Import volume". Should I try to import the individual disks? Is there something to pay special attention to?

View attachment 11512
View attachment 11513


Here is the output of some commands that seem to be requested in similar questions in the forum:

camcontrol devlist

Code:

<ST3000DM001-1CH166 CC24>          at scbus3 target 0 lun 0 (ada0,pass0)
<ST3000DM001-1CH166 CC24>          at scbus4 target 0 lun 0 (ada1,pass1)
<ST3000DM001-9YN166 CC4H>          at scbus5 target 0 lun 0 (ada2,pass2)
<TSSTcorp CDDVDW SH-S203D SB00>    at scbus6 target 0 lun 0 (pass3,cd0)
<LaCie iamaKey 0.00>               at scbus8 target 0 lun 0 (da0,pass4)

gpart show

Code:

=>        34  5860533101  ada0  GPT  (2.7T)
          34          94        - free -  (47K)
         128     4194304     1  freebsd-swap  (2.0G)
     4194432  5856338696     2  freebsd-zfs  (2.7T)
  5860533128           7        - free -  (3.5K)

=>        34  5860533101  ada1  GPT  (2.7T)
          34          94        - free -  (47K)
         128     4194304     1  freebsd-swap  (2.0G)
     4194432  5856338696     2  freebsd-zfs  (2.7T)
  5860533128           7        - free -  (3.5K)

=>        34  5860533101  ada2  GPT  (2.7T)
          34          94        - free -  (47K)
         128     4194304     1  freebsd-swap  (2.0G)
     4194432  5856338696     2  freebsd-zfs  (2.7T)
  5860533128           7        - free -  (3.5K)

=>      34  15794109  da0  GPT  (7.5G)
        34      1024    1  bios-boot  (512K)
      1058         6       - free -  (3.0K)
      1064  15793072    2  freebsd-zfs  (7.5G)
  15794136         7       - free -  (3.5K)

glabel status

Code:

                                      Name  Status  Components
gptid/555680e8-6f05-11e2-a223-00173188bc07     N/A  ada0p1
gptid/556cbc05-6f05-11e2-a223-00173188bc07     N/A  ada0p2
gptid/55c5fb38-6f05-11e2-a223-00173188bc07     N/A  ada1p1
gptid/55d28e6f-6f05-11e2-a223-00173188bc07     N/A  ada1p2
gptid/f2043045-f77f-11e5-b90b-00173188bc07     N/A  ada2p1
gptid/f21e00cb-f77f-11e5-b90b-00173188bc07     N/A  ada2p2
gptid/91d5596e-0956-11e6-bb75-00173188bc07     N/A  da0p1

zpool history

Code:

History for 'dNAS':

[...]

2016-03-11.03:00:11 zpool scrub dNAS
2016-03-21.03:00:10 zpool scrub dNAS
2016-03-26.06:03:04 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 951189869556734306
2016-03-26.06:03:04 zpool set cachefile=/data/zfs/zpool.cache dNAS
2016-03-26.09:27:54 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 951189869556734306
2016-03-26.09:27:54 zpool set cachefile=/data/zfs/zpool.cache dNAS
2016-03-26.10:32:08 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 951189869556734306
2016-03-26.10:32:08 zpool set cachefile=/data/zfs/zpool.cache dNAS
2016-03-31.22:12:31 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 951189869556734306
2016-03-31.22:12:31 zpool set cachefile=/data/zfs/zpool.cache dNAS
2016-03-31.22:31:23 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 951189869556734306
2016-03-31.22:31:23 zpool set cachefile=/data/zfs/zpool.cache dNAS
2016-03-31.22:34:34 zpool replace dNAS 148634962024130135 gptid/f21e00cb-f77f-11e5-b90b-00173188bc07
2016-03-31.22:36:04 zpool detach dNAS 148634962024130135
2016-04-04.20:10:36 zpool import -c /data/zfs/zpool.cache.saved -o cachefile=none -R /mnt -f 951189869556734306
2016-04-04.20:10:36 zpool set cachefile=/data/zfs/zpool.cache dNAS
2016-04-06.03:00:10 zpool scrub dNAS
2016-04-16.03:00:10 zpool scrub dNAS
2016-04-23.15:47:04 zpool import -f dNAS
2016-04-23.16:01:34 zpool upgrade dNAS

History for 'freenas-boot':
2016-04-23.15:23:41 zpool create -f -o cachefile=/tmp/zpool.cache -o version=28 -O mountpoint=none -O atime=off -O canmount=off freenas-boot da0p2
2016-04-23.15:23:46 zpool set feature@async_destroy=enabled freenas-boot
2016-04-23.15:23:48 zpool set feature@empty_bpobj=enabled freenas-boot
2016-04-23.15:23:49 zpool set feature@lz4_compress=enabled freenas-boot
2016-04-23.15:23:50 zfs set compress=lz4 freenas-boot
2016-04-23.15:23:54 zfs create -o canmount=off freenas-boot/ROOT
2016-04-23.15:23:56 zfs create -o mountpoint=legacy freenas-boot/ROOT/default
2016-04-23.15:24:03 zfs create -o mountpoint=legacy freenas-boot/grub
2016-04-23.15:27:21 zpool set bootfs=freenas-boot/ROOT/default freenas-boot
2016-04-23.15:27:21 zpool set cachefile=/boot/zfs/rpool.cache freenas-boot
2016-04-23.15:30:29 zfs set beadm:nickname=default freenas-boot/ROOT/default
2016-04-23.15:34:29 zfs snapshot -r freenas-boot/ROOT/default@2016-04-23-06:34:21
2016-04-23.15:34:36 zfs clone -o canmount=off -o mountpoint=legacy freenas-boot/ROOT/default@2016-04-23-06:34:21 freenas-boot/ROOT/Initial-Install
2016-04-23.15:34:42 zfs set beadm:nickname=Initial-Install freenas-boot/ROOT/Initial-Install

I would appreciate any help.

You imported from the CLI? Well, there's your problem. Export it from the CLI and import it from the GUI.

 

[derio]

Hi Ericloewe, as strange as it seems to me, that was it! I did what you instructed and got the volume back.

I'm now in the process of setting everything back up.

Thank you all for your help!

 

[Ericloewe]

as strange as it seems to me

It's quite easy to explain, really.

The GUI is a messy blob of code that's rather limited in its interactions with the underlying system. This means it's unaware of many changes made directly on the OS - namely via the CLI.

FreeNAS 10 will provide better integration and hide the real OS CLI away from most users, presenting instead the new, improved GUI, a new CLI, as well as third-party apps, all interacting with the middleware that is the secret sauce in FreeNAS.