Imported FreeNAS Corral zpool into FreeNAS 11 - multiple volumes missing? =(

Status
Not open for further replies.
victorhooi

victorhooi

Contributor
Joined
Mar 16, 2012
Messages
184
I had a FreeNAS system at my parent's place that was previously running on FreeNAS Corral.

It has a single RAIDZ-1 volume, comprised of four 4TB disks.

I checked today, and it appeared to be at the GRUB screen, so I assumed it had tried to update and borked - or the USB stick had failed. Anyhow, I downloaded the latest FreeNAS 11 build (FreeNAS-11-MASTER-201706020409-373d389.iso), and wrote it to the USB stick.

I booted up fine. I cancelled the initial setup wizard, and went to import my existing ZFS volumes. It seemed to pull the ZFS volume (datastore) fine.

It also prompted me to upgrade the zpool version via the GUI, which I did.

However, I just checked, and there is only a ZFS dataset (ablage) whereas before I had around 10 or so under datastore *sad face*

Output of zfs list:
Code:
root@freenas:/mnt/datastore # zfs list
NAME														 USED  AVAIL  REFER  MOUNTPOINT
datastore												   1.20T  9.01T   128K  /mnt/datastore
datastore/.system										   8.81M  9.01T   140K  legacy
datastore/.system/configs-127b3ac88ebd45cc8152eb1d50a360da   128K  9.01T   128K  legacy
datastore/.system/cores									  128K  9.01T   128K  legacy
datastore/.system/rrd-127b3ac88ebd45cc8152eb1d50a360da	  7.90M  9.01T  7.90M  legacy
datastore/.system/samba4									 279K  9.01T   279K  legacy
datastore/.system/syslog-127b3ac88ebd45cc8152eb1d50a360da	256K  9.01T   256K  legacy
datastore/ablage											1.20T  9.01T  1.20T  /mnt/datastore/ablage
freenas-boot												 743M  6.24G	64K  none
freenas-boot/ROOT											736M  6.24G	29K  none
freenas-boot/ROOT/Initial-Install							  1K  6.24G   734M  legacy
freenas-boot/ROOT/default									736M  6.24G   734M  legacy
freenas-boot/grub										   6.29M  6.24G  6.29M  legacy


Output of zpool status:
Code:
root@freenas:/mnt/datastore # zpool status
  pool: datastore
state: ONLINE
  scan: none requested
config:

	NAME											STATE	 READ WRITE CKSUM
	datastore									   ONLINE	   0	 0	 0
	  raidz1-0									  ONLINE	   0	 0	 0
		gptid/896b991d-a974-11e6-b04f-a0b3cce4fa18  ONLINE	   0	 0	 0
		gptid/88ace9fe-a974-11e6-b04f-a0b3cce4fa18  ONLINE	   0	 0	 0
		gptid/8957be15-a974-11e6-b04f-a0b3cce4fa18  ONLINE	   0	 0	 0
		gptid/8cbda7a2-a974-11e6-b04f-a0b3cce4fa18  ONLINE	   0	 0	 0

errors: No known data errors

  pool: freenas-boot
state: ONLINE
  scan: none requested
config:

	NAME		STATE	 READ WRITE CKSUM
	freenas-boot  ONLINE	   0	 0	 0
	  da0p2	 ONLINE	   0	 0	 0

errors: No known data errors


I am hoping it's somehow still on the disks. Any advice on how to check, and/or recover?

What is very concerning is that the datastore volume has 9.0 TB free....?

That doesn't bode well, surely.

Regards,
Victor
 
Last edited:
victorhooi

victorhooi

Contributor
Joined
Mar 16, 2012
Messages
184
I did notice on the actual FreeNAS terminal, there is a line that say;

Code:
Jun 4 17:01:29 freeness savecore: /dev/ada2p1: Operation not permitted
Jun 4 17:01:29 freeness savecore: /dev/ada1p1: Operation not permitted
Jun 4 17:01:29 freeness savecore: /dev/ada0p1: Operation not permitted


Screenshots of disks and volumes:

bVx3DAK.png


BB9q61q.png
 
victorhooi

victorhooi

Contributor
Joined
Mar 16, 2012
Messages
184
I was going to use ZDB to list recent transaction groups. However:
Code:
sudo zdb -h-h freenas-boot

works fine and returns output.

However:
Code:
sudo zdb -hh datastore
zdb; can't open 'datastore': No such file or directory


I'm not sure why - I assume perhaps I'm using the syntax wrong?

But zpool list lists both datastore and freenas-boot as pools.
 
victorhooi

victorhooi

Contributor
Joined
Mar 16, 2012
Messages
184
Aha, I think I hit https://bugs.freenas.org/issues/14536, which is why "zdb" didn't work.

Output of "zpool history -il":
Code:
% sudo zpool history -il datastore
Password:
History for 'datastore':
2016-11-13.18:41:21 [txg:5] create pool version 5000; software version 5000/5; uts freenas.local 10.3-STABLE 1003510 amd64 [on freenas.local]
2016-11-13.18:41:21 [txg:5] set datastore (21) compression=15 [on freenas.local]
2016-11-13.18:41:21 [txg:5] set datastore (21) aclmode=3 [on freenas.local]
2016-11-13.18:41:21 [txg:5] set datastore (21) mountpoint=/mnt/datastore [on freenas.local]
2016-11-13.18:41:21 [txg:5] set datastore (21) aclinherit=3 [on freenas.local]
2016-11-13.18:41:24 <dispatcher> zpool create -o feature@embedded_data=enabled -o feature@filesystem_limits=enabled -o feature@enabled_txg=enabled -o feature@multi_vdev_crash_dump=enabled -o feature@extensible_dataset=enabled -o feature@large_blocks=enabled -o feature@empty_bpobj=enabled -o feature@bookmarks=enabled -o feature@hole_birth=enabled -o cachefile=/data/zfs/volume.cache -o autoexpand=on -o feature@lz4_compress=enabled -o failmode=continue -o feature@async_destroy=enabled -o feature@spacemap_histogram=enabled -O aclmode=passthrough -O mountpoint=/mnt/datastore -O aclinherit=passthrough -O compression=lz4 datastore raidz1 /dev/gptid/896b991d-a974-11e6-b04f-a0b3cce4fa18 /dev/gptid/88ace9fe-a974-11e6-b04f-a0b3cce4fa18 /dev/gptid/8957be15-a974-11e6-b04f-a0b3cce4fa18 /dev/gptid/8cbda7a2-a974-11e6-b04f-a0b3cce4fa18 [user 0 (root) on freenas.local]
2016-11-13.18:41:24 [txg:6] set datastore (21) org.freenas:permissions_type=PERM [on freenas.local]
2016-11-13.18:41:29 <dispatcher> zfs set org.freenas:permissions_type=PERM datastore [user 0 (root) on freenas.local]
2016-11-13.18:45:45 [txg:59] set datastore (21) atime=0 [on freenas.local]
2016-11-13.18:45:50 <dispatcher> zfs set atime=off datastore [user 0 (root) on freenas.local]
2016-11-13.18:46:18 [txg:66] create datastore/ablage (42)  [on freenas.local]
2016-11-13.18:46:18 [txg:67] set datastore/ablage (42) compression=15 [on freenas.local]
2016-11-13.18:46:18 [txg:67] set datastore/ablage (42) org.freenas:uuid=502dd243-a1aa-444d-9e98-435766fcf206 [on freenas.local]
2016-11-13.18:46:18 [txg:67] set datastore/ablage (42) aclmode=3 [on freenas.local]
2016-11-13.18:46:18 [txg:67] set datastore/ablage (42) org.freenas:permissions_type=PERM [on freenas.local]
2016-11-13.18:46:23 ioctl create
	input:
		type: 2
		props:
			compression: 15
			org.freenas:uuid: '502dd243-a1aa-444d-9e98-435766fcf206'
			aclmode: 3
			org.freenas:permissions_type: 'PERM'
 [user 0 (root) on freenas.local]
2016-11-13.18:46:23 <dispatcher> zfs mount datastore/ablage [user 0 (root) on freenas.local]
2017-06-04.17:01:25 [txg:20779] open pool version 5000; software version 5000/5; uts freenas.local 11.0-STABLE 1100512 amd64 [on freenas.local]
2017-06-04.17:01:26 [txg:20781] import pool version 5000; software version 5000/5; uts freenas.local 11.0-STABLE 1100512 amd64 [on freenas.local]
2017-06-04.17:01:26 zpool import -f -R /mnt 1196016705705258035 [user 0 (root) on freenas.local]
2017-06-04.17:01:26 [txg:20783] inherit datastore (21) mountpoint=/ [on freenas.local]
2017-06-04.17:01:26 [txg:20784] inherit datastore/ablage (42) mountpoint=/ [on freenas.local]
2017-06-04.17:01:26 zfs inherit -r mountpoint datastore [user 0 (root) on freenas.local]
2017-06-04.17:01:26 zpool set cachefile=/data/zfs/zpool.cache datastore [user 0 (root) on freenas.local]
2017-06-04.17:01:26 [txg:20785] set datastore (21) aclmode=3 [on freenas.local]
2017-06-04.17:01:26 zfs set aclmode=passthrough datastore [user 0 (root) on freenas.local]
2017-06-04.17:01:26 [txg:20786] set datastore (21) aclinherit=3 [on freenas.local]
2017-06-04.17:01:31 zfs set aclinherit=passthrough datastore [user 0 (root) on freenas.local]
2017-06-04.17:01:31 [txg:20787] create datastore/.system (71)  [on freenas.local]
2017-06-04.17:01:31 [txg:20788] set datastore/.system (71) mountpoint=legacy [on freenas.local]
2017-06-04.17:01:31 ioctl create
	input:
		type: 2
		props:
			mountpoint: 'legacy'
 [user 0 (root) on freenas.local]
2017-06-04.17:01:31 zfs create -o mountpoint=legacy datastore/.system [user 0 (root) on freenas.local]
2017-06-04.17:01:31 [txg:20789] create datastore/.system/cores (77)  [on freenas.local]
2017-06-04.17:01:31 [txg:20790] set datastore/.system/cores (77) mountpoint=legacy [on freenas.local]
2017-06-04.17:01:31 ioctl create
	input:
		type: 2
		props:
			mountpoint: 'legacy'
 [user 0 (root) on freenas.local]
2017-06-04.17:01:31 zfs create -o mountpoint=legacy datastore/.system/cores [user 0 (root) on freenas.local]
2017-06-04.17:01:31 [txg:20791] create datastore/.system/samba4 (83)  [on freenas.local]
2017-06-04.17:01:31 [txg:20792] set datastore/.system/samba4 (83) mountpoint=legacy [on freenas.local]
2017-06-04.17:01:31 ioctl create
	input:
		type: 2
		props:
			mountpoint: 'legacy'
 [user 0 (root) on freenas.local]
2017-06-04.17:01:31 zfs create -o mountpoint=legacy datastore/.system/samba4 [user 0 (root) on freenas.local]
2017-06-04.17:01:31 [txg:20793] create datastore/.system/syslog-127b3ac88ebd45cc8152eb1d50a360da (89)  [on freenas.local]
2017-06-04.17:01:32 [txg:20794] set datastore/.system/syslog-127b3ac88ebd45cc8152eb1d50a360da (89) mountpoint=legacy [on freenas.local]
2017-06-04.17:01:32 ioctl create
	input:
		type: 2
		props:
			mountpoint: 'legacy'
 [user 0 (root) on freenas.local]
2017-06-04.17:01:32 zfs create -o mountpoint=legacy datastore/.system/syslog-127b3ac88ebd45cc8152eb1d50a360da [user 0 (root) on freenas.local]
2017-06-04.17:01:32 [txg:20795] create datastore/.system/rrd-127b3ac88ebd45cc8152eb1d50a360da (95)  [on freenas.local]
2017-06-04.17:01:32 [txg:20796] set datastore/.system/rrd-127b3ac88ebd45cc8152eb1d50a360da (95) mountpoint=legacy [on freenas.local]
2017-06-04.17:01:32 ioctl create
	input:
		type: 2
		props:
			mountpoint: 'legacy'
 [user 0 (root) on freenas.local]
2017-06-04.17:01:32 zfs create -o mountpoint=legacy datastore/.system/rrd-127b3ac88ebd45cc8152eb1d50a360da [user 0 (root) on freenas.local]
2017-06-04.17:01:32 [txg:20797] create datastore/.system/configs-127b3ac88ebd45cc8152eb1d50a360da (101)  [on freenas.local]
2017-06-04.17:01:32 [txg:20798] set datastore/.system/configs-127b3ac88ebd45cc8152eb1d50a360da (101) mountpoint=legacy [on freenas.local]
2017-06-04.17:01:37 ioctl create
	input:
		type: 2
		props:
			mountpoint: 'legacy'
 [user 0 (root) on freenas.local]
2017-06-04.17:01:37 zfs create -o mountpoint=legacy datastore/.system/configs-127b3ac88ebd45cc8152eb1d50a360da [user 0 (root) on freenas.local]
2017-06-04.17:12:04 [txg:20925] set feature@sha512=enabled [on freenas.local]
2017-06-04.17:12:04 [txg:20926] set feature@skein=enabled [on freenas.local]
2017-06-04.17:12:09 zpool upgrade datastore [user 0 (root) on freenas.local]


Hopefully the above is helpful...

I'm not sure what activity has happened on this system since November 2016 - but honestly, rolling back to then at this stage is better than nothing.

I am still super confused as to why only the first dataset is showing - does the fact that it's the first alphabetically mean anything?
 
Ericloewe

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,194
You're better off filing a new bug report and linking to the old one, otherwise your comment will be missed.
 
victorhooi

victorhooi

Contributor
Joined
Mar 16, 2012
Messages
184
Ericloewe said:
You're better off filing a new bug report and linking to the old one, otherwise your comment will be missed.
Click to expand...

Sure, I've just filed https://bugs.freenas.org/issues/24357 and mentioned https://bugs.freenas.org/issues/14536 - I didn't see any option to re-open closed bugs, but I assume that's by design.

Do you have any thoughts on the main issue though? (i.e. what's going on with my parent's NAS). I included the zpool history - hopefully that may shed some light. Let me know if you need any other diagnostic or troubleshooting information.
 
Ericloewe

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,194
I don't see anything that could be of help in the zpool history...
 
victorhooi

victorhooi

Contributor
Joined
Mar 16, 2012
Messages
184
Ericloewe said:
I don't see anything that could be of help in the zpool history...
Click to expand...

Is there anything I should save or pull from the machine now, before I do anything further? (I don't know if I have enough space somewhere to clone all 4 x 5.0 TB disks - but I was thinking more caches, or metadata things).

Which leads me to - would say, rebooting the machine, or exporting/re-importing the ZFS pool (possibly on a stable FreeNAS build) possibly help? Or could that do harm?

And thirdly is there some way of (non-destructively) rolling back to one of those earlier transaction groups? (e.g. November 2016).
 
Ericloewe

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,194
I don't even see other datasets, so I have few options for you. Whoever picks up your bug ticket will have a better idea of where to take this.
 
Status
Not open for further replies.

Important Announcement for the TrueNAS Community.

The TrueNAS Community has now been moved. This forum will now become READ-ONLY for historical purposes. Please feel free to join us on the new TrueNAS Community Forums.

Related topics on forums.truenas.com for thread: "Imported FreeNAS Corral zpool into FreeNAS 11 - multiple volumes missing? =("

Similar threads

melloa
  • Locked
Is this a two raidz2 in mirror?
Replies
9
Views
2K
Chris Moore
Chris Moore
victorhooi
  • Locked
Volume went from DEGRADED to ONLINE - but zpool status shows resilvering of 166 MB, and CHKSUM of 2?
Replies
5
Views
1K
Apollo
A
victorhooi
  • Locked
Added more devices to ZFS stripe - dataset volume stays the same?
Replies
1
Views
3K
Dusan
Dusan
victorhooi
  • Locked
Unable to import ZFS volume into FreeNAS 11
Replies
12
Views
10K
indivision
indivision
D
  • Locked
Yet another thread about how to assess disk usage on ZFS
Replies
3
Views
4K
dtom10
D
Top