User Error - Need help adding a disk back to an unmounted zpool

[Jared Stoker]

Ok, I'm new to freenas and I have a small pool of 4 random sized drives. I was planning on removing one of the smaller ones and replacing it with a larger one. First I detached the pool, then I wiped the drive in question. <insert expletives>. I did in fact look up ways to do this first. Anyways, I need help to add the wiped drive back into the zpool so that I can reattach the pool. The wipe was a quick one and didn't wipe any of the hard data. - ada3 is the drive I need to add back in.

[root@freenas] /# camcontrol devlist
<COMPAQ RAID 1(1+0) OK> at scbus0 target 0 lun 0 (da0,pass0)
<COMPAQ RAID 0 OK> at scbus0 target 1 lun 0 (da1,pass1)
<ST31500341AS CC1H> at scbus3 target 0 lun 0 (ada0,pass2)
<ST3250823AS 3.03> at scbus3 target 1 lun 0 (ada1,pass3)
<WDC WD3200AAJS-00L7A0 01.03E01> at scbus4 target 0 lun 0 (ada2,pass4)
<ST3160811AS 3.AAE> at scbus4 target 1 lun 0 (ada3,pass5)
<SanDisk Cruzer 1.26> at scbus6 target 0 lun 0 (pass6,da2)

[root@freenas] /# gpart show ada3
=> 34 312581741 ada3 GPT (149G)
34 94 - free - (47k)
128 4194304 1 freebsd-swap (2.0G)
4194432 308387336 2 freebsd-zfs (147G)
312581768 7 - free - (3.5k)


[root@freenas] /# zpool import
pool: Backup
id: 7530902688017962925
state: UNAVAIL
status: One or more devices are missing from the system.
action: The pool cannot be imported. Attach the missing
devices and try again.
see: http://illumos.org/msg/ZFS-8000-6X
config:

Backup UNAVAIL missing device
gptid/c2511c04-1c30-11e4-b749-001517294f28 ONLINE
gptid/c2d7fe55-1c30-11e4-b749-001517294f28 ONLINE
gptid/c3479575-1c30-11e4-b749-001517294f28 ONLINE
cache
18258670900347583938

Additional devices are known to be part of this pool, though their
exact configuration cannot be determined.


[root@freenas] /# glabel status
Name Status Components
gptid/c3479575-1c30-11e4-b749-001517294f28 N/A ada0p2
gptid/c2511c04-1c30-11e4-b749-001517294f28 N/A ada1p2
gptid/c2d7fe55-1c30-11e4-b749-001517294f28 N/A ada2p2
gptid/082aede9-1c31-11e4-b749-001517294f28 N/A da0p2
gptid/f00908e9-1c30-11e4-b749-001517294f28 N/A da1p2
ufs/FreeNASs3 N/A da2s3
ufs/FreeNASs4 N/A da2s4
ufs/FreeNASs1a N/A da2s1a
gptid/c19a6a37-1c30-11e4-b749-001517294f28 N/A ada3p2

 

[danb35]

It looks like your pool was striped disks, i.e., not in any sort of redundant configuration (RAIDZ or mirrors). If that's the case, your pool and all its data are toast.

 

[Ericloewe]

What in the world made you think that you could even think of removing a drive from a stripe of four drives!?

Please post that output again, with CODE tags (little text box button in the forum's edit bar) - but it looks like a hopeless situation.

 

[Jared Stoker]

Yes it was one of a striped volume :( The part that makes me hope is that the gpart shows the data is still there, just minus the disk index. Would it be possible to recover it at all? Here is a repost of the code.

Code:

[root@freenas] /# camcontrol devlist
<COMPAQ RAID 1(1+0) OK> at scbus0 target 0 lun 0 (da0,pass0)
<COMPAQ RAID 0 OK> at scbus0 target 1 lun 0 (da1,pass1)
<ST31500341AS CC1H> at scbus3 target 0 lun 0 (ada0,pass2)
<ST3250823AS 3.03> at scbus3 target 1 lun 0 (ada1,pass3)
<WDC WD3200AAJS-00L7A0 01.03E01> at scbus4 target 0 lun 0 (ada2,pass4)
<ST3160811AS 3.AAE> at scbus4 target 1 lun 0 (ada3,pass5)
<SanDisk Cruzer 1.26> at scbus6 target 0 lun 0 (pass6,da2)

[root@freenas] /# gpart show ada3
=> 34 312581741 ada3 GPT (149G)
34 94 - free - (47k)
128 4194304 1 freebsd-swap (2.0G)
4194432 308387336 2 freebsd-zfs (147G)
312581768 7 - free - (3.5k)


[root@freenas] /# zpool import
pool: Backup
id: 7530902688017962925
state: UNAVAIL
status: One or more devices are missing from the system.
action: The pool cannot be imported. Attach the missing
devices and try again.
see: http://illumos.org/msg/ZFS-8000-6X
config:

Backup UNAVAIL missing device
gptid/c2511c04-1c30-11e4-b749-001517294f28 ONLINE
gptid/c2d7fe55-1c30-11e4-b749-001517294f28 ONLINE
gptid/c3479575-1c30-11e4-b749-001517294f28 ONLINE
cache
18258670900347583938

Additional devices are known to be part of this pool, though their
exact configuration cannot be determined.


[root@freenas] /# glabel status
Name Status Components
gptid/c3479575-1c30-11e4-b749-001517294f28 N/A ada0p2
gptid/c2511c04-1c30-11e4-b749-001517294f28 N/A ada1p2
gptid/c2d7fe55-1c30-11e4-b749-001517294f28 N/A ada2p2
gptid/082aede9-1c31-11e4-b749-001517294f28 N/A da0p2
gptid/f00908e9-1c30-11e4-b749-001517294f28 N/A da1p2
ufs/FreeNASs3 N/A da2s3
ufs/FreeNASs4 N/A da2s4
ufs/FreeNASs1a N/A da2s1a
gptid/c19a6a37-1c30-11e4-b749-001517294f28 N/A ada3p2

 

[Jared Stoker]

The data isn't irreplaceable, it will just take a while to transfer it again. I just assumed that it might be recoverable given that no new data has been written over it and just the disk index was 'deleted'

 

[Ericloewe]

The data isn't irreplaceable, it will just take a while to transfer it again. I just assumed that it might be recoverable given that no new data has been written over it and just the disk index was 'deleted'

Maybe theoretically possible, but there are no (trustworthy) tools to do so.

 

[cyberjock]

Yeah, recovery for that pool will cost you serious money. If it's not irreplaceable I'd just learn the lesson (don't do single-disk vdevs) and restore the data from backups and such.

Glad you didn't lose anything important.

 

[danb35]

I'd also point out that, unless I'm badly mistaken, what you were trying to do in the first place just isn't possible. You can replace disks in RAIDZ vdevs, but not in stripes. So even if you hadn't wiped the disk, you couldn't swap in a larger one to replace the smaller disk (and thereby increase the capacity of your pool).

 

[cyberjock]

Technically, if you are really smart and dip into the source code you can reproduce commands that will let you turn a single disk vdev into a mirrored vdev. This can only be done from the CLI, the WebGUI doesn't support this feature. The problem is that if you are doing this because your single disk vdev is going bad you're already f*cked. So at that point your doing recovery and not simple "replace a failed disk" scenario.

 

[danb35]

Well, Jared didn't say a disk was failing, he said he wanted a bigger pool. So you could mirror a single disk with a larger one, then once that had finished syncing (would that still be resilvering?), remove the original (smaller) disk? And be left with a larger, single-disk vdev (rather than a degraded mirror)? All from the CLI, of course... In that case, I was somewhat correct (in that ZFS doesn't directly allow you to replace one disk in a striped pool with another disk), but mostly incorrect (in that there's an end run around that). None of which helps Jared now, but is still good info.

Do I correctly recall your mentioning that the logic to add a mirror to a single-disk vdev would be added to the GUI in 9.3?

 

[cyberjock]

I don't think that converting single disk to mirrors is in 9.3. But I'm a little out of it right now. Been a very long day. :P I'm pretty sure there's a feature request in at bugs.freenas.org that, if found, will give an indicator on if this is scheduled for 9.3.