Trouble replacing failed HDD in mirrored pool

[ovig]

I have a FreeNAS 8.2 (r11950) box running a mirrored ZFS pool (4x 2TB altogether) which has been running fine for months - if it makes a difference, it's a HP Proliant Microserver with 8GB of RAM

I notice lately problems with one disk and have taken it out (ada0) to replace with a new one - I checked serial numbers twice and am pretty sure that the disk I took out was the failing one.

In an unusual (for me) move I read the documentation (both for FreeNAS and the NFS bit of the FreeBSD doc) to figure out how to proceed...
After rebooting (the disks are not hot-swapable), the pool looked good considering the situation, i.e. DEGRADED with one disk offline (ada0p2) - all others where online - so I tried to Replace it and got an error:

Code:

Feb 14 10:46:00 freenas notifier: swapoff: /dev/ada0p1: Invalid argument
Feb 14 10:46:04 freenas notifier: 1+0 records in
Feb 14 10:46:04 freenas notifier: 1+0 records out
Feb 14 10:46:04 freenas notifier: 1048576 bytes transferred in 0.076377 secs (13728965 bytes/sec)
Feb 14 10:46:04 freenas notifier: dd: /dev/ada0: short write on character device
Feb 14 10:46:04 freenas notifier: dd: /dev/ada0: end of device
Feb 14 10:46:04 freenas notifier: 5+0 records in
Feb 14 10:46:04 freenas notifier: 4+1 records out
Feb 14 10:46:04 freenas notifier: 4284416 bytes transferred in 0.349663 secs (12252986 bytes/sec)
Feb 14 10:46:05 freenas notifier: cannot online ada0p2: no such device in pool
Feb 14 10:46:05 freenas notifier: swapon: /dev/ada0p1: device already in use

I tried to use the "zpool replace DATA ada0p2" (DATA is the name of my pool) and end up with

Code:

[root@freenas ~]# zpool replace -f DATA ada0p2                                  
invalid vdev specification                                                      
the following errors must be manually repaired:                                 
/dev/ada0p2 is part of active pool 'DATA' 

I have to confess being reasonably "green" on ZFS and FreeNAS but it looks like the swap partition (I have 4x 2GB swap partitions, one on each disk) is what causes the disk replacement to fail - am I right?
What is the best way forward to get the new disk to resilver?

Thanks in advance for the help & suggestions

Olivier

 

[paleoN]

I have a FreeNAS 8.2 (r11950) box running a mirrored ZFS pool (4x 2TB altogether) which has been running fine for months - if it makes a difference, it's a HP Proliant Microserver with 8GB of RAM

Upgrade to FreeNAS 8.3.

I tried to use the "zpool replace DATA ada0p2" (DATA is the name of my pool) and end up with

No, you used -f which is stupid. You should not have to force the replace except in select few circumstances. Do not blindly run it. Where is this "new" disk from and was it ever part of a zpool?

I have to confess being reasonably "green" on ZFS and FreeNAS but it looks like the swap partition (I have 4x 2GB swap partitions, one on each disk) is what causes the disk replacement to fail - am I right?

Not enough information:

Code:

zpool status -v

camcontrol devlist

glabel status

gpart show

 

[ovig]

Thanks for the pointers.

The "zpool replace -f" was run after a "zpool replace" errored and part of the error message suggested trying again with "-f"
Disk had been lying around my desk - can't remember all the details but I think it was NTFS formatted and had definitely not been part of my ZFS pool.

I've had on my ToDo to upgrade to 8.3 so will give that a go first and see how far I get.

 

[cyberjock]

As paleoN said, using -f is almost stupid. -f is like telling the system "make this work and I don't care if I lose data and I need this to work regardless of anything". Post the output of the 4 commands paleoN provided inside of code tags and we'll see how you should proceed.

 

[paleoN]

The "zpool replace -f" was run after a "zpool replace" errored and part of the error message suggested trying again with "-f"
Disk had been lying around my desk - can't remember all the details but I think it was NTFS formatted and had definitely not been part of my ZFS pool.

Then AFAICT, based on what information you provided, you attempted to force replace the wrong disk which IMO is stupid. It would explain the swapon & zpool replace errors you saw.

Post the output of the 4 commands paleoN provided inside of code tags and we'll see how you should proceed.

I've had on my ToDo to upgrade to 8.3 so will give that a go first and see how far I get.

Hopefully the upgrade will take care of it. Otherwise, I would need the output.