I'm having several issues replacing a disk on a FreeNAS 11.2 U7 setup that's been running solid for at least the last two years now.
My setup is as follows:
Dell R730 with a LSI 2308 HBA connected to a single NetApp 24 drive shelf. All disks are multipath disks.
I am running FreeNAS 11.2 in a VM (ESXi 7.0), doing PCI pass through to my VM. This has worked and I do not believe this is related to my problem.
I have a 24 disk pool based on two 12 disk RAIDZ3 vDevs.
I had a disk start throwing smart errors so replaced it. here are my steps:
in the GUI, offline the disk with smart errors (validated by Serial Number)
physically swap the disk in the shelf
in CLI find the new disk with
in the GUI i went to the pool, status, and tried to replace the disk. There were two disks still shown, da2 and da12. da12 was the active disk in the multipath so I selected that. It did not work, gave me an error to select a valid disk. da2 also didn't work. I searched the forums and found there's a bug in 11.2 with this, at least based on some posts. doing it in the CLI worked.
The resilveringi process for ~140Tib at ~80% took roughly 36 hours which was fine. completed without errors and the pool showed as healthy.
However after doing this I noticed that if I go to disks, da2 (multipath/disk1 is active on da14 and passive on da2) still shows as an available disk. if i go to pools, the GUI lets me attempt to create a new pool with da2.
I was going to start replacing disks in this vDev with larger ones - but this seems to be not right so before proceeding thought I should post here to see how to clean this up or if i need to remove disk1 and re-do multipathing.
here is the result of gmultipath list on the newly replaced disk. the results look similar to other disks, the only difference being the mode is r2w2e2 on the provider, and r3w3e3 on the consumers. all multipath disks are in an active passive state.
Questions:
why is a member of a multipath disk showing as an available/unused disk.
how do I fix it to ensure i can safely resilver my pool to larger disks
Thanks all!
My setup is as follows:
Dell R730 with a LSI 2308 HBA connected to a single NetApp 24 drive shelf. All disks are multipath disks.
I am running FreeNAS 11.2 in a VM (ESXi 7.0), doing PCI pass through to my VM. This has worked and I do not believe this is related to my problem.
I have a 24 disk pool based on two 12 disk RAIDZ3 vDevs.
I had a disk start throwing smart errors so replaced it. here are my steps:
in the GUI, offline the disk with smart errors (validated by Serial Number)
physically swap the disk in the shelf
in CLI find the new disk with
camcontrol rescan allin the GUI i went to the pool, status, and tried to replace the disk. There were two disks still shown, da2 and da12. da12 was the active disk in the multipath so I selected that. It did not work, gave me an error to select a valid disk. da2 also didn't work. I searched the forums and found there's a bug in 11.2 with this, at least based on some posts. doing it in the CLI worked.
The resilveringi process for ~140Tib at ~80% took roughly 36 hours which was fine. completed without errors and the pool showed as healthy.
zpool replace tank /dev/gptid/440095da-a217-11e8-bf5f-e41f13e5b4c8 multipath/disk1
However after doing this I noticed that if I go to disks, da2 (multipath/disk1 is active on da14 and passive on da2) still shows as an available disk. if i go to pools, the GUI lets me attempt to create a new pool with da2.
I was going to start replacing disks in this vDev with larger ones - but this seems to be not right so before proceeding thought I should post here to see how to clean this up or if i need to remove disk1 and re-do multipathing.
here is the result of gmultipath list on the newly replaced disk. the results look similar to other disks, the only difference being the mode is r2w2e2 on the provider, and r3w3e3 on the consumers. all multipath disks are in an active passive state.
root@freenas[/]# gmultipath list disk1
Geom name: disk1
Type: AUTOMATIC
Mode: Active/Passive
UUID: 3be252a6-bb41-11ec-92ff-005056a863bd
State: OPTIMAL
Providers:
1. Name: multipath/disk1
Mediasize: 14000519642624 (13T)
Sectorsize: 512
Stripesize: 4096
Stripeoffset: 0
Mode: r1w1e1
State: OPTIMAL
Consumers:
1. Name: da14
Mediasize: 14000519643136 (13T)
Sectorsize: 512
Stripesize: 4096
Stripeoffset: 0
Mode: r2w2e2
State: ACTIVE
2. Name: da2
Mediasize: 14000519643136 (13T)
Sectorsize: 512
Stripesize: 4096
Stripeoffset: 0
Mode: r2w2e2
State: PASSIVE
Questions:
why is a member of a multipath disk showing as an available/unused disk.
how do I fix it to ensure i can safely resilver my pool to larger disks
Thanks all!
Last edited:
