EXPAND AND ADD/REPLACE

[Asimov1973]

Hello guys,
My Freenas system (9.10 U6) have a ZPOOL composed by TWO VDEV of 6 DISKS each in RAID Z2
Thanks to the case SC-4324S and the HBA 9305-24i I still have 12 slots free to add disks.

Now, in my first VDEV (composed by the 6 x 3TB) there is a disk that it's going to die so I would like to replace all 3TB disks with 8TB disks (already purchased 6 X ST8000NM000A).

I saw that basically I have the choice of replacing each 3TB old disk with 8TB new disk and have my VDEV in degraded mode OR put my 6 new 8TB disks in other slots and replace the 3TB disks but during the resilver the pool will not be degraded because the 3TB disks will remain all connected until the end.
I saw also some people talking about CLI and strange commands and UID and serial numbers and other stuff but I didn't went too much into it.

Can somebody explain to me (step by step if it's possible) the best and safe way to replace my 6 x 3TB disks with 6 x 8TB disks from the GUI either replacing them one by one or connect all the 8TB disks on the free SATA slots and then migrate/resilver ?
I also add three screenshots of my ZPOOL situation to avoid misunderstanding in case I did some mistake in explaining how my VDEVs are configured.

 

[sretalla]

Because you have a pool with 2 VDEVs, it's not the same as you have seen in other cases with pools with only a single VDEV.

Your old version of FreeNAS also takes top-level VDEV removal off the table (even though it would never be on there with RAIDZ2 members anyway).

Your only option of keeping the pool contents on the 6x6TB VDEV while swapping out the 6x3TB is to replace disks.

You don't need to remove a disk to replace it, so you don't have to degrade the pool to do that, just insert the new disk (you mention you have the room in the case for that) and do a replace on the oldest/least healthy disk (which will stay alive in the system while the resilver happens)... Remove the 3TB disk when the resilver completes. Continue to do that until all 3TB disks are gone, then you get a pool of 6x6 and 6x8 with all the 3TB removed.

 

[Asimov1973]

Because you have a pool with 2 VDEVs, it's not the same as you have seen in other cases with pools with only a single VDEV.

Your old version of FreeNAS also takes top-level VDEV removal off the table (even though it would never be on there with RAIDZ2 members anyway).

Your only option of keeping the pool contents on the 6x6TB VDEV while swapping out the 6x3TB is to replace disks.

You don't need to remove a disk to replace it, so you don't have to degrade the pool to do that, just insert the new disk (you mention you have the room in the case for that) and do a replace on the oldest/least healthy disk (which will stay alive in the system while the resilver happens)... Remove the 3TB disk when the resilver completes. Continue to do that until all 3TB disks are gone, then you get a pool of 6x6 and 6x8 with all the 3TB removed.

Thanks for the explanation, can you give me more details about the whole procedure of replacing a disk like how I tell to my zpool to start migrating the data from let’s say da9 to the new disk da12 ? (being da9 an old 3TB disk and da12 a new 8TB disk inserted in a free slot)

 

[sretalla]

how I tell to my zpool to start migrating the data from let’s say da9 to the new disk da12

8. Storage — FreeNAS® User Guide 9.10.2-U2 Table of Contents

www.ixsystems.com

If you have hot-plug bays, you can skip the shutdown/reboot steps.

 

[Asimov1973]

Big Thank you to Sretalla!
The procedure was absolutely correct. Resilvering lasted a week from monday 10-08-2020 evening to sunday 16-08-2020 evening. The first disk had 333 bad sectors and took three days to resilver, the other were all old but with no errors.
I added each disk one by one using the free sata ports and letting the pool always whole (not degraded). For each disk "migrated" I turned off my NAS, extracted the old disk, put the new disk in the same place of the old and added a new single disk on the first available sata port for the next one to be migrated.
Now I just need to change boot drive from USB to SSD....I'm still not convinced that the new freenas is for me...I find the 9.10 U6 really good and the GUI easy for find everything.....before taking the decision to install on ssd FREENAS 9.10 or 11.3 I think I will try it in a virtual machine