modify 2 disk stripe+ZIL to 4 disk (2xmirrored) zpool

[dreadpirate roberts]

First time posting here, though I've had an (incorrectly configured) freenas home server for about 3.5 years. I first started with just one HDD, then (thought) I had a mirror with the addition of 2nd HDD after initial setup/config. About 1.5 years ago, I added 3rd HDD, believing I'd get a RAID5 equivalent config, with ultimate goal of adding 4th HDD to get 2x mirrored vdevs. Well, only when I finally tried to add 4th HDD (all HDDs are WD Red 3TB) last week via volume manager, and got following message:

... did I realize my first 3 disks were not configured like I thought (entirely my fault as I hadn't bothered to check carefully after each HDD addition). In particular, I have no idea how I ended up with a ZIL (too much time has passed so I can't honestly remember the exact steps I took, other than using volume manager to add the 2nd and 3rd disks). I realize I need to remove the ZIL per info I found here. Since this is a ZIL, I should be able to remove without affecting my current 2 disk striped vdev, right?

I found steps here and here with info on how to change a single disk stripe into a mirror. It's not clear to me whether the steps in the links would work for my 2 disk stripe. I'd appreciate any/all suggestions on how best to convert the 2 disk stripe to a 2 disk mirror, assuming it can be done.

If it cannot be done, I'm guessing I would need to use the (recovered) 3rd HDD, along with 4th HDD to create a new mirrored vdev and then copy the data over from my current 2 disk stripe, before wiping the stripe and then reconfigure that as a 2nd mirror, before adding both mirrors to a zpool? Or is it better to create a new zpool at same time as I create the new mirrored vdev (using 3rd/4th HDDs) and add the 2nd mirrored vdev to the zpool? Sorry if I'm asking a very obvious question, but, as you can see, I've not created/used a zpool before.

Thanks in advance.

 

[Robert Trevellyan]

this is a ZIL, I should be able to remove

Yes.

Further, it is possible to convert a 2-disk stripe into a pair of mirrors, but unless things have changed with Corral, you can't do it from the GUI. Instead, you need to use the zpool attach command from the FreeBSD shell. The tricky part is, by default the command will use the whole disk, but FreeNAS likes to partition its disks. If you're not super fussy, you can live with the added disks not being partitioned the same way as the original disks.

Take a look at the documentation to get a sense for what's involved: https://www.freebsd.org/cgi/man.cgi?zpool(8). If possible, practice with a VM.

It isn't complicated, but if using the shell makes you uncomfortable, you might consider backing up everything and starting over instead.

If it cannot be done, I'm guessing I would need to use the (recovered) 3rd HDD, along with 4th HDD to create a new mirrored vdev and then copy the data over from my current 2 disk stripe, before wiping the stripe and then reconfigure that as a 2nd mirror, before adding both mirrors to a zpool?

You appear to be overthinking this scenario:

1. Create new pool as 2-disk mirror.
2. Move data to new pool.
3. Destroy old pool, marking disks as new.
4. Extend new pool with 2-disk mirror.

 

[dreadpirate roberts]

Y

1. Create new pool as 2-disk mirror.
2. Move data to new pool.
3. Destroy old pool, marking disks as new.
4. Extend new pool with 2-disk mirror.

I'm not super adept at using the CLI, so I opted to create a new 2 disk mirror instead. I'm currently using send/receive of snapshot (of 2 disk stripe) to move data over to the mirrored pool (taking a while but seems to be progressing smoothly). Thanks so much for the prompt response. Much appreciated.

 

[dreadpirate roberts]

Further, it is possible to convert a 2-disk stripe into a pair of mirrors...you need to use the zpool attach command from the FreeBSD shell.

Using send/receive to transfer snapshot of 2 disk striped data did not work (I ran out of space on the newly created 3TB mirrored pool as original 2 disk stripe now shows up 5.44TB after removal of ZIL, but <3TB prior to that; don't understand this).

So, I then tried using your suggested 'zpool attach' command (my exact command was "zpool attach -f WD30EFRX1 WD30EFRX1 WD3TBmirror1"), which resulted in an error message of "cannot open 'WD3TBmirror1': no such GEOM provider must be a full path or shorthand device name".

Perhaps I don't need to create the mirrored pool first and 'zpool attach' will do it automatically, but in that case, how do I specify both empty HDDs as the 'new device' in the command line. I'd appreciate any pointers as to what I'm doing wrong here/need to do. Thanks again.

 

[Robert Trevellyan]

Since your new pool is not large enough to backup the data, the next step is to destroy it so you can reuse the disks. The screenshot you posted appears to show a pool named WD3TB consisting of a single mirror vdev.

Once you've destroyed that pool, you need to figure out the device names for all four disks in order to use the zpool attach command. On FreeBSD, SATA disks usually have names like /dev/ada0, /dev/ada1 etc.

For each disk in the original 2-disk stripe, you will issue a zpool attach to pair it with one of the new disks.

 

[dreadpirate roberts]

On FreeBSD, SATA disks usually have names like /dev/ada0, /dev/ada1 etc.

For each disk in the original 2-disk stripe, you will issue a zpool attach to pair it with one of the new disks.

Thanks for clarifying; makes perfect sense now. For whatever reason, I thought it was one command operating on the stripe/both disks at same time.

So, just went to try it and still no joy :-( I have 4 disks, ada0 thru ada3. Unless I'm mistaken, the stripe uses ada0 & ada2 (see above). I'm assuming for the pool in the attach command, I should use the pool name of the existing stripe. When I issued "zpool attach -f WD30EFRX1 /dev/ada0 /dev/ada1" command, I got an error msg "cannot attach /dev/ada1 to /dev/ada0: no such device in pool"; same result using the other pair of striped/empty disks. Since the striped volume status shows the disks/devices as ada0p2 & ada2p2, I also tried using the "p2 suffix" to specify the 1st device in command and got same error. I know, grasping at straws here. I even tried giving a new pool name in the command, but that didn't help.

Sorry to be such a pain, with what appears pretty straight forward, but I'm at a loss at this point. Appreciate that last bit of help to get me over the hump. By the way, is it necessary to wait for operation on 1st disk to completely finish before proceeding with 2nd disk? Thank you.

 

[Robert Trevellyan]

You probably need to use a different method to identify the original disks. It might be as simple as using the IDs displayed by zpool status. If not, look for subfolders in /dev, e.g. /dev/by-id or similar.

 

[dreadpirate roberts]

You probably need to use a different method to identify the original disks.

Ended up using gptid to identify disks. Took a while for both striped disks to resilver, but here is the end result

Thank you SO MUCH for all your help. Much appreciated.

 

[Robert Trevellyan]

 

[Robert Trevellyan]

It looks like you added the new mirror to the SATA_V1 pool, which is where zvol2 lives. This seems correct. Maybe you just need to read up on how to change the space allocated to a ZVOL?

 

[Robert Trevellyan]

Have you tried changing the settings for the zvol?
http://doc.freenas.org/9.10/storage.html#create-zvol

I don't use zvols, so if that doesn't work, maybe someone else can chime in.