Planning replication...?

[BigPapaPSP]

Okay -- A few months ago a spent a lot of time (and $$) moving everything to new hardware and upgrading to latest FreeNas 11.2. I started with four 10TB drives in a pool. Which is reaching capacity. My server is a 6-bay dual node server. And FreeNAS is running on both nodes. Node "A" has the 40TB drive pool. My PLAN was to start with four drives, and add the other two as needed to expand my pool. WELL, That didn't work as planned as I didn't plan on adding them as STRIPEd but as part of my Z2 pool. SOoooo

I'm now getting to the point where I need to add space -- so to fix my issue with not being able to add 2 more drives to the same Z2 drive pool, I'm considering adding 6 NEW 10TB drives on NODE "B" -- and replicate my pool from NODE "A" to NODE "B" -- end result being my pool will be on 60TB (6x10 @ Z2) versus 60TB (4x10 @ Z2 + 10tb stripe + 10tb stipe).

Any issues with changing or creating the same shares and plugins? Do I need to re-install the plugin? Or will the IOcage and configurations, etc transfer over as well? Bottom line is I screwed up -- I should have started with 6 drives instead of 4 -- but I thought I had growth options to add the other 2 drives. I have 2.5TB free on the 40tb server... How should I expect this to work out? Any hurdles? Is there an easier solution?

Please advise.

 

[PhiloEpisteme]

(6x10 @ Z2)

This is a much preferred setup to...

4x10 @ Z2 + 10tb stripe + 10tb stipe

This is not an ideal pool setup. Keep in mind that pools are made of vdev and vdevs are made of disks. You lose your pool if you lose a vdev and you loose a vdev if you lose enough disks in a vdev such that it cannot deliver your data. With the 4x10TB RAIDZ2 + 1x10TB + 1x10TB you could lose your entire pool any of those three vdevs fail. If you lose 3 drives in the RAIDZ2 vdev (unlikely) or either of the single 10TB drives in the striped vdev (very likely over time) you'd lose your pool. Avoid single-points of failure for data you care about. Stick with RAIDZ2 or mirrors where possible.

I thought I had growth options to add the other 2 drives

It seems like you've recently discovered this but it is worth being clear about it. You do have growth options. You can grow a pool by adding vdevs to the pool, grow a vdev by replacing each drive in it with larger drives, or add another pool with entirely new vdevs. You can't grow a pool by just adding new disks to a RAIDZ vdev though. :/

I'm considering adding 6 NEW 10TB drives on NODE "B" -- and replicate my pool from NODE "A" to NODE "B" -- end result being my pool will be on 60TB (6x10 @ Z2) versus 60TB (4x10 @ Z2 + 10tb stripe + 10tb stipe).

I think this is reasonable. You may have to update some configs etc once you do the move but you'll still have all of your data in a much safer setup.

If I'm not mistaken, once this process is done you'll have 10x10TB drives, yes? You want to try to avoid filling your pools beyond 80% capacity as that will impact your performance. If you grow too full you can run into other serious issues such as not being able to mount the pool. If your storage needs grow again you may consider adding another RADIZ2 vdev out of 6x10TB drives to that pool.

 

[BigPapaPSP]

It seems like you've recently discovered this but it is worth being clear about it. You do have growth options. You can grow a pool by adding vdevs to the pool, grow a vdev by replacing each drive in it with larger drives, or add another pool with entirely new vdevs. You can't grow a pool by just adding new disks to a RAIDZ vdev though. :/


I think this is reasonable. You may have to update some configs etc once you do the move but you'll still have all of your data in a much safer setup.

If I'm not mistaken, once this process is done you'll have 10x10TB drives, yes? You want to try to avoid filling your pools beyond 80% capacity as that will impact your performance. If you grow too full you can run into other serious issues such as not being able to mount the pool. If your storage needs grow again you may consider adding another RADIZ2 vdev out of 6x10TB drives to that pool.

Yes I am now aware I can add Vdev's, but problem is I only have 6 bays in each node. So if I wanted to add another 4-disk vdev to that node, I wouldn't have the open drive bays.

And yes, that would mean 10x10tb -- if I kept both nodes up, but once I got the 6x10tb online and all plugin/data configured -- I'd re-do the vdevs on node "A" and do it with 6 drives as well. I looked into expanding the 10tb to 12tb or larger, but at this point the larger drives beyond 10tb are just not cost effective yet.

 

[PhiloEpisteme]

Yes I am now aware I can add Vdev's, but problem is I only have 6 bays in each node. So if I wanted to add another 4-disk vdev to that node, I wouldn't have the open drive bays.

Yeah, that happens. You can do as you mentioned and add drives to the other node OR you could pick up a chassis with a bunch of bays and hook it up via SAS expander to your primary server. This second chassis only really houses drives and has no need for a full motherboard, memory, etc. Either way you'll be able to make good use of those drives. :)

 

[artlessknave]

wile not recommended, it is possible to attach a mirror to a raidz pool. this would be better than turning your whole pool into a stripe, and would give you some room to work with until you have everything sorted out (zfs doesn't like having no free space). you would need a total of 12x 10TB drives though, in order to properly migrate over to 6 drive raidz2. you would have:
tank
raidz2 (4x10TB)
mirror (2x10TB)
you don't seem to list any if your hardware which is a forum rule, and is one way to ensure no replies by the Guru's here. I'm not clear what you mean by "node"; do you have 2 full servers?
if you replicate the entire pool everything will go with it storage wise, but your freenas specific configuration wont. the GUI should see your plugins and jails on your storage. if by "node", you mean you have 2 identical servers, you ought to be able to replicate the storage, save the config, shutdown both, and then upload your config to the 2nd. that should restore everything exactly the same as restoring the OS any other time. then you can blow away the other raidz2 and either use it as another pool or backup (recommended)

if your undefined "2nd node" is less hardware, you should still be able to do this but it would be a bit more complicated.
clone atank to btank.
save anode config
swap all 10-12 disks
import btank on anode.
restore anode config
blow atank away.

I've never tried the above, but there's a chance of that working.