Advice on RAIDZ1 reconfiguration

[alex740il]

Hi All, my first post here, hopefully can get some advice.

I have been running a TrueNAS Scales system for some time now, and everything has been great. I built it with 4x 4TB drives in RAIDZ1. However, my use-case has changed, and now I am adding two additional 4TB drives, but I also would like to make the system more resilient to drive failures. This brings to be a question, is it possible to change to RAIDZ2 without data loss? If not, any other options/recommendations? Also, can I add a cache drive (NVME) and remove it later without breaking the storage pool?

Show : See My freeNAS system

OS:TrueNAS-SCALE-23.10.1
CPU:12th Gen Intel(R) Core(TM) i5-12600K
Memory: 62 GiB
GPU: NVIDIA Tesla P4
OS Drive: NVME 512GB
Data VDEV: 1 x RAIDZ1 | 4 wide | 3.64 TiB

 

[Arwen]

...
This brings to be a question, is it possible to change to RAIDZ2 without data loss?
...

No.

...
If not, any other options/recommendations?
...

The only real option is backup, destroy, create as RAID-Z2 and restore.

...
Also, can I add a cache drive (NVME) and remove it later without breaking the storage pool?

Yes.

Note that in general, Cache / L2ARC drives should be about 5 times memory size. With 62GByte of RAM, that is about 300GByte. Some say you can go up to 10 times, (600GByte in your case). Cache / L2ARC data requires some RAM for pointers and metadata for the Cache / L2ARC entries. So excessively large Cache / L2ARC could starve the ARC / L1ARC, which exists solely in RAM.

 

[alex740il]

No.


The only real option is backup, destroy, create as RAID-Z2 and restore.


Yes.

Note that in general, Cache / L2ARC drives should be about 5 times memory size. With 62GByte of RAM, that is about 300GByte. Some say you can go up to 10 times, (600GByte in your case). Cache / L2ARC data requires some RAM for pointers and metadata for the Cache / L2ARC entries. So excessively large Cache / L2ARC could starve the ARC / L1ARC, which exists solely in RAM.

Appreciate the quick response... In this case, correct me if I am wrong, but I can add a 4TB disk, create a storage pool with the single drive to temporarily store 3TB of data on it, then rebuild current as RAID-Z2, move the data back, format the temp drive used for backup, and add it to the new pool? What is the best way to backup everything? Short of creating smb shares and copying between the two..

Is there anything I can use the 256 GB NVME for in this system? It's just hanging around, doing nothing... heh.

 

[ChrisRJ]

Before adding a L2ARC it makes sense to first check the ARC hit rate. In my case, as an example, it would be a complete waste of money because my ARC hit rate has a mean of 99.58 and a minimum of 92.69 over the last 6 months.

Also L2ARC works differently than what most people associate with cache. It helps when you read the same stuff multiple times, but not the first time. The good thing, though, is that you can easily remove it (other then e.g. a special metadata VDEV).

I don't want to discourage you, all I'm saying is that the use-case must match or L2ARC will not bring much or any benefit.

 

[Arwen]

Appreciate the quick response... In this case, correct me if I am wrong, but I can add a 4TB disk, create a storage pool with the single drive to temporarily store 3TB of data on it, then rebuild current as RAID-Z2, move the data back, format the temp drive used for backup, and add it to the new pool? What is the best way to backup everything? Short of creating smb shares and copying between the two..
...

Yes, you can add a 4TB disk to your NAS and make a single disk pool for backup purposes. Then use ZFS Send & Receive to copy the data.

But, no, at present there is no way to expand the RAID-Z2 with a single disk. It's a planned feature and will likely start beta testing towards the end of this year, (2024). But, it is not available at present.

Look, their are tricks and things that can be done for that single drive & adding it to your RAID-Z2 vDev. BUT, they are ALL high risk. So, the general rule here in the forum is that if you have to ask, you don't have the skill set to perform the task without potentially loosing your data. None of us want that to happen, nor take responsibility for that outcome.

...
Is there anything I can use the 256 GB NVME for in this system? It's just hanging around, doing nothing... heh.

Depends on your use case. A single NVMe drive would not have any drive redundancy, so less useful for applications. And too big for a boot pool Mirror device.