Replace disk got wrong on raidz1-0 :-(

[jeepee]

Hey,

Something went wrong in an attempt to replace a disc.
I did something wrong but I don't know what.
Disk 1 of the raidz1 configuration (total of 4 disks) gave errors and I wanted to replace them.
I have put the server down and connected another disk in the same place. (in retrospect not the right way)
Then with the (old) web interface added this disk to my "raid" pool. (pool is called raid)

Only the raid is still degraded and the disk I have replaced has not been added in the right place.

Remove the disk,
gptid / 8c2ea1e4-f268-11e9-95f8-001cc09874e6,
failed because this is not allowed.

Is there still a possibility to reverse this or do I have to set up the 4 disk raid configuration again.

I still have a config file of the correct configuration.

I have replaced a disk more often and always went well but what I have done now was not smart. Too much rush.

thanks in advance for advice,
Jan

Code:

root@fnas[~]# zpool status
  pool: freenas-boot
 state: ONLINE
  scan: scrub repaired 0 in 0 days 00:01:08 with 0 errors on Thu Oct 17 03:46:09 2019
config:

    NAME        STATE     READ WRITE CKSUM
    freenas-boot  ONLINE       0     0     0
      ada0p2    ONLINE       0     0     0

errors: No known data errors

  pool: raid
 state: DEGRADED
status: One or more devices has been taken offline by the administrator.
    Sufficient replicas exist for the pool to continue functioning in a
    degraded state.
action: Online the device using 'zpool online' or replace the device with
    'zpool replace'.
  scan: scrub canceled on Sat Oct 19 13:10:21 2019
config:

    NAME                                            STATE     READ WRITE CKSUM
    raid                                            DEGRADED     0     0     0
      raidz1-0                                      DEGRADED     0     0     0
        3091878060739501801                         OFFLINE      0     0     0  was /dev/gptid/56130691-4c01-11e9-a9a7-001cc09874e6
        ada2p2                                      ONLINE       0     0     0
        gptid/f5e72458-74f3-11e9-8d4a-001cc09874e6  ONLINE       0     0     0
        gptid/5c822db3-4c01-11e9-a9a7-001cc09874e6  ONLINE       0     0     0
      gptid/8c2ea1e4-f268-11e9-95f8-001cc09874e6    ONLINE       0     0     0

errors: No known data errors

 

[PhiloEpisteme]

Yes, you made a mistake. You added your new drive as a single-drive vdev which you striped to your RAIDZ1 pool. You're in a very bad place now because if any other drive fails you lose your entire pool; even if you replace 3091878060739501801 you're still in trouble if gptid/8c2ea1e4-f268-11e9-95f8-001cc09874e6 fails.

It looks like it may be that you did some work via the CLI as well? I'm guessing because I see one drive labeled as ada2p2 and the others by gptid. I highly recommend that you stick to the GUI and follow all guidelines and procedures as outlined in the User Guide.

If you have a way to back up your data or you already have good backups you should be able to recover from this fairly easily. I hope this answers your question, let us know if not.

It may be worth reviewing some of the documentation/information about the relationships between pools, vdevs, etc as you proceed as well; it may help you avoid getting into tricky spots like this in the future.

 

[Heracles]

Hey Jeepee,

Unfortunately, as Philo mentioned, you did put yourself in deep trouble. The complete solution will be to empty your pool by moving all of your data to another support. Once the pool empty, you destroy it and re-create it.

Until you can do that, you can mitigate the risk if you can add 2 drives to your system. One drive needs to mirror your single freshly added drive to transform it to a mirror. The second drive will be to re-silver your raid-z1 vDev.

If you can not add these 2 drives, be sure to backup everything critical out of the pool because as of now, you made it very very shaky.

Good luck,

 

[jeepee]

Thanks for the answers.
I know. Expected that this was irreversible
Adding extra disks is not possible with this configuration. Good tip to still be able to save the case in this way.
I have a backup and will delete and recreate the pool.

Jan