partitioning of disks in freenas

[bra1n]

I replaced a failed disk in a raidZ2 config (6 disks) some time ago, and I'm just noticing that the raidZ2 is composed of partitions on the original 5 disks, and the whole disk on the replacement disk:

Code:

  pool: tank
 state: ONLINE
  scan: resilvered 4.39M in 0 days 00:00:01 with 0 errors on Fri Nov 30 10:31:16 2018
config:

		NAME											STATE	 READ WRITE CKSUM
		tank											ONLINE	   0	 0	 0
		  raidz2-0									  ONLINE	   0	 0	 0
			gptid/b0215cb9-fd8c-11e4-85c4-d0509964a7f6  ONLINE	   0	 0	 0
			gptid/b0b9c307-fd8c-11e4-85c4-d0509964a7f6  ONLINE	   0	 0	 0
			gptid/b14f3f13-fd8c-11e4-85c4-d0509964a7f6  ONLINE	   0	 0	 0
			ada3										ONLINE	   0	 0	 0
			gptid/b279adfb-fd8c-11e4-85c4-d0509964a7f6  ONLINE	   0	 0	 0
			gptid/b30bba6f-fd8c-11e4-85c4-d0509964a7f6  ONLINE	   0	 0	 0

errors: No known data errors

After digging around it seems like FreeNAS sets up disks by default using two partitions (one with type freebsd-swap with 2G size and one with type freebsd-ZFS containing the rest of the storage on the disk). When I replaced the failed disk on ada3, I must have done it in such a way that those partitions were not recreated and FreeNAS was happy to just use the whole disk (I guess).

The system is working fine, but I would think there would be a way to remove ada3, partition it, and re-add it as a housekeeping effort if nothing else. Does anyone have any thoughts on how to do this?

 

[m0nkey_]

This is a tell-tail way that you use the CLI opposed to the GUI to replace the disk. The UI does a whole lot more, include partitioning the disk into ZFS and swap.

If it was me, I would probably offline it in the UI. Your pool will be degraded but you still have one disk redundancy. Once you've off-lined it, wipe the disk in another machine, put it back in and use the replace function in the UI.

Before you begin, make sure you have a full backup.

https://www.ixsystems.com/documentation/freenas/11/storage.html#replacing-a-failed-drive

 

[bra1n]

This is a tell-tail way that you use the CLI opposed to the GUI to replace the disk. The UI does a whole lot more, include partitioning the disk into ZFS and swap.

If it was me, I would probably offline it in the UI. Your pool will be degraded but you still have one disk redundancy. Once you've off-lined it, wipe the disk in another machine, put it back in and use the replace function in the UI.

Before you begin, make sure you have a full backup.

https://www.ixsystems.com/documentation/freenas/11/storage.html#replacing-a-failed-drive

Yeah that sounds right. It feels like there should be a way to do this without pulling the drive, but it's not a huge deal.

 

[bra1n]

This is a tell-tail way that you use the CLI opposed to the GUI to replace the disk. The UI does a whole lot more, include partitioning the disk into ZFS and swap.

If it was me, I would probably offline it in the UI. Your pool will be degraded but you still have one disk redundancy. Once you've off-lined it, wipe the disk in another machine, put it back in and use the replace function in the UI.

Before you begin, make sure you have a full backup.

https://www.ixsystems.com/documentation/freenas/11/storage.html#replacing-a-failed-drive

With the the disk out of the machine and sitting in a SATA-USB cradle, what is the best way to wipe it so that freenas sees it as a completely new disk? It doesn't seem like just doing a repartition and quick format in Windows is sufficient.

 

[pro lamer]

Correct me if I'm wrong: Some sectors at the end of the disk need to be wiped too. If you're limited to Windows, you can try DD cygwin's command or use some disk sanitation software.

Sent from my mobile phone

 

[bra1n]

Correct me if I'm wrong: Some sectors at the end of the disk need to be wiped too. If you're limited to Windows, you can try DD cygwin's command or use some disk sanitation software.

Sent from my mobile phone

I can plug the cradle into a Linux box too. Would something like

Code:

dd if=/dev/zero of=/dev/sdX bs=1M

Be sufficient?

 

[danb35]

Really, you can do it all on the FreeNAS box, there's just the danger of your inadvertently specifying the wrong disk. Once you offline the disk in the GUI, gpart destroy ada3 should do the job.

 

[bra1n]

Really, you can do it all on the FreeNAS box, there's just the danger of your inadvertently specifying the wrong disk. Once you offline the disk in the GUI, gpart destroy ada3 should do the job.

gpart doesn't recognize the device anymore:

Code:

sudo gpart destroy ada3
gpart: arg0 'ada3': Invalid argument

So I've gone through a few iterations trying to get freenas to not see this disk as the same one as before.

After using dd to zero out the last MB of the disk, the first MB, the last 100MB, the first 100MB, and still after onlining the disk through the UI I end up with:

Code:

  pool: tank
 state: DEGRADED
status: One or more devices could not be opened.  Sufficient replicas exist for
        the pool to continue functioning in a degraded state.
action: Attach the missing device and online it using 'zpool online'.
   see: http://illumos.org/msg/ZFS-8000-2Q
  scan: scrub repaired 0 in 0 days 02:52:34 with 0 errors on Fri Dec  7 15:19:19 2018
config:

        NAME                                            STATE     READ WRITE CKSUM
        tank                                            DEGRADED     0     0     0
          raidz2-0                                      DEGRADED     0     0     0
            gptid/b0215cb9-fd8c-11e4-85c4-d0509964a7f6  ONLINE       0     0     0
            gptid/b0b9c307-fd8c-11e4-85c4-d0509964a7f6  ONLINE       0     0     0
            gptid/b14f3f13-fd8c-11e4-85c4-d0509964a7f6  ONLINE       0     0     0
            1875012985446647245                         UNAVAIL      0     0     0  was /dev/ada3
            gptid/b279adfb-fd8c-11e4-85c4-d0509964a7f6  ONLINE       0     0     0
            gptid/b30bba6f-fd8c-11e4-85c4-d0509964a7f6  ONLINE       0     0     0

errors: No known data errors

Any ideas what I can do to this disk to get freenas to think it is completely new?

 

[danb35]

Any ideas what I can do to this disk to get freenas to think it is completely new?

Well, I guess one way would be to manually partition the disk as FreeNAS does and do the replacement at the CLI (instructions for the former step can be found here; the latter step would be, IIRC, zpool replace tank 1875012985446647245 gptid/whatever, using the gptid for the second partition. I don't think that's necessarily the best thing to do, but it would get you back to where FreeNAS would expect to be.

 

[bra1n]

Well, I guess one way would be to manually partition the disk as FreeNAS does and do the replacement at the CLI (instructions for the former step can be found here; the latter step would be, IIRC, zpool replace tank 1875012985446647245 gptid/whatever, using the gptid for the second partition. I don't think that's necessarily the best thing to do, but it would get you back to where FreeNAS would expect to be.

That looks to have worked, thanks!

pool: tank
state: DEGRADED
status: One or more devices is currently being resilvered. The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scan: resilver in progress since Sat Dec 8 12:09:38 2018
809G scanned at 805M/s, 567M issued at 3.35M/s, 1.43T total
0 resilvered, 0.04% done, no estimated completion time
config:

NAME STATE READ WRITE CKSUM
tank DEGRADED 0 0 0
raidz2-0 DEGRADED 0 0 0
gptid/b0215cb9-fd8c-11e4-85c4-d0509964a7f6 ONLINE 0 0 0
gptid/b0b9c307-fd8c-11e4-85c4-d0509964a7f6 ONLINE 0 0 0
gptid/b14f3f13-fd8c-11e4-85c4-d0509964a7f6 ONLINE 0 0 0
replacing-3 UNAVAIL 0 0 0
1875012985446647245 UNAVAIL 0 0 0 was /dev/ada3
gptid/9853c717-fb0b-11e8-add2-d05099192ca7 ONLINE 0 0 0
gptid/b279adfb-fd8c-11e4-85c4-d0509964a7f6 ONLINE 0 0 0
gptid/b30bba6f-fd8c-11e4-85c4-d0509964a7f6 ONLINE 0 0 0

 

[Allan Jude]

Note: this command should only be used when you are VERY sure you know what you are doing. It will make this disk unrecoverable.

zpool labelclear -f /dev/ada3

 

[Redcoat]

Note: this command should only be used when you are VERY sure you know what you are doing. It will make this disk unrecoverable.

zpool labelclear -f /dev/ada3

For my education, can you define/qualify "unrecoverable" in this context, please?

 

[Allan Jude]

For my education, can you define/qualify "unrecoverable" in this context, please?

It spoils (purposely corrupts) the 4 copies of the ZFS label on the disk, so that it is not consider part of a pool anymore. It is how you remove the ZFS data that identifies a disk as part of a pool, ahead of adding it to a different pool, without having to spend large amounts of time overwriting the drive with 0s, or trying to exactly overwrite just the labels etc.

 

[Redcoat]

Thanks. That's helpful background.