ZFS Pool Resilvered, Upon Reboot forgets new drive

Status
Not open for further replies.
G

GregSC2

Dabbler
Joined
Aug 17, 2013
Messages
13
I had a drive fail in a storage pool. The storage pool is 3x 1.5 TB drives in the motherboard SATA ports. I replaced the drive, resilvered the new drive, then performed the scrub. All was well, until I rebooted. Upon reboot, i ran a zpool status -v oneterrorbyte. It showed the old drive listed as unavailable ratherthan the new drive, which has already been resilvered and scrubbed as a member of the pool.


Code:
[root@FreeNAS] ~# zpool status -v oneterrorbyte
pool: oneterrorbyte
state: DEGRADED
status: One or more devices could not be opened. Sufficient replicas exist for
the pool to continue functioning in a degraded state.
action: Attach the missing device and online it using 'zpool online'.
see: http://illumos.org/msg/ZFS-8000-2Q
scan: scrub repaired 24K in 7h6m with 0 errors on Sat Aug 17 15:10:26 2013
config:
 
NAME STATE READ WRITE CKSUM
oneterrorbyte DEGRADED 0 0 0
raidz1-0 DEGRADED 0 0 0
4548809083459598154 UNAVAIL 0 0 0 was /dev/ada1
gpt/da2 ONLINE 0 0 0
gpt/da3 ONLINE 0 0 0


I have tried a zfs attach oneterrorbyte 4548809083459598154 ada1. However, it says that new drive is part of the pool already?

Code:
[root@FreeNAS] ~# zpool attach -f oneterrorbyte 4548809083459598154 ada1
invalid vdev specification
the following errors must be manually repaired:
/dev/ada1 is part of active pool 'oneterrorbyte'


Two questions:

1. Why isn't freenas remembering my new pool and how do I get it to remember in the future?
2. How do I re-attach the new drive which is plugged in and has been resilvered.?
 
paleoN

paleoN

Wizard
Joined
Apr 22, 2012
Messages
1,403
First, stop quoting and start using [code][/code] tags for CLI output. What's the FreeNAS version?

GregSC2 said:
1. Why isn't freenas remembering my new pool and how do I get it to remember in the future?
Click to expand...
Speculating, you may have some bits of an old label on the drive. Where/how was this pool originally created.

GregSC2 said:
2. How do I re-attach the new drive which is plugged in and has been resilvered.?
Click to expand...
Using [code] tags the output of:
Code:
zpool status -v

camcontrol devlist

gpart show

glabel status

zdb -l /dev/ada1

zdb -l /dev/gpt/da2
 
G

GregSC2

Dabbler
Joined
Aug 17, 2013
Messages
13
The volume was originally created on FreeNAS 8 using the ZFS Volume Manager. I am currently on FreeNAS 9.1.0. The current /dev/ada1 is the drive that has been resilvered by FreeNAS 9.1.0 and was working and listed prior to reboot.

zpool status -v:
Code:
  pool: oneterrorbyte
state: DEGRADED
status: One or more devices could not be opened.  Sufficient replicas exist for
        the pool to continue functioning in a degraded state.
action: Attach the missing device and online it using 'zpool online'.
  see: http://illumos.org/msg/ZFS-8000-2Q
  scan: scrub repaired 24K in 7h6m with 0 errors on Sat Aug 17 15:10:26 2013
config:
 
        NAME                    STATE    READ WRITE CKSUM
        oneterrorbyte            DEGRADED    0    0    0
          raidz1-0              DEGRADED    0    0    0
            4548809083459598154  UNAVAIL      0    0    0  was /dev/ada1
            gpt/da2              ONLINE      0    0    0
            gpt/da3              ONLINE      0    0    0
 
errors: No known data errors
 
  pool: twoterrorbyte
state: ONLINE
  scan: resilvered 22.0M in 0h6m with 0 errors on Sat Aug 10 20:39:43 2013
config:
 
        NAME          STATE    READ WRITE CKSUM
        twoterrorbyte  ONLINE      0    0    0
          raidz1-0    ONLINE      0    0    0
            gpt/ada0  ONLINE      0    0    0
            gpt/ada1  ONLINE      0    0    0
            gpt/da0  ONLINE      0    0    0
 
errors: No known data errors


camcontrol devlist:
Code:
<WDC WD15EARS-00Z5B1 80.00A80>    at scbus0 target 0 lun 0 (ada0,pass0)
<WDC WD15EZRX-00DC0B0 80.00A80>    at scbus1 target 0 lun 0 (ada1,pass1)
<WDC WD15EARS-00Z5B1 80.00A80>    at scbus2 target 0 lun 0 (ada2,pass2)
<WDC WD20EARS-00MVWB0 51.0AB51>    at scbus3 target 0 lun 0 (ada3,pass3)
<WDC WD20EARS-00MVWB0 51.0AB51>    at scbus4 target 0 lun 0 (ada4,pass4)
<WDC WD20EARS-00MVWB0 51.0AB51>    at scbus5 target 0 lun 0 (ada5,pass5)
<Kingston DataTraveler G3 PMAP>    at scbus6 target 0 lun 0 (pass6,da0)


gpart show:
Code:
=>      63  30481089  da0  MBR  (14G)
        63  1930257    1  freebsd  [active]  (942M)
  1930320        63      - free -  (31k)
  1930383  1930257    2  freebsd  (942M)
  3860640      3024    3  freebsd  (1.5M)
  3863664    41328    4  freebsd  (20M)
  3904992  26576160      - free -  (12G)
 
=>      0  1930257  da0s1  BSD  (942M)
        0      16        - free -  (8.0k)
      16  1930241      1  !0  (942M)
 
=>        34  2930277101  ada0  GPT  (1.4T)
          34          94        - free -  (47k)
        128    4194304    1  freebsd-swap  (2.0G)
    4194432  2926082703    2  freebsd-zfs  (1.4T)
 
=>        34  2930277101  ada1  GPT  (1.4T) [CORRUPT]
          34          94        - free -  (47k)
        128    4194304    1  freebsd-swap  (2.0G)
    4194432  2926082696    2  freebsd-zfs  (1.4T)
  2930277128          7        - free -  (3.5k)
 
=>        34  2930277101  ada2  GPT  (1.4T)
          34          94        - free -  (47k)
        128    4194304    1  freebsd-swap  (2.0G)
    4194432  2926082703    2  freebsd-zfs  (1.4T)
 
=>        34  3907029101  ada3  GPT  (1.8T)
          34          94        - free -  (47k)
        128    4194304    1  freebsd-swap  (2.0G)
    4194432  3902834703    2  freebsd-zfs  (1.8T)
 
=>        34  3907029101  ada4  GPT  (1.8T)
          34          94        - free -  (47k)
        128    4194304    1  freebsd-swap  (2.0G)
    4194432  3902834703    2  freebsd-zfs  (1.8T)
 
=>        34  3907029101  ada5  GPT  (1.8T) [CORRUPT]
          34          94        - free -  (47k)
        128    4194304    1  freebsd-swap  (2.0G)
    4194432  3902834703    2  freebsd-zfs  (1.8T)


glabel status:
Code:
                            ufs/FreeNASs3    N/A  da0s3
                            ufs/FreeNASs4    N/A  da0s4
                            ufs/FreeNASs1a    N/A  da0s1a
                                  gpt/da3    N/A  ada0p2
gptid/b114f041-0540-11e3-84d8-0025903bc102    N/A  ada1p2
                                  gpt/da2    N/A  ada2p2
                                  gpt/ada0    N/A  ada3p2
                                  gpt/ada1    N/A  ada4p2
                                  gpt/da0    N/A  ada5p2


zdb -l /dev/ada1:
Code:
--------------------------------------------
LABEL 0
--------------------------------------------
    version: 15
    name: 'oneterrorbyte'
    state: 0
    txg: 1541528
    pool_guid: 8210305841187571992
    hostid: 1318763924
    hostname: 'FreeNAS.local'
    top_guid: 16858560762606922925
    guid: 4548809083459598154
    vdev_children: 1
    vdev_tree:
        type: 'raidz'
        id: 0
        guid: 16858560762606922925
        nparity: 1
        metaslab_array: 23
        metaslab_shift: 35
        ashift: 9
        asize: 4494448656384
        is_log: 0
        children[0]:
            type: 'disk'
            id: 0
            guid: 4548809083459598154
            path: '/dev/ada1'
            phys_path: '/dev/ada1'
            whole_disk: 0
            DTL: 111
        children[1]:
            type: 'disk'
            id: 1
            guid: 316550613392296400
            path: '/dev/gpt/da2'
            phys_path: '/dev/gpt/da2'
            whole_disk: 0
            DTL: 31
        children[2]:
            type: 'disk'
            id: 2
            guid: 11515767252682951891
            path: '/dev/gpt/da3'
            phys_path: '/dev/gpt/da3'
            whole_disk: 0
            DTL: 30
    features_for_read:
--------------------------------------------
LABEL 1
--------------------------------------------
    version: 15
    name: 'oneterrorbyte'
    state: 0
    txg: 1541528
    pool_guid: 8210305841187571992
    hostid: 1318763924
    hostname: 'FreeNAS.local'
    top_guid: 16858560762606922925
    guid: 4548809083459598154
    vdev_children: 1
    vdev_tree:
        type: 'raidz'
        id: 0
        guid: 16858560762606922925
        nparity: 1
        metaslab_array: 23
        metaslab_shift: 35
        ashift: 9
        asize: 4494448656384
        is_log: 0
        children[0]:
            type: 'disk'
            id: 0
            guid: 4548809083459598154
            path: '/dev/ada1'
            phys_path: '/dev/ada1'
            whole_disk: 0
            DTL: 111
        children[1]:
            type: 'disk'
            id: 1
            guid: 316550613392296400
            path: '/dev/gpt/da2'
            phys_path: '/dev/gpt/da2'
            whole_disk: 0
            DTL: 31
        children[2]:
            type: 'disk'
            id: 2
            guid: 11515767252682951891
            path: '/dev/gpt/da3'
            phys_path: '/dev/gpt/da3'
            whole_disk: 0
            DTL: 30
    features_for_read:
--------------------------------------------
LABEL 2
--------------------------------------------
    version: 15
    name: 'oneterrorbyte'
    state: 0
    txg: 1541528
    pool_guid: 8210305841187571992
    hostid: 1318763924
    hostname: 'FreeNAS.local'
    top_guid: 16858560762606922925
    guid: 4548809083459598154
    vdev_children: 1
    vdev_tree:
        type: 'raidz'
        id: 0
        guid: 16858560762606922925
        nparity: 1
        metaslab_array: 23
        metaslab_shift: 35
        ashift: 9
        asize: 4494448656384
        is_log: 0
        children[0]:
            type: 'disk'
            id: 0
            guid: 4548809083459598154
            path: '/dev/ada1'
            phys_path: '/dev/ada1'
            whole_disk: 0
            DTL: 111
        children[1]:
            type: 'disk'
            id: 1
            guid: 316550613392296400
            path: '/dev/gpt/da2'
            phys_path: '/dev/gpt/da2'
            whole_disk: 0
            DTL: 31
        children[2]:
            type: 'disk'
            id: 2
            guid: 11515767252682951891
            path: '/dev/gpt/da3'
            phys_path: '/dev/gpt/da3'
            whole_disk: 0
            DTL: 30
    features_for_read:
--------------------------------------------
LABEL 3
--------------------------------------------
    version: 15
    name: 'oneterrorbyte'
    state: 0
    txg: 1541528
    pool_guid: 8210305841187571992
    hostid: 1318763924
    hostname: 'FreeNAS.local'
    top_guid: 16858560762606922925
    guid: 4548809083459598154
    vdev_children: 1
    vdev_tree:
        type: 'raidz'
        id: 0
        guid: 16858560762606922925
        nparity: 1
        metaslab_array: 23
        metaslab_shift: 35
        ashift: 9
        asize: 4494448656384
        is_log: 0
        children[0]:
            type: 'disk'
            id: 0
            guid: 4548809083459598154
            path: '/dev/ada1'
            phys_path: '/dev/ada1'
            whole_disk: 0
            DTL: 111
        children[1]:
            type: 'disk'
            id: 1
            guid: 316550613392296400
            path: '/dev/gpt/da2'
            phys_path: '/dev/gpt/da2'
            whole_disk: 0
            DTL: 31
        children[2]:
            type: 'disk'
            id: 2
            guid: 11515767252682951891
            path: '/dev/gpt/da3'
            phys_path: '/dev/gpt/da3'
            whole_disk: 0
            DTL: 30
    features_for_read:
 
paleoN

paleoN

Wizard
Joined
Apr 22, 2012
Messages
1,403
GregSC2 said:
The volume was originally created on FreeNAS 8 using the ZFS Volume Manager. I am currently on FreeNAS 9.1.0.
Click to expand...
Care to share which version of FreeNAS 8 used gpt labels. IIRC, it was none.

Code:
zdb -l /dev/ada1p2
 
G

GregSC2

Dabbler
Joined
Aug 17, 2013
Messages
13
Perhaps it could have been FreeNAS 7 then. It has been quite a while since I originally created the volumes. It was definitely no earlier than 7.

zdb -l /dev/ada1p2:
Code:
zdb -l /dev/ada1p2
--------------------------------------------
LABEL 0
--------------------------------------------
failed to unpack label 0
--------------------------------------------
LABEL 1
--------------------------------------------
failed to unpack label 1
--------------------------------------------
LABEL 2
--------------------------------------------
failed to unpack label 2
--------------------------------------------
LABEL 3
--------------------------------------------
failed to unpack label 3
 
paleoN

paleoN

Wizard
Joined
Apr 22, 2012
Messages
1,403
How exactly did you do the replace? The GUI doesn't use whole drives, and yet ada1 has both a swap partition & a zfs partition on it, but the whole disk was added to the pool. The active swap partition is why the drive isn't available. IMO, the best way forward is to quick wipe ada1 and resilver it, via the GUI, back into the pool. This will clear the existing ZFS labels, simplify future replacements and allow ZFS to access the disk after a reboot.
 
G

GregSC2

Dabbler
Joined
Aug 17, 2013
Messages
13
At first I attempted to replace the drive through the GUI. It was throwing all kinds of errors. I then reverted to the version of FreeNAS I was running, which was FreeNAS 8. This was giving me its own set of errors through the GUI. So I then ran a replace via the console using the following command:

Code:
zpool replace oneterrorbyte 4548809083459598154 ada1


I just tried to use the FreeNAS 9.1.0 GUI to replace the drive, but it gives me the following error: (which shows up and disappears really quickly)
Code:
Error: Disk replacement failed: "cannot replace 4548809083459598154 with gptid/57a97a4d-07b5-11e3-8372-0025903bc102: devices have different sector alignment, "
 
paleoN

paleoN

Wizard
Joined
Apr 22, 2012
Messages
1,403
GregSC2 said:
So I then ran a replace via the console using the following command:
Click to expand...
Next time wipe the partitions first.

GregSC2 said:
Code:
Error: Disk replacement failed: "cannot replace 4548809083459598154 with gptid/57a97a4d-07b5-11e3-8372-0025903bc102: devices have different sector alignment, "
Click to expand...
Heh, seven sectors smaller, and it's less than 4K. Quick wipe the drive and rerun your replace command.
 
G

GregSC2

Dabbler
Joined
Aug 17, 2013
Messages
13
Still getting the same error, despite a quick wipe of ada1 (the replacement disk).
Code:
Error: Disk replacement failed: "cannot replace 4548809083459598154 with gptid/8586914d-07b8-11e3-8372-0025903bc102: devices have different sector alignment, "
 
paleoN

paleoN

Wizard
Joined
Apr 22, 2012
Messages
1,403
GregSC2 said:
Still getting the same error, despite a quick wipe of ada1 (the replacement disk).
Click to expand...
Yes. I guess I wasn't clear. Quick wipe the drive again which should destroy the partitions. If it doesn't be sure to destroy them manually. Then from the CLI run zpool replace oneterrorbyte 4548809083459598154 ada1.
 
G

GregSC2

Dabbler
Joined
Aug 17, 2013
Messages
13
Sorry, I'm not too familiar with zfs/freeBSD. Clean wiping several times had no effect. How do I ago about manually destroying the partitions on ada1? fdisk gave me a few errors so I hope thats not the tool I need to use -.-
 
paleoN

paleoN

Wizard
Joined
Apr 22, 2012
Messages
1,403
GregSC2 said:
Sorry, I'm not too familiar with zfs/freeBSD. Clean wiping several times had no effect. How do I ago about manually destroying the partitions on ada1? fdisk gave me a few errors so I hope thats not the tool I need to use -.-
Click to expand...
Code:
swapctl -d /dev/ada1p1

gpart delete -i 2 ada1
gpart delete -i 1 ada1

gpart destroy ada1
 
G

GregSC2

Dabbler
Joined
Aug 17, 2013
Messages
13
Code:
[root@FreeNAS] ~# swapctl -d /dev/ada1p1
swapctl: /dev/ada1p1: No such file or directory
[root@FreeNAS] ~# gpart delete -i 2 ada1
gpart: arg0 'ada1': Invalid argument
[root@FreeNAS] ~# gpart delete -i 1 ada1
gpart: arg0 'ada1': Invalid argument
[root@FreeNAS] ~# gpart destroy ada1
gpart: arg0 'ada1': Invalid argument
 
paleoN

paleoN

Wizard
Joined
Apr 22, 2012
Messages
1,403
Code:
swapctl -ls
The gpart likely want's the full path, /dev/ada1.
 
G

GregSC2

Dabbler
Joined
Aug 17, 2013
Messages
13
I tried /dev/ada1 as well with no luck.

swapctl -ls
Code:
[root@FreeNAS] ~# swapctl -ls
Device:       1024-blocks     Used:
/dev/ada0p1.eli   2097152         0
/dev/ada2p1.eli   2097152         0
/dev/ada3p1.eli   2097152         0
/dev/ada4p1.eli   2097152         0
/dev/ada5p1.eli   2097152         0
Total:         10485760         0
 
paleoN

paleoN

Wizard
Joined
Apr 22, 2012
Messages
1,403
paleoN said:
The gpart likely want's the full path, /dev/ada1.
Click to expand...
Actually, the GPT table is damaged for this drive. You may only be able to recover or destroy it. Keep forgetting swap is encrypted now. Try this:
Code:
geli list

If ada1 is listed replace, DEV with the above:
Code:
geli detach DEV


Code:
gpart destroy -F /dev/ada1
 
G

GregSC2

Dabbler
Joined
Aug 17, 2013
Messages
13
ada1 is not listed anywhere in the output of "geli list" :-/

Code:
 
[root@FreeNAS] ~# geli list
Geom name: ada0p1.eli
State: ACTIVE
EncryptionAlgorithm: AES-XTS
KeyLength: 256
Crypto: software
Flags: ONETIME, W-DETACH, W-OPEN
KeysAllocated: 1
KeysTotal: 1
Providers:
1. Name: ada0p1.eli
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 4096
   Mode: r1w1e0
Consumers:
1. Name: ada0p1
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e1
 
Geom name: ada2p1.eli
State: ACTIVE
EncryptionAlgorithm: AES-XTS
KeyLength: 256
Crypto: software
Flags: ONETIME, W-DETACH, W-OPEN
KeysAllocated: 1
KeysTotal: 1
Providers:
1. Name: ada2p1.eli
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 4096
   Mode: r1w1e0
Consumers:
1. Name: ada2p1
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e1
 
Geom name: ada3p1.eli
State: ACTIVE
EncryptionAlgorithm: AES-XTS
KeyLength: 256
Crypto: software
Flags: ONETIME, W-DETACH, W-OPEN
KeysAllocated: 1
KeysTotal: 1
Providers:
1. Name: ada3p1.eli
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 4096
   Mode: r1w1e0
Consumers:
1. Name: ada3p1
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e1
 
Geom name: ada4p1.eli
State: ACTIVE
EncryptionAlgorithm: AES-XTS
KeyLength: 256
Crypto: software
Flags: ONETIME, W-DETACH, W-OPEN
KeysAllocated: 1
KeysTotal: 1
Providers:
1. Name: ada4p1.eli
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 4096
   Mode: r1w1e0
Consumers:
1. Name: ada4p1
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e1
 
Geom name: ada5p1.eli
State: ACTIVE
EncryptionAlgorithm: AES-XTS
KeyLength: 256
Crypto: software
Flags: ONETIME, W-DETACH, W-OPEN
KeysAllocated: 1
KeysTotal: 1
Providers:
1. Name: ada5p1.eli
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 4096
   Mode: r1w1e0
Consumers:
1. Name: ada5p1
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e1
 
[root@FreeNAS] ~# geli list
Geom name: ada0p1.eli
State: ACTIVE
EncryptionAlgorithm: AES-XTS
KeyLength: 256
Crypto: software
Flags: ONETIME, W-DETACH, W-OPEN
KeysAllocated: 1
KeysTotal: 1
Providers:
1. Name: ada0p1.eli
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 4096
   Mode: r1w1e0
Consumers:
1. Name: ada0p1
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e1
 
Geom name: ada2p1.eli
State: ACTIVE
EncryptionAlgorithm: AES-XTS
KeyLength: 256
Crypto: software
Flags: ONETIME, W-DETACH, W-OPEN
KeysAllocated: 1
KeysTotal: 1
Providers:
1. Name: ada2p1.eli
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 4096
   Mode: r1w1e0
Consumers:
1. Name: ada2p1
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e1
 
Geom name: ada3p1.eli
State: ACTIVE
EncryptionAlgorithm: AES-XTS
KeyLength: 256
Crypto: software
Flags: ONETIME, W-DETACH, W-OPEN
KeysAllocated: 1
KeysTotal: 1
Providers:
1. Name: ada3p1.eli
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 4096
   Mode: r1w1e0
Consumers:
1. Name: ada3p1
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e1
 
Geom name: ada4p1.eli
State: ACTIVE
EncryptionAlgorithm: AES-XTS
KeyLength: 256
Crypto: software
Flags: ONETIME, W-DETACH, W-OPEN
KeysAllocated: 1
KeysTotal: 1
Providers:
1. Name: ada4p1.eli
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 4096
   Mode: r1w1e0
Consumers:
1. Name: ada4p1
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e1
 
Geom name: ada5p1.eli
State: ACTIVE
EncryptionAlgorithm: AES-XTS
KeyLength: 256
Crypto: software
Flags: ONETIME, W-DETACH, W-OPEN
KeysAllocated: 1
KeysTotal: 1
Providers:
1. Name: ada5p1.eli
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 4096
   Mode: r1w1e0
Consumers:
1. Name: ada5p1
   Mediasize: 2147483648 (2.0G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r1w1e1
 
paleoN

paleoN

Wizard
Joined
Apr 22, 2012
Messages
1,403
GregSC2 said:
ada1 is not listed anywhere in the output of "geli list" :-/
Click to expand...
Well something is holding the geom open. I have to go, but I will be around tomorrow. In the mean time you can either pull the drive and wipe it in a different machine, e.g. with DBAN. Or reboot into single user mode, I think it's still option 6, wipe the partitions, reboot and replace. It might even see the drive without the partitions on it depending. If so do scrub.
 
G

GregSC2

Dabbler
Joined
Aug 17, 2013
Messages
13
Makes sense. I appreciate all the help. This has been a nightmare for such a simple replace and resilver. I'll give those few things a try and post my results. Thanks a lot!
 
paleoN

paleoN

Wizard
Joined
Apr 22, 2012
Messages
1,403
GregSC2 said:
I'll give those few things a try and post my results.
Click to expand...
I did a poor job reading this error message, devices have different sector alignment, last night. I saw it failed the replace and it was slightly smaller. However, that error is because of newer code in 9-STABLE, or trueos, automatically detecting a 4K disk, but with an ashift=9 pool. If you haven't already destroyed the partitions then this should let you do the replace via the GUI:
Code:
sysctl vfs.zfs.vdev.larger_ashift_disable=1
Current ZFS code should be able to handle a seven 512b sector difference. You can read in this post the fine print for more information about the sysctl. Keep in mind an ashift=9 pool has a significant write penalty on 4K drives and inflates small reads. Ideally, you would have a current tested backup, destroy the pool and recreate the pool with ashift=12, i.e. 4K pool.
 
Status
Not open for further replies.

Important Announcement for the TrueNAS Community.

The TrueNAS Community has now been moved. This forum will now become READ-ONLY for historical purposes. Please feel free to join us on the new TrueNAS Community Forums.

Related topics on forums.truenas.com for thread: "ZFS Pool Resilvered, Upon Reboot forgets new drive"

Similar threads

U
  • Locked
Lost one disk, resilvered, but lost data
Replies
9
Views
3K
ben
ben
D
  • Locked
How do I replace a drive?
Replies
4
Views
2K
sympton
S
T
  • Locked
ZFS problems after a drive change
Replies
0
Views
1K
tlongarms
T
P
  • Locked
Lost a disk, can't online new disk
Replies
9
Views
3K
PierceIt
P
Bobo Moreno
  • Locked
zpool replace leaves pool degraded
Replies
4
Views
1K
Robert Smith
Robert Smith
Top