[Solved] db> Prompt After Reboot

kreg · Jul 7, 2019

Motherboard: ASUS E35M1-I
CPU: Fusion AMD E-350 (onboard)
Memory: 8GB
Boot Drive: 32GB USB Flash Drive. Silicon Power
Hard Drives: 6 Drives. Each model is a WD30EZRX
RAID: ZFS2
Hard Disk Control: Motherboard onboard.
Network Card: Motherboard onboard.

FreeNas 11.0-U2

I have been using this system for several years. Today I was browsing my files via the network, and the share stopped. I went to look at my server. I turned on the KVM and Monitor. I saw a db> prompt. I had never seen such a prompt before.

After doing some basic web searching, I decided to replace the USB flash drive with a fresh install of 11.0-U2. The installation went fine. I was able to access the web gui. When I imported the pool, the wizard found the pool name. I clicked next, and waited. Eventually an error was claimed at the web gui. I went to look at my server's screen, and it was in the middle of a reboot.

More searching of this forum suggested to try importing the pool as read only. This would allow me to backup data via ssh, ftp, etc. I edited GRUB to use "set kFreeBSD.vfs.zfs.recover=1" and "set kFreeBSD.vfs.zfs.debug=1", to prevent the pool from being mounted (to prevent a reboot). Using the basic /bin/sh I was able to attempt importing the pool as read only. Again, I landed at a db> prompt.

In both cases, I see similar errors. The blkptr messages with addresses look potentially helpful. Just a guess.

Some further thread wandering proposed faulty hardware. I happen to have an extra identical motherboard. I swapped it before posting this, but I get the same results.
Can anyone please advise what I should try next? I do have some of this pool backed up but... it's not extensive. So much of my past would be lost :(

I also saw today stones being thrown for using desktop motherboards and non-ECC memory. Is this a fatal decision? If so, I've never seen any warnings appear that no ECC memory was detected when using my system. I've never known this to be concern. I would find it odd that you can even run FreeNAS on non-ECC if that's the case...

If anyone can please help, I would be greatly in debt to you. At the least, it would bring peace to know without any doubt that the pool truly is corrupt beyond any data recovery. I am comfortable in command lines and working with various tools, but that's mostly my linux side. I do not use FreeBSD beyond FreeNAS.

Thank you

SweetAndLow · Jul 8, 2019

What version of freenas did you have running initially? Use the exact same version you had used initially. If not you should try the latest version not something old and I think that version was never even a stable release. 11.2-U5 is the current stable release.

Basically your system is having a kernel crash when importing the pool. This is either a bug or your pool is corrupt.

Run smart tests on all your driver's and or post the smart data for us to see.

What is the output of zpool import?

danb35 · Jul 8, 2019

kreg said:
I've never known this to be concern.

So you've never read the manual or any of the hardware recommendation guides--all of them strongly recommend server-grade hardware using ECC memory. It isn't required, in that FreeNAS will run without it, but it adds a layer of protection for your data that you simply don't have without it. Whether that's the cause of your problem is impossible to say at this point, though.

kreg · Jul 8, 2019

SweetAndLow said:
What version of freenas did you have running initially? Use the exact same version you had used initially.

I'm not positive the exact initial version used to birth the pool. Can the system tell me? I built it back in 2011, and have steadily upgraded since. If I recall, I did a fresh install at one point, since a direct upgrade path wasn't really possible. Something 8.x to 9.x if I remember. My best guess is that this pool was born from 9.x. But I could be wrong :/ Can trying random versions hurt it on import?

SweetAndLow said:
If not you should try the latest version.

I did try a fresh install of 11.1-U7 before posting. Same problem at import. I can try 11.2-U5 specifically.

SweetAndLow said:
Run smart tests on all your driver's and or post the smart data for us to see.

I will try to lookup how to do that, and share. I will also post the zpool import info.

danb35 said:
So you've never read the manual or any of the hardware recommendation guides

Well, ok I see it now :) It's right there in the introduction. First page basically. I have read the manuals before, but for other questions. Such as upgrade paths. Good to know now! Thank you.

To setup a saftey net, can I "dd" a copy of each drive by hand, and attach them to the same or different motherboard? Not sure how the ID's of the system works, or if it would let me do that.

I am tempted to build a completely new system, since 8GB of memory is below the recommend. I am grasping for straws, hoping that maybe a memory limit is causing the crash at import combined with the tiny CPU and chipset. It's a 12TB pool that is well over 90% full. The motherboard is a just an old AMD Fusion.

Thank you for some initial thoughts. I'll collect the requested data soon.

kreg · Jul 8, 2019

zpool import info. All of the commands below were run after I Installed 9.10.2, for science. Import still caused a reboot. However, it did not land in a db> prompt. Same physical machine.

Code:

[root@freenas] ~# uname -a
FreeBSD freenas.local 10.3-STABLE FreeBSD 10.3-STABLE #0 r295946+47645f1(9.10.2-STABLE): Mon Dec 19 08:30:01 UTC 2016     root@gauntlet:/freenas-9.10-releng/_BE/objs/freenas-9.10-releng/_BE/os/sys/FreeNAS.amd64  amd64


[root@freenas] ~# zpool import
   pool: archive
     id: 1636185473559245456
  state: ONLINE
status: The pool was last accessed by another system.
action: The pool can be imported using its name or numeric identifier and
        the '-f' flag.
   see: http://illumos.org/msg/ZFS-8000-EY
config:

        archive     ONLINE
          raidz2-0  ONLINE
            ada0p2  ONLINE
            ada1p2  ONLINE
            ada2p2  ONLINE
            ada3p2  ONLINE
            ada4p2  ONLINE
            ada5p2  ONLINE
[root@freenas] ~#

kreg · Jul 8, 2019

Smart data for drives 0-2

Code:

[root@freenas] ~# smartctl -a /dev/ada0
smartctl 6.5 2016-05-07 r4318 [FreeBSD 10.3-STABLE amd64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Green
Device Model:     WDC WD30EZRX-00MMMB0
Serial Number:    WD-WCAWZ1163096
LU WWN Device Id: 5 0014ee 25b802501
Firmware Version: 80.00A80
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ATA8-ACS (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Mon Jul  8 23:07:04 2019 PDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (52800) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        ( 507) minutes.
Conveyance self-test routine
recommended polling time:        (   5) minutes.
SCT capabilities:              (0x3035) SCT Status supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       0
  3 Spin_Up_Time            0x0027   156   152   021    Pre-fail  Always       -       9191
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       129
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   200   200   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   026   026   000    Old_age   Always       -       54214
 10 Spin_Retry_Count        0x0032   100   100   000    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   100   100   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       127
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       67
193 Load_Cycle_Count        0x0032   001   001   000    Old_age   Always       -       2787444
194 Temperature_Celsius     0x0022   122   109   000    Old_age   Always       -       30
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   200   200   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   200   200   000    Old_age   Offline      -       0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%     16423         -
# 2  Extended offline    Completed without error       00%     16087         -
# 3  Extended offline    Completed without error       00%     15703         -
# 4  Extended offline    Completed without error       00%     15368         -
# 5  Extended offline    Completed without error       00%     11466         -
# 6  Extended offline    Completed without error       00%     11131         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

[root@freenas] ~#


[root@freenas] ~# smartctl -a /dev/ada1
smartctl 6.5 2016-05-07 r4318 [FreeBSD 10.3-STABLE amd64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Green
Device Model:     WDC WD30EZRX-00MMMB0
Serial Number:    WD-WCAWZ0835917
LU WWN Device Id: 5 0014ee 205fe4cc6
Firmware Version: 80.00A80
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ATA8-ACS (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Mon Jul  8 23:08:16 2019 PDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (52560) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        ( 505) minutes.
Conveyance self-test routine
recommended polling time:        (   5) minutes.
SCT capabilities:              (0x3035) SCT Status supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       2
  3 Spin_Up_Time            0x0027   152   148   021    Pre-fail  Always       -       9383
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       115
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   200   200   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   026   026   000    Old_age   Always       -       54192
 10 Spin_Retry_Count        0x0032   100   100   000    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   100   100   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       114
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       57
193 Load_Cycle_Count        0x0032   001   001   000    Old_age   Always       -       2791616
194 Temperature_Celsius     0x0022   121   107   000    Old_age   Always       -       31
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   200   200   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       1
200 Multi_Zone_Error_Rate   0x0008   200   200   000    Old_age   Offline      -       0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%     16401         -
# 2  Extended offline    Completed without error       00%     16065         -
# 3  Extended offline    Completed without error       00%     15682         -
# 4  Extended offline    Completed without error       00%     15346         -
# 5  Extended offline    Completed without error       00%     11445         -
# 6  Extended offline    Completed without error       00%     11110         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

[root@freenas] ~#


[root@freenas] ~# smartctl -a /dev/ada2
smartctl 6.5 2016-05-07 r4318 [FreeBSD 10.3-STABLE amd64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Green
Device Model:     WDC WD30EZRX-00MMMB0
Serial Number:    WD-WCAWZ1081635
LU WWN Device Id: 5 0014ee 2062259cc
Firmware Version: 80.00A80
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ATA8-ACS (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Mon Jul  8 23:08:21 2019 PDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (51480) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        ( 494) minutes.
Conveyance self-test routine
recommended polling time:        (   5) minutes.
SCT capabilities:              (0x3035) SCT Status supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       1
  3 Spin_Up_Time            0x0027   154   149   021    Pre-fail  Always       -       9300
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       117
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   200   200   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   026   026   000    Old_age   Always       -       54169
 10 Spin_Retry_Count        0x0032   100   100   000    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   100   100   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       116
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       58
193 Load_Cycle_Count        0x0032   001   001   000    Old_age   Always       -       2827062
194 Temperature_Celsius     0x0022   122   108   000    Old_age   Always       -       30
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   200   200   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   200   200   000    Old_age   Offline      -       0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%     16377         -
# 2  Extended offline    Completed without error       00%     16041         -
# 3  Extended offline    Completed without error       00%     15658         -
# 4  Extended offline    Completed without error       00%     15322         -
# 5  Extended offline    Completed without error       00%     11421         -
# 6  Extended offline    Completed without error       00%     11086         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

[root@freenas] ~#

kreg · Jul 8, 2019

Smart data for drives 3-5

Code:

[root@freenas] ~# smartctl -a /dev/ada3
smartctl 6.5 2016-05-07 r4318 [FreeBSD 10.3-STABLE amd64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Green
Device Model:     WDC WD30EZRX-00MMMB0
Serial Number:    WD-WCAWZ1157733
LU WWN Device Id: 5 0014ee 25b803f9c
Firmware Version: 80.00A80
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ATA8-ACS (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Mon Jul  8 23:08:23 2019 PDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (51360) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        ( 494) minutes.
Conveyance self-test routine
recommended polling time:        (   5) minutes.
SCT capabilities:              (0x3035) SCT Status supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       1
  3 Spin_Up_Time            0x0027   151   146   021    Pre-fail  Always       -       9408
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       122
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   200   200   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   026   026   000    Old_age   Always       -       54143
 10 Spin_Retry_Count        0x0032   100   100   000    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   100   100   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       120
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       64
193 Load_Cycle_Count        0x0032   001   001   000    Old_age   Always       -       2785076
194 Temperature_Celsius     0x0022   122   107   000    Old_age   Always       -       30
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   200   200   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   200   200   000    Old_age   Offline      -       0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%     16351         -
# 2  Extended offline    Completed without error       00%     16016         -
# 3  Extended offline    Completed without error       00%     15632         -
# 4  Extended offline    Completed without error       00%     15297         -
# 5  Extended offline    Completed without error       00%     11395         -
# 6  Extended offline    Completed without error       00%     11060         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

[root@freenas] ~#


[root@freenas] ~# smartctl -a /dev/ada4
smartctl 6.5 2016-05-07 r4318 [FreeBSD 10.3-STABLE amd64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Green
Device Model:     WDC WD30EZRX-00MMMB0
Serial Number:    WD-WCAWZ0842006
LU WWN Device Id: 5 0014ee 25b544cbe
Firmware Version: 80.00A80
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ATA8-ACS (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Mon Jul  8 23:08:25 2019 PDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (52380) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        ( 503) minutes.
Conveyance self-test routine
recommended polling time:        (   5) minutes.
SCT capabilities:              (0x3035) SCT Status supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       0
  3 Spin_Up_Time            0x0027   154   150   021    Pre-fail  Always       -       9291
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       113
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   200   200   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   027   027   000    Old_age   Always       -       53868
 10 Spin_Retry_Count        0x0032   100   100   000    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   100   100   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       111
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       58
193 Load_Cycle_Count        0x0032   001   001   000    Old_age   Always       -       2801095
194 Temperature_Celsius     0x0022   122   107   000    Old_age   Always       -       30
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   200   200   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   200   200   000    Old_age   Offline      -       0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%     16076         -
# 2  Extended offline    Completed without error       00%     15741         -
# 3  Extended offline    Completed without error       00%     15357         -
# 4  Extended offline    Completed without error       00%     15022         -
# 5  Extended offline    Completed without error       00%     11118         -
# 6  Extended offline    Completed without error       00%     10784         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

[root@freenas] ~#


[root@freenas] ~# smartctl -a /dev/ada5
smartctl 6.5 2016-05-07 r4318 [FreeBSD 10.3-STABLE amd64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Green
Device Model:     WDC WD30EZRX-00MMMB0
Serial Number:    WD-WCAWZ0526977
LU WWN Device Id: 5 0014ee 2b062d744
Firmware Version: 80.00A80
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ATA8-ACS (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Mon Jul  8 23:08:28 2019 PDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (52380) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        ( 503) minutes.
Conveyance self-test routine
recommended polling time:        (   5) minutes.
SCT capabilities:              (0x3035) SCT Status supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       0
  3 Spin_Up_Time            0x0027   148   146   021    Pre-fail  Always       -       9566
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       483
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   200   200   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   026   026   000    Old_age   Always       -       54206
 10 Spin_Retry_Count        0x0032   100   100   000    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   100   100   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       481
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       194
193 Load_Cycle_Count        0x0032   001   001   000    Old_age   Always       -       2787063
194 Temperature_Celsius     0x0022   122   094   000    Old_age   Always       -       30
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   200   200   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   200   200   000    Old_age   Offline      -       0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%     16415         -
# 2  Extended offline    Completed without error       00%     16080         -
# 3  Extended offline    Completed without error       00%     15696         -
# 4  Extended offline    Completed without error       00%     15361         -
# 5  Extended offline    Completed without error       00%     11458         -
# 6  Extended offline    Completed without error       00%     11123         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

[root@freenas] ~#

SweetAndLow · Jul 8, 2019

Ada1 and ada3 could probably be failing. You have not run smart tests in over 4 years. They should be setup to run daily and weekly.

You should run smart long tests on all those drives. And also short tested. Then learn to setup the smart task in freenas. Then return back with new smartctl -a info.

After that you can try zpool import -f archive from the cli to see what happens and report back with the output.

danb35 · Jul 9, 2019

Also noticing load cycle counts near 3 million on all disks; they obviously haven't had the timer reset using the WDIDLE utility. And the disks are 6 years old--not that I favor proactive disk replacement, but they're getting up there in years.

FWIW, I'd favor trying the import on the latest 11.2 version--there have been a lot of updates to the ZFS code since 9.10, and one of the results is that it's supposed to handle degraded pools better.

kreg · Jul 9, 2019

SweetAndLow said:
You have not run smart tests in over 4 years.

Looking over the data, I'm not sure where the 4 years stands out. Where is this shown? You're right though, I've rarely ever run such a test. I'll learn to perform this regularly instead now. I will also perform the tests you asked for and share.

danb35 said:
...not that I favor proactive disk replacement, but they're getting up there in years.

I can provide a replacement for all drives. Since I can not import the pool, how will I replace them? I'm assuming there is no way to replace them using FreeNAS without the pool imported and online.

As an alternative, can I dd them to new disks by hand using external bays and a different system? It would give me a "copy/backup" of this problematic pool to work with, instead of the original. It would also "replace" the discs. Am I dreaming, or would this work to duplicate the pool (regardless if it's corrupted or not) onto a new set of drives?

danb35 · Jul 9, 2019

kreg said:
I'm not sure where the 4 years stands out. Where is this shown?

In the math between the power-on hours (SMART attribute 9) and the timestamp for the last test.

kreg said:
Since I can not import the pool, how will I replace them?

I wasn't clear; that was more of a longer-term idea. You'd need to be able to import the pool first.

kreg · Jul 15, 2019

Sorry for absence. I'm back.

Short tests are done ( I think). The aborted test on /dev/ada0 was me. (Learning how to drive still). I do not know why there are 6 Extended tests listed for each. Is an Extended the same as -t long? Maybe those were from long ago. I don't recall performing one.

I'll run the long tests next. Safe to start all 6 at the same time?

Really appreciate each of you looking at this for me.

Code:

root@freenas[~]# smartctl -l selftest /dev/ada0
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Completed without error       00%     54228         -
# 2  Extended offline    Aborted by host               90%     54228         -
# 3  Extended offline    Completed without error       00%     16423         -
# 4  Extended offline    Completed without error       00%     16087         -
# 5  Extended offline    Completed without error       00%     15703         -
# 6  Extended offline    Completed without error       00%     15368         -
# 7  Extended offline    Completed without error       00%     11466         -
# 8  Extended offline    Completed without error       00%     11131         -

root@freenas[~]#
root@freenas[~]#
root@freenas[~]#
root@freenas[~]# smartctl -l selftest /dev/ada1
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Completed without error       00%     54207         -
# 2  Extended offline    Completed without error       00%     16401         -
# 3  Extended offline    Completed without error       00%     16065         -
# 4  Extended offline    Completed without error       00%     15682         -
# 5  Extended offline    Completed without error       00%     15346         -
# 6  Extended offline    Completed without error       00%     11445         -
# 7  Extended offline    Completed without error       00%     11110         -

root@freenas[~]#
root@freenas[~]# smartctl -l selftest /dev/ada2
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Completed without error       00%     54171         -
# 2  Extended offline    Completed without error       00%     16377         -
# 3  Extended offline    Completed without error       00%     16041         -
# 4  Extended offline    Completed without error       00%     15658         -
# 5  Extended offline    Completed without error       00%     15322         -
# 6  Extended offline    Completed without error       00%     11421         -
# 7  Extended offline    Completed without error       00%     11086         -

root@freenas[~]#
root@freenas[~]# smartctl -l selftest /dev/ada3
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Completed without error       00%     54145         -
# 2  Extended offline    Completed without error       00%     16351         -
# 3  Extended offline    Completed without error       00%     16016         -
# 4  Extended offline    Completed without error       00%     15632         -
# 5  Extended offline    Completed without error       00%     15297         -
# 6  Extended offline    Completed without error       00%     11395         -
# 7  Extended offline    Completed without error       00%     11060         -

root@freenas[~]#
root@freenas[~]# smartctl -l selftest /dev/ada4
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Completed without error       00%     53870         -
# 2  Extended offline    Completed without error       00%     16076         -
# 3  Extended offline    Completed without error       00%     15741         -
# 4  Extended offline    Completed without error       00%     15357         -
# 5  Extended offline    Completed without error       00%     15022         -
# 6  Extended offline    Completed without error       00%     11118         -
# 7  Extended offline    Completed without error       00%     10784         -

root@freenas[~]#
root@freenas[~]# smartctl -l selftest /dev/ada5
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Completed without error       00%     54209         -
# 2  Extended offline    Completed without error       00%     16415         -
# 3  Extended offline    Completed without error       00%     16080         -
# 4  Extended offline    Completed without error       00%     15696         -
# 5  Extended offline    Completed without error       00%     15361         -
# 6  Extended offline    Completed without error       00%     11458         -
# 7  Extended offline    Completed without error       00%     11123         -

root@freenas[~]#

kreg · Jul 15, 2019

FWIW, I put together a new system. Each of the original disks have been moved into it. The above short tests were performed on the new system. it is running 11.2-U5 booted as UEFI.

Hopefully this will be an approved system. I do believe ECC is in play now. Below is what I put together.

Supermicro X11SSH-LN4F-O
Intel Xeon E3-1230 V6
Crucial 32GB (2x16GB) (CT16G4WFD8266)
6x 6TB WD Reds are on standby waiting to be installed.

kreg · Jul 16, 2019

Long tests complete.

Code:

root@freenas[~]# smartctl -l selftest /dev/ada0
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%     54237         -
# 2  Short offline       Completed without error       00%     54228         -
# 3  Extended offline    Aborted by host               90%     54228         -
# 4  Extended offline    Completed without error       00%     16423         -
# 5  Extended offline    Completed without error       00%     16087         -
# 6  Extended offline    Completed without error       00%     15703         -
# 7  Extended offline    Completed without error       00%     15368         -
# 8  Extended offline    Completed without error       00%     11466         -
# 9  Extended offline    Completed without error       00%     11131         -

root@freenas[~]# smartctl -l selftest /dev/ada1
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%     54216         -
# 2  Short offline       Completed without error       00%     54207         -
# 3  Extended offline    Completed without error       00%     16401         -
# 4  Extended offline    Completed without error       00%     16065         -
# 5  Extended offline    Completed without error       00%     15682         -
# 6  Extended offline    Completed without error       00%     15346         -
# 7  Extended offline    Completed without error       00%     11445         -
# 8  Extended offline    Completed without error       00%     11110         -

root@freenas[~]# smartctl -l selftest /dev/ada2
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%     54180         -
# 2  Short offline       Completed without error       00%     54171         -
# 3  Extended offline    Completed without error       00%     16377         -
# 4  Extended offline    Completed without error       00%     16041         -
# 5  Extended offline    Completed without error       00%     15658         -
# 6  Extended offline    Completed without error       00%     15322         -
# 7  Extended offline    Completed without error       00%     11421         -
# 8  Extended offline    Completed without error       00%     11086         -

root@freenas[~]# smartctl -l selftest /dev/ada3
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%     54154         -
# 2  Short offline       Completed without error       00%     54145         -
# 3  Extended offline    Completed without error       00%     16351         -
# 4  Extended offline    Completed without error       00%     16016         -
# 5  Extended offline    Completed without error       00%     15632         -
# 6  Extended offline    Completed without error       00%     15297         -
# 7  Extended offline    Completed without error       00%     11395         -
# 8  Extended offline    Completed without error       00%     11060         -

root@freenas[~]# smartctl -l selftest /dev/ada4
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%     53879         -
# 2  Short offline       Completed without error       00%     53870         -
# 3  Extended offline    Completed without error       00%     16076         -
# 4  Extended offline    Completed without error       00%     15741         -
# 5  Extended offline    Completed without error       00%     15357         -
# 6  Extended offline    Completed without error       00%     15022         -
# 7  Extended offline    Completed without error       00%     11118         -
# 8  Extended offline    Completed without error       00%     10784         -

root@freenas[~]# smartctl -l selftest /dev/ada5
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%     54218         -
# 2  Short offline       Completed without error       00%     54209         -
# 3  Extended offline    Completed without error       00%     16415         -
# 4  Extended offline    Completed without error       00%     16080         -
# 5  Extended offline    Completed without error       00%     15696         -
# 6  Extended offline    Completed without error       00%     15361         -
# 7  Extended offline    Completed without error       00%     11458         -
# 8  Extended offline    Completed without error       00%     11123         -

kreg · Jul 31, 2019

All data recovered. Used a Windows tool called ZFS Recovery. Worked with the author. Was very helpful.

https://www.klennet.com/zfs-recovery/default.aspx

Important Announcement for The TrueNAS Community.

[Solved] db> Prompt After Reboot

kreg

Dabbler

SweetAndLow

Sweet'NASty

danb35

Hall of Famer

kreg

Dabbler

kreg

Dabbler

kreg

Dabbler

kreg

Dabbler

SweetAndLow

Sweet'NASty

danb35

Hall of Famer

kreg

Dabbler

danb35

Hall of Famer

kreg

Dabbler

kreg

Dabbler

kreg

Dabbler

kreg

Dabbler

Similar threads

Important Announcement for The TrueNAS Community.