Hello,
this follows up a post I made earlier, reflecting where things are now. I believe that I'm in a situation where I will need to rebuild the pool entirely with two replacement disks. Can anyone comment or confirm this based on what I will post below.
The box was originally inaccessible. After a reboot it will begin to resilver but never completes, as far as I can see, because it will either lock up or reboot after a few hours. The zpool status details vary from invocation to invocation, always having data errors; many now. Two drives, ada1 and ada3 show errors in the log files. The same drives show bad results with a smartctl check. I thought that I might to try to offline what I felt was the worse of the two drives, attempting to take the drive offline isn't allowed due to no valid replicas; because it's resilvering I can go further with that.
TIA
ada1
ada3
this follows up a post I made earlier, reflecting where things are now. I believe that I'm in a situation where I will need to rebuild the pool entirely with two replacement disks. Can anyone comment or confirm this based on what I will post below.
The box was originally inaccessible. After a reboot it will begin to resilver but never completes, as far as I can see, because it will either lock up or reboot after a few hours. The zpool status details vary from invocation to invocation, always having data errors; many now. Two drives, ada1 and ada3 show errors in the log files. The same drives show bad results with a smartctl check. I thought that I might to try to offline what I felt was the worse of the two drives, attempting to take the drive offline isn't allowed due to no valid replicas; because it's resilvering I can go further with that.
TIA
Code:
Sep 12 09:11:17 freenas smartd[24138]: Device: /dev/ada3, 4832 Offline uncorrectable sectors Sep 12 09:11:17 freenas smartd[24138]: Device: /dev/ada1, 41784 Currently unreadable (pending) sectors
Code:
Sep 12 01:41:47 freenas siisch3: Error while READ LOG EXT Sep 12 01:41:47 freenas (ada3:siisch3:0:0:0): READ_FPDMA_QUEUED. ACB: 60 28 80 4a 6f 40 1c 01 00 00 00 00 Sep 12 01:41:47 freenas (ada3:siisch3:0:0:0): CAM status: ATA Status Error Sep 12 01:41:47 freenas (ada3:siisch3:0:0:0): ATA status: 00 () Sep 12 01:41:47 freenas (ada3:siisch3:0:0:0): RES: 00 00 00 00 00 00 00 00 00 00 00 Sep 12 01:41:47 freenas (ada3:siisch3:0:0:0): Error 5, Periph was invalidated
ada1
Code:
SMART Attributes Data Structure revision number: 10 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x000f 113 092 006 Pre-fail Always - 51837688 3 Spin_Up_Time 0x0003 095 094 000 Pre-fail Always - 0 4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 22 5 Reallocated_Sector_Ct 0x0033 078 078 010 Pre-fail Always - 29064 7 Seek_Error_Rate 0x000f 079 060 030 Pre-fail Always - 99101519 9 Power_On_Hours 0x0032 092 092 000 Old_age Always - 7352 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0 12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 22 183 Runtime_Bad_Block 0x0032 100 100 000 Old_age Always - 0 184 End-to-End_Error 0x0032 100 100 099 Old_age Always - 0 187 Reported_Uncorrect 0x0032 001 001 000 Old_age Always - 621 188 Command_Timeout 0x0032 100 092 000 Old_age Always - 28 29 39 189 High_Fly_Writes 0x003a 084 084 000 Old_age Always - 16 190 Airflow_Temperature_Cel 0x0022 060 044 045 Old_age Always In_the_past 40 (Min/Max 39/46 #13) 191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 0 192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 11 193 Load_Cycle_Count 0x0032 100 100 000 Old_age Always - 76 194 Temperature_Celsius 0x0022 040 056 000 Old_age Always - 40 (0 18 0 0 0) 197 Current_Pending_Sector 0x0012 001 001 000 Old_age Always - 41784 198 Offline_Uncorrectable 0x0010 001 001 000 Old_age Offline - 41784 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 5 240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 7351h+39m+01.385s 241 Total_LBAs_Written 0x0000 100 253 000 Old_age Offline - 25648930986 242 Total_LBAs_Read 0x0000 100 253 000 Old_age Offline - 3578276333359
ada3
Code:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x000f 105 082 006 Pre-fail Always - 76064911 3 Spin_Up_Time 0x0003 095 094 000 Pre-fail Always - 0 4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 22 5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail Always - 0 7 Seek_Error_Rate 0x000f 079 060 030 Pre-fail Always - 89673874 9 Power_On_Hours 0x0032 091 091 000 Old_age Always - 7996 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0 12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 23 183 Runtime_Bad_Block 0x0032 100 100 000 Old_age Always - 0 184 End-to-End_Error 0x0032 100 100 099 Old_age Always - 0 187 Reported_Uncorrect 0x0032 001 001 000 Old_age Always - 9091 188 Command_Timeout 0x0032 100 092 000 Old_age Always - 282 282 311 189 High_Fly_Writes 0x003a 085 085 000 Old_age Always - 15 190 Airflow_Temperature_Cel 0x0022 062 047 045 Old_age Always - 38 (Min/Max 37/44) 191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 0 192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 11 193 Load_Cycle_Count 0x0032 100 100 000 Old_age Always - 80 194 Temperature_Celsius 0x0022 038 053 000 Old_age Always - 38 (0 18 0 0 0) 197 Current_Pending_Sector 0x0012 071 071 000 Old_age Always - 4832 198 Offline_Uncorrectable 0x0010 071 071 000 Old_age Offline - 4832 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 29 240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 7995h+55m+45.376s 241 Total_LBAs_Written 0x0000 100 253 000 Old_age Offline - 13181833043 242 Total_LBAs_Read 0x0000 100 253 000 Old_age Offline - 364588048186
Code:
pool: main_volume state: ONLINE status: One or more devices is currently being resilvered. The pool will continue to function, possibly in a degraded state. action: Wait for the resilver to complete. scan: resilver in progress since Thu Sep 12 09:20:30 2019 909G scanned at 303M/s, 427G issued at 142M/s, 4.53T total 20.0G resilvered, 9.19% done, 0 days 08:26:28 to go config: NAME STATE READ WRITE CKSUM main_volume ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 gptid/4074d0a8-cc10-11e8-9942-d05099aa523b ONLINE 0 0 0 gptid/41779cad-cc10-11e8-9942-d05099aa523b ONLINE 0 0 0 gptid/427882a1-cc10-11e8-9942-d05099aa523b ONLINE 0 0 0 gptid/4388574d-cc10-11e8-9942-d05099aa523b ONLINE 0 0 23 errors: 2563253 data errors, use '-v' for a list