Pool TM state is ONLINE: One or more devices has experienced an unrecoverable error.

TerryVog

Explorer
Joined
Dec 5, 2015
Messages
55
My alert system gave an error:
CRITICAL: Sept. 8, 2019, 12:28 a.m. - Pool TM state is ONLINE: One or more devices has experienced an unrecoverable error. An attempt was made to correct the error. Applications are unaffected.

I tried to find a solution on this forum and I noticed that the first step is to run zpool status -v, so I did. But I don't know what to do next.

Code:
[root@freenas ~]# zpool status -v                                                                                                   
  pool: TM                                                                                                                          
state: ONLINE
status: One or more devices has experienced an unrecoverable error. An
attempt was made to correct the error. Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
using 'zpool clear' or replace the device with 'zpool replace'.
see: http://illumos.org/msg/ZFS-8000-9P
scan: scrub repaired 0 in 1 days 21:05:14 with 0 errors on Mon Oct 21 21:05:15 2019
config:

NAME STATE READ WRITE CKSUM
TM ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
gptid/66c6f9a2-cc0f-11e5-b9f3-60a44caefdbd ONLINE 0 0 2
gptid/f48b4bd1-3c00-11e5-b5da-60a44caefdbd ONLINE 0 0 2
mirror-1 ONLINE 0 0 0
gptid/b50d8539-9c33-11e5-82d9-60a44caefdbd ONLINE 0 0 2
gptid/b5bbf4c4-9c33-11e5-82d9-60a44caefdbd ONLINE 0 0 1

errors: No known data errors

pool: freenas-boot
state: ONLINE
scan: scrub repaired 0 in 0 days 00:03:03 with 0 errors on Tue Nov 12 03:48:03 2019
config:

NAME STATE READ WRITE CKSUM
freenas-boot ONLINE 0 0 0
da0p2 ONLINE 0 0 0

errors: No known data errors


Build: FreeNAS-11.2-U5
Platform: Intel(R) Core(TM) i3-3240 CPU @ 3.40GHz
Memory: 16020MB
 

Jessep

Patron
Joined
Aug 19, 2018
Messages
379
Next step would be to check the SMART data on the drives.
  • smartctl /dev/ada0
You will need to check how your drives are named, they may not be ada0/1/etc.

Do you have periodic SMART tests scheduled?
 

TerryVog

Explorer
Joined
Dec 5, 2015
Messages
55
Thank you for responding, Jessep.

Under Services I see that S.M.A.R.T. is running.

I tried smartctl on all 4 of my drives:

Code:
[root@freenas ~]# smartctl /dev/ada0                                            
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-STABLE amd64] (local build)         
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org     
                                                                                
ATA device successfully opened                                                  
                                                                                
Use 'smartctl -a' (or '-x') to print SMART (and more) information               
                                                                                
[root@freenas ~]# smartctl /dev/ada1                                            
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-STABLE amd64] (local build)         
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org     
                                                                                
ATA device successfully opened                                                  
                                                                                
Use 'smartctl -a' (or '-x') to print SMART (and more) information               
                                                                                
[root@freenas ~]# smartctl /dev/ada2                                            
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-STABLE amd64] (local build)         
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org     
                                                                                
ATA device successfully opened                                                  
                                                                                
Use 'smartctl -a' (or '-x') to print SMART (and more) information               
                                                                                
[root@freenas ~]# smartctl /dev/ada3                                            
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-STABLE amd64] (local build)         
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org     
                                                                                
ATA device successfully opened                                                  
                                                                                
Use 'smartctl -a' (or '-x') to print SMART (and more) information 
 

TerryVog

Explorer
Joined
Dec 5, 2015
Messages
55
So can anyone see what is wrong or what could be the next step diagnosing it?
 

TerryVog

Explorer
Joined
Dec 5, 2015
Messages
55
This command gave me too much text to fit on the screen; even when I set it to 132x50. Oh well; here is at least the bottom part of smartctl -x for each of my disks:
Code:
[root@freenas ~]# smartctl -x /dev/ada0

SCT Temperature History Version:     2                                                                                              
Temperature Sampling Period:         1 minute                                                                                       
Temperature Logging Interval:        1 minute                                                                                       
Min/Max recommended Temperature:      0/60 Celsius                                                                                  
Min/Max Temperature Limit:           -41/85 Celsius                                                                                 
Temperature History Size (Index):    478 (424)                                                                                      
                                                                                                                                    
Index    Estimated Time   Temperature Celsius                                                                                       
 425    2019-11-15 04:13    29  **********                                                                                          
 ...    ..( 63 skipped).    ..  **********                                                                                          
  11    2019-11-15 05:17    29  **********                                                                                          
  12    2019-11-15 05:18    30  ***********                                                                                         
 ...    ..( 43 skipped).    ..  ***********                                                                                         
  56    2019-11-15 06:02    30  ***********                                                                                         
  57    2019-11-15 06:03    31  ************                                                                                        
 ...    ..(266 skipped).    ..  ************                                                                                        
 324    2019-11-15 10:30    31  ************                                                                                        
 325    2019-11-15 10:31    32  *************                                                                                       
 ...    ..( 14 skipped).    ..  *************                                                                                       
 340    2019-11-15 10:46    32  *************                                                                                       
 341    2019-11-15 10:47    29  **********                                                                                          
 ...    ..( 82 skipped).    ..  **********                                                                                          
 424    2019-11-15 12:10    29  **********                                                                                          
                                                                                                                                    
SCT Error Recovery Control:                                                                                                         
           Read:     70 (7.0 seconds)                                                                                               
          Write:     70 (7.0 seconds)                                                                                               
                                                                                                                                    
Device Statistics (GP/SMART Log 0x04) not supported                                                                                 
                                                                                                                                    
Pending Defects log (GP Log 0x0c) not supported                                                                                     
                                                                                                                                    
SATA Phy Event Counters (GP Log 0x11)                                                                                               
ID      Size     Value  Description                                                                                                 
0x0001  2            0  Command failed due to ICRC error                                                                            
0x0002  2            0  R_ERR response for data FIS                                                                                 
0x0003  2            0  R_ERR response for device-to-host data FIS                                                                  
0x0004  2            0  R_ERR response for host-to-device data FIS                                                                  
0x0005  2            0  R_ERR response for non-data FIS                                                                             
0x0006  2            0  R_ERR response for device-to-host non-data FIS                                                              
0x0007  2            0  R_ERR response for host-to-device non-data FIS                                                              
0x0008  2            0  Device-to-host non-data FIS retries                                                                         
0x0009  2            3  Transition from drive PhyRdy to drive PhyNRdy                                                               
0x000a  2            3  Device-to-host register FISes sent due to a COMRESET                                                        
0x000b  2            0  CRC errors within host-to-device FIS                                                                        
0x000f  2            0  R_ERR response for host-to-device data FIS, CRC                                                             
0x0012  2            0  R_ERR response for host-to-device non-data FIS, CRC                                                         
0x8000  4      7513873  Vendor specific 

[root@freenas ~]# smartctl -x /dev/ada1

  53    2019-11-12 22:24    35  ****************                                                                                    
  54    2019-11-12 23:58    34  ***************                                                                                     
  55    2019-11-13 01:32    34  ***************                                                                                     
  56    2019-11-13 03:06    35  ****************                                                                                    
  57    2019-11-13 04:40    34  ***************                                                                                     
  58    2019-11-13 06:14    33  **************                                                                                      
  59    2019-11-13 07:48    35  ****************                                                                                    
 ...    ..(  7 skipped).    ..  ****************                                                                                    
  67    2019-11-13 20:20    35  ****************                                                                                    
  68    2019-11-13 21:54    34  ***************                                                                                     
  69    2019-11-13 23:28    33  **************                                                                                      
  70    2019-11-14 01:02    33  **************                                                                                      
  71    2019-11-14 02:36    32  *************                                                                                       
  72    2019-11-14 04:10    32  *************                                                                                       
  73    2019-11-14 05:44    31  ************                                                                                        
  74    2019-11-14 07:18    32  *************                                                                                       
  75    2019-11-14 08:52    33  **************                                                                                      
 ...    ..(  4 skipped).    ..  **************                                                                                      
  80    2019-11-14 16:42    33  **************                                                                                      
  81    2019-11-14 18:16    34  ***************                                                                                     
  82    2019-11-14 19:50    34  ***************                                                                                     
  83    2019-11-14 21:24    34  ***************                                                                                     
  84    2019-11-14 22:58    33  **************                                                                                      
  85    2019-11-15 00:32    32  *************                                                                                       
  86    2019-11-15 02:06    32  *************                                                                                       
  87    2019-11-15 03:40    32  *************                                                                                       
  88    2019-11-15 05:14    31  ************                                                                                        
  89    2019-11-15 06:48    31  ************                                                                                        
  90    2019-11-15 08:22    32  *************                                                                                       
  91    2019-11-15 09:56    33  **************                                                                                      
  92    2019-11-15 11:30    33  **************                                                                                      
                                                                                                                                    
SCT Error Recovery Control:                                                                                                         
           Read: Disabled                                                                                                           
          Write: Disabled                                                                                                           
                                                                                                                                    
Device Statistics (GP/SMART Log 0x04) not supported                                                                                 
                                                                                                                                    
Pending Defects log (GP Log 0x0c) not supported                                                                                     
                                                                                                                                    
SATA Phy Event Counters (GP Log 0x11)                                                                                               
ID      Size     Value  Description                                                                                                 
0x000a  2            3  Device-to-host register FISes sent due to a COMRESET                                                        
0x0001  2            0  Command failed due to ICRC error                                                                            
0x0003  2            0  R_ERR response for device-to-host data FIS                                                                  
0x0004  2            0  R_ERR response for host-to-device data FIS                                                                  
0x0006  2            0  R_ERR response for device-to-host non-data FIS                                                              
0x0007  2            0  R_ERR response for host-to-device non-data FIS

[root@freenas ~]# smartctl -x /dev/ada2

  64    2019-11-13 01:32    34  ***************                                                                                     
  65    2019-11-13 03:06    35  ****************                                                                                    
  66    2019-11-13 04:40    33  **************                                                                                      
  67    2019-11-13 06:14    33  **************                                                                                      
  68    2019-11-13 07:48    35  ****************                                                                                    
 ...    ..(  5 skipped).    ..  ****************                                                                                    
  74    2019-11-13 17:12    35  ****************                                                                                    
  75    2019-11-13 18:46    36  *****************                                                                                   
  76    2019-11-13 20:20    35  ****************                                                                                    
  77    2019-11-13 21:54    34  ***************                                                                                     
  78    2019-11-13 23:28    33  **************                                                                                      
  79    2019-11-14 01:02    32  *************                                                                                       
  80    2019-11-14 02:36    32  *************                                                                                       
  81    2019-11-14 04:10    32  *************                                                                                       
  82    2019-11-14 05:44    31  ************                                                                                        
  83    2019-11-14 07:18    32  *************                                                                                       
  84    2019-11-14 08:52    33  **************                                                                                      
  85    2019-11-14 10:26    33  **************                                                                                      
  86    2019-11-14 12:00    33  **************                                                                                      
  87    2019-11-14 13:34    34  ***************                                                                                     
 ...    ..(  4 skipped).    ..  ***************                                                                                     
  92    2019-11-14 21:24    34  ***************                                                                                     
  93    2019-11-14 22:58    33  **************                                                                                      
  94    2019-11-15 00:32    32  *************                                                                                       
  95    2019-11-15 02:06    32  *************                                                                                       
  96    2019-11-15 03:40    32  *************                                                                                       
  97    2019-11-15 05:14    31  ************                                                                                        
  98    2019-11-15 06:48    31  ************                                                                                        
  99    2019-11-15 08:22    33  **************                                                                                      
 100    2019-11-15 09:56    33  **************                                                                                      
 101    2019-11-15 11:30    33  **************                                                                                      
                                                                                                                                    
SCT Error Recovery Control:                                                                                                         
           Read: Disabled                                                                                                           
          Write: Disabled                                                                                                           
                                                                                                                                    
Device Statistics (GP/SMART Log 0x04) not supported                                                                                 
                                                                                                                                    
Pending Defects log (GP Log 0x0c) not supported                                                                                     
                                                                                                                                    
SATA Phy Event Counters (GP Log 0x11)                                                                                               
ID      Size     Value  Description                                                                                                 
0x000a  2            3  Device-to-host register FISes sent due to a COMRESET                                                        
0x0001  2            0  Command failed due to ICRC error                                                                            
0x0003  2            0  R_ERR response for device-to-host data FIS                                                                  
0x0004  2            0  R_ERR response for host-to-device data FIS                                                                  
0x0006  2            0  R_ERR response for device-to-host non-data FIS                                                              
0x0007  2            0  R_ERR response for host-to-device non-data FIS   

[root@freenas ~]# smartctl -x /dev/ada3

Min/Max recommended Temperature:      0/60 Celsius                                                                                  
Min/Max Temperature Limit:           -41/85 Celsius                                                                                 
Temperature History Size (Index):    478 (150)                                                                                      
                                                                                                                                    
Index    Estimated Time   Temperature Celsius                                                                                       
 151    2019-11-15 04:17    32  *************                                                                                       
 ...    ..( 87 skipped).    ..  *************                                                                                       
 239    2019-11-15 05:45    32  *************                                                                                       
 240    2019-11-15 05:46    33  **************                                                                                      
 ...    ..( 25 skipped).    ..  **************                                                                                      
 266    2019-11-15 06:12    33  **************                                                                                      
 267    2019-11-15 06:13    31  ************                                                                                        
 ...    ..( 21 skipped).    ..  ************                                                                                        
 289    2019-11-15 06:35    31  ************                                                                                        
 290    2019-11-15 06:36    30  ***********                                                                                         
 ...    ..(110 skipped).    ..  ***********                                                                                         
 401    2019-11-15 08:27    30  ***********                                                                                         
 402    2019-11-15 08:28    31  ************                                                                                        
 ...    ..(121 skipped).    ..  ************                                                                                        
  46    2019-11-15 10:30    31  ************                                                                                        
  47    2019-11-15 10:31    32  *************                                                                                       
 ...    ..(102 skipped).    ..  *************                                                                                       
 150    2019-11-15 12:14    32  *************                                                                                       
                                                                                                                                    
SCT Error Recovery Control:                                                                                                         
           Read:     70 (7.0 seconds)                                                                                               
          Write:     70 (7.0 seconds)                                                                                               
                                                                                                                                    
Device Statistics (GP/SMART Log 0x04) not supported                                                                                 
                                                                                                                                    
Pending Defects log (GP Log 0x0c) not supported                                                                                     
                                                                                                                                    
SATA Phy Event Counters (GP Log 0x11)                                                                                               
ID      Size     Value  Description                                                                                                 
0x0001  2            0  Command failed due to ICRC error                                                                            
0x0002  2            0  R_ERR response for data FIS                                                                                 
0x0003  2            0  R_ERR response for device-to-host data FIS                                                                  
0x0004  2            0  R_ERR response for host-to-device data FIS                                                                  
0x0005  2            0  R_ERR response for non-data FIS                                                                             
0x0006  2            0  R_ERR response for device-to-host non-data FIS                                                              
0x0007  2            0  R_ERR response for host-to-device non-data FIS                                                              
0x0008  2            0  Device-to-host non-data FIS retries                                                                         
0x0009  2            3  Transition from drive PhyRdy to drive PhyNRdy                                                               
0x000a  2            3  Device-to-host register FISes sent due to a COMRESET                                                        
0x000b  2            0  CRC errors within host-to-device FIS                                                                        
0x000f  2            0  R_ERR response for host-to-device data FIS, CRC                                                             
0x0012  2            0  R_ERR response for host-to-device non-data FIS, CRC                                                         
0x8000  4      7514194  Vendor specific  
 

TerryVog

Explorer
Joined
Dec 5, 2015
Messages
55
Because of the character limit, I will post each disk separately. Here is the first one:
Code:
root@freenas:~ # smartctl -x /dev/ada0
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Red
Device Model:     WDC WD30EFRX-68EUZN0
Serial Number:    WD-WCC4N3RRFEUX
LU WWN Device Id: 5 0014ee 20c28af7b
Firmware Version: 82.00A82
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    5400 rpm
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-2 (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Fri Nov 15 14:18:51 2019 CET
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
AAM feature is:   Unavailable
APM feature is:   Unavailable
Rd look-ahead is: Enabled
Write cache is:   Enabled
DSN feature is:   Unavailable
ATA Security is:  Disabled, frozen [SEC2]
Wt Cache Reorder: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00)    Offline data collection activity
                    was never started.
                    Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0)    The previous self-test routine completed
                    without error or no self-test has ever 
                    been run.
Total time to complete Offline 
data collection:         (39900) seconds.
Offline data collection
capabilities:              (0x7b) SMART execute Offline immediate.
                    Auto Offline data collection on/off support.
                    Suspend Offline collection upon new
                    command.
                    Offline surface scan supported.
                    Self-test supported.
                    Conveyance Self-test supported.
                    Selective Self-test supported.
SMART capabilities:            (0x0003)    Saves SMART data before entering
                    power-saving mode.
                    Supports SMART auto save timer.
Error logging capability:        (0x01)    Error logging supported.
                    General Purpose Logging supported.
Short self-test routine 
recommended polling time:      (   2) minutes.
Extended self-test routine
recommended polling time:      ( 400) minutes.
Conveyance self-test routine
recommended polling time:      (   5) minutes.
SCT capabilities:            (0x703d)    SCT Status supported.
                    SCT Error Recovery Control supported.
                    SCT Feature Control supported.
                    SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE
  1 Raw_Read_Error_Rate     POSR-K   200   200   051    -    0
  3 Spin_Up_Time            POS--K   183   181   021    -    5816
  4 Start_Stop_Count        -O--CK   100   100   000    -    103
  5 Reallocated_Sector_Ct   PO--CK   200   200   140    -    0
  7 Seek_Error_Rate         -OSR-K   100   253   000    -    0
  9 Power_On_Hours          -O--CK   059   059   000    -    30534
 10 Spin_Retry_Count        -O--CK   100   100   000    -    0
 11 Calibration_Retry_Count -O--CK   100   100   000    -    0
 12 Power_Cycle_Count       -O--CK   100   100   000    -    103
192 Power-Off_Retract_Count -O--CK   200   200   000    -    88
193 Load_Cycle_Count        -O--CK   200   200   000    -    811
194 Temperature_Celsius     -O---K   118   106   000    -    32
196 Reallocated_Event_Count -O--CK   200   200   000    -    0
197 Current_Pending_Sector  -O--CK   200   200   000    -    0
198 Offline_Uncorrectable   ----CK   100   253   000    -    0
199 UDMA_CRC_Error_Count    -O--CK   200   200   000    -    0
200 Multi_Zone_Error_Rate   ---R--   100   253   000    -    0
                            ||||||_ K auto-keep
                            |||||__ C event count
                            ||||___ R error rate
                            |||____ S speed/performance
                            ||_____ O updated online
                            |______ P prefailure warning

General Purpose Log Directory Version 1
SMART           Log Directory Version 1 [multi-sector log support]
Address    Access  R/W   Size  Description
0x00       GPL,SL  R/O      1  Log Directory
0x01           SL  R/O      1  Summary SMART error log
0x02           SL  R/O      5  Comprehensive SMART error log
0x03       GPL     R/O      6  Ext. Comprehensive SMART error log
0x06           SL  R/O      1  SMART self-test log
0x07       GPL     R/O      1  Extended self-test log
0x09           SL  R/W      1  Selective self-test log
0x10       GPL     R/O      1  NCQ Command Error log
0x11       GPL     R/O      1  SATA Phy Event Counters log
0x21       GPL     R/O      1  Write stream error log
0x22       GPL     R/O      1  Read stream error log
0x80-0x9f  GPL,SL  R/W     16  Host vendor specific log
0xa0-0xa7  GPL,SL  VS      16  Device vendor specific log
0xa8-0xb7  GPL,SL  VS       1  Device vendor specific log
0xbd       GPL,SL  VS       1  Device vendor specific log
0xc0       GPL,SL  VS       1  Device vendor specific log
0xc1       GPL     VS      93  Device vendor specific log
0xe0       GPL,SL  R/W      1  SCT Command/Status
0xe1       GPL,SL  R/W      1  SCT Data Transfer

SMART Extended Comprehensive Error Log Version: 1 (6 sectors)
No Errors Logged

SMART Extended Self-test Log Version: 1 (1 sectors)
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

SCT Status Version:                  3
SCT Version (vendor specific):       258 (0x0102)
SCT Support Level:                   1
Device State:                        Active (0)
Current Temperature:                    32 Celsius
Power Cycle Min/Max Temperature:     29/42 Celsius
Lifetime    Min/Max Temperature:      2/44 Celsius
Under/Over Temperature Limit Count:   0/0
Vendor specific:
01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

SCT Temperature History Version:     2
Temperature Sampling Period:         1 minute
Temperature Logging Interval:        1 minute
Min/Max recommended Temperature:      0/60 Celsius
Min/Max Temperature Limit:           -41/85 Celsius
Temperature History Size (Index):    478 (74)

Index    Estimated Time   Temperature Celsius
  75    2019-11-15 06:21    31  ************
 ...    ..(248 skipped).    ..  ************
 324    2019-11-15 10:30    31  ************
 325    2019-11-15 10:31    32  *************
 ...    ..(142 skipped).    ..  *************
 468    2019-11-15 12:54    32  *************
 469    2019-11-15 12:55    29  **********
 ...    ..( 19 skipped).    ..  **********
  11    2019-11-15 13:15    29  **********
  12    2019-11-15 13:16    30  ***********
 ...    ..( 43 skipped).    ..  ***********
  56    2019-11-15 14:00    30  ***********
  57    2019-11-15 14:01    31  ************
 ...    ..( 16 skipped).    ..  ************
  74    2019-11-15 14:18    31  ************

SCT Error Recovery Control:
           Read:     70 (7.0 seconds)
          Write:     70 (7.0 seconds)

Device Statistics (GP/SMART Log 0x04) not supported

Pending Defects log (GP Log 0x0c) not supported

SATA Phy Event Counters (GP Log 0x11)
ID      Size     Value  Description
0x0001  2            0  Command failed due to ICRC error
0x0002  2            0  R_ERR response for data FIS
0x0003  2            0  R_ERR response for device-to-host data FIS
0x0004  2            0  R_ERR response for host-to-device data FIS
0x0005  2            0  R_ERR response for non-data FIS
0x0006  2            0  R_ERR response for device-to-host non-data FIS
0x0007  2            0  R_ERR response for host-to-device non-data FIS
0x0008  2            0  Device-to-host non-data FIS retries
0x0009  2            3  Transition from drive PhyRdy to drive PhyNRdy
0x000a  2            3  Device-to-host register FISes sent due to a COMRESET
0x000b  2            0  CRC errors within host-to-device FIS
0x000f  2            0  R_ERR response for host-to-device data FIS, CRC
0x0012  2            0  R_ERR response for host-to-device non-data FIS, CRC
0x8000  4      7521560  Vendor specific



root@freenas:~ #
 

TerryVog

Explorer
Joined
Dec 5, 2015
Messages
55
Code:
root@freenas:~ # smartctl -x /dev/ada1
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Seagate NAS HDD
Device Model:     ST3000VN000-1HJ166
Serial Number:    W6A0V1SH
LU WWN Device Id: 5 000c50 08b8b8c46
Firmware Version: SC60
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    5900 rpm
Form Factor:      3.5 inches
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-2, ACS-3 T13/2161-D revision 3b
SATA Version is:  SATA 3.1, 6.0 Gb/s (current: 3.0 Gb/s)
Local Time is:    Fri Nov 15 14:19:16 2019 CET
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
AAM feature is:   Unavailable
APM feature is:   Disabled
Rd look-ahead is: Enabled
Write cache is:   Enabled
DSN feature is:   Unavailable
ATA Security is:  Disabled, frozen [SEC2]
Wt Cache Reorder: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82)    Offline data collection activity
                    was completed without error.
                    Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0)    The previous self-test routine completed
                    without error or no self-test has ever 
                    been run.
Total time to complete Offline 
data collection:         (   97) seconds.
Offline data collection
capabilities:              (0x7b) SMART execute Offline immediate.
                    Auto Offline data collection on/off support.
                    Suspend Offline collection upon new
                    command.
                    Offline surface scan supported.
                    Self-test supported.
                    Conveyance Self-test supported.
                    Selective Self-test supported.
SMART capabilities:            (0x0003)    Saves SMART data before entering
                    power-saving mode.
                    Supports SMART auto save timer.
Error logging capability:        (0x01)    Error logging supported.
                    General Purpose Logging supported.
Short self-test routine 
recommended polling time:      (   1) minutes.
Extended self-test routine
recommended polling time:      ( 376) minutes.
Conveyance self-test routine
recommended polling time:      (   2) minutes.
SCT capabilities:            (0x10bd)    SCT Status supported.
                    SCT Error Recovery Control supported.
                    SCT Feature Control supported.
                    SCT Data Table supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE
  1 Raw_Read_Error_Rate     POSR--   118   099   006    -    182976960
  3 Spin_Up_Time            PO----   094   094   000    -    0
  4 Start_Stop_Count        -O--CK   100   100   020    -    126
  5 Reallocated_Sector_Ct   PO--CK   100   100   010    -    0
  7 Seek_Error_Rate         POSR--   082   060   030    -    197077377
  9 Power_On_Hours          -O--CK   064   064   000    -    31845
 10 Spin_Retry_Count        PO--C-   100   100   097    -    0
 12 Power_Cycle_Count       -O--CK   100   100   020    -    126
184 End-to-End_Error        -O--CK   100   100   099    -    0
187 Reported_Uncorrect      -O--CK   100   100   000    -    0
188 Command_Timeout         -O--CK   100   099   000    -    12885098499
189 High_Fly_Writes         -O-RCK   001   001   000    -    209
190 Airflow_Temperature_Cel -O---K   066   054   045    -    34 (Min/Max 31/45)
191 G-Sense_Error_Rate      -O--CK   100   100   000    -    0
192 Power-Off_Retract_Count -O--CK   100   100   000    -    110
193 Load_Cycle_Count        -O--CK   100   100   000    -    1412
194 Temperature_Celsius     -O---K   034   046   000    -    34 (0 16 0 0 0)
197 Current_Pending_Sector  -O--C-   100   100   000    -    0
198 Offline_Uncorrectable   ----C-   100   100   000    -    0
199 UDMA_CRC_Error_Count    -OSRCK   200   200   000    -    0
                            ||||||_ K auto-keep
                            |||||__ C event count
                            ||||___ R error rate
                            |||____ S speed/performance
                            ||_____ O updated online
                            |______ P prefailure warning

General Purpose Log Directory Version 1
SMART           Log Directory Version 1 [multi-sector log support]
Address    Access  R/W   Size  Description
0x00       GPL,SL  R/O      1  Log Directory
0x01           SL  R/O      1  Summary SMART error log
0x02           SL  R/O      5  Comprehensive SMART error log
0x03       GPL     R/O      5  Ext. Comprehensive SMART error log
0x06           SL  R/O      1  SMART self-test log
0x07       GPL     R/O      1  Extended self-test log
0x09           SL  R/W      1  Selective self-test log
0x10       GPL     R/O      1  NCQ Command Error log
0x11       GPL     R/O      1  SATA Phy Event Counters log
0x15       GPL     R/W      1  Rebuild Assist log
0x21       GPL     R/O      1  Write stream error log
0x22       GPL     R/O      1  Read stream error log
0x80-0x9f  GPL,SL  R/W     16  Host vendor specific log
0xa1       GPL,SL  VS      20  Device vendor specific log
0xa2       GPL     VS    4496  Device vendor specific log
0xa8       GPL,SL  VS     129  Device vendor specific log
0xa9       GPL,SL  VS       1  Device vendor specific log
0xab       GPL     VS       1  Device vendor specific log
0xb0       GPL     VS    5176  Device vendor specific log
0xbe-0xbf  GPL     VS   65535  Device vendor specific log
0xc0       GPL,SL  VS       1  Device vendor specific log
0xc1       GPL,SL  VS      10  Device vendor specific log
0xc3       GPL,SL  VS       8  Device vendor specific log
0xc4       GPL,SL  VS       5  Device vendor specific log
0xd1       GPL,SL  VS       8  Device vendor specific log
0xe0       GPL,SL  R/W      1  SCT Command/Status
0xe1       GPL,SL  R/W      1  SCT Data Transfer

SMART Extended Comprehensive Error Log Version: 1 (5 sectors)
No Errors Logged

SMART Extended Self-test Log Version: 1 (1 sectors)
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Completed without error       00%       960         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

SCT Status Version:                  3
SCT Version (vendor specific):       522 (0x020a)
SCT Support Level:                   1
Device State:                        Active (0)
Current Temperature:                    34 Celsius
Power Cycle Min/Max Temperature:     31/45 Celsius
Lifetime    Min/Max Temperature:     16/46 Celsius
Under/Over Temperature Limit Count:   0/0

SCT Temperature History Version:     2
Temperature Sampling Period:         1 minute
Temperature Logging Interval:        94 minutes
Min/Max recommended Temperature:      1/61 Celsius
Min/Max Temperature Limit:            2/60 Celsius
Temperature History Size (Index):    128 (93)

Index    Estimated Time   Temperature Celsius
  94    2019-11-07 06:06    33  **************
  95    2019-11-07 07:40    34  ***************
  96    2019-11-07 09:14    35  ****************
 ...    ..(  7 skipped).    ..  ****************
 104    2019-11-07 21:46    35  ****************
 105    2019-11-07 23:20    34  ***************
 106    2019-11-08 00:54    34  ***************
 107    2019-11-08 02:28    33  **************
 108    2019-11-08 04:02    33  **************
 109    2019-11-08 05:36    33  **************
 110    2019-11-08 07:10    34  ***************
 ...    ..(  4 skipped).    ..  ***************
 115    2019-11-08 15:00    34  ***************
 116    2019-11-08 16:34    35  ****************
 ...    ..(  4 skipped).    ..  ****************
 121    2019-11-09 00:24    35  ****************
 122    2019-11-09 01:58    34  ***************
 ...    ..( 14 skipped).    ..  ***************
   9    2019-11-10 01:28    34  ***************
  10    2019-11-10 03:02    33  **************
 ...    ..(  6 skipped).    ..  **************
  17    2019-11-10 14:00    33  **************
  18    2019-11-10 15:34    34  ***************
  19    2019-11-10 17:08    34  ***************
  20    2019-11-10 18:42    34  ***************
  21    2019-11-10 20:16    35  ****************
  22    2019-11-10 21:50    34  ***************
  23    2019-11-10 23:24    34  ***************
  24    2019-11-11 00:58    33  **************
  25    2019-11-11 02:32    33  **************
  26    2019-11-11 04:06    34  ***************
 ...    ..( 11 skipped).    ..  ***************
  38    2019-11-11 22:54    34  ***************
  39    2019-11-12 00:28    33  **************
  40    2019-11-12 02:02    32  *************
  41    2019-11-12 03:36    32  *************
  42    2019-11-12 05:10    33  **************
  43    2019-11-12 06:44    33  **************
  44    2019-11-12 08:18    34  ***************
 ...    ..(  4 skipped).    ..  ***************
  49    2019-11-12 16:08    34  ***************
  50    2019-11-12 17:42    35  ****************
 ...    ..(  2 skipped).    ..  ****************
  53    2019-11-12 22:24    35  ****************
  54    2019-11-12 23:58    34  ***************
  55    2019-11-13 01:32    34  ***************
  56    2019-11-13 03:06    35  ****************
  57    2019-11-13 04:40    34  ***************
  58    2019-11-13 06:14    33  **************
  59    2019-11-13 07:48    35  ****************
 ...    ..(  7 skipped).    ..  ****************
  67    2019-11-13 20:20    35  ****************
  68    2019-11-13 21:54    34  ***************
  69    2019-11-13 23:28    33  **************
  70    2019-11-14 01:02    33  **************
  71    2019-11-14 02:36    32  *************
  72    2019-11-14 04:10    32  *************
  73    2019-11-14 05:44    31  ************
  74    2019-11-14 07:18    32  *************
  75    2019-11-14 08:52    33  **************
 ...    ..(  4 skipped).    ..  **************
  80    2019-11-14 16:42    33  **************
  81    2019-11-14 18:16    34  ***************
  82    2019-11-14 19:50    34  ***************
  83    2019-11-14 21:24    34  ***************
  84    2019-11-14 22:58    33  **************
  85    2019-11-15 00:32    32  *************
  86    2019-11-15 02:06    32  *************
  87    2019-11-15 03:40    32  *************
  88    2019-11-15 05:14    31  ************
  89    2019-11-15 06:48    31  ************
  90    2019-11-15 08:22    32  *************
  91    2019-11-15 09:56    33  **************
  92    2019-11-15 11:30    33  **************
  93    2019-11-15 13:04    34  ***************

SCT Error Recovery Control:
           Read: Disabled
          Write: Disabled

Device Statistics (GP/SMART Log 0x04) not supported

Pending Defects log (GP Log 0x0c) not supported

SATA Phy Event Counters (GP Log 0x11)
ID      Size     Value  Description
0x000a  2            3  Device-to-host register FISes sent due to a COMRESET
0x0001  2            0  Command failed due to ICRC error
0x0003  2            0  R_ERR response for device-to-host data FIS
0x0004  2            0  R_ERR response for host-to-device data FIS
0x0006  2            0  R_ERR response for device-to-host non-data FIS
0x0007  2            0  R_ERR response for host-to-device non-data FIS

root@freenas:~ #
 

TerryVog

Explorer
Joined
Dec 5, 2015
Messages
55
Code:
root@freenas:~ smartctl -x /dev/ada2
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Seagate NAS HDD
Device Model:     ST3000VN000-1HJ166
Serial Number:    W6A0V22F
LU WWN Device Id: 5 000c50 08b8b6833
Firmware Version: SC60
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    5900 rpm
Form Factor:      3.5 inches
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-2, ACS-3 T13/2161-D revision 3b
SATA Version is:  SATA 3.1, 6.0 Gb/s (current: 3.0 Gb/s)
Local Time is:    Fri Nov 15 14:19:21 2019 CET
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
AAM feature is:   Unavailable
APM feature is:   Disabled
Rd look-ahead is: Enabled
Write cache is:   Enabled
DSN feature is:   Unavailable
ATA Security is:  Disabled, frozen [SEC2]
Wt Cache Reorder: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82)    Offline data collection activity
                    was completed without error.
                    Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0)    The previous self-test routine completed
                    without error or no self-test has ever 
                    been run.
Total time to complete Offline 
data collection:         (  107) seconds.
Offline data collection
capabilities:              (0x7b) SMART execute Offline immediate.
                    Auto Offline data collection on/off support.
                    Suspend Offline collection upon new
                    command.
                    Offline surface scan supported.
                    Self-test supported.
                    Conveyance Self-test supported.
                    Selective Self-test supported.
SMART capabilities:            (0x0003)    Saves SMART data before entering
                    power-saving mode.
                    Supports SMART auto save timer.
Error logging capability:        (0x01)    Error logging supported.
                    General Purpose Logging supported.
Short self-test routine 
recommended polling time:      (   1) minutes.
Extended self-test routine
recommended polling time:      ( 383) minutes.
Conveyance self-test routine
recommended polling time:      (   2) minutes.
SCT capabilities:            (0x10bd)    SCT Status supported.
                    SCT Error Recovery Control supported.
                    SCT Feature Control supported.
                    SCT Data Table supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE
  1 Raw_Read_Error_Rate     POSR--   119   099   006    -    234547360
  3 Spin_Up_Time            PO----   094   094   000    -    0
  4 Start_Stop_Count        -O--CK   100   100   020    -    127
  5 Reallocated_Sector_Ct   PO--CK   100   100   010    -    0
  7 Seek_Error_Rate         POSR--   083   060   030    -    203711907
  9 Power_On_Hours          -O--CK   064   064   000    -    31865
 10 Spin_Retry_Count        PO--C-   100   100   097    -    0
 12 Power_Cycle_Count       -O--CK   100   100   020    -    127
184 End-to-End_Error        -O--CK   100   100   099    -    0
187 Reported_Uncorrect      -O--CK   100   100   000    -    0
188 Command_Timeout         -O--CK   100   099   000    -    12885098499
189 High_Fly_Writes         -O-RCK   001   001   000    -    350
190 Airflow_Temperature_Cel -O---K   066   054   045    -    34 (Min/Max 31/44)
191 G-Sense_Error_Rate      -O--CK   100   100   000    -    0
192 Power-Off_Retract_Count -O--CK   100   100   000    -    110
193 Load_Cycle_Count        -O--CK   100   100   000    -    1412
194 Temperature_Celsius     -O---K   034   046   000    -    34 (0 17 0 0 0)
197 Current_Pending_Sector  -O--C-   100   100   000    -    0
198 Offline_Uncorrectable   ----C-   100   100   000    -    0
199 UDMA_CRC_Error_Count    -OSRCK   200   200   000    -    0
                            ||||||_ K auto-keep
                            |||||__ C event count
                            ||||___ R error rate
                            |||____ S speed/performance
                            ||_____ O updated online
                            |______ P prefailure warning

General Purpose Log Directory Version 1
SMART           Log Directory Version 1 [multi-sector log support]
Address    Access  R/W   Size  Description
0x00       GPL,SL  R/O      1  Log Directory
0x01           SL  R/O      1  Summary SMART error log
0x02           SL  R/O      5  Comprehensive SMART error log
0x03       GPL     R/O      5  Ext. Comprehensive SMART error log
0x06           SL  R/O      1  SMART self-test log
0x07       GPL     R/O      1  Extended self-test log
0x09           SL  R/W      1  Selective self-test log
0x10       GPL     R/O      1  NCQ Command Error log
0x11       GPL     R/O      1  SATA Phy Event Counters log
0x15       GPL     R/W      1  Rebuild Assist log
0x21       GPL     R/O      1  Write stream error log
0x22       GPL     R/O      1  Read stream error log
0x80-0x9f  GPL,SL  R/W     16  Host vendor specific log
0xa1       GPL,SL  VS      20  Device vendor specific log
0xa2       GPL     VS    4496  Device vendor specific log
0xa8       GPL,SL  VS     129  Device vendor specific log
0xa9       GPL,SL  VS       1  Device vendor specific log
0xab       GPL     VS       1  Device vendor specific log
0xb0       GPL     VS    5176  Device vendor specific log
0xbe-0xbf  GPL     VS   65535  Device vendor specific log
0xc0       GPL,SL  VS       1  Device vendor specific log
0xc1       GPL,SL  VS      10  Device vendor specific log
0xc3       GPL,SL  VS       8  Device vendor specific log
0xc4       GPL,SL  VS       5  Device vendor specific log
0xd1       GPL,SL  VS       8  Device vendor specific log
0xe0       GPL,SL  R/W      1  SCT Command/Status
0xe1       GPL,SL  R/W      1  SCT Data Transfer

SMART Extended Comprehensive Error Log Version: 1 (5 sectors)
No Errors Logged

SMART Extended Self-test Log Version: 1 (1 sectors)
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Completed without error       00%     24617         -
# 2  Short offline       Completed without error       00%     24449         -
# 3  Short offline       Completed without error       00%     24290         -
# 4  Short offline       Completed without error       00%     24122         -
# 5  Extended offline    Completed without error       00%     24096         -
# 6  Conveyance offline  Completed without error       00%     24089         -
# 7  Short offline       Completed without error       00%     23954         -
# 8  Short offline       Completed without error       00%     23786         -
# 9  Short offline       Completed without error       00%     23618         -
#10  Short offline       Completed without error       00%     23450         -
#11  Extended offline    Completed without error       00%     23376         -
#12  Conveyance offline  Completed without error       00%     23369         -
#13  Short offline       Completed without error       00%     23282         -
#14  Short offline       Completed without error       00%     23114         -
#15  Short offline       Completed without error       00%     22946         -
#16  Short offline       Completed without error       00%     22778         -
#17  Extended offline    Completed without error       00%     22631         -
#18  Conveyance offline  Completed without error       00%     22624         -
#19  Short offline       Completed without error       00%     22609         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

SCT Status Version:                  3
SCT Version (vendor specific):       522 (0x020a)
SCT Support Level:                   1
Device State:                        Active (0)
Current Temperature:                    34 Celsius
Power Cycle Min/Max Temperature:     31/44 Celsius
Lifetime    Min/Max Temperature:     17/48 Celsius
Under/Over Temperature Limit Count:   0/0

SCT Temperature History Version:     2
Temperature Sampling Period:         1 minute
Temperature Logging Interval:        94 minutes
Min/Max recommended Temperature:      1/61 Celsius
Min/Max Temperature Limit:            2/60 Celsius
Temperature History Size (Index):    128 (102)

Index    Estimated Time   Temperature Celsius
 103    2019-11-07 06:06    33  **************
 104    2019-11-07 07:40    34  ***************
 105    2019-11-07 09:14    35  ****************
 ...    ..(  7 skipped).    ..  ****************
 113    2019-11-07 21:46    35  ****************
 114    2019-11-07 23:20    34  ***************
 115    2019-11-08 00:54    34  ***************
 116    2019-11-08 02:28    33  **************
 117    2019-11-08 04:02    33  **************
 118    2019-11-08 05:36    34  ***************
 119    2019-11-08 07:10    34  ***************
 120    2019-11-08 08:44    34  ***************
 121    2019-11-08 10:18    35  ****************
 122    2019-11-08 11:52    34  ***************
 123    2019-11-08 13:26    35  ****************
 ...    ..(  5 skipped).    ..  ****************
   1    2019-11-08 22:50    35  ****************
   2    2019-11-09 00:24    34  ***************
 ...    ..( 13 skipped).    ..  ***************
  16    2019-11-09 22:20    34  ***************
  17    2019-11-09 23:54    33  **************
 ...    ..(  8 skipped).    ..  **************
  26    2019-11-10 14:00    33  **************
  27    2019-11-10 15:34    34  ***************
  28    2019-11-10 17:08    34  ***************
  29    2019-11-10 18:42    34  ***************
  30    2019-11-10 20:16    35  ****************
  31    2019-11-10 21:50    34  ***************
  32    2019-11-10 23:24    34  ***************
  33    2019-11-11 00:58    33  **************
  34    2019-11-11 02:32    33  **************
  35    2019-11-11 04:06    34  ***************
 ...    ..(  9 skipped).    ..  ***************
  45    2019-11-11 19:46    34  ***************
  46    2019-11-11 21:20    35  ****************
  47    2019-11-11 22:54    34  ***************
  48    2019-11-12 00:28    32  *************
  49    2019-11-12 02:02    32  *************
  50    2019-11-12 03:36    33  **************
  51    2019-11-12 05:10    33  **************
  52    2019-11-12 06:44    34  ***************
 ...    ..(  5 skipped).    ..  ***************
  58    2019-11-12 16:08    34  ***************
  59    2019-11-12 17:42    35  ****************
  60    2019-11-12 19:16    36  *****************
  61    2019-11-12 20:50    35  ****************
  62    2019-11-12 22:24    35  ****************
  63    2019-11-12 23:58    35  ****************
  64    2019-11-13 01:32    34  ***************
  65    2019-11-13 03:06    35  ****************
  66    2019-11-13 04:40    33  **************
  67    2019-11-13 06:14    33  **************
  68    2019-11-13 07:48    35  ****************
 ...    ..(  5 skipped).    ..  ****************
  74    2019-11-13 17:12    35  ****************
  75    2019-11-13 18:46    36  *****************
  76    2019-11-13 20:20    35  ****************
  77    2019-11-13 21:54    34  ***************
  78    2019-11-13 23:28    33  **************
  79    2019-11-14 01:02    32  *************
  80    2019-11-14 02:36    32  *************
  81    2019-11-14 04:10    32  *************
  82    2019-11-14 05:44    31  ************
  83    2019-11-14 07:18    32  *************
  84    2019-11-14 08:52    33  **************
  85    2019-11-14 10:26    33  **************
  86    2019-11-14 12:00    33  **************
  87    2019-11-14 13:34    34  ***************
 ...    ..(  4 skipped).    ..  ***************
  92    2019-11-14 21:24    34  ***************
  93    2019-11-14 22:58    33  **************
  94    2019-11-15 00:32    32  *************
  95    2019-11-15 02:06    32  *************
  96    2019-11-15 03:40    32  *************
  97    2019-11-15 05:14    31  ************
  98    2019-11-15 06:48    31  ************
  99    2019-11-15 08:22    33  **************
 100    2019-11-15 09:56    33  **************
 101    2019-11-15 11:30    33  **************
 102    2019-11-15 13:04    34  ***************

SCT Error Recovery Control:
           Read: Disabled
          Write: Disabled

Device Statistics (GP/SMART Log 0x04) not supported

Pending Defects log (GP Log 0x0c) not supported

SATA Phy Event Counters (GP Log 0x11)
ID      Size     Value  Description
0x000a  2            3  Device-to-host register FISes sent due to a COMRESET
0x0001  2            0  Command failed due to ICRC error
0x0003  2            0  R_ERR response for device-to-host data FIS
0x0004  2            0  R_ERR response for host-to-device data FIS
0x0006  2            0  R_ERR response for device-to-host non-data FIS
0x0007  2            0  R_ERR response for host-to-device non-data FIS

root@freenas:~ #
 

TerryVog

Explorer
Joined
Dec 5, 2015
Messages
55
And the last one:
Code:
root@freenas:~ # smartctl -x /dev/ada3
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Red
Device Model:     WDC WD30EFRX-68EUZN0
Serial Number:    WD-WCC4NEKJ1J4T
LU WWN Device Id: 5 0014ee 20aaf0e1b
Firmware Version: 82.00A82
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    5400 rpm
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-2 (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 3.0 Gb/s)
Local Time is:    Fri Nov 15 14:19:25 2019 CET
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
AAM feature is:   Unavailable
APM feature is:   Unavailable
Rd look-ahead is: Enabled
Write cache is:   Enabled
DSN feature is:   Unavailable
ATA Security is:  Disabled, frozen [SEC2]
Wt Cache Reorder: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00)    Offline data collection activity
                    was never started.
                    Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0)    The previous self-test routine completed
                    without error or no self-test has ever 
                    been run.
Total time to complete Offline 
data collection:         (41040) seconds.
Offline data collection
capabilities:              (0x7b) SMART execute Offline immediate.
                    Auto Offline data collection on/off support.
                    Suspend Offline collection upon new
                    command.
                    Offline surface scan supported.
                    Self-test supported.
                    Conveyance Self-test supported.
                    Selective Self-test supported.
SMART capabilities:            (0x0003)    Saves SMART data before entering
                    power-saving mode.
                    Supports SMART auto save timer.
Error logging capability:        (0x01)    Error logging supported.
                    General Purpose Logging supported.
Short self-test routine 
recommended polling time:      (   2) minutes.
Extended self-test routine
recommended polling time:      ( 412) minutes.
Conveyance self-test routine
recommended polling time:      (   5) minutes.
SCT capabilities:            (0x703d)    SCT Status supported.
                    SCT Error Recovery Control supported.
                    SCT Feature Control supported.
                    SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE
  1 Raw_Read_Error_Rate     POSR-K   200   200   051    -    0
  3 Spin_Up_Time            POS--K   182   179   021    -    5858
  4 Start_Stop_Count        -O--CK   100   100   000    -    154
  5 Reallocated_Sector_Ct   PO--CK   200   200   140    -    0
  7 Seek_Error_Rate         -OSR-K   100   253   000    -    0
  9 Power_On_Hours          -O--CK   053   053   000    -    34913
 10 Spin_Retry_Count        -O--CK   100   100   000    -    0
 11 Calibration_Retry_Count -O--CK   100   100   000    -    0
 12 Power_Cycle_Count       -O--CK   100   100   000    -    154
192 Power-Off_Retract_Count -O--CK   200   200   000    -    136
193 Load_Cycle_Count        -O--CK   200   200   000    -    861
194 Temperature_Celsius     -O---K   117   105   000    -    33
196 Reallocated_Event_Count -O--CK   200   200   000    -    0
197 Current_Pending_Sector  -O--CK   200   200   000    -    0
198 Offline_Uncorrectable   ----CK   100   253   000    -    0
199 UDMA_CRC_Error_Count    -O--CK   200   200   000    -    0
200 Multi_Zone_Error_Rate   ---R--   200   200   000    -    0
                            ||||||_ K auto-keep
                            |||||__ C event count
                            ||||___ R error rate
                            |||____ S speed/performance
                            ||_____ O updated online
                            |______ P prefailure warning

General Purpose Log Directory Version 1
SMART           Log Directory Version 1 [multi-sector log support]
Address    Access  R/W   Size  Description
0x00       GPL,SL  R/O      1  Log Directory
0x01           SL  R/O      1  Summary SMART error log
0x02           SL  R/O      5  Comprehensive SMART error log
0x03       GPL     R/O      6  Ext. Comprehensive SMART error log
0x06           SL  R/O      1  SMART self-test log
0x07       GPL     R/O      1  Extended self-test log
0x09           SL  R/W      1  Selective self-test log
0x10       GPL     R/O      1  NCQ Command Error log
0x11       GPL     R/O      1  SATA Phy Event Counters log
0x21       GPL     R/O      1  Write stream error log
0x22       GPL     R/O      1  Read stream error log
0x80-0x9f  GPL,SL  R/W     16  Host vendor specific log
0xa0-0xa7  GPL,SL  VS      16  Device vendor specific log
0xa8-0xb7  GPL,SL  VS       1  Device vendor specific log
0xbd       GPL,SL  VS       1  Device vendor specific log
0xc0       GPL,SL  VS       1  Device vendor specific log
0xc1       GPL     VS      93  Device vendor specific log
0xe0       GPL,SL  R/W      1  SCT Command/Status
0xe1       GPL,SL  R/W      1  SCT Data Transfer

SMART Extended Comprehensive Error Log Version: 1 (6 sectors)
Device Error Count: 2
    CR     = Command Register
    FEATR  = Features Register
    COUNT  = Count (was: Sector Count) Register
    LBA_48 = Upper bytes of LBA High/Mid/Low Registers ]  ATA-8
    LH     = LBA High (was: Cylinder High) Register    ]   LBA
    LM     = LBA Mid (was: Cylinder Low) Register      ] Register
    LL     = LBA Low (was: Sector Number) Register     ]
    DV     = Device (was: Device/Head) Register
    DC     = Device Control Register
    ER     = Error register
    ST     = Status register
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.

Error 2 [1] occurred at disk power-on lifetime: 28046 hours (1168 days + 14 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER -- ST COUNT  LBA_48  LH LM LL DV DC
  -- -- -- == -- == == == -- -- -- -- --
  10 -- 51 00 00 00 00 f1 b8 98 e8 40 00  Error: IDNF at LBA = 0xf1b898e8 = 4055406824

  Commands leading to the command that caused the error were:
  CR FEATR COUNT  LBA_48  LH LM LL DV DC  Powered_Up_Time  Command/Feature_Name
  -- == -- == -- == == == -- -- -- -- --  ---------------  --------------------
  61 00 10 00 98 00 00 f1 b8 98 e8 40 08 10d+23:01:46.335  WRITE FPDMA QUEUED
  61 00 10 00 90 00 00 f1 b8 98 d0 40 08 10d+23:01:46.335  WRITE FPDMA QUEUED
  61 00 08 00 88 00 00 f1 b8 98 c0 40 08 10d+23:01:46.335  WRITE FPDMA QUEUED
  61 00 08 00 80 00 00 98 4e c8 20 40 08 10d+23:01:46.335  WRITE FPDMA QUEUED
  61 00 10 00 78 00 00 98 4e c8 00 40 08 10d+23:01:46.335  WRITE FPDMA QUEUED

Error 1 [0] occurred at disk power-on lifetime: 4376 hours (182 days + 8 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER -- ST COUNT  LBA_48  LH LM LL DV DC
  -- -- -- == -- == == == -- -- -- -- --
  10 -- 51 00 08 00 00 3c 51 d0 30 40 00  Error: IDNF 8 sectors at LBA = 0x3c51d030 = 1011994672

  Commands leading to the command that caused the error were:
  CR FEATR COUNT  LBA_48  LH LM LL DV DC  Powered_Up_Time  Command/Feature_Name
  -- == -- == -- == == == -- -- -- -- --  ---------------  --------------------
  35 00 00 00 08 00 00 3c 51 d0 30 40 00     00:31:09.197  WRITE DMA EXT
  ea 00 00 00 00 00 00 00 00 00 00 40 00     00:31:09.113  FLUSH CACHE EXT
  35 00 00 00 18 00 00 3c 4c 2a c8 40 00     00:31:09.113  WRITE DMA EXT
  35 00 00 01 00 00 00 3c 4c 29 c8 40 00     00:31:09.112  WRITE DMA EXT
  ea 00 00 00 00 00 00 00 00 00 00 40 00     00:31:09.100  FLUSH CACHE EXT

SMART Extended Self-test Log Version: 1 (1 sectors)
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Completed without error       00%     27714         -
# 2  Short offline       Completed without error       00%     27546         -
# 3  Short offline       Completed without error       00%     27387         -
# 4  Short offline       Completed without error       00%     27219         -
# 5  Extended offline    Completed without error       00%     27171         -
# 6  Conveyance offline  Completed without error       00%     27162         -
# 7  Short offline       Completed without error       00%     27051         -
# 8  Short offline       Completed without error       00%     26884         -
# 9  Short offline       Completed without error       00%     26716         -
#10  Short offline       Completed without error       00%     26548         -
#11  Extended offline    Completed without error       00%     26452         -
#12  Conveyance offline  Completed without error       00%     26443         -
#13  Short offline       Completed without error       00%     26380         -
#14  Short offline       Completed without error       00%     26212         -
#15  Short offline       Completed without error       00%     26044         -
#16  Short offline       Completed without error       00%     25877         -
#17  Short offline       Completed without error       00%     25708         -
#18  Extended offline    Completed without error       00%     25707         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

SCT Status Version:                  3
SCT Version (vendor specific):       258 (0x0102)
SCT Support Level:                   1
Device State:                        Active (0)
Current Temperature:                    33 Celsius
Power Cycle Min/Max Temperature:     30/43 Celsius
Lifetime    Min/Max Temperature:      2/45 Celsius
Under/Over Temperature Limit Count:   0/0
Vendor specific:
01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

SCT Temperature History Version:     2
Temperature Sampling Period:         1 minute
Temperature Logging Interval:        1 minute
Min/Max recommended Temperature:      0/60 Celsius
Min/Max Temperature Limit:           -41/85 Celsius
Temperature History Size (Index):    478 (274)

Index    Estimated Time   Temperature Celsius
 275    2019-11-15 06:22    33  **************
 ...    ..(114 skipped).    ..  **************
 390    2019-11-15 08:17    33  **************
 391    2019-11-15 08:18    30  ***********
 ...    ..(  9 skipped).    ..  ***********
 401    2019-11-15 08:28    30  ***********
 402    2019-11-15 08:29    31  ************
 ...    ..(121 skipped).    ..  ************
  46    2019-11-15 10:31    31  ************
  47    2019-11-15 10:32    32  *************
 ...    ..(191 skipped).    ..  *************
 239    2019-11-15 13:44    32  *************
 240    2019-11-15 13:45    33  **************
 ...    ..( 33 skipped).    ..  **************
 274    2019-11-15 14:19    33  **************

SCT Error Recovery Control:
           Read:     70 (7.0 seconds)
          Write:     70 (7.0 seconds)

Device Statistics (GP/SMART Log 0x04) not supported

Pending Defects log (GP Log 0x0c) not supported

SATA Phy Event Counters (GP Log 0x11)
ID      Size     Value  Description
0x0001  2            0  Command failed due to ICRC error
0x0002  2            0  R_ERR response for data FIS
0x0003  2            0  R_ERR response for device-to-host data FIS
0x0004  2            0  R_ERR response for host-to-device data FIS
0x0005  2            0  R_ERR response for non-data FIS
0x0006  2            0  R_ERR response for device-to-host non-data FIS
0x0007  2            0  R_ERR response for host-to-device non-data FIS
0x0008  2            0  Device-to-host non-data FIS retries
0x0009  2            3  Transition from drive PhyRdy to drive PhyNRdy
0x000a  2            3  Device-to-host register FISes sent due to a COMRESET
0x000b  2            0  CRC errors within host-to-device FIS
0x000f  2            0  R_ERR response for host-to-device data FIS, CRC
0x0012  2            0  R_ERR response for host-to-device non-data FIS, CRC
0x8000  4      7521671  Vendor specific

root@freenas:~ #
 
Joined
May 10, 2017
Messages
838
Disks look fine, but you're not running regular SMART tests, be sure to schedule them, there are no set rules for frequency but IMHO set at least one extended SMART test per month, and like one short per week.
 
Joined
Jul 3, 2015
Messages
926
Unlikely that all four of your disks have suffered an error at the same time which has resulted in CKSUM errors. Much more likely to be a cable, mobo or power supply issue. Make sure your data is backed up as something is not right.
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,457
Top