That isn't how it works. You can take drives from any pool and plug them into another system running FreeNAS and import them (so long as the FreeNAs versions are compatible). The system config saves certain share settings etc but isn't required to import a new pool and typically isn't stored on your pool.my biggest concern, is there any chance the Freenas config gets embedded to the 8 hard disks ? making it unrecognized which ever hardware changes i make.
I wouldn't necessarily try this right away. If the system is otherwise stable it is likely not the issue.will re installing and uploading the back up config help ?
I don't know anything about this specific bit of hardware but I doubt it is related; it is 3 drives and 3 SATA plugs so likely each has its own plug. Though, it is probably good to just go with the onboard SATA ports.i have taken out the thermaltake 3503 hard disks bay https://www.thermaltake.com/C_00002934.htm and plug sata all direct to raid card and motherboard ports. with only bad slog, new slog not plugged in.
smartctl -a /dev/{device}
for every drive in the pool and see what is up. Specifically I'd look for any failing smart tests. If you've never run any smart tests now is a good time to do a long test on those drives. To identify failures look for actual test errors but also look for values > 0 for the following lines.. . . 5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail Always - 0 . . . 197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0 . . .
OK i feel x10 better knowing this and it isn't a software issue, many thanks for thisThat isn't how it works. You can take drives from any pool and plug them into another system running FreeNAS and import them (so long as the FreeNAs versions are compatible). The system config saves certain share settings etc but isn't required to import a new pool and typically isn't stored on your pool.
OK got it, if i do will try on a totally different hardware=processor, motherboard, rams, raid cardI wouldn't necessarily try this right away. If the system is otherwise stable it is likely not the issue.
yes i have already removed the disks and they are directly plugged to SATA portsI don't know anything about this specific bit of hardware but I doubt it is related; it is 3 drives and 3 SATA plugs so likely each has its own plug. Though, it is probably good to just go with the onboard SATA ports.
OK i will do this, but just let me try the different hardware to recognize the pools first, i would like that first level of comfort. Here is a screenshot of da0 which is one of the diskAt this point I'd go throughsmartctl -a /dev/{device}
for every drive in the pool and see what is up. Specifically I'd look for any failing smart tests. If you've never run any smart tests now is a good time to do a long test on those drives. To identify failures look for actual test errors but also look for values > 0 for the following lines.
Again x10 felt better if it is hardware issue (assuming not the disks or software issue). yes i do have 2 different machines which i can alternate to plug in to see if the pool is recognized. Will go on the assumption of hardware issue. Let me set up those hardware now and i will post again.Do you have another FreeNAS machine you could plug these drives into to try to import the pool?
Right now my guess is that the issue is hardware; either that RAID card, some of the drives, or the onoard SATA controller. If we exhaust every option you may want to try to get those drives into a machine without any RAID card whatsoever directly connected via SATA or an HBA and see if you could import them.
Focus on the mobo and RAID cards first.OK got it, if i do will try on a totally different hardware=processor, motherboard, rams, raid card
SATA ports on your motherboard or via that card?yes i have already removed the disks and they are directly plugged to SATA ports
I would suggest you do this step as soon as you're able. Running the smart tests on your drives may tell us how many are failing or having issues etc.OK i will do this, but just let me try the different hardware to recognize the pools first, i would like that first level of comfort. Here is a screenshot of da0 which is one of the disk
OK i will, i only have 2 raid cards flashed to IT, the others i have not flashed(will take time). As i have 2 spare mobo (1 which is a lab status, let's call it AMD965 and is already in case ready to deploy). i can move the 8 disks Data Storage pool to this PC and see if the pool is detected for import. if is then all is well and the Asus X99 mobo have a problem. From here i will assemble another production server ( let's call it IntelE2680, couple of hours only), put all the 12 disks + 2 slog, install fresh 11.2 U6, upload config, and hope all goes well. that's my plan, all depends if the Data Storage pool can be recognized by AMD965 for pool import. i have not done a pool import before and current config has lots of settings which i may not be able to repeat.Focus on the mobo and RAID cards first.
both, as mobo only has 8 ports and i have 12 hdd + 2 slog. To test both the raid cards i will alternate and plug in Data Media, if this pool is not recognized then i know it's the raid card and which one.SATA ports on your motherboard or via that card?
yes i will, i just want to be comfortable that the Data Storage pool is recognized first. if i do smart tests first are we suspecting some disks have totally failed which is why the pool is not recognized ? there are no prior alerts to this. but if this is a real concern then i will do it first and post here for all the 8 disk.I would suggest you do this step as soon as you're able. Running the smart tests on your drives may tell us how many are failing or having issues etc.
i am doing it via GUI and i have tried to highlight the test and copy-paste to a note pad but it does not work, no option to do that.How are you accessing your terminal? If you're doing it via the GUI could you just highlight the text and copy-paste the values into these forums? It'll save you having to take so many screen shots and truncating results. For example, the image above doesn't include the full output such as test results, or reallocated sector count. It really is best not to truncate information where possible.
Yes, that is why I want to look atyes i will, i just want to be comfortable that the Data Storage pool is recognized first. if i do smart tests first are we suspecting some disks have totally failed which is why the pool is not recognized ? there are no prior alerts to this. but if this is a real concern then i will do it first and post here for all the 8 disk.
smartctl -a
and run long tests. If drives are hosed that would explain your situation.alright i will do the smart test first. can you please guide me on using shell command to do it ?Yes, that is why I want to look atsmartctl -a
and run long tests. If drives are hosed that would explain your situation.
smartctl -t {short|long} {device}
. I do encourage you to check the forums for this kind of information though, there are a lot of great resources around here. :)OK but i am not very familiar with this and ran into this problemFor things like that you'll find great resources looking to the manual, which is here, or even a good old search of this this forum or googling. In general it issmartctl -t {short|long} {device}
. I do encourage you to check the forums for this kind of information though, there are a lot of great resources around here. :)
SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. [root@freenas ~]# smartctl -t long /dev/da0 smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-STABLE amd64] (local build) Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org === START OF OFFLINE IMMEDIATE AND SELF-TEST SECTION === Sending command: "Execute SMART Extended self-test routine immediately in off-line mode". Drive command "Execute SMART Extended self-test routine immediately in off-line mode" successful. Testing has begun. Please wait 631 minutes for test to complete. Test will complete after Fri Oct 18 15:37:04 2019 Use smartctl -X to abort test. [root@freenas ~]#
0x06 0x010 4 144 --- Number of ASR Events 0x06 0x018 4 667 --- Number of Interface CRC Errors |||_ C monitored condition met ||__ D supports DSN |___ N normalized value Pending Defects log (GP Log 0x0c) not supported SATA Phy Event Counters (GP Log 0x11) ID Size Value Description 0x000a 2 2 Device-to-host register FISes sent due to a COMRESET 0x0001 2 0 Command failed due to ICRC error 0x0003 2 0 R_ERR response for device-to-host data FIS 0x0004 2 0 R_ERR response for host-to-device data FIS 0x0006 2 0 R_ERR response for device-to-host non-data FIS 0x0007 2 0 R_ERR response for host-to-device non-data FIS [root@freenas ~]# smartctl -t short /dev/da0 smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-STABLE amd64] (local build) Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org === START OF OFFLINE IMMEDIATE AND SELF-TEST SECTION === Can't start self-test without aborting current test (90% remaining), add '-t force' option to override, or run 'smartctl -X' to abort test. [root@freenas ~]#
199 UDMA_CRC_Error_Count 0x003e 200 192 000 Old_age Always - 178 240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 16849 (208 62 0) 241 Total_LBAs_Written 0x0000 100 253 000 Old_age Offline - 65529703758 242 Total_LBAs_Read 0x0000 100 253 000 Old_age Offline - 46229415874 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Extended offline Self-test routine in progress 90% 16858 - # 2 Short offline Completed without error 00% 16857 - SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. [root@freenas ~]#
0x05 0x048 1 30 --- Lowest Average Long Term Temperature 0x05 0x050 4 0 --- Time in Over-Temperature 0x05 0x058 1 70 --- Specified Maximum Operating Temperature 0x05 0x060 4 0 --- Time in Under-Temperature 0x05 0x068 1 0 --- Specified Minimum Operating Temperature 0x06 ===== = = === == Transport Statistics (rev 1) == 0x06 0x008 4 1626 --- Number of Hardware Resets 0x06 0x010 4 144 --- Number of ASR Events 0x06 0x018 4 667 --- Number of Interface CRC Errors |||_ C monitored condition met ||__ D supports DSN |___ N normalized value Pending Defects log (GP Log 0x0c) not supported SATA Phy Event Counters (GP Log 0x11) ID Size Value Description 0x000a 2 2 Device-to-host register FISes sent due to a COMRESET 0x0001 2 0 Command failed due to ICRC error 0x0003 2 0 R_ERR response for device-to-host data FIS 0x0004 2 0 R_ERR response for host-to-device data FIS 0x0006 2 0 R_ERR response for device-to-host non-data FIS 0x0007 2 0 R_ERR response for host-to-device non-data FIS
The tests can be slow.631 minutes is like too long, then to abort i did smartctl -x /dev/da0. to check if the smart test did stop i did smartctl -t short /dev/da0 and results below
smartctl -a {device}
to get a sense of what state the tests are currently in. The output above is also truncated. Keep in mind that when you use the command line via the GUI long output doesn't fit on one screen and you have to scroll.199 UDMA_CRC_Error_Count 0x003e 200 192 000 Old_age Always - 178 240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 16849 (113 200 0) 241 Total_LBAs_Written 0x0000 100 253 000 Old_age Offline - 65529703758 242 Total_LBAs_Read 0x0000 100 253 000 Old_age Offline - 46229415874 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Extended offline Self-test routine in progress 80% 16859 - # 2 Short offline Completed without error 00% 16857 - SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. [root@freenas ~]#
OK got it and i apologize for this, bottom of my heart. This is the first time i am doing shell CLI so i tend to be confusing.& it's a steep learning curve. but i will do best so that information is correct.The tests can be slow.
To be honest; it is hard to keep up with what you're doing. We go back and forth but between messages sometimes I can't tell if you've done only what has been suggested or a bunch of other commands. When you run a bunch of other commands it makes it hard to know what state you're in and so I feel the need to try to sync up again. It is tempting to panic and try a bunch of stuff but that runs the risk of actually taking longer and possibly messing things up. If you're pool is already in a bad state it is very important that you not accidentally put it in a worse state. This is why I'm being slow with what I'm recommending; because I don't know exactly what state you're in and I don't want to suggest something that will make it impossible for you to get your data back.
i am not sure how to do long test in parallel therefore i am doing it one by one. On the output being truncated i will learn to scroll to copy. Perhaps after this drive of da0 has completed i will do short testsI recommend you run long tests on every drive in that pool; they can run in parallel. If you want, you can copy-paste the entire outputsmartctl -a {device}
to get a sense of what state the tests are currently in. The output above is also truncated. Keep in mind that when you use the command line via the GUI long output doesn't fit on one screen and you have to scroll.
You are the best person so far and please continue to be happy ya. i really need your help. Appreciate your patienceIt may be that someone with more experience than me will recognize a situation like this more clearly and have better solutions.
Anyway, happy to keep helping, but please try to be as explicit about exactly what you do and provide as much output as you can without truncating results.
Issue the command for each drive individually. The tests run int he background. If you do them 1 by 1 it will take many days. if you do them in parallel they won't take nearly as long.i am not sure how to do long test in parallel therefore i am doing it one by one. On the output being truncated i will learn to scroll to copy. Perhaps after this drive of da0 has completed i will do short tests
OK let me first learn how to copy & paste the whole output, i see what you mean by truncating, totally agree data are left out.Issue the command for each drive individually. The tests run int he background. If you do them 1 by 1 it will take many days. if you do them in parallel they won't take nearly as long.
Spin_Up_Time 0x0003 093 093 000 Pre-fail Always - 0 4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 540 5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail Always - 0 7 Seek_Error_Rate 0x000f 090 060 045 Pre-fail Always - 904816146 9 Power_On_Hours 0x0032 081 081 000 Old_age Always - 16860 (6 228 0) 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0 12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 545 184 End-to-End_Error 0x0032 100 100 099 Old_age Always - 0 187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0 188 Command_Timeout 0x0032 100 001 000 Old_age Always - 667 189 High_Fly_Writes 0x003a 100 100 000 Old_age Always - 0 190 Airflow_Temperature_Cel 0x0022 056 056 040 Old_age Always - 44 (Min/Max 33/44) 191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 0 192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 205 193 Load_Cycle_Count 0x0032 100 100 000 Old_age Always - 891 194 Temperature_Celsius 0x0022 044 044 000 Old_age Always - 44 (0 21 0 0 0) 197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x003e 200 192 000 Old_age Always - 178 240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 16850 (179 234 0) 241 Total_LBAs_Written 0x0000 100 253 000 Old_age Offline - 65529703758 242 Total_LBAs_Read 0x0000 100 253 000 Old_age Offline - 46229415874 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Extended offline Self-test routine in progress 70% 16860 - # 2 Short offline Completed without error 00% 16857 - SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. [root@freenas ~]# SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. 199 UDMA_CRC_Error_Count 0x003e 200 192 000 Old_age Always - 178 240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 16850 (143 113 0) 241 Total_LBAs_Written 0x0000 100 253 000 Old_age Offline - 65529703758 242 Total_LBAs_Read 0x0000 100 253 000 Old_age Offline - 46229415874 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Extended offline Self-test routine in progress 80% 16859 - # 2 Short offline Completed without error 00% 16857 - SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay.
root@freenas:~ # smartctl -a /dev/da0 smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-STABLE amd64] (local build) Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org === START OF INFORMATION SECTION === Model Family: Seagate IronWolf Device Model: ST4000VN008-2DR166 Serial Number: ZDH1E7MA LU WWN Device Id: 5 000c50 0a2ca53df Firmware Version: SC60 User Capacity: 4,000,787,030,016 bytes [4.00 TB] Sector Sizes: 512 bytes logical, 4096 bytes physical Rotation Rate: 5980 rpm Form Factor: 3.5 inches Device is: In smartctl database [for details use: -P show] ATA Version is: ACS-3 T13/2161-D revision 5 SATA Version is: SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s) Local Time is: Fri Oct 18 07:49:08 2019 +08 SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED General SMART Values: Offline data collection status: (0x82) Offline data collection activity was completed without error. Auto Offline Data Collection: Enabled. Self-test execution status: ( 246) Self-test routine in progress... 60% of test remaining. Total time to complete Offline data collection: ( 581) seconds. Offline data collection capabilities: (0x7b) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 1) minutes. Extended self-test routine recommended polling time: ( 631) minutes. Conveyance self-test routine recommended polling time: ( 2) minutes. SCT capabilities: (0x50bd) SCT Status supported. SCT Error Recovery Control supported. SCT Feature Control supported. SCT Data Table supported. SMART Attributes Data Structure revision number: 10 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x000f 082 064 044 Pre-fail Always - 176119726 3 Spin_Up_Time 0x0003 093 093 000 Pre-fail Always - 0 4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 540 5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail Always - 0 7 Seek_Error_Rate 0x000f 090 060 045 Pre-fail Always - 905038420 9 Power_On_Hours 0x0032 081 081 000 Old_age Always - 16860 (186 67 0) 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0 12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 545 184 End-to-End_Error 0x0032 100 100 099 Old_age Always - 0 187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0 188 Command_Timeout 0x0032 100 001 000 Old_age Always - 667 189 High_Fly_Writes 0x003a 100 100 000 Old_age Always - 0 190 Airflow_Temperature_Cel 0x0022 056 056 040 Old_age Always - 44 (Min/Max 33/44) 191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 0 192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 205 193 Load_Cycle_Count 0x0032 100 100 000 Old_age Always - 891 194 Temperature_Celsius 0x0022 044 044 000 Old_age Always - 44 (0 21 0 0 0) 197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x003e 200 192 000 Old_age Always - 178 240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 16851 (120 200 0) 241 Total_LBAs_Written 0x0000 100 253 000 Old_age Offline - 65529703758 242 Total_LBAs_Read 0x0000 100 253 000 Old_age Offline - 46229415874 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Extended offline Self-test routine in progress 60% 16860 - # 2 Short offline Completed without error 00% 16857 - SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. root@freenas:~ # root@freenas:~ #