SOLVED Need help please-FN 11.2.U5 Unusable Pool-ports using Sata Express (backward to Sata only ports)

1kokies

Contributor
Joined
Oct 7, 2017
Messages
138
Sharing my experience with Asus X99-e WS/USB3.1, Xeon E260x V3, 112GB ECC, IT Raid card, 12 hdd.

Excellent stable board. The board has 8xSata & 2xSata Express=technically 10 Sata ports can be recognized. However the Sata Express ports will make the pool as unusable( the whole pool !, disk is detected) especially if i musical chair the ports for case changes, rewire etc. i have resolve to not using the Sata Express ports at all, if needed add an IT raid card. Hope this helps


ps once it is unusable just plug back all the ports back to the Sata only ports & reboot.
 
Joined
Oct 18, 2018
Messages
969
Hey @1kokies thanks for sharing your experience. For the benefit of other members and to help search engines find this thread when folks use the same hardware would you mind elaborating with exact model numbers and firmware numbers where appropriate?
 

Jessep

Patron
Joined
Aug 19, 2018
Messages
379
This is likely due to the eSATA ports not being Intel chipset

8 x SATA 6Gb/s port(s), gray, *4
Intel® Rapid Storage Technology supports*5
ASMedia® SATA Express controller : *6
1 x SATA Express port, gray, , Compatible with 2 x SATA 6.0 Gb/s ports
ASMedia® SATA 6Gb/s controller :
2 x eSATA 6Gb/s port(s), Compatible with 2 x SATA 6.0 Gb/s ports
 

1kokies

Contributor
Joined
Oct 7, 2017
Messages
138
OK guys i think this needs correction. i am not not using those sata ports, and perhaps it is not due to those ports. But i need help, i can't see my main pool.

i have 2 pools
1. Data Storage which has 8x4TB in raidz2 + an slog ssd
2. Data Media which has 4x4TB in raidz1.

3 hours ago the slog failed but pool status is offline ( it is not really offline, i forgot what is says but basically not available ), pool is degraded.
1. i powered off, took out bad disk and inserted good disk
2. Powered on & pool immediately detected good ssd and included it in the pool without the need to offline or replace process.
3. i was curious as to how does Freenas knows that new ssd is for slog of Data Storage
4. Therefore i wanted to do replace process to Data Storage pool. i powered off, inserted the bad ssd without taking out the good ssd.
5. In GUI i i can see both ssd and i did the replace process of bad ssd with good ssd. Freenas says the new ssd size is not big enough (both of identical brand and size), i also did force replace but same result.
6. There seems no other option so fine, i powered off and took out the bad ssd.
7. When i powered up the Data Storage pool does how show up !!!

Please help me urgently, greatly appreciate, all data are in this pool. In pool import option, Data Storage pool does not exist.

i swapped all ports, changed raid cards, re install 11.2 U6 to upload old config, all came back with the same results. The disks are detected with serial numbers but the pool is not.
 

Attachments

  • FNpool1.PNG
    FNpool1.PNG
    39.7 KB · Views: 278
Last edited:
Joined
Oct 18, 2018
Messages
969
Hi @1kokies, would you mind please checking my suggestion above and include exact model numbers of all potentially relevant hardware if you haven't already done so? We can take guesses as to what is going on but knowing exactly what you're using would speed up the process. For example, how are your drives connected to your system? What model of RAID card are you using (is it that SATA card you referred to)? Is it flashed to IT mode? What Firmware version?

Also, the output of zpool status would be very helpful. Also, showing us what disks are recognized by the os would be useful; you could check the Storage->Disks screen perhaps.

Final question, do you use encryption?

I know I asked a lot of questions here. If you could go through and answer each of them someone can likely help you get access to your data back quicker. :)
 

1kokies

Contributor
Joined
Oct 7, 2017
Messages
138
Hi Philo,

Please ask more questions, i need to recover this pool. i will post shortly
 

1kokies

Contributor
Joined
Oct 7, 2017
Messages
138
proc: Xeon E2609 V3
mobo: Asus X99-e WS/USB3.1. Using only the 8 sata ports, not the AS Media ones
raid card: 1x 9220i (LSI i think) flashed to IT Mode, all 8 ports are used, sata. not sure where to see the firmware but it may be 500605B 0-03CF-6840
hdd: 12 x ironwolf 4TB. 4xraidz2 + 4xraidz2 + 1slog128GB Sandisk for Data Storage. For Data Media 4xraidz1. All connected with sata via raid card and on board sata.
Network: intel 2x nic card, not using the onboard ones
graphic: nvidia card for console
rams: 112GB ECC
power supply: 850W
hard disks drive bays: thermaltake 3503
encryption: none

again please help. screenshot as requested.
 

Attachments

  • FNpool2.PNG
    FNpool2.PNG
    24.1 KB · Views: 288
  • FNpool3.PNG
    FNpool3.PNG
    60.6 KB · Views: 272
  • FNpool4.PNG
    FNpool4.PNG
    57.2 KB · Views: 284

1kokies

Contributor
Joined
Oct 7, 2017
Messages
138
screenshot of pool unknown, please help guys.
 

Attachments

  • FNpool5.PNG
    FNpool5.PNG
    45.4 KB · Views: 266
Joined
Oct 18, 2018
Messages
969
please help guys.
I am helping. :)

Your screenshot seems to truncate a lot of information. Is there any way you can copy-paste the entire output of zpool status? Any truncated results you post here may delay anyone's ability to help you due to going back and forth trying to get all of the information.

From the looks of it your slog devices are probably those 111GB drives currently at ada3 and ada4. Could you give the output of smartctl -i /dev/ada3 and smartctl -i /dev/ada3? Please don't truncate the results. Feel free to copy-paste, and if the screen scrolls when you hit the command please scroll back to get the full contents. You can paste them in code tags like this.

[CODE]
output here
[/CODE]

Also, looks like your data drives are da0-da7.

raid card: 1x 9220i (LSI i think) flashed to IT Mode, all 8 ports are used, sata. not sure where to see the firmware but it may be 500605B 0-03CF-6840
What is the output of sas2flash -listall?

In general, what I'm looking to discover is what devices your system expects to be in the pool and which are malfunctioning. If it is the SLOG device it may be as simple as removing the bad SLOG device and adding the new one again; though this will depend somewhat on the situation.
 

1kokies

Contributor
Joined
Oct 7, 2017
Messages
138
thanks Philo, really appreciate it. i will provide shortly however i am not sure how to copy & paste from shell but i will use screenshots in 2-3 parts if that is ok.
 

1kokies

Contributor
Joined
Oct 7, 2017
Messages
138
Hi Philo,

yes the slog's are ada3 and ada4. ada3 is the bad one based on the serial #.
zpoolstatus attached 1&2 screenshot
smartctl -i for ada3 & ada4 attached screenshot
sasflash -listall attached screenshot
yes, my Data Storage pool is da0-da7

Is this suspected of hardware error ? i have
1. removed ada3, left ada4
2. the reverse of the above
3. removed both

i have tried to use 2 x raid of the same model to isolate mobo sata ports issue as well, all with same results.
 

Attachments

  • zpoolstatus1.PNG
    zpoolstatus1.PNG
    25.8 KB · Views: 279
  • zpoolstatus2.PNG
    zpoolstatus2.PNG
    15.7 KB · Views: 287
  • smartada3.PNG
    smartada3.PNG
    44.6 KB · Views: 267
  • smartada4.PNG
    smartada4.PNG
    39.9 KB · Views: 288
  • sas.PNG
    sas.PNG
    20.7 KB · Views: 288
Joined
Oct 18, 2018
Messages
969
Is this suspected of hardware error ? i have
1. removed ada3, left ada4
2. the reverse of the above
3. removed both
It is hard to tell. From here on out don't remove drives, swap them, etc. Doing so may complicate the situation.

Okay, so my guess is that one issue MAY be that the two SSDs appear to be different model numbers. This could prevent the new one from replacing the old one. No worries though; what we will do is remove the old one from the pool and add the new one.



First, if you pull the new SSD and reboot the machine, what happens?
6. There seems no other option so fine, i powered off and took out the bad ssd.
7. When i powered up the Data Storage pool does how show up !!!
It sounds like from the above that if you do that, the data pool will show up but in a degrated state? If so, what is the output of zpool status Data-Storage?
 

1kokies

Contributor
Joined
Oct 7, 2017
Messages
138
It is hard to tell. From here on out don't remove drives, swap them, etc. Doing so may complicate the situation.

Okay, so my guess is that one issue MAY be that the two SSDs appear to be different model numbers. This could prevent the new one from replacing the old one. No worries though; what we will do is remove the old one from the pool and add the new one.
Both the SSD's are bought for slog and they are identical models. old + 0, new + 0, old + new, 0+0=pool unknown

First, if you pull the new SSD and reboot the machine, what happens?
It sounds like from the above that if you do that, the data pool will show up but in a degrated state? If so, what is the output of zpool status Data-Storage?
which ever way old or new ssd or both physically in Freenas, the pool remains unknown, zpool status says 'no such pool'. i can't access the pool at all.
 

Attachments

  • zpoolstatusStorage1.PNG
    zpoolstatusStorage1.PNG
    4.8 KB · Views: 297
Last edited:

1kokies

Contributor
Joined
Oct 7, 2017
Messages
138
the slog ssd & dashboard
 

Attachments

  • slogSandisk.PNG
    slogSandisk.PNG
    436.1 KB · Views: 294
  • FNpool6.PNG
    FNpool6.PNG
    44.3 KB · Views: 290
Last edited:

1kokies

Contributor
Joined
Oct 7, 2017
Messages
138
really appreciate your help Philo, i am really stuck here.
 
Joined
Oct 18, 2018
Messages
969
which ever way old or new ssd or both physically in Freenas, the pool remains unknown, zpool status says 'no such pool'. i can't access the pool at all.
Ah, I misunderstood. Is there any way in which you can get the pool to be recognized, even if degraded?
 

1kokies

Contributor
Joined
Oct 7, 2017
Messages
138
that is where i am still stuck, couldn't get the pool to be recognized. is it a hardware or software issue ?

so far i have

1. use different but some model of raid cards to isolate board sata issue
2. re install Freenas and upload config (even this returns the same results)
3. alternate bad and good disk
4. alternate all the sata ports

any other suggestions is welcomed

Never had this problem in the 3 years that i have been using Freenas 11.1U7
 

1kokies

Contributor
Joined
Oct 7, 2017
Messages
138
will re installing and uploading the back up config help ?
 

1kokies

Contributor
Joined
Oct 7, 2017
Messages
138
i have taken out the thermaltake 3503 hard disks bay https://www.thermaltake.com/C_00002934.htm and plug sata all direct to raid card and motherboard ports. with only bad slog, new slog not plugged in.

same result, pool unknown
 

1kokies

Contributor
Joined
Oct 7, 2017
Messages
138
my biggest concern, is there any chance the Freenas config gets embedded to the 8 hard disks ? making it unrecognized which ever hardware changes i make.
 
Top