TrueNAS Core cannot see all disks, but Ubuntu Live can (and possibly Scale)

J

Jorsher

Explorer
Joined
Jul 8, 2018
Messages
88
I posted a separate thread for assistance with my drives not showing up after an upgrade. I replaced the old 92xx-8e + HP Expanders with a 9400-16e + Adaptec expanders so I could add more drives. mprutils would show all expanders and disks, but TrueNAS doesn't seem to although it assigns mpr1 to the controller and ses0/1/2/3/4 to the expanders. TrueNAS would only see ~12 of the drives, on different expanders.

Someone suggested that I try to live boot Ubuntu because TrueNAS Core may have old drivers that don't play nice. I tried it, and Ubuntu had no problem seeing all 40 disks...

I'd prefer not to switch to Scale for a number of reasons, but I'm unaware of any way to update the drivers due to TrueNAS non-persistence. Are there any workarounds, or are my only options to migrate to Scale or wait until Core is updated?

Thanks
 
M

morganL

Captain Morgan
Administrator
Moderator
iXsystems
Joined
Mar 10, 2018
Messages
2,694
Jorsher said:
I posted a separate thread for assistance with my drives not showing up after an upgrade. I replaced the old 92xx-8e + HP Expanders with a 9400-16e + Adaptec expanders so I could add more drives. mprutils would show all expanders and disks, but TrueNAS doesn't seem to although it assigns mpr1 to the controller and ses0/1/2/3/4 to the expanders. TrueNAS would only see ~12 of the drives, on different expanders.

Someone suggested that I try to live boot Ubuntu because TrueNAS Core may have old drivers that don't play nice. I tried it, and Ubuntu had no problem seeing all 40 disks...

I'd prefer not to switch to Scale for a number of reasons, but I'm unaware of any way to update the drivers due to TrueNAS non-persistence. Are there any workarounds, or are my only options to migrate to Scale or wait until Core is updated?

Thanks
Click to expand...
Lets start with full hardware description and specific CORE version.
Were the drives formatted in TrueNAS CORE?
 
J

Jorsher

Explorer
Joined
Jul 8, 2018
Messages
88
Version:
TrueNAS-13.0-U3.1

Drives were the ones I was using before changing the HBA. Didn't do anything with them except export the pool before shutting down.

Replaced 92xx-8i with 9400-16e, replaced old HP SAS expanders with Adaptec 82885T Expanders. Powered it back on. During post/boot, the 9400-16e detects all 40 drives. Once TrueNAS loaded, I went to import the pool, it couldn't find it. I went to Storage -> Disks, and it doesn't show them all. Shows about 12 out of 40, from different expanders.

mprutil -u 1 show all - shows all controllers, expanders, and disks! Great!

storcli /c0 show - shows the controller
storcli /c0/eall show - doesn't find any controllers
storcli /c0/pall show - shows all active links to the expanders

sesutil map -u /dev/ses0 - shows the expander, but all elements (disks?) say 'not installed'

camcontrol devlist - shows all drives from the first expander, only shows one drive for each of the other 4 expanders

... Some other things were tried that I am probably forgetting.

Made an Ubuntu live boot disk, loaded it, Ubuntu picked up all 40 drives. Might try booting TrueNAS Scale to see if it also detects them all as well, but I'd prefer to stick with Core.

Not sure if it's a driver or FreeBSD / TrueNAS Core issue.

mprutil output:
Code:
root@nas[~]# mprutil -u 1 show all
Adapter:
mpr1 Adapter:
       Board Name: HBA 9400-16e
   Board Assembly: 05-50013-00
        Chip Name: SAS3416
    Chip Revision: ALL
    BIOS Revision: 9.47.00.00
Firmware Revision: 24.00.00.00
  Integrated RAID: no
         SATA NCQ: ENABLED
 PCIe Width/Speed: x8 (8.0 GB/sec)
        IOC Speed: Full
      Temperature: 56 C

PhyNum  CtlrHandle  DevHandle  Disabled  Speed   Min    Max    Device
0       0001        0021       N         12      3.0    12     SAS Initiator 
1       0001        0021       N         12      3.0    12     SAS Initiator 
2       0003        0036       N         12      3.0    12     SAS Initiator 
3       0003        0036       N         12      3.0    12     SAS Initiator 
4       0002        0035       N         12      3.0    12     SAS Initiator 
5       0002        0035       N         12      3.0    12     SAS Initiator 
6                              N                 3.0    12     SAS Initiator 
7                              N                 3.0    12     SAS Initiator 
8       0009        0017       N         12      3.0    12     SAS Initiator 
9       0009        0017       N         12      3.0    12     SAS Initiator 
10                             N                 3.0    12     SAS Initiator 
11                             N                 3.0    12     SAS Initiator 
12      000a        0022       N         12      3.0    12     SAS Initiator 
13      000a        0022       N         12      3.0    12     SAS Initiator 
14                             N                 3.0    12     SAS Initiator 
15                             N                 3.0    12     SAS Initiator 
16                             N                 ???    ???    No Device     
17                             N                 ???    ???    No Device     
18                             N                 ???    ???    No Device     
19                             N                 ???    ???    No Device     
20                             N                 ???    ???    No Device     

Devices:
B____T    SAS Address      Handle  Parent    Device        Speed Enc  Slot  Wdt
          50000d1702abed3f 0017    0009      SMP Target    12    0002 00    2
          50000d1702abed00 0018    0017      SATA Target   6.0              1
          50000d1702abed01 0019    0017      SATA Target   6.0              1
          50000d1702abed02 001a    0017      SATA Target   6.0              1
          50000d1702abed03 001b    0017      SATA Target   6.0              1
          50000d1702abed04 001c    0017      SATA Target   6.0              1
          50000d1702abed05 001d    0017      SATA Target   6.0              1
          50000d1702abed06 001e    0017      SATA Target   6.0              1
00   91   50000d1702abed07 001f    0017      SATA Target   6.0              1
00   115  50000d1702abed3e 0020    0017      SEP Target    12    0002 24    1
          50000d1700e649bf 0021    0001      SMP Target    12    0003 00    2
          50000d170110d43f 0022    000a      SMP Target    12    0004 00    2
          50000d1700e64980 0023    0021      SATA Target   6.0              1
          50000d1700e64981 0024    0021      SATA Target   6.0              1
          50000d1700e64982 0025    0021      SATA Target   6.0              1
          50000d1700e64983 0026    0021      SATA Target   6.0              1
          50000d1700e64984 0027    0021      SATA Target   6.0              1
          50000d1700e64985 0028    0021      SATA Target   6.0              1
          50000d1700e64986 0029    0021      SATA Target   6.0              1
00   16   50000d1700e64987 002a    0021      SATA Target   6.0              1
          50000d170110d400 002b    0022      SATA Target   6.0              1
          50000d170110d401 002c    0022      SATA Target   6.0              1
          50000d170110d402 002d    0022      SATA Target   6.0              1
          50000d170110d403 002e    0022      SATA Target   6.0              1
          50000d170110d404 002f    0022      SATA Target   6.0              1
          50000d170110d405 0030    0022      SATA Target   6.0              1
          50000d170110d406 0031    0022      SATA Target   6.0              1
00   116  50000d170110d407 0032    0022      SATA Target   6.0              1
00   40   50000d1700e649be 0033    0021      SEP Target    12    0003 24    1
00   140  50000d170110d43e 0034    0022      SEP Target    12    0004 24    1
          50000d17015a29bf 0035    0002      SMP Target    12    0005 00    2
          50000d170052f53f 0036    0003      SMP Target    12    0006 00    2
          50000d17015a2980 0037    0035      SATA Target   6.0              1
          50000d17015a2981 0038    0035      SATA Target   6.0              1
          50000d17015a2982 0039    0035      SATA Target   6.0              1
          50000d17015a2983 003a    0035      SATA Target   6.0              1
          50000d17015a2984 003b    0035      SATA Target   6.0              1
          50000d17015a2985 003c    0035      SATA Target   6.0              1
          50000d17015a2986 003d    0035      SATA Target   6.0              1
00   41   50000d17015a2987 003e    0035      SATA Target   6.0              1
          50000d170052f500 003f    0036      SATA Target   6.0              1
          50000d170052f501 0040    0036      SATA Target   6.0              1
          50000d170052f502 0041    0036      SATA Target   6.0              1
          50000d170052f503 0042    0036      SATA Target   6.0              1
          50000d170052f504 0043    0036      SATA Target   6.0              1
          50000d170052f505 0044    0036      SATA Target   6.0              1
          50000d170052f506 0045    0036      SATA Target   6.0              1
00   66   50000d170052f507 0046    0036      SATA Target   6.0              1
00   65   50000d17015a29be 0047    0035      SEP Target    12    0005 24    1
00   90   50000d170052f53e 0048    0036      SEP Target    12    0006 24    1

Enclosures:
Slots      Logical ID     SEPHandle  EncHandle    Type
  17    300605b00e119fe7    0000       0001     Direct Attached SES-2
  25    50000d1702abed3e    0020       0002     External SES-2
  25    50000d1700e649be    0033       0003     External SES-2
  25    50000d170110d43e    0034       0004     External SES-2
  25    50000d17015a29be    0047       0005     External SES-2
  25    50000d170052f53e    0048       0006     External SES-2

Expanders:
NumPhys   SAS Address     DevHandle   Parent  EncHandle  SAS Level
  37    50000d1702abed3f    0017       0009     0002       1

     Phy  RemotePhy  DevHandle  Speed  Min   Max    Device
     00      00        0018      6.0   1.5   12    SATA Target   
     01      00        0019      6.0   1.5   12    SATA Target   
     02      00        001a      6.0   1.5   12    SATA Target   
     03      00        001b      6.0   1.5   12    SATA Target   
     04      00        001c      6.0   1.5   12    SATA Target   
     05      00        001d      6.0   1.5   12    SATA Target   
     06      00        001e      6.0   1.5   12    SATA Target   
     07      00        001f      6.0   1.5   12    SATA Target   
     08                                1.5   12    No Device     
     09                                1.5   12    No Device     
     10                                1.5   12    No Device     
     11                                1.5   12    No Device     
     12                                1.5   12    No Device     
     13                                1.5   12    No Device     
     14                                1.5   12    No Device     
     15                                1.5   12    No Device     
     16                                1.5   12    No Device     
     17                                1.5   12    No Device     
     18                                1.5   12    No Device     
     19                                1.5   12    No Device     
     20                                1.5   12    No Device     
     21                                1.5   12    No Device     
     22                                1.5   12    No Device     
     23                                1.5   12    No Device     
     24                                1.5   12    No Device     
     25                                1.5   12    No Device     
     26                                1.5   12    No Device     
     27                                1.5   12    No Device     
     28                                1.5   12    No Device     
     29                                1.5   12    No Device     
     30                                1.5   12    No Device     
     31                                1.5   12    No Device     
     32      09        0009      12    1.5   12    SAS Initiator 
     33      08        0009      12    1.5   12    SAS Initiator 
     34                                1.5   12    No Device     
     35                                1.5   12    No Device     
     36      36        0020      12    12    12    SEP Target    
  37    50000d1700e649bf    0021       0001     0003       1

     Phy  RemotePhy  DevHandle  Speed  Min   Max    Device
     00      00        0023      6.0   1.5   12    SATA Target   
     01      00        0024      6.0   1.5   12    SATA Target   
     02      00        0025      6.0   1.5   12    SATA Target   
     03      00        0026      6.0   1.5   12    SATA Target   
     04      00        0027      6.0   1.5   12    SATA Target   
     05      00        0028      6.0   1.5   12    SATA Target   
     06      00        0029      6.0   1.5   12    SATA Target   
     07      00        002a      6.0   1.5   12    SATA Target   
     08                                1.5   12    No Device     
     09                                1.5   12    No Device     
     10                                1.5   12    No Device     
     11                                1.5   12    No Device     
     12                                1.5   12    No Device     
     13                                1.5   12    No Device     
     14                                1.5   12    No Device     
     15                                1.5   12    No Device     
     16                                1.5   12    No Device     
     17                                1.5   12    No Device     
     18                                1.5   12    No Device     
     19                                1.5   12    No Device     
     20                                1.5   12    No Device     
     21                                1.5   12    No Device     
     22                                1.5   12    No Device     
     23                                1.5   12    No Device     
     24                                1.5   12    No Device     
     25                                1.5   12    No Device     
     26                                1.5   12    No Device     
     27                                1.5   12    No Device     
     28                                1.5   12    No Device     
     29                                1.5   12    No Device     
     30                                1.5   12    No Device     
     31                                1.5   12    No Device     
     32      01        0001      12    1.5   12    SAS Initiator 
     33      00        0001      12    1.5   12    SAS Initiator 
     34                                1.5   12    No Device     
     35                                1.5   12    No Device     
     36      36        0033      12    12    12    SEP Target    
  37    50000d170110d43f    0022       000a     0004       1

     Phy  RemotePhy  DevHandle  Speed  Min   Max    Device
     00      00        002b      6.0   1.5   12    SATA Target   
     01      00        002c      6.0   1.5   12    SATA Target   
     02      00        002d      6.0   1.5   12    SATA Target   
     03      00        002e      6.0   1.5   12    SATA Target   
     04      00        002f      6.0   1.5   12    SATA Target   
     05      00        0030      6.0   1.5   12    SATA Target   
     06      00        0031      6.0   1.5   12    SATA Target   
     07      00        0032      6.0   1.5   12    SATA Target   
     08                                1.5   12    No Device     
     09                                1.5   12    No Device     
     10                                1.5   12    No Device     
     11                                1.5   12    No Device     
     12                                1.5   12    No Device     
     13                                1.5   12    No Device     
     14                                1.5   12    No Device     
     15                                1.5   12    No Device     
     16                                1.5   12    No Device     
     17                                1.5   12    No Device     
     18                                1.5   12    No Device     
     19                                1.5   12    No Device     
     20                                1.5   12    No Device     
     21                                1.5   12    No Device     
     22                                1.5   12    No Device     
     23                                1.5   12    No Device     
     24                                1.5   12    No Device     
     25                                1.5   12    No Device     
     26                                1.5   12    No Device     
     27                                1.5   12    No Device     
     28                                1.5   12    No Device     
     29                                1.5   12    No Device     
     30                                1.5   12    No Device     
     31                                1.5   12    No Device     
     32      13        000a      12    1.5   12    SAS Initiator 
     33      12        000a      12    1.5   12    SAS Initiator 
     34                                1.5   12    No Device     
     35                                1.5   12    No Device     
     36      36        0034      12    12    12    SEP Target    
  37    50000d17015a29bf    0035       0002     0005       1

     Phy  RemotePhy  DevHandle  Speed  Min   Max    Device
     00      00        0037      6.0   1.5   12    SATA Target   
     01      00        0038      6.0   1.5   12    SATA Target   
     02      00        0039      6.0   1.5   12    SATA Target   
     03      00        003a      6.0   1.5   12    SATA Target   
     04      00        003b      6.0   1.5   12    SATA Target   
     05      00        003c      6.0   1.5   12    SATA Target   
     06      00        003d      6.0   1.5   12    SATA Target   
     07      00        003e      6.0   1.5   12    SATA Target   
     08                                1.5   12    No Device     
     09                                1.5   12    No Device     
     10                                1.5   12    No Device     
     11                                1.5   12    No Device     
     12                                1.5   12    No Device     
     13                                1.5   12    No Device     
     14                                1.5   12    No Device     
     15                                1.5   12    No Device     
     16                                1.5   12    No Device     
     17                                1.5   12    No Device     
     18                                1.5   12    No Device     
     19                                1.5   12    No Device     
     20                                1.5   12    No Device     
     21                                1.5   12    No Device     
     22                                1.5   12    No Device     
     23                                1.5   12    No Device     
     24                                1.5   12    No Device     
     25                                1.5   12    No Device     
     26                                1.5   12    No Device     
     27                                1.5   12    No Device     
     28                                1.5   12    No Device     
     29                                1.5   12    No Device     
     30                                1.5   12    No Device     
     31                                1.5   12    No Device     
     32      05        0002      12    1.5   12    SAS Initiator 
     33      04        0002      12    1.5   12    SAS Initiator 
     34                                1.5   12    No Device     
     35                                1.5   12    No Device     
     36      36        0047      12    12    12    SEP Target    
  37    50000d170052f53f    0036       0003     0006       1

     Phy  RemotePhy  DevHandle  Speed  Min   Max    Device
     00      00        003f      6.0   1.5   12    SATA Target   
     01      00        0040      6.0   1.5   12    SATA Target   
     02      00        0041      6.0   1.5   12    SATA Target   
     03      00        0042      6.0   1.5   12    SATA Target   
     04      00        0043      6.0   1.5   12    SATA Target   
     05      00        0044      6.0   1.5   12    SATA Target   
     06      00        0045      6.0   1.5   12    SATA Target   
     07      00        0046      6.0   1.5   12    SATA Target   
     08                                1.5   12    No Device     
     09                                1.5   12    No Device     
     10                                1.5   12    No Device     
     11                                1.5   12    No Device     
     12                                1.5   12    No Device     
     13                                1.5   12    No Device     
     14                                1.5   12    No Device     
     15                                1.5   12    No Device     
     16                                1.5   12    No Device     
     17                                1.5   12    No Device     
     18                                1.5   12    No Device     
     19                                1.5   12    No Device     
     20                                1.5   12    No Device     
     21                                1.5   12    No Device     
     22                                1.5   12    No Device     
     23                                1.5   12    No Device     
     24                                1.5   12    No Device     
     25                                1.5   12    No Device     
     26                                1.5   12    No Device     
     27                                1.5   12    No Device     
     28                                1.5   12    No Device     
     29                                1.5   12    No Device     
     30                                1.5   12    No Device     
     31                                1.5   12    No Device     
     32      02        0003      12    1.5   12    SAS Initiator 
     33      03        0003      12    1.5   12    SAS Initiator 
     34                                1.5   12    No Device     
     35                                1.5   12    No Device     
     36      36        0048      12    12    12    SEP Target


camcontrol devlist
Code:
root@nas[~]# camcontrol devlist
<ATA CT1000MX500SSD1 045>          at scbus0 target 0 lun 0 (pass0,da0)
<ATA CT1000MX500SSD1 045>          at scbus0 target 1 lun 0 (pass1,da1)
<ATA HGST HDN728080AL W91X>        at scbus1 target 0 lun 0 (pass2,da2)
<ATA HGST HDN728080AL W91X>        at scbus1 target 1 lun 0 (pass3,da3)
<ATA HGST HDN728080AL W91X>        at scbus1 target 2 lun 0 (pass4,da4)
<ATA HGST HDN728080AL W91X>        at scbus1 target 3 lun 0 (pass5,da5)
<ATA HGST HDN728080AL W91X>        at scbus1 target 4 lun 0 (pass6,da6)
<ATA HGST HDN728080AL W91X>        at scbus1 target 5 lun 0 (pass7,da7)
<ATA HGST HDN728080AL W91X>        at scbus1 target 6 lun 0 (pass8,da8)
<ATA HGST HDN728080AL W91X>        at scbus1 target 16 lun 0 (pass9,da9)
<ADAPTEC AEC-82885T B025>          at scbus1 target 40 lun 0 (ses0,pass10)
<ATA ST16000NM001G-2K SN03>        at scbus1 target 41 lun 0 (pass11,da10)
<ADAPTEC AEC-82885T B025>          at scbus1 target 65 lun 0 (ses1,pass12)
<ATA ST14000NM001G-2K SN02>        at scbus1 target 66 lun 0 (pass13,da11)
<ADAPTEC AEC-82885T B025>          at scbus1 target 90 lun 0 (ses2,pass14)
<ATA ST16000NM001G-2K SN03>        at scbus1 target 91 lun 0 (pass15,da12)
<ADAPTEC AEC-82885T B025>          at scbus1 target 115 lun 0 (ses3,pass16)
<ATA ST16000NM001G-2K SN03>        at scbus1 target 116 lun 0 (pass17,da13)
<ADAPTEC AEC-82885T B025>          at scbus1 target 140 lun 0 (ses4,pass18)


geom disk list
Code:
root@nas[~]# geom disk list
Geom name: nvd0
Providers:
1. Name: nvd0
   Mediasize: 256060514304 (238G)
   Sectorsize: 512
   Mode: r1w1e2
   descr: SAMSUNG MZVLB256HAHQ-00000
   lunid: 00253886810102ca
   ident: S444NE0K603420
   rotationrate: 0
   fwsectors: 0
   fwheads: 0

Geom name: da0
Providers:
1. Name: da0
   Mediasize: 1000204886016 (932G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r2w2e5
   descr: ATA CT1000MX500SSD1
   lunid: 500a0751e6999b7f
   ident: 2301E6999B7F
   rotationrate: 0
   fwsectors: 63
   fwheads: 255

Geom name: da1
Providers:
1. Name: da1
   Mediasize: 1000204886016 (932G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r2w2e5
   descr: ATA CT1000MX500SSD1
   lunid: 500a0751e6999042
   ident: 2301E6999042
   rotationrate: 0
   fwsectors: 63
   fwheads: 255

Geom name: da7
Providers:
1. Name: da7
   Mediasize: 8001563222016 (7.3T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   descr: ATA HGST HDN728080AL
   lunid: 5000cca263c1a46b
   ident: R6G3LZVY
   rotationrate: 7200
   fwsectors: 63
   fwheads: 255

Geom name: da4
Providers:
1. Name: da4
   Mediasize: 8001563222016 (7.3T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   descr: ATA HGST HDN728080AL
   lunid: 5000cca263c076d6
   ident: R6G10NBY
   rotationrate: 7200
   fwsectors: 63
   fwheads: 255

Geom name: da9
Providers:
1. Name: da9
   Mediasize: 8001563222016 (7.3T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   descr: ATA HGST HDN728080AL
   lunid: 5000cca263c1a40a
   ident: R6G3LWRY
   rotationrate: 7200
   fwsectors: 63
   fwheads: 255

Geom name: da2
Providers:
1. Name: da2
   Mediasize: 8001563222016 (7.3T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   descr: ATA HGST HDN728080AL
   lunid: 5000cca263c02dbd
   ident: R6G0D5RY
   rotationrate: 7200
   fwsectors: 63
   fwheads: 255

Geom name: da6
Providers:
1. Name: da6
   Mediasize: 8001563222016 (7.3T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   descr: ATA HGST HDN728080AL
   lunid: 5000cca263c0965b
   ident: R6G191NY
   rotationrate: 7200
   fwsectors: 63
   fwheads: 255

Geom name: da8
Providers:
1. Name: da8
   Mediasize: 8001563222016 (7.3T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   descr: ATA HGST HDN728080AL
   lunid: 5000cca263c19aa0
   ident: R6G3JBZY
   rotationrate: 7200
   fwsectors: 63
   fwheads: 255

Geom name: da3
Providers:
1. Name: da3
   Mediasize: 8001563222016 (7.3T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   descr: ATA HGST HDN728080AL
   lunid: 5000cca263c0621c
   ident: R6G0V46Y
   rotationrate: 7200
   fwsectors: 63
   fwheads: 255

Geom name: da5
Providers:
1. Name: da5
   Mediasize: 8001563222016 (7.3T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   descr: ATA HGST HDN728080AL
   lunid: 5000cca263c18fef
   ident: R6G3EJPY
   rotationrate: 7200
   fwsectors: 63
   fwheads: 255

Geom name: da12
Providers:
1. Name: da12
   Mediasize: 16000900661248 (15T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   descr: ATA ST16000NM001G-2K
   lunid: 5000c500c978d7ee
   ident: ZL2G7WAR
   rotationrate: 7200
   fwsectors: 63
   fwheads: 255

Geom name: da13
Providers:
1. Name: da13
   Mediasize: 16000900661248 (15T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   descr: ATA ST16000NM001G-2K
   lunid: 5000c500dbc8fbb2
   ident: ZL2K74H3
   rotationrate: 7200
   fwsectors: 63
   fwheads: 255

Geom name: da10
Providers:
1. Name: da10
   Mediasize: 16000900661248 (15T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   descr: ATA ST16000NM001G-2K
   lunid: 5000c500dbb81547
   ident: ZL2JVZ6H
   rotationrate: 7200
   fwsectors: 63
   fwheads: 255

Geom name: da11
Providers:
1. Name: da11
   Mediasize: 14000519643136 (13T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   descr: ATA ST14000NM001G-2K
   lunid: 5000c500b6d4a0b4
   ident: ZL20ALDM
   rotationrate: 7200
   fwsectors: 63
   fwheads: 255


Seems like all drives from the first listed expander are listed/detected by the OS, but each additional expander only presents one. Is there a drive mapping issue from multiple expanders?

EDIT: TrueNAS Scale has no issue detecting all drives. Wanted to stick with Core but I'm not sure what else to check and need to get to my data. If anyone has any suggestions, I will certainly try them, but something about detecting drives on FreeBSD doesn't seem to work with my configuration.
 
Last edited:
M

morganL

Captain Morgan
Administrator
Moderator
iXsystems
Joined
Mar 10, 2018
Messages
2,694
Jorsher said:
Version:
TrueNAS-13.0-U3.1

Drives were the ones I was using before changing the HBA. Didn't do anything with them except export the pool before shutting down.

Replaced 92xx-8i with 9400-16e, replaced old HP SAS expanders with Adaptec 82885T Expanders. Powered it back on. During post/boot, the 9400-16e detects all 40 drives. Once TrueNAS loaded, I went to import the pool, it couldn't find it. I went to Storage -> Disks, and it doesn't show them all. Shows about 12 out of 40, from different expanders.

mprutil -u 1 show all - shows all controllers, expanders, and disks! Great!

storcli /c0 show - shows the controller
storcli /c0/eall show - doesn't find any controllers
storcli /c0/pall show - shows all active links to the expanders

sesutil map -u /dev/ses0 - shows the expander, but all elements (disks?) say 'not installed'

camcontrol devlist - shows all drives from the first expander, only shows one drive for each of the other 4 expanders

... Some other things were tried that I am probably forgetting.

Made an Ubuntu live boot disk, loaded it, Ubuntu picked up all 40 drives. Might try booting TrueNAS Scale to see if it also detects them all as well, but I'd prefer to stick with Core.

Not sure if it's a driver or FreeBSD / TrueNAS Core issue.

mprutil output:
Code:
root@nas[~]# mprutil -u 1 show all
Adapter:
mpr1 Adapter:
       Board Name: HBA 9400-16e
   Board Assembly: 05-50013-00
        Chip Name: SAS3416
    Chip Revision: ALL
    BIOS Revision: 9.47.00.00
Firmware Revision: 24.00.00.00
  Integrated RAID: no
         SATA NCQ: ENABLED
 PCIe Width/Speed: x8 (8.0 GB/sec)
        IOC Speed: Full
      Temperature: 56 C

PhyNum  CtlrHandle  DevHandle  Disabled  Speed   Min    Max    Device
0       0001        0021       N         12      3.0    12     SAS Initiator
1       0001        0021       N         12      3.0    12     SAS Initiator
2       0003        0036       N         12      3.0    12     SAS Initiator
3       0003        0036       N         12      3.0    12     SAS Initiator
4       0002        0035       N         12      3.0    12     SAS Initiator
5       0002        0035       N         12      3.0    12     SAS Initiator
6                              N                 3.0    12     SAS Initiator
7                              N                 3.0    12     SAS Initiator
8       0009        0017       N         12      3.0    12     SAS Initiator
9       0009        0017       N         12      3.0    12     SAS Initiator
10                             N                 3.0    12     SAS Initiator
11                             N                 3.0    12     SAS Initiator
12      000a        0022       N         12      3.0    12     SAS Initiator
13      000a        0022       N         12      3.0    12     SAS Initiator
14                             N                 3.0    12     SAS Initiator
15                             N                 3.0    12     SAS Initiator
16                             N                 ???    ???    No Device    
17                             N                 ???    ???    No Device    
18                             N                 ???    ???    No Device    
19                             N                 ???    ???    No Device    
20                             N                 ???    ???    No Device    

Devices:
B____T    SAS Address      Handle  Parent    Device        Speed Enc  Slot  Wdt
          50000d1702abed3f 0017    0009      SMP Target    12    0002 00    2
          50000d1702abed00 0018    0017      SATA Target   6.0              1
          50000d1702abed01 0019    0017      SATA Target   6.0              1
          50000d1702abed02 001a    0017      SATA Target   6.0              1
          50000d1702abed03 001b    0017      SATA Target   6.0              1
          50000d1702abed04 001c    0017      SATA Target   6.0              1
          50000d1702abed05 001d    0017      SATA Target   6.0              1
          50000d1702abed06 001e    0017      SATA Target   6.0              1
00   91   50000d1702abed07 001f    0017      SATA Target   6.0              1
00   115  50000d1702abed3e 0020    0017      SEP Target    12    0002 24    1
          50000d1700e649bf 0021    0001      SMP Target    12    0003 00    2
          50000d170110d43f 0022    000a      SMP Target    12    0004 00    2
          50000d1700e64980 0023    0021      SATA Target   6.0              1
          50000d1700e64981 0024    0021      SATA Target   6.0              1
          50000d1700e64982 0025    0021      SATA Target   6.0              1
          50000d1700e64983 0026    0021      SATA Target   6.0              1
          50000d1700e64984 0027    0021      SATA Target   6.0              1
          50000d1700e64985 0028    0021      SATA Target   6.0              1
          50000d1700e64986 0029    0021      SATA Target   6.0              1
00   16   50000d1700e64987 002a    0021      SATA Target   6.0              1
          50000d170110d400 002b    0022      SATA Target   6.0              1
          50000d170110d401 002c    0022      SATA Target   6.0              1
          50000d170110d402 002d    0022      SATA Target   6.0              1
          50000d170110d403 002e    0022      SATA Target   6.0              1
          50000d170110d404 002f    0022      SATA Target   6.0              1
          50000d170110d405 0030    0022      SATA Target   6.0              1
          50000d170110d406 0031    0022      SATA Target   6.0              1
00   116  50000d170110d407 0032    0022      SATA Target   6.0              1
00   40   50000d1700e649be 0033    0021      SEP Target    12    0003 24    1
00   140  50000d170110d43e 0034    0022      SEP Target    12    0004 24    1
          50000d17015a29bf 0035    0002      SMP Target    12    0005 00    2
          50000d170052f53f 0036    0003      SMP Target    12    0006 00    2
          50000d17015a2980 0037    0035      SATA Target   6.0              1
          50000d17015a2981 0038    0035      SATA Target   6.0              1
          50000d17015a2982 0039    0035      SATA Target   6.0              1
          50000d17015a2983 003a    0035      SATA Target   6.0              1
          50000d17015a2984 003b    0035      SATA Target   6.0              1
          50000d17015a2985 003c    0035      SATA Target   6.0              1
          50000d17015a2986 003d    0035      SATA Target   6.0              1
00   41   50000d17015a2987 003e    0035      SATA Target   6.0              1
          50000d170052f500 003f    0036      SATA Target   6.0              1
          50000d170052f501 0040    0036      SATA Target   6.0              1
          50000d170052f502 0041    0036      SATA Target   6.0              1
          50000d170052f503 0042    0036      SATA Target   6.0              1
          50000d170052f504 0043    0036      SATA Target   6.0              1
          50000d170052f505 0044    0036      SATA Target   6.0              1
          50000d170052f506 0045    0036      SATA Target   6.0              1
00   66   50000d170052f507 0046    0036      SATA Target   6.0              1
00   65   50000d17015a29be 0047    0035      SEP Target    12    0005 24    1
00   90   50000d170052f53e 0048    0036      SEP Target    12    0006 24    1

Enclosures:
Slots      Logical ID     SEPHandle  EncHandle    Type
  17    300605b00e119fe7    0000       0001     Direct Attached SES-2
  25    50000d1702abed3e    0020       0002     External SES-2
  25    50000d1700e649be    0033       0003     External SES-2
  25    50000d170110d43e    0034       0004     External SES-2
  25    50000d17015a29be    0047       0005     External SES-2
  25    50000d170052f53e    0048       0006     External SES-2

Expanders:
NumPhys   SAS Address     DevHandle   Parent  EncHandle  SAS Level
  37    50000d1702abed3f    0017       0009     0002       1

     Phy  RemotePhy  DevHandle  Speed  Min   Max    Device
     00      00        0018      6.0   1.5   12    SATA Target  
     01      00        0019      6.0   1.5   12    SATA Target  
     02      00        001a      6.0   1.5   12    SATA Target  
     03      00        001b      6.0   1.5   12    SATA Target  
     04      00        001c      6.0   1.5   12    SATA Target  
     05      00        001d      6.0   1.5   12    SATA Target  
     06      00        001e      6.0   1.5   12    SATA Target  
     07      00        001f      6.0   1.5   12    SATA Target  
     08                                1.5   12    No Device    
     09                                1.5   12    No Device    
     10                                1.5   12    No Device    
     11                                1.5   12    No Device    
     12                                1.5   12    No Device    
     13                                1.5   12    No Device    
     14                                1.5   12    No Device    
     15                                1.5   12    No Device    
     16                                1.5   12    No Device    
     17                                1.5   12    No Device    
     18                                1.5   12    No Device    
     19                                1.5   12    No Device    
     20                                1.5   12    No Device    
     21                                1.5   12    No Device    
     22                                1.5   12    No Device    
     23                                1.5   12    No Device    
     24                                1.5   12    No Device    
     25                                1.5   12    No Device    
     26                                1.5   12    No Device    
     27                                1.5   12    No Device    
     28                                1.5   12    No Device    
     29                                1.5   12    No Device    
     30                                1.5   12    No Device    
     31                                1.5   12    No Device    
     32      09        0009      12    1.5   12    SAS Initiator
     33      08        0009      12    1.5   12    SAS Initiator
     34                                1.5   12    No Device    
     35                                1.5   12    No Device    
     36      36        0020      12    12    12    SEP Target   
  37    50000d1700e649bf    0021       0001     0003       1

     Phy  RemotePhy  DevHandle  Speed  Min   Max    Device
     00      00        0023      6.0   1.5   12    SATA Target  
     01      00        0024      6.0   1.5   12    SATA Target  
     02      00        0025      6.0   1.5   12    SATA Target  
     03      00        0026      6.0   1.5   12    SATA Target  
     04      00        0027      6.0   1.5   12    SATA Target  
     05      00        0028      6.0   1.5   12    SATA Target  
     06      00        0029      6.0   1.5   12    SATA Target  
     07      00        002a      6.0   1.5   12    SATA Target  
     08                                1.5   12    No Device    
     09                                1.5   12    No Device    
     10                                1.5   12    No Device    
     11                                1.5   12    No Device    
     12                                1.5   12    No Device    
     13                                1.5   12    No Device    
     14                                1.5   12    No Device    
     15                                1.5   12    No Device    
     16                                1.5   12    No Device    
     17                                1.5   12    No Device    
     18                                1.5   12    No Device    
     19                                1.5   12    No Device    
     20                                1.5   12    No Device    
     21                                1.5   12    No Device    
     22                                1.5   12    No Device    
     23                                1.5   12    No Device    
     24                                1.5   12    No Device    
     25                                1.5   12    No Device    
     26                                1.5   12    No Device    
     27                                1.5   12    No Device    
     28                                1.5   12    No Device    
     29                                1.5   12    No Device    
     30                                1.5   12    No Device    
     31                                1.5   12    No Device    
     32      01        0001      12    1.5   12    SAS Initiator
     33      00        0001      12    1.5   12    SAS Initiator
     34                                1.5   12    No Device    
     35                                1.5   12    No Device    
     36      36        0033      12    12    12    SEP Target   
  37    50000d170110d43f    0022       000a     0004       1

     Phy  RemotePhy  DevHandle  Speed  Min   Max    Device
     00      00        002b      6.0   1.5   12    SATA Target  
     01      00        002c      6.0   1.5   12    SATA Target  
     02      00        002d      6.0   1.5   12    SATA Target  
     03      00        002e      6.0   1.5   12    SATA Target  
     04      00        002f      6.0   1.5   12    SATA Target  
     05      00        0030      6.0   1.5   12    SATA Target  
     06      00        0031      6.0   1.5   12    SATA Target  
     07      00        0032      6.0   1.5   12    SATA Target  
     08                                1.5   12    No Device    
     09                                1.5   12    No Device    
     10                                1.5   12    No Device    
     11                                1.5   12    No Device    
     12                                1.5   12    No Device    
     13                                1.5   12    No Device    
     14                                1.5   12    No Device    
     15                                1.5   12    No Device    
     16                                1.5   12    No Device    
     17                                1.5   12    No Device    
     18                                1.5   12    No Device    
     19                                1.5   12    No Device    
     20                                1.5   12    No Device    
     21                                1.5   12    No Device    
     22                                1.5   12    No Device    
     23                                1.5   12    No Device    
     24                                1.5   12    No Device    
     25                                1.5   12    No Device    
     26                                1.5   12    No Device    
     27                                1.5   12    No Device    
     28                                1.5   12    No Device    
     29                                1.5   12    No Device    
     30                                1.5   12    No Device    
     31                                1.5   12    No Device    
     32      13        000a      12    1.5   12    SAS Initiator
     33      12        000a      12    1.5   12    SAS Initiator
     34                                1.5   12    No Device    
     35                                1.5   12    No Device    
     36      36        0034      12    12    12    SEP Target   
  37    50000d17015a29bf    0035       0002     0005       1

     Phy  RemotePhy  DevHandle  Speed  Min   Max    Device
     00      00        0037      6.0   1.5   12    SATA Target  
     01      00        0038      6.0   1.5   12    SATA Target  
     02      00        0039      6.0   1.5   12    SATA Target  
     03      00        003a      6.0   1.5   12    SATA Target  
     04      00        003b      6.0   1.5   12    SATA Target  
     05      00        003c      6.0   1.5   12    SATA Target  
     06      00        003d      6.0   1.5   12    SATA Target  
     07      00        003e      6.0   1.5   12    SATA Target  
     08                                1.5   12    No Device    
     09                                1.5   12    No Device    
     10                                1.5   12    No Device    
     11                                1.5   12    No Device    
     12                                1.5   12    No Device    
     13                                1.5   12    No Device    
     14                                1.5   12    No Device    
     15                                1.5   12    No Device    
     16                                1.5   12    No Device    
     17                                1.5   12    No Device    
     18                                1.5   12    No Device    
     19                                1.5   12    No Device    
     20                                1.5   12    No Device    
     21                                1.5   12    No Device    
     22                                1.5   12    No Device    
     23                                1.5   12    No Device    
     24                                1.5   12    No Device    
     25                                1.5   12    No Device    
     26                                1.5   12    No Device    
     27                                1.5   12    No Device    
     28                                1.5   12    No Device    
     29                                1.5   12    No Device    
     30                                1.5   12    No Device    
     31                                1.5   12    No Device    
     32      05        0002      12    1.5   12    SAS Initiator
     33      04        0002      12    1.5   12    SAS Initiator
     34                                1.5   12    No Device    
     35                                1.5   12    No Device    
     36      36        0047      12    12    12    SEP Target   
  37    50000d170052f53f    0036       0003     0006       1

     Phy  RemotePhy  DevHandle  Speed  Min   Max    Device
     00      00        003f      6.0   1.5   12    SATA Target  
     01      00        0040      6.0   1.5   12    SATA Target  
     02      00        0041      6.0   1.5   12    SATA Target  
     03      00        0042      6.0   1.5   12    SATA Target  
     04      00        0043      6.0   1.5   12    SATA Target  
     05      00        0044      6.0   1.5   12    SATA Target  
     06      00        0045      6.0   1.5   12    SATA Target  
     07      00        0046      6.0   1.5   12    SATA Target  
     08                                1.5   12    No Device    
     09                                1.5   12    No Device    
     10                                1.5   12    No Device    
     11                                1.5   12    No Device    
     12                                1.5   12    No Device    
     13                                1.5   12    No Device    
     14                                1.5   12    No Device    
     15                                1.5   12    No Device    
     16                                1.5   12    No Device    
     17                                1.5   12    No Device    
     18                                1.5   12    No Device    
     19                                1.5   12    No Device    
     20                                1.5   12    No Device    
     21                                1.5   12    No Device    
     22                                1.5   12    No Device    
     23                                1.5   12    No Device    
     24                                1.5   12    No Device    
     25                                1.5   12    No Device    
     26                                1.5   12    No Device    
     27                                1.5   12    No Device    
     28                                1.5   12    No Device    
     29                                1.5   12    No Device    
     30                                1.5   12    No Device    
     31                                1.5   12    No Device    
     32      02        0003      12    1.5   12    SAS Initiator
     33      03        0003      12    1.5   12    SAS Initiator
     34                                1.5   12    No Device    
     35                                1.5   12    No Device    
     36      36        0048      12    12    12    SEP Target


camcontrol devlist
Code:
root@nas[~]# camcontrol devlist
<ATA CT1000MX500SSD1 045>          at scbus0 target 0 lun 0 (pass0,da0)
<ATA CT1000MX500SSD1 045>          at scbus0 target 1 lun 0 (pass1,da1)
<ATA HGST HDN728080AL W91X>        at scbus1 target 0 lun 0 (pass2,da2)
<ATA HGST HDN728080AL W91X>        at scbus1 target 1 lun 0 (pass3,da3)
<ATA HGST HDN728080AL W91X>        at scbus1 target 2 lun 0 (pass4,da4)
<ATA HGST HDN728080AL W91X>        at scbus1 target 3 lun 0 (pass5,da5)
<ATA HGST HDN728080AL W91X>        at scbus1 target 4 lun 0 (pass6,da6)
<ATA HGST HDN728080AL W91X>        at scbus1 target 5 lun 0 (pass7,da7)
<ATA HGST HDN728080AL W91X>        at scbus1 target 6 lun 0 (pass8,da8)
<ATA HGST HDN728080AL W91X>        at scbus1 target 16 lun 0 (pass9,da9)
<ADAPTEC AEC-82885T B025>          at scbus1 target 40 lun 0 (ses0,pass10)
<ATA ST16000NM001G-2K SN03>        at scbus1 target 41 lun 0 (pass11,da10)
<ADAPTEC AEC-82885T B025>          at scbus1 target 65 lun 0 (ses1,pass12)
<ATA ST14000NM001G-2K SN02>        at scbus1 target 66 lun 0 (pass13,da11)
<ADAPTEC AEC-82885T B025>          at scbus1 target 90 lun 0 (ses2,pass14)
<ATA ST16000NM001G-2K SN03>        at scbus1 target 91 lun 0 (pass15,da12)
<ADAPTEC AEC-82885T B025>          at scbus1 target 115 lun 0 (ses3,pass16)
<ATA ST16000NM001G-2K SN03>        at scbus1 target 116 lun 0 (pass17,da13)
<ADAPTEC AEC-82885T B025>          at scbus1 target 140 lun 0 (ses4,pass18)


geom disk list
Code:
root@nas[~]# geom disk list
Geom name: nvd0
Providers:
1. Name: nvd0
   Mediasize: 256060514304 (238G)
   Sectorsize: 512
   Mode: r1w1e2
   descr: SAMSUNG MZVLB256HAHQ-00000
   lunid: 00253886810102ca
   ident: S444NE0K603420
   rotationrate: 0
   fwsectors: 0
   fwheads: 0

Geom name: da0
Providers:
1. Name: da0
   Mediasize: 1000204886016 (932G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r2w2e5
   descr: ATA CT1000MX500SSD1
   lunid: 500a0751e6999b7f
   ident: 2301E6999B7F
   rotationrate: 0
   fwsectors: 63
   fwheads: 255

Geom name: da1
Providers:
1. Name: da1
   Mediasize: 1000204886016 (932G)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r2w2e5
   descr: ATA CT1000MX500SSD1
   lunid: 500a0751e6999042
   ident: 2301E6999042
   rotationrate: 0
   fwsectors: 63
   fwheads: 255

Geom name: da7
Providers:
1. Name: da7
   Mediasize: 8001563222016 (7.3T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   descr: ATA HGST HDN728080AL
   lunid: 5000cca263c1a46b
   ident: R6G3LZVY
   rotationrate: 7200
   fwsectors: 63
   fwheads: 255

Geom name: da4
Providers:
1. Name: da4
   Mediasize: 8001563222016 (7.3T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   descr: ATA HGST HDN728080AL
   lunid: 5000cca263c076d6
   ident: R6G10NBY
   rotationrate: 7200
   fwsectors: 63
   fwheads: 255

Geom name: da9
Providers:
1. Name: da9
   Mediasize: 8001563222016 (7.3T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   descr: ATA HGST HDN728080AL
   lunid: 5000cca263c1a40a
   ident: R6G3LWRY
   rotationrate: 7200
   fwsectors: 63
   fwheads: 255

Geom name: da2
Providers:
1. Name: da2
   Mediasize: 8001563222016 (7.3T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   descr: ATA HGST HDN728080AL
   lunid: 5000cca263c02dbd
   ident: R6G0D5RY
   rotationrate: 7200
   fwsectors: 63
   fwheads: 255

Geom name: da6
Providers:
1. Name: da6
   Mediasize: 8001563222016 (7.3T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   descr: ATA HGST HDN728080AL
   lunid: 5000cca263c0965b
   ident: R6G191NY
   rotationrate: 7200
   fwsectors: 63
   fwheads: 255

Geom name: da8
Providers:
1. Name: da8
   Mediasize: 8001563222016 (7.3T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   descr: ATA HGST HDN728080AL
   lunid: 5000cca263c19aa0
   ident: R6G3JBZY
   rotationrate: 7200
   fwsectors: 63
   fwheads: 255

Geom name: da3
Providers:
1. Name: da3
   Mediasize: 8001563222016 (7.3T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   descr: ATA HGST HDN728080AL
   lunid: 5000cca263c0621c
   ident: R6G0V46Y
   rotationrate: 7200
   fwsectors: 63
   fwheads: 255

Geom name: da5
Providers:
1. Name: da5
   Mediasize: 8001563222016 (7.3T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   descr: ATA HGST HDN728080AL
   lunid: 5000cca263c18fef
   ident: R6G3EJPY
   rotationrate: 7200
   fwsectors: 63
   fwheads: 255

Geom name: da12
Providers:
1. Name: da12
   Mediasize: 16000900661248 (15T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   descr: ATA ST16000NM001G-2K
   lunid: 5000c500c978d7ee
   ident: ZL2G7WAR
   rotationrate: 7200
   fwsectors: 63
   fwheads: 255

Geom name: da13
Providers:
1. Name: da13
   Mediasize: 16000900661248 (15T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   descr: ATA ST16000NM001G-2K
   lunid: 5000c500dbc8fbb2
   ident: ZL2K74H3
   rotationrate: 7200
   fwsectors: 63
   fwheads: 255

Geom name: da10
Providers:
1. Name: da10
   Mediasize: 16000900661248 (15T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   descr: ATA ST16000NM001G-2K
   lunid: 5000c500dbb81547
   ident: ZL2JVZ6H
   rotationrate: 7200
   fwsectors: 63
   fwheads: 255

Geom name: da11
Providers:
1. Name: da11
   Mediasize: 14000519643136 (13T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r0w0e0
   descr: ATA ST14000NM001G-2K
   lunid: 5000c500b6d4a0b4
   ident: ZL20ALDM
   rotationrate: 7200
   fwsectors: 63
   fwheads: 255


Seems like all drives from the first listed expander are listed/detected by the OS, but each additional expander only presents one. Is there a drive mapping issue from multiple expanders?

EDIT: TrueNAS Scale has no issue detecting all drives. Wanted to stick with Core but I'm not sure what else to check and need to get to my data. If anyone has any suggestions, I will certainly try them, but something about detecting drives on FreeBSD doesn't seem to work with my configuration.
Click to expand...

My guess is its a driver issue with Broadcom/LSI 9400 and freeBSD ... Glad that SCALE works.

Internally, we have many products that use multiple expanders. We typically use the 9300 series and 9500 series HBAs.
The 9400 series FreeBSD drivers were never quite as good. (I can't remember any hardware issues).

Feel free to report a bug if you are prepared to test any fixes we get.
 
J

Jorsher

Explorer
Joined
Jul 8, 2018
Messages
88
Hi, i'm glad that Scale works too. I feel in the minority on this, but I miss jails... Firejail looks like a potential Linux equivalent but I haven't tried it yet.

Kind of surprised nobody else seems to have run into this issue. I wonder someone with more knowledge than me figured it out (not a high bar), or they don't play nice with the Adaptec 82885T expander (I know Adapted doesn't have a great reputation), or just not a lot of people are using the HBA with Core.

I'll likely stick with Scale now that I've put time into setting everything up again, but I'd be willing to test any fixes if testing on Core could be done by simply booting to Core from another drive and seeing if the drives appear... Will post in bugs tomorrow.
 
You must log in or register to reply here.

Similar threads

J
SOLVED: Drives detected during boot, but not showing in TrueNAS
Replies
1
Views
2K
Jorsher
J
R
X399 TrueNAS SCALE "web interface could not be accessed"
Replies
3
Views
1K
ren
R
K
After TrueNas Core install pc won't boot
Replies
1
Views
187
kiwiOverlord
K
hiro5id
SOLVED Install TrueNAS SCALE and restore backup config made from TrueNAS CORE
Replies
1
Views
5K
hiro5id
hiro5id
D
SOLVED TrueNAS-SCALE-21.08-BETA.1 hang on install with nvme disks
Replies
0
Views
1K
Diff
D
Top