FreeNAS Not Showing All Drives Connected with SAS9217-4i4e HBA

[shwan_kr1]

Hello FreeNAS Community,

I have FreeNAS version 11.3-U2.1 installed on a Dell PowerEdge R730xd with 2 CPUs and 64 GB of RAM. I am using a PCIe Broadcom SAS9217-4i4e HBA to connect 4 daisy chained external disk enclosures to the FreeNAS server. There are a collection of SAS and SATA disks and from different vendors (Toshiba, Western Digital and Hitachi).
When launching the SAS9217-4i4e HBA utility, I see all disks in the enclosures but when I boot FreeNAS, I see it detects all the SATA disks but only 4 of the SAS disks. So from a total of 52 disks, FreeNAS only detects 28 disks (24 SATA + 4 SAS).
Any idea about what could be the reason? I checked Kernel logs and system logs but could not figure out anything that could give me a clue. The kernel logs and system logs since the last startup are attached.

Could you please help?

Thanks
Shwan

 

[Fredda]

Could you tell us a little bit more about the enclosures. From the logs it looks like they are NetApps.
Jun 30 01:18:42 freenas2 da10: <NETAPP X316_HAKPE06TA07 NA00> Fixed Direct Access SPC-4 SCSI device
Were the disks prior used in a NetApp? NetApp uses a 520 byte sector size and the disks need to be reformated to 512 so that FreeNAS can handle them.
Is your HBA flashed to IT mode? Please post the output of sas2flash -list.
Also please post the of camcontrol devlist
Put the output in [code][/code] tags.

 

[shwan_kr1]

Hi Fredda,

Many thanks for your help!

The enclosures are 2 x Symantec Promise JX30 + 2 x Dell PowerVault MD1200
The following:

Code:

Jun 30 01:18:42 freenas2 da10: <NETAPP X316_HAKPE06TA07 NA00> Fixed Direct Access SPC-4 SCSI device

is for /dev/da10 that is already detected by FreeNAS. There are /dev/da0 to /dev/da41, the first 14 drives are internal and da10 is already a member of a FreeNAS pool and ZFS volume. The last 28 drives (da14 - d41) are external and still unused but there are 24 external disks that are detected by the SAS9217-4i4e HBA utility but NOT by FreeNAS. I am not sure where the NETAPP label has originated from to be honest!

>>>NetApp uses a 520 byte sector size and the disks need to be reformated to 512 so that FreeNAS can handle them.
I'm sorry if my question is not smart, but when I create a pool and a ZFS volume, doesn't that also involve reformatting all the disks in the pool?

Below is the output of the commands you requested:

Code:

root@freenas2[~]# sas2flash -list
LSI Corporation SAS2 Flash Utility
Version 16.00.00.00 (2013.03.01)
Copyright (c) 2008-2013 LSI Corporation. All rights reserved

        Adapter Selected is a LSI SAS: SAS2308_2(D1)

        Controller Number              : 0
        Controller                     : SAS2308_2(D1)
        PCI Address                    : 00:04:00:00
        SAS Address                    : 500605b-0-0c38-e1d0
        NVDATA Version (Default)       : 11.00.00.02
        NVDATA Version (Persistent)    : 11.00.00.02
        Firmware Product ID            : 0x2714 (IR)
        Firmware Version               : 18.00.00.00
        NVDATA Vendor                  : LSI
        NVDATA Product ID              : SAS9217-4i4e
        BIOS Version                   : 07.35.02.00
        UEFI BSD Version               : 07.25.01.00
        FCODE Version                  : N/A
        Board Name                     : SAS9217-4i4e
        Board Assembly                 : 03-25597-01B
        Board Tracer Number            : SP64539295

        Finished Processing Commands Successfully.
        Exiting SAS2Flash.
root@freenas2[~]#

It seems it is in IR mode, should I convert it to IT mode? Do you think that could solve the problem? With a Google search, I've found the following: https://nguvu.org/freenas/Convert-LSI-HBA-card-to-IT-mode/, can I do right from FreeNAS knowing that it is booted from another HBA or it is a must to boot a minimal image from a bootable flash?
The following is the output of camcontrol command:

Code:

root@freenas2[~]# camcontrol devlist
<NETAPP X316_HAKPE06TA07 NA00>     at scbus1 target 0 lun 0 (pass0,da0)
<NETAPP X316_HAKPE06TA07 NA00>     at scbus1 target 1 lun 0 (pass1,da1)
<NETAPP X316_HAKPE06TA07 NA00>     at scbus1 target 2 lun 0 (pass2,da2)
<NETAPP X316_HAKPE06TA07 NA00>     at scbus1 target 3 lun 0 (pass3,da3)
<NETAPP X316_HAKPE06TA07 NA00>     at scbus1 target 4 lun 0 (pass4,da4)
<NETAPP X316_HAKPE06TA07 NA00>     at scbus1 target 5 lun 0 (pass5,da5)
<NETAPP X316_HAKPE06TA07 NA00>     at scbus1 target 6 lun 0 (pass6,da6)
<NETAPP X316_HAKPE06TA07 NA00>     at scbus1 target 7 lun 0 (pass7,da7)
<NETAPP X316_HAKPE06TA07 NA00>     at scbus1 target 8 lun 0 (pass8,da8)
<NETAPP X316_HAKPE06TA07 NA00>     at scbus1 target 9 lun 0 (pass9,da9)
<NETAPP X316_HAKPE06TA07 NA00>     at scbus1 target 10 lun 0 (pass10,da10)
<NETAPP X316_HAKPE06TA07 NA00>     at scbus1 target 11 lun 0 (pass11,da11)
<ATA MTFDDAK256TBN-1A 0014>        at scbus1 target 12 lun 0 (pass12,da12)
<ATA MTFDDAK256TBN-1A 0014>        at scbus1 target 13 lun 0 (pass13,da13)
<DP BP13G+EXP 3.03>                at scbus1 target 32 lun 0 (pass14,ses0)
<HITACHI HUS723030ALS640 A222>     at scbus2 target 0 lun 0 (pass15,da14)
<HITACHI HUS723030ALS640 A222>     at scbus2 target 1 lun 0 (pass16,da15)
<HITACHI HUS723030ALS640 A222>     at scbus2 target 2 lun 0 (pass17,da16)
<HITACHI HUS723030ALS640 A222>     at scbus2 target 3 lun 0 (pass18,da17)
<ATA WDC WD2003FYYS-0 1D01>        at scbus2 target 4 lun 0 (pass19,da18)
<ATA WDC WD2003FYYS-0 1D02>        at scbus2 target 5 lun 0 (pass20,da19)
<ATA WDC WD2003FYYS-0 1D02>        at scbus2 target 6 lun 0 (pass21,da20)
<ATA WDC WD2003FYYS-0 1D01>        at scbus2 target 7 lun 0 (pass22,da21)
<ATA WDC WD2003FYYS-0 1D01>        at scbus2 target 8 lun 0 (pass23,da22)
<ATA WDC WD2003FYYS-0 1D01>        at scbus2 target 9 lun 0 (pass24,da23)
<ATA WDC WD2003FYYS-0 1D01>        at scbus2 target 10 lun 0 (pass25,da24)
<ATA WDC WD2003FYYS-0 1D01>        at scbus2 target 11 lun 0 (pass26,da25)
<ATA MB2000EAZNL HPG2>             at scbus2 target 12 lun 0 (pass27,da26)
<ATA WDC WD2003FYYS-0 1D02>        at scbus2 target 13 lun 0 (pass28,da27)
<ATA WDC WD2003FYYS-0 1D02>        at scbus2 target 14 lun 0 (pass29,da28)
<ATA WDC WD2003FYYS-0 1D02>        at scbus2 target 15 lun 0 (pass30,da29)
<ATA WDC WD2003FYYS-0 1D02>        at scbus2 target 17 lun 0 (pass31,da30)
<ATA WDC WD2003FYYS-3 1D01>        at scbus2 target 18 lun 0 (pass32,da31)
<ATA MB2000EAZNL HPG2>             at scbus2 target 19 lun 0 (pass33,da32)
<ATA WDC WD2003FYYS-0 1D01>        at scbus2 target 20 lun 0 (pass34,da33)
<ATA WDC WD2003FYYS-0 1D01>        at scbus2 target 21 lun 0 (pass35,da34)
<ATA WDC WD2003FYYS-0 1D01>        at scbus2 target 22 lun 0 (pass36,da35)
<ATA TOSHIBA DT01ACA2 ABB0>        at scbus2 target 23 lun 0 (pass37,da36)
<ATA MB2000EAZNL HPG2>             at scbus2 target 24 lun 0 (pass38,da37)
<ATA WDC WD2003FYYS-0 1D02>        at scbus2 target 25 lun 0 (pass39,da38)
<ATA WDC WD2003FYYS-0 1D01>        at scbus2 target 26 lun 0 (pass40,da39)
<ATA WDC WD2003FYYS-3 1D01>        at scbus2 target 27 lun 0 (pass41,da40)
<ATA TOSHIBA DT01ACA2 ABB0>        at scbus2 target 28 lun 0 (pass42,da41)
<Promise J630s 060=>               at scbus2 target 36 lun 0 (pass43,ses1)
<DELL MD1200 1.01>                 at scbus2 target 40 lun 0 (pass44,ses2)
<DELL MD1200 1.01>                 at scbus2 target 53 lun 0 (pass45,ses3)

Strangely I see only one Promise J630s, there should be 2!

Regards,
Shwan

 

[Fredda]

It seems it is in IR mode, should I convert it to IT mode?

The IT version is the preferred version for FreeNAS. The IR version has some basic RAID functionality.
You can simply donwload the firmware from the broadcom website.
It usually includes both IT and IR firmware and has a DOS tool to flash the firmware.
You simpy boot a FreeDOS USB drive with the FW and the flash tool.
I think there a some guides here in this forum.

If I read it correctly your card then will turn into a SAS9207 card.

But I'm not sure if that will help as you said on of the shelfs does not show up.

 

[shwan_kr1]

Hi Fredda,

Just wanted to update this ticket so that other people may benefit from the result and to thank you for your support. I've just converted the card to IT mode and now all the drives are detected by FreeNAS!

Regards,
Shwan

 

[Fredda]

Glad all worked out