Only 7 of 8 disks detecting, FreeNAS 8.0.4-p1-x64 with LSI SAS9211-8i controller.

[steinhilberm]

Hello, maybe somebody might have some input on a problem I'm encountering.

Here are my server specifications:

Chassis: Supermicro SC825TQ-R700LPB
Motherboard: Supermicro X7DB3
CPU: Intel Xeon 5150
RAM: 8GB
Controller: LSI SAS9211-8i HBA
HDD: 8x 2TB HDD's
Boot device: HP 4GB USB

Here are the symptoms:

System posts and the LSI controller detects all 8 HDD's. FreeNAS boots properly and allowed me to configure the onboard two gigabit ethernet ports as LACP. I login to the web GUI and attempt to create a volume but find FreeNAS only recognizes 7 of the 8 HDD's (da0 - da6). Four of the 8 HDD's were previously used in another server of the exact same specifications running Server 2008 R2 so I wiped the drives clean before proceeding with the FreeNAS configuration, the remaining 4 HDD's were new in box.

Here is what I have tried so far:

I removed the 4 HDD's along the top of the server to see if FreeNAS would detect all 4 of the HDD's in the lower bays of the server. Only three of the HDD's would detect in the lower bays. I shutdown the server and swapped the drives in the lower bays with the drives I previously removed from the top bays (and left the upper 4 bays empty), boot up the server and still only 3 of the 4 drives in the lower bays detected. At this point I assumed perhaps a cable issue, checked to verify none had become unplugged and all were secure so I replaced the SAS cable and the same problem persisted. At this point I thought it may be an issue with the SAS backplane even though Server 2008 R2 detected disks in all 8 bays. I have a half dozen servers of the same specification so I grabbed one off the shelf that had also been previously configured as a 2008 R2 server with 8 HDD's and swapped my HDD's into the server and installed my USB stick containing my FreeNAS install - boot the server up and only 7 of the 8 HDD's were detected again. I swapped a series of 4 HDD's at a time into the lower HDD bays and it only detected 3 of the 4 HDD's as it had before in the previous server.

I don't believe the issue to be related to my motherboard, controller or HDD's given the troubleshooting I have done thus far. I have found posts here of others using the same LSI controller as me with 8 drives connected as well, so I don't believe it to be a problem with that. I attempted the latest firmware for the LSI controller as well and also updated the BIOS on the motherboard to the latest. Does anybody have any ideas? I was hoping to turn a bulk of these servers I have sitting here into FreeNAS servers if I can manage to get all 8 HDD's to detect. Any and all help is greatly appreciated!

 

[ProtoSD]

Can you post the output of:

camcontrol devlist

gpart show

 

[steinhilberm]

Can you post the output of:

camcontrol devlist

gpart show

Sure can, here is the output requested:

Code:

[root@freenas] ~# camcontrol devlist
<ATA Hitachi HDS72302 A800>        at scbus0 target 0 lun 0 (da0,pass0)
<ATA Hitachi HDS72302 A800>        at scbus0 target 1 lun 0 (da1,pass1)
<ATA Hitachi HDS72302 A800>        at scbus0 target 2 lun 0 (da2,pass2)
<ATA WDC WD2002FAEX-0 1D05>        at scbus0 target 3 lun 0 (da3,pass3)
<ATA WDC WD2002FAEX-0 1D05>        at scbus0 target 4 lun 0 (da4,pass4)
<ATA WDC WD2002FAEX-0 1D05>        at scbus0 target 5 lun 0 (da5,pass5)
<ATA WDC WD2002FAEX-0 1D05>        at scbus0 target 6 lun 0 (da6,pass6)
<hp v165w 1100>                    at scbus7 target 0 lun 0 (da7,pass7)
[root@freenas] ~# gpart show
=>     63  7831467  da7  MBR  (3.7G)
       63  1930257    1  freebsd  [active]  (943M)
  1930320       63       - free -  (32K)
  1930383  1930257    2  freebsd  (943M)
  3860640     3024    3  freebsd  (1.5M)
  3863664    41328    4  freebsd  (20M)
  3904992  3926538       - free -  (1.9G)

=>      0  1930257  da7s1  BSD  (943M)
        0       16         - free -  (8.0K)
       16  1930241      1  !0  (943M)

 

[ProtoSD]

My guess is it's the firmware. What firmware are you using, I know you said it was the latest. I'm pretty sure this is one of those controllers that you need to flash in IT Mode.

 

[steinhilberm]

My guess is it's the firmware. What firmware are you using, I know you said it was the latest. I'm pretty sure this is one of those controllers that you need to flash in IT Mode.

Thanks much for both of your quick responses. I'm running firmware 13.00.01.00 however it is the IR revision so I will try the IT version once I'm back in the office tomorrow and report back. Thank you again!

 

[survive]

Hi steinhilberm,

What happens if you go into the RAID card's BIOS? Can you see the drives there? It looks like FreeNAS can see all 8 drives on some level, maybe there is some residual partition information that's causing trouble. If you have some time you can grab the DBAN (Darik's Boot And Nuke) boot disk from:

http://www.dban.org/

and let it nuke the data on the drives. It's kind of a big hammer to hit the problem with, there are certainly more elegant ways to get in and erase the partition data on the drives....DBAN is more a boot it, start it & go do something else for a couple of hours kind of thing. Please be careful with the DBAN disk....it won't autonuke anything, but it is after all a wipe-all-your-data boot disk!

I'd get the drive situation sorted out first, but once you can see all 8 drives take a look at this page:

http://www.servethehome.com/ibm-serveraid-m1015-part-4/

There are instructions there that will let you "dumb" your card down into a simple HBA with no RAID abilities to get in between ZFS & the drives. ZFS is happiest when it has unfettered access to the disk drives.

-Will

 

[steinhilberm]

Hi steinhilberm,

What happens if you go into the RAID card's BIOS? Can you see the drives there? It looks like FreeNAS can see all 8 drives on some level, maybe there is some residual partition information that's causing trouble. If you have some time you can grab the DBAN (Darik's Boot And Nuke) boot disk from:

http://www.dban.org/

and let it nuke the data on the drives. It's kind of a big hammer to hit the problem with, there are certainly more elegant ways to get in and erase the partition data on the drives....DBAN is more a boot it, start it & go do something else for a couple of hours kind of thing. Please be careful with the DBAN disk....it won't autonuke anything, but it is after all a wipe-all-your-data boot disk!

I'd get the drive situation sorted out first, but once you can see all 8 drives take a look at this page:

http://www.servethehome.com/ibm-serveraid-m1015-part-4/

There are instructions there that will let you "dumb" your card down into a simple HBA with no RAID abilities to get in between ZFS & the drives. ZFS is happiest when it has unfettered access to the disk drives.

-Will

Four of the drives were wiped using an alternate utility prior to being installed, the drive it isn't picking up (a 4th Hitachi drive) is new out of the box. Only 7 of the 8 were showing up in the output of "camcontrol devlist" as well as looking through the output of "dmesg | less". Going to give the IT firmware a go tomorrow morning and see if I have any success.

 

[steinhilberm]

Flashing the IT mode firmware corrected the issue, thanks for the help!