Multiple HBAs on a server

[Oko]

I have an old file server with 24 bays which I inherited from a system admin who left our lab 2 years ago. The server is currently running Red Hat 6.5 and having two large softraids 6. The RAIDs are on the life support as the server originally was running Ubuntu 8.0.4 and then "upgraded" to Red Hat. Each power down cycle is a night mare and requires partition search and manual RAID assembly. As soon as the data is migrated the server will be running FreeNAS or TrueOS. I found current RAID card to be supper low quality LSI Symbios Logic SAS1068E Fusion-MPT SAS which currently works in JBOD mode. I was thinking to replace is with two IBM ServeRAID M1015, crossflashed to IT mode. . Why two. Well because my understanding is that M1015 supports up to 14 HDDs. Will having two HBA controllers be a problem. I could buy one new HBA controller from our regular vendor but I would like to keep the purchase low profile as some people might question my decision to remove Linux.

 

[cyberjock]

Umm.. the M1015 does not support up to 14 HDDs. It supports 8 disks, and with SAS expanders, up to 128 drives. Might want to read up some more. ;)

 

[mjws00]

So you want. 1 M1015 and 1 Sas Expander. You will connect the expander to the card via one 8087 -> 8087. The other port on the M1015 will drive the last 4 drives. Or you could grab 3 x M1015. Unfortunately that IBM doc you linked to is a little misleading.

 

[Ericloewe]

Umm.. the M1015 does not support up to 14 HDDs. It supports 8 disks, and with SAS expanders, up to 128 drives. Might want to read up some more. ;)

SAS claims 256 SAS devices, which should mean a fair bit more drives, after accounting for expanders. Anything I'm missing?

I have an old file server with 24 bays which I inherited from a system admin who left our lab 2 years ago. The server is currently running Red Hat 6.5 and having two large softraids 6. The RAIDs are on the life support as the server originally was running Ubuntu 8.0.4 and then "upgraded" to Red Hat. Each power down cycle is a night mare and requires partition search and manual RAID assembly. As soon as the data is migrated the server will be running FreeNAS or TrueOS. I found current RAID card to be supper low quality LSI Symbios Logic SAS1068E Fusion-MPT SAS which currently works in JBOD mode. I was thinking to replace is with two IBM ServeRAID M1015, crossflashed to IT mode. . Why two. Well because my understanding is that M1015 supports up to 14 HDDs. Will having two HBA controllers be a problem. I could buy one new HBA controller from our regular vendor but I would like to keep the purchase low profile as some people might question my decision to remove Linux.

Having multiple HBAs is no problem at all, but expanders are also something to consider.

 

[Oko]

I was thinking of two M1015 with no SAS expanders. However I would swear that M1015 had 14 connections default. I might be better off just shelling $300 and getting a new HBA which supports 16 HDD. I don't even have 24 enterprise grade level HDDin that server. I might be able to scrap to RAIDZ3 of 7 or 2 RIADZ2 of 6 drives and leave few as spare.

 

[Ericloewe]

I was thinking of two M1015 with no SAS expanders. However I would swear that M1015 had 14 connections default. I might be better off just shelling $300 and getting a new HBA which supports 16 HDD. I don't even have 24 enterprise grade level HDDin that server. I might be able to scrap to RAIDZ3 of 7 or 2 RIADZ2 of 6 drives and leave few as spare.

Nope, 8 SAS channels. Same goes for all LSI 2008, 2308 and 3008 controllers. For 24 drives, I'd probably go with one HBA and an expander.
Do keep in mind that the 16-port stuff isn't as tested around here and is typically a lot more expensive than it should be.

 

[cyberjock]

@Ericloewe

The box for the 9211-8i says 128 I believe. I threw out my box or I'd look on it again to verify.

@Oko

You can definitely go find a 16 port controller to use. Do be warned that, like Ericloewe said, none are recommended. If you want to stick to recommended stuff I'd say you're looking at an M1015 (or 9211-8i) and an Intel SAS expander. The cost will probabably be within $20 of whatever 16 port controller you're going to buy anyway.

 

[Ericloewe]

@Ericloewe

The box for the 9211-8i says 128 I believe. I threw out my box or I'd look on it again to verify.

I'll keep that in mind if someone comes along with 128+ SAS devices off an LSI 2008. Fortunately, it's probably an academic question, since 120ish disks off one controller is insane.

 

[Darren Myers]

I managed to get 3 M1015's for $290 and was going to ask the same question. There shouldnt be any issues what so ever then correct?

 

[Ericloewe]

I managed to get 3 M1015's for $290 and was going to ask the same question. There shouldnt be any issues what so ever then correct?

Use as many as you'd like, in any combination.

The only thing to keep in mind is that boot ROMs may interfere with one another. This has been observed when mixing LSI SAS 2308 controllers with 2008 controllers.
Ideally, flash all controllers without the boot ROM, since you won't be booting from them.

 

[Darren Myers]

Use as many as you'd like, in any combination.

The only thing to keep in mind is that boot ROMs may interfere with one another. This has been observed when mixing LSI SAS 2308 controllers with 2008 controllers.
Ideally, flash all controllers without the boot ROM, since you won't be booting from them.

Thanks for the reply Ericloewe, the gentleman i got them from sold them as already flashed to LSI 9211. The original item discription is
"I am selling an IBM M1015 / SAS9220-8i RAID Controller Card. This card has been successfully cross-flashed with the LSI 9211 firmware running in IT mode"

 

[Ericloewe]

Thanks for the reply Ericloewe, the gentleman i got them from sold them as already flashed to LSI 9211. The original item discription is
"I am selling an IBM M1015 / SAS9220-8i RAID Controller Card. This card has been successfully cross-flashed with the LSI 9211 firmware running in IT mode"

You need P16, so you'll most likely have to flash the cards to P16.

 

[cyberjock]

What Ericloewe said. Unless you told him you are going to need P16, he almost certainly used the latest (which is P19) so you're going to have to reflash them again anyway.

 

[Darren Myers]

You need P16, so you'll most likely have to flash the cards to P16.

I shot the gentleman a email who im getting them from, he says theyre flashed to P19, is there a link handy with the P16 firmware? A google search didnt yield me any exact links that no longer work

 

[cyberjock]

Nope. Go to the LSI website and you'll find it in their archived downloads.

 

[Darren Myers]

Perfect, i got it now i was searching the wrong thing sorry about that. I found the P16 firmware for the i8 MSDOS :)

 

[Ericloewe]

Perfect, i got it now i was searching the wrong thing sorry about that. I found the P16 firmware for the i8 MSDOS :)

You may need the UEFI flasher, if your motherboard is recent.

 

[Darren Myers]

You may need the UEFI flasher, if your motherboard is recent.

I see a UEFI installer for P19, but not for P16. The LSI zip i downloaded is "Package_P16_Firmware_BIOS_for_MSDOS_Windows"

 

[cyberjock]

Google it. There's a special set of executables if your system uses UEFI. ;)

 

[pjc]

Make sure you click on the "Archived" link after you expand the "Drivers" in the results. Otherwise it'll only show you current versions:

http://www.lsi.com/support/Pages/download-search.aspx

Separately, downgrading is scary and requires DOS or UEFI. You have to wipe the NVRAM and everything else from your HBA, and if you lose power or screw up, you brick your HBA. The following were helpful for me:

http://mycusthelp.info/LSI/_cs/AnswerDetail.aspx?s&inc=7954
http://justanothergeeks.blogspot.com/2013/06/flashing-it-firmware-on-lsi-9211-8i-hba.html
http://mycusthelp.info/LSI/_cs/AnswerPreview.aspx?inc=8352