Unable to see raw disks attach to RAID card LSISAS 2108

Status
Not open for further replies.

datnus

Contributor
Joined
Jan 25, 2013
Messages
102
Hi guy,
I'm using FreeNas 8.3.1 in SuperMicro box.
I have 10+ HDDs attach to a hardware RAID card LSISAS 2108.
And I want to use ZFS software RAID.

If I create Virtual drive (aka RAID disk) in the LSI GUI, FreeNAS can see these RAID disks.
But if I let the raw disk unconfigured, FreeNAS can't see any of them.

I would like to have FreeNAS to see raw disks and create RAID-Z from them.
Any help is appreciated.;)
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,680
Right. See this post. You have an intelligent RAID controller with cache memory. You want an HBA. They're not the same thing, and I don't think the 2108 can be taught to act as an HBA (some of the entry-level RAID controllers can).

Put simply, the problem is that LSI assumes that if you buy a controller with features like cache, you intend for them to be used. The only way to use cache is for the controller to sit in the communications/data path between the server and the drive, and handle the requests.
 

datnus

Contributor
Joined
Jan 25, 2013
Messages
102
Is there any way to flash the card or something?
Haiz, the card is more expensive than the normal HBA.

FreeNas has put the card in the supported card without any warning.
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
FreeNas has put the card in the supported card without any warning.
You are correct. It's only really an issue if you use ZFS. If you want to use UFS then it would be a fine card to use. If you're someone that doesn't trust software RAID and insists on using a hardware RAID you are still welcome to use ZFS on a RAID array. But its entirely at your risk and peril.

As you'll learn quickly in the Linux/FreeBSD world(and probably everything that's non-Windows/Mac since they decide what's "best" for you instead of letting you choose) is that its not about getting warnings when you're going to do something stupid, its about giving the administrator full control to do whatever he/she wants. There is always that one situation where something that would normally be incredibly stupid is actually incredibly smart. Instead most non-Windows OSes seem to give you the versatility and choice for you to do whatever you want and assume you have the experience and education necessary to make those decisions. If you choose to not do your homework..well.. that's your choice. ZFS will never be available for Windows because Windows expects you to be so incompetent that you couldn't possibly make smart choices for yourself. There's a reason why Windows users are often called "zombies".

Yes, I am a Windows user.. and Linux user.. and a FreeNAS user. So I'm the worst of all of them put together :P
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,680
Is there any way to flash the card or something?

I am not aware of any. The information I am reasonably certain of, via hearsay, or know from direct experience, I stick into that sticky that I referenced. If you discover a way to flash a 2108 to something that supports an HBA mode, provide a link and I will certainly look at it.

Haiz, the card is more expensive than the normal HBA.

Ah, yes, the irony. I feel your pain.

FreeNas has put the card in the supported card without any warning.

The documented Hardware Recommendations clearly states:

NOTE: instead of mixing ZFS RAID with hardware RAID, it is recommended that you place your hardware RAID controller in JBOD mode and let ZFS handle the RAID. According to Wikipedia: "ZFS can not fully protect the user's data when using a hardware RAID controller, as it is not able to perform the automatic self-healing unless it controls the redundancy of the disks and data. ZFS prefers direct, exclusive access to the disks, with nothing in between that interferes. If the user insists on using hardware-level RAID, the controller should be configured as JBOD mode (i.e. turn off RAID-functionality) for ZFS to be able to guarantee data integrity. Note that hardware RAID configured as JBOD may still detach disks that do not respond in time; and as such may require TLER/CCTL/ERC-enabled disks to prevent drive dropouts. These limitations do not apply when using a non-RAID controller, which is the preferred method of supplying disks to ZFS."

My sticky in this forum states:

The IBM ServeRAID M1015, crossflashed to IT mode, is probably one of the best choices available if you need a bunch of extra SATA ports. It does take an extra ~12 watts however. Don't buy it new. They're available cheap on eBay all the time.

Random RAID controllers that are operating in RAID mode (showing virtual or logical devices to FreeNAS) are a very bad idea. If your controller costs more than a few hundred dollars, it may not be a good choice for FreeNAS.

So I am going to say that your claim of "without any warning" is also demonstrably untrue. I will gladly concede, however, that the learning curve here is high, and that it is easy to miss these details. One normally assumes that the pricier, more capable product is the better choice. This isn't always true, and that's what prompted me to write that sticky.

The problem is that there is hardware that works, and there's the hardware that works best. From a high level point of view, ZFS is your RAID controller. It has access to eye-melting amounts of memory and CPU resources. When you put a dinky little "hardware" RAID controller after ZFS, you aren't getting anything of value from it, and you may be losing things, even, such as the ability to speak SMART with the drive.

However, let me also say this: the 2108 will work. It isn't the best pick though. You should thoroughly read the section of the LSI sticky in this forum, and you'll get most of what you "need to know." The most important bit is that the LSI will throw an LSI label on the disk and I'm pretty sure you won't be able to migrate the disks to a non-LSI controller if ever you might need to (like if the LSI controller starts on fire and you have no spare).
 

datnus

Contributor
Joined
Jan 25, 2013
Messages
102
Haha, I spent triple the $, check the card on supported list and still hit the trap. :smile:
I don't want to use hardware raid as if I add in new disk, I need to reboot the server to BIOS/LSI GUI, aka. downtime.

For software raid, it's much simpler, just plug in the HDDs and configure them while the server is still online.

- - - Updated - - -

@jgreco: If I change to IBM ServeRAID M1015 and don't want to flash the card, will the card....work?
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,680
No, the IBM firmware is MFI. Covered in sticky:

It is important to crossflash this card! If you do, it works under the "mps" driver, and it becomes a plain HBA. If you don't, then it is under the "mfi" driver and it is a RAID card.

Well, actually to answer your question, if you choose the M1015 but don't flash the card, yes, it will work, but it will work with the same caveats you have on the 2108. In that case, the 2108 is maybe more advantageous because you might actually get some marginal benefit from the cache.

But you should also be aware that FreeNAS does supply mfiutil from the command line, and it seems complete and competent from my minor explorations. You shouldn't actually need to reboot to do maintenance if you can figure out the correct mfiutil commands. I've successfully set up a RAID1 using it, as I recall.
 

datnus

Contributor
Joined
Jan 25, 2013
Messages
102
Haiz, so what I could believe the "HBA" suitable for software RAID is not widely available (without modding/flashing the card)?
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,680
Haiz, so what I could believe the "HBA" suitable for software RAID is not widely available (without modding/flashing the card)?

There seems to be a reasonable selection of them, though I believe some of the PCIe 3.0 ones are only supported under newer FreeBSD's.

The question is, though, do you go for something like the LSI SAS 9211-8i, which goes for about $350, and may need to be flashed to IT mode, or do you just get one of the many M1015's available on eBay, that people pull out of their IBM servers while upgrading them, and are usually selling for less than $100?

I mean, either way, you're taking the card and bludgeoning the MFI firmware out of it. And I don't know about you, but I can think of clever things to do with an extra $250. The other bonus to the M1015 is that there are very specific guides available to flashing the M1015, which may be helpful if you are not an LSI guru.
 

datnus

Contributor
Joined
Jan 25, 2013
Messages
102
With my raid card, is it better to create raid 0 for EACH of the HDD, and then use ZFS to create software raid on top of these hardware raid 0?

https://calomel.org/zfs_raid_speed_capacity.html

ZFS prefers to be in full control of the drives without any interference from drive controllers or caching mechanisms. In order to get the most performance out of ZFS and the LSI card we need to allow direct access to the drives by disabling raid card cache.

You need to setup each disk as a separate RAID0 array in the LSI raid controller. This is similar to a JBOD mode, but this method allows us to use all the caching and efficiency algorithms the LSI card can offer
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,680
Yes, that at least allows you to leverage some of the features of ZFS.
 
Status
Not open for further replies.
Top