LSI MegaRAID SAS 9261-8i timeout with write-back

[freibuis]

Long time Reader. First time Poster!.
(end awkward moment)


I have an LSI mfi based storage controller LSI MegaRAID SAS 9261-8i and I get the following issues

mfi0: COMMAND 0xffffff80005b1ee0 TIMEOUT AFTER 123 SECONDS -->
mfi0: COMMAND 0xffffff80005b1ee0 TIMEOUT AFTER x SECONDS
during this time. any services attached to the ZFS volume crashes
Feb 14 10:19:03 freenas cnid_dbd[3067]: read: Connection reset by peer
Feb 14 10:19:03 freenas cnid_dbd[3067]: error reading message header: Connection reset by peer

But if I run a
mfiutil show adapter
the time out stops counting up and resets to 0 and all is ok for 20 seconds or so then the
mfi0: COMMAND 0xffffff80005b1ee0 TIMEOUT starts again.


here is the cache settings on one of the disks

mfiutil cache mfid0
mfi0 volume mfid0 cache settings:
I/O caching: disabled
write caching: write-back
read ahead: none
drive write cache: disabled



I changed the caching method from write-back to write-through (cahce - write straight to disk)


mfiutil cache mfid0
mfi0 volume mfid0 cache settings:
I/O caching: disabled
write caching: write-through
read ahead: none
drive write cache: disabled

Issue Stopped..


okay I would really like to know is why having write-back option Fail with time outs? hardware/firmware issue? freenas mfi drivers out of date?

I would love to use the write-back feature as performace boost is awesome?


any takers?

 

[freibuis]

so I installed Freebsd 8.2 and same issue... then I install freebsd 9.0 this issue went away..

so when is the freeNas going to be based off freebsd 9?

 

[peterh]

The other solution is to get rid of your megaraid ( sell it on ebay !)
Zfs does not need it, you don't need it. In fact your currect cpu(s) has more power then
megaraid ever had. Not to mention the ease of administration when all discs are
visible and manageble from the os itself. No need for strange utils, no need for
getting down to BIOS to change settings.

 

[freibuis]

I have used Zfs for a couple of years. I do agree to some extent with what you are saying. But hardware raid is still better then Zfs atm. I love ZFS but because its tied to the CPU its not always the best solution. I know that I am using the megaraid disks in RAID0 so I can use Zfs. maybe in my situation you would be right.

I will not change this card. This card is pretty darn good for the project it is using it for. Just have to wait until freenas is using freebsd 9.0

anyway I would like to see Zfs work with hardware raid and not work against it. sure software raid has its benefits but so does hardware raid. Imagine Zfs using next to no cpu but giving better read/write performace. A man gotta dream.

 

[survive]

Hi freibuis,

A couple of thoughts:

If you have the disks in RAID0 with ZFS on top of that what are you doing for redundancy? I don't see how ZFS being tied to the CPU is a problem, clock cycles are cheaper than ever these days!

You will be waiting for a while for FreeNAS based on FreeBSD 9.0. It might make more sense to be a little more flexible if you want the box to actually work. If you return or sell the card you can pick up a simple HBA for around $100 and solve your whole problem.

I don't know that you will ever see "Zfs work with hardware raid and not work against it" because that idea really seems contradictory to some of the fundamental objectives of ZFS. Personally the whole idea of paying several hundred bucks for a controller card just so I don't use the CPU I already have just doesn't make sense to me.

-Will

 

[Milhouse]

I know that I am using the megaraid disks in RAID0 so I can use Zfs.

That sounds like a recipe for disaster.

I will not change this card. This card is pretty darn good for the project it is using it for. Just have to wait until freenas is using freebsd 9.0

Which is all well and good, except that, by your own admission, it doesn't work properly.

As others have already commented, ZFS does not need or benefit from hardware RAID accelerated controllers. In the context of ZFS, hardware RAID controllers are generally bad news. If you like fancy controllers, a "dumb" HBA like the LSI 9211-8i (once flashed with IT firmware) works perfectly with FreeNAS 8.x and ZFS.

Not only is your current card misbehaving under FreeNAS 8.x, but in order to get it to work you've configured the controller-based disk array in such a way that it may eliminate any redundancy protection ZFS would otherwise give you. It's adding risk, complexity and overhead where it's not needed, and certainly not recommended. And all for the sake of a hundred bucks or so?

 

[dneculai]

If i may, try to build the kernel with this mfi.c patch instead - http://freshbsd.org/commit/freebsd/r227905 - as it helped me with timeouts on a machine running FreeNAS 8.0.4-RELEASE.

Test environment recommended, of course.