HP P800 controller

[valvolainen]

Hi,
My system consists of a HP 360 G6 32gb ram, with a MSA70 shelf and a P800 controller. (not set up yet) Including internal drivers I have 29 drives in total.
My use will be solely as a home nas, movie streaming and shared document folders. Not very exiting, nor i/o intensive.

Going for hardware raid, I'd like to create groups of rougly 5-7 drives so it'll be cheaper to replace a raid set with a larger one. It'll waste more space, but be more affordable to upgrade. There is also a limitation of 1,5gb/s on sata performance, but it'll have to do for now.

Since JBOD is not an option on this controller, do I want to run freenas/zfs on hardware raid sets, or would it be better just to run some linux distro with lvm and ext4 on this? Alternatively freenas with other filesystem?
If there is a controller swap I could do, then that could be a great alternative, what sort of controller could I use? Sorry for stupid questions, but are the external connectors on the P800 / msa70 shelf proprietary?

Thanks!

 

[Darren Myers]

huge noooooo for hardware raid ontop of ZFS....you will run into so many problems, you could pick up a regular HBA, either 9211-8i or a M1015 and crossflash it to IT mode, this would in theory allow each drive to be seen in JBOD that is in the disk shelf...but i am not sure if HP did any proprietary shiz between there controller and the disk shelf/shelves.

As for the external connections no, it should be SFF-8088 cables

 

[valvolainen]

Hi,
Thanks for answering. Neither the 9211 nor m1015 seems to have external connectors.
I'll have to check to see if there is an extra slot available for a conversion bracket.
Removing the low profile network adapter, and there is room for one more card so I can add an internal-to-external converter.
Ordered the 1015 card, bracket with external connectors and internal cables.
I got the M1015 on ebay, not thinking that i could perhaps aquire this locally, and in my local area I found not less than four such items for sale:(
instead I have to wait 3-6 weeks for delivery. Note to self: ebay is not always my only choice.

So now I hope that I've got sas expanders that works with this card (if that can be an issue).
Theroretically, i should also have support for larger-than 2tb drives as well with this setup?
Not that it's needed, but it would be nice.

Internally in my dl360 (4 slots) I can perhaps pick up an ssd or two and use for memory offloading? (don't remember what it's called.)
Is this something that can be added later on or do I need to set it up before using zfs? Sorry, newbie on this.

 

[sfcredfox]

I was using a DL 380 G5 for a while.

The G5 had 8484 connectors on the internal backplane that could be connected to the internal 8087 connectors of the M1015 (LSI 9211-8i).
It makes a good place for SSDs if you need to use them.
You need to read some posts from jgreco for insights into SLOG and L2ARC. If you don't have the right amount of memory or an understanding of what they are really used for, it will be a waste of time or cause performance to get worse.

HP rebrands an LSI 9207-8e as the HP H221 which is a 6G dual external 8088. You can use that to connect the MSA70. That's what I'm using. I also just got a D2700 to see if the 6G disks work better than the crappy old 3G disks.

If you get an HP H221, you might end up getting an 9205-8e card instead of the 9207-8e, they used both. The problem with the 9205-8e is with flashing the firmware. A guy has a blog where he figured out to use the P14 sas2flash utility to flash the 9205-8e with newer firmware.

Hope that helps.

 

[valvolainen]

Thanks,
I'm going for the 9220-8i, hopefully that'll give me what I want. Meanwhile, since the lsi card is far far away, I could play a bit with the HP P800 card, I've got a spare one and I'm using neither. Noticed there was an lsi chip there. Maybe I can use stock lsi firmware. Worth a try.
Exciting days!

 

[Ericloewe]

LSI SAS 9211s, 9207s and 9201s are readily available as 4i4e or 8e versions, at a premium, since they are niche products.

 

[valvolainen]

Hi all,
Got my 9220-8i in the mail yesterday, and decided to test it.
It worked, and my internal-to-external sas card fit as well. So, for the drivers.
I didn't find much on LSI' site regarding the 9220-8i, and during post, the bios for the card was a graphical one.
After much internal debate, I decided to try it with the 9211-8i sas2008 P15 drivers. I should have done this research beforehand, but you know...
The lsi flash program didn't complain at all during flashing, and I tried first without embedding the hba bios user interface, and after booting my newly installed freenas, all 25 drives came up. I could also do smart status on random selected drives.
Decided to add the bios user interface for the card, and I also had to program the sas address (what's this for? the card seemed to work without it..) So after re-flashing with sas address and all, no drives shoved after booting.
This was, however, due to erroneous patching of disk shelf. After finding the correct opening for the sas cable, voila! all 25 drives popped back in.

At this point I had to power down, the waf factor on these things are as you'd expect them to be.
_low_.
The rest of the week I have to empty a place in the toolshed and do some wiring.
Next week is the RAM and disk stress testing! (Home alone for a week!).

 

[Ericloewe]

You need P16 IT mode for FreeNAS 9.3 and P20 for 9.3.1

 

[valvolainen]

Done the flashing to p20, doing some initial testing and verifying.

 

[valvolainen]

So, I realize the topic title is slightly off now, but hey..

Anyway, I've set up my system, tried different things, tested a bit here and there.
I'm running raidz2 clusters of 6 drives. I figured this gives good redundancy and less money is needed to replace a vdev.

Yesterday and today I got a message about one of the drives in the array.

Sep 23 14:07:53 freenas (da12:mps0:0:20:0): WRITE(10). CDB: 2a 00 03 b3 0d 48 00 00 18 00
Sep 23 14:07:53 freenas (da12:mps0:0:20:0): CAM status: Command timeout
Sep 23 14:07:53 freenas (da12:mps0:0:20:0): Retrying command


From what I could understand this may be related to an error in the msa70 shelf. To test this I reseated the drive in another bay.
Waiting to see if the message returns, and if so I'll replace the chassis.
I've been thinking a bit about daisy chaining an msa60 on here as well. It's cheaper to work with in the long run, given the cost of 3.5" drives and the additional 'not-above-80%-fill-grade' makes the price difference bigger. We'll see, for now i'll have to bye 6 x 2tb 2.5 to make the cluster uable for the next year or so.

This is what happened during the timeout - note the total lack of data.

The peaks are about 840 mb/s, and the lower part of the graph is about 200mbit.


Things are looking good so far.
I'm trying to keep it in way that I can replace the server and/or server parts between os and drives without any hassle. I'm still on the right track.