Expand or not to expand, that is the question

[AVB]

(EDIT) I decided to go with 3HBAs so you don't have to read any further unless you want too.


I just bought an 846TQ (like an hour ago) and am trying to decide if 3 HBAs or an HBA and an expander card is the way to go. I like the idea of 3 HBAs (Dell Perc H310) because they will have 6GB connections (and I already own 2 of them) but just having less things internally and generating less heat has benefits too. A 12GBs SAS expander which would allow Sata III connection is not in the budget. My understanding is that the inexpensive HP expander cards floating around will only do 6GB when hooked to SAS drives and not Sata drives. The plan is to populate with 20 7200 RPM 3TB drives (I have 18 already all of which are Sata III)

Any other pros and cons of going one way over the other?

 

[jgreco]

"6GB" connections? Do you mean 6Gbps? "B" means bytes.

I would point out that the very fastest hard drives today are capable of pushing about 225MBytes/sec, or around 1.8Gbits/sec.

You've already bought the train wreck model chassis ("TQ") so at this point an expander isn't all that helpful. I believe the PMC Sierra expander is limited to 3Gbps for SATA in the manner you describe, so that's basically just another disrecommendation for going that route.

 

[AVB]

"6GB" connections? Do you mean 6Gbps? "B" means bytes.

I would point out that the very fastest hard drives today are capable of pushing about 225MBytes/sec, or around 1.8Gbits/sec.

You've already bought the train wreck model chassis ("TQ") so at this point an expander isn't all that helpful. I believe the PMC Sierra expander is limited to 3Gbps for SATA in the manner you describe, so that's basically just another disrecommendation for going that route.

Yes, 6Gbps. I would differ that the TQ is a "train wreck" especially for the reason given. Just because you can't deal with figuring out which cable is bad doesn't mean the rest of us can't. It takes some planning and preparation which you seem to have the ability to do so I don't know why this would elude you. The cables that I already use in my 16 bay server with forward breakout cables have latching connectors so using that as a point of contention is really moot. Is multilane easier? Certainly but not necessarily better, just ask all the people using SAS1 backplanes. For my home server that isn't on 24/7 and was never meant to be, TQ will work just fine. For production - then you have a point.

Now do you have a preference of 3 HBAs or an expander or do you just want to poke at my equipment choices?

 

[jgreco]

Yes, 6Gbps. I would differ that the TQ is a "train wreck" especially for the reason given. Just because you can't deal with figuring out which cable is bad doesn't mean the rest of us can't. It takes some planning and preparation which you seem to have the ability to do so I don't know why this would elude you.

When you're dealing with 11 systems per rack and rows of racks, minor issues become major quickly. Unnecessary cabling is definitely problematic. Cables come loose for the stupidest reasons. Drive connectors on backplanes go bad and the backplane needs replacing. One multilane cable with two total connections trumps six breakout cables and thirty-plus individual connections every time.

Now do you have a preference of 3 HBAs or an expander or do you just want to poke at my equipment choices?

I do have a preference, but you've already made it clear that you're not really that interested in hearing it.

 

[AVB]

Giving a choice that is not one of the options is just indulging in how you think things should be. This isn't corporate nor production and if you want to talk about that then you would be on TrueNas boards and forums. Your 11 systems per rack is a red herring for home use and FreeNAS is home use which you seem to be forgetting in your responses. As I said for production you have a point and I agree with it. When and if I ever have need of a home server with the same requirements I'll take it under consideration.

 

[jgreco]

You asked if an expander or three HBA's was a better idea. I provided an answer that an expander was going to be a bad idea with the train wreck that is a TQ chassis; you're just adding more cables and more things to go wrong. You didn't like the answer. That's not really my problem. However, since you seem to be more interested in creating an argument, I'm happy to close this thread.

 

[JoshDW19]

Please make sure that we keep the thread on topic. If anyone here is getting heated or frustrated take a break and come back later. If there's nothing to add to the topic don't say anything :).

 

[AVB]

I'm happy to reopen this thread if anyone else has some thoughts on why 3 HBAs may be better or worse than a single HBA and an expander. The train is leaving the station and I'll have to make a decision one way or another fairly soon. Thanks.

 

[Valdhor]

In my particular server I replaced the SAS-1 backplane with a SAS-2 and used a single IBM M1015 HBA. Works great with my 12 x 4TB RAIDZ3 pool.

I'm not saying it's better or worse than 3 HBA's.

You may want to check your current backplane supports drives larger than 2TB.

 

[AVB]

In my particular server I replaced the SAS-1 backplane with a SAS-2 and used a single IBM M1015 HBA. Works great with my 12 x 4TB RAIDZ3 pool.

I'm not saying it's better or worse than 3 HBA's.

You may want to check your current backplane supports drives larger than 2TB.

The TQ backplane being a pass through design, based on everything I've found, does support greater than 2TB drives. An upgrade may be something to look at for the future but seeing that I only bought this 12 hours ago I'm going to wait to cross that bridge.

 

[ttabbal]

TQ works fine with >2TB and various speeds I use one for my home server. Unless you have a shortage of slots, 3 HBAs is cheaper and less complex.

I can see why some people don't like the TQ. If I were managing more than one machine, I would avoid them too. I might even pick up a backplane at some point just to simplify things.

 

[AVB]

TQ works fine with >2TB and various speeds I use one for my home server. Unless you have a shortage of slots, 3 HBAs is cheaper and less complex.

I can see why some people don't like the TQ. If I were managing more than one machine, I would avoid them too. I might even pick up a backplane at some point just to simplify things.

Good to know from a real user. I decided to go with the 3 HBAs and had just finished ordering it when your message popped up. I was looking for a new chassis for awhile and when I found this one for $150 I had to jump on it. TQ was perfectly acceptable for me for a home unit. I don't know anyone who has a rack of 11 in their house where they would have to agonized over wiring.

 

[Stux]

3HBAs uses up 3 PCIe slots. 1 HBA and an Expander uses 1 slot.

 

[AVB]

3HBAs uses up 3 PCIe slots. 1 HBA and an Expander uses 1 slot.

Good point and I thought of that although the HP expander would use a second slot but a backplane one wouldn't. Since I have 6 empty PCIe X8 slots that didn't matter much in this instance.

 

[Stux]

Many expanders (most? all?) don't actually use any lanes, the slot is just a convenient place to locate them and supply power from. If they have an auxillary power input (say a 4 pin molex connector), then you can place them wherever you want.

 

[Valdhor]

I bought my SAS2 4u 24 port backplane for $120 and (The recommended) IBM M1015 for $90. Three M1015's would have cost $270 plus the cabling. For me it was cheaper to go with a backplane upgrade.

 

[Ericloewe]

Many expanders (most? all?)

All.

 

[AVB]

I bought my SAS2 4u 24 port backplane for $120 and (The recommended) IBM M1015 for $90. Three M1015's would have cost $270 plus the cabling. For me it was cheaper to go with a backplane upgrade.

I've paid $50 for each of my Perc H310s. I think people are talking about expanders on a backplane while was talking about PCIe card style like the HP 36 port that is easily found. Moot now since I have the 3rd H310 on the way in.