Basic Question re: IBM M1015 Controller

[Steve Beschakis]

Hi, total noob here. Sorry if this is a thread that's been beaten to death, but my question is really very basic.
Just getting ready to complete the assembly of components for a FreeNAS server, and I'm wondering what exactly I will gain if I decide to crossflash and install the IBM M1015 I bought (oem version, no dox).
The main page of the FreeNAS documentation (Hardware Recommendations) is a little unclear. First, they tell you not to bother with a controller, then they say:

"If you need reliable disk alerting, immediate reporting of a failed drive, and or swapping, use a fully manageable hardware RAID controller such as a LSI MegaRAID controller or a 3Ware twa-compatible controller."

...sounds great, right? And then they steer you to a forum post which builds the case for a controller with claims like:

" If you get an M1015 and crossflash it to IT mode, you end up with one of the best HBA controllers available for FreeNAS, in my opinion. "

OK, so I included the M1015 in my parts list, and it arrived yesterday...nice PCI-E card, ready to pop in. Got the breakout cables, etc. I have the driver ready for the crossflash procedure, etc.. But before I do this, I'd like to know exactly why there seems to be a convergence of the FreeNAS community on this option. IOW, what exactly do you gain by doing it? Are the "reliable disk alerting, immediate reporting of a failed drive, and or swapping" features of the card's driver? How are they accessed and managed? Are people willing to introduce an addition point of failure for a performance gain? Even as a pass-through, the card isn't going to speed up throughput to an appreciable degree in a home user media archive scenario (lots of large, infrequently-accessed files), right?

Thanks. I appreciate any advice on this.

 

[Lordadmiral Drake]

As far as I understand it when used with the standard (RAID) firmware the controller abstracts the disks to the system, loosing some functionality in the process, while in IT mode (acting as plain HBA) it lets FreeNAS/ZFS directly access the drives as raw disks

 

[jgreco]

They're like your mainboard ports when used in IT mode. Except if you have a problem like cheap SATA ports on your mainboard, it is better. And if you don't have enough ports, it's a useful fix.

 

[Steve Beschakis]

Hmmm. Thanks for the replies! I might just forget the card, then. This mobo is supposed to be a decent server board (SUPERMICRO MBD-X9SCL-F-O LGA 1155 Intel C202 Micro ATX Intel Xeon E3) and I'm giving the system 16GB of ECC ram. So, if the only reason to use the IBM pass-through card it to correct mainboard deficiencies, I'll prolly just leave it out.
Thanks again!

 

[jgreco]

The X9SCL's onboard ports are perfectly fine for standard hard drives. Someone will inevitably point out that some are only SATA-II, which is irrelevant because a 3Gbps SATA port is still faster than current hard drives can handle. The moment you want to add beyond six drives, the M1015 is one of the best ways to do it.

 

[Steve Beschakis]

Ah..great. I'll simplify the current setup and build without the controller. Can't see growing beyond the useable space under RAIDZ2 with 6 X 4T HDDs, but you never know..especially with higher video rez standards emerging...
--cheers

 

[Steve Beschakis]

heh...the IBM card is going back in. I just finished a thorough chin-wag with several of the company's uber-SA's and Sysprogs who all basically insisted that there are sound data-integrity reasons for bypassing your mobo's SATA controller and letting IBM handle things, especially for a larger pool of drives. Maxing out the available channels on a mainboard is apparently not the best idea since you can't assume that they're all created equal. The way it was presented to me, going with the enterprise-grade controller in IT-mode is basically an insurance policy, and it doesn't introduce another point of potential failure, since you're completely bypassing the native controller. My head hurts.

 

[Steve Beschakis]

...basically, the guys at the office just confirmed what jgreco & others were saying, so thanks again for the replies, folks! :D

 

[jgreco]

Motherboard SATA ports are indeed not all created equal, but in the case of the X9SCL it isn't a big concern. On the other hand, your data is in good hands with an M1015. It is an enterprise grade HBA and works accordingly well.

 

[Lordadmiral Drake]

And what does "not created equal" mean for the average Joe?

Gesendet von meinem C6903 mit Tapatalk

 

[jgreco]

Crappy chipset (whatever the SATA equivalent of a Realtek ethernet is!) with a less-than-ideal driver could mean poor performance or bad error recovery or whatever.

 

[scurrier]

Crappy chipset might mean either or both of two things: poor performance or poor reliability. I can deal with poor performance, but not poor reliability.

Post #7 seems to suggest that there could be a reliability concern with using the onboard SATA on a server board. I don't think I know more than the average SA, but I really have to question whether this is a data-based claim of unreliability or if it is just a case of an IBM card being "cheap insurance" for an enterprise. It's not "cheap" to me.

What evidence do we have that, say, a typical SuperMicro server board would have an unreliable SATA chipset? If I could get away without the IBM card it would be nice. I want to cut cost, but not cut corners. I think it would really suck if we could not trust a "server grade" board with being "server grade reliable" on the sata chipset.

 

[cyberjock]

First, there is no "typical supermicro board" here. Tons of different boards, some with Intel SATA, others with LSI SAS, still others are AMD solutions with Marvell and other brands I can't think of off the top of my head. Intel SATA seems to be pretty good. LSI SAS seems to be pretty good assuming it's compatible with FreeNAS and assuming it's got a jbod mode. As for the other brands and models, it's very hit and miss. Just check out my sticky on Highpoint controllers. They range from non-functional to "can be hacked to make it work" to "just a very very bad idea for ZFS".

So as a person with the knowledge and ability to write stickies on hardware I can either test all the hardware myself and come up with a list, or I can find 1 or 2 models that definitely work well(and hopefully are inexpensive).

Generally, I recommend Intel based server grade Supermicro boards. That means you will have Intel SATA onboard. That's a great start. I don't even have to tell you to buy a board with Intel SATA because it WILL come with it.

We've got stickies discussing a few other model of cards that have never failed anyone. By far, the least expensive on a per-SAS port is the M1015. The M1015 offers 8x6Gbps SAS ports on a PCI-express version and number of lanes that will no bottleneck 99.99% of users, supports >2TB drives, and has been rock solid in performance and reliability. To boot, it's basically *the* cheapest 8 port controller you're going to find that has all of those things. There's just no other controller that offers all of those things for little over $100.

Another thing to keep in mind scurrier is that ZFS really works best with huge pools. If you want to do a 2 disk mirrored pool, the costs for ZFS is rather high. Gotta buy server grade stuff, ECC RAM, etc. Quite often it's cheaper and simpler to just make a mirrored RAID on a Windows 7 machine and share it from a desktop. ZFS was designed specifically for huge pools, period. There was no thought to cost at all. It was going to be an ultra-expensive enterprise class solution. And that's exactly what it is. Many people can build a ZFS server for less than an Oracle machine. That's fine and dandy. But I think you should reevaluate your expectations for ZFS in relation to the intended crowd that ZFS was designed for. The people that go with ZFS aren't doing it because its cheap, or fast. It can be made to be fast, but it can also be outrageously expensive to accomplish that. They're doing ZFS because they want a relatively cheap server with corruption protection that isn't offered anywhere else. And they're willing to pay extra for that feature. They'll buy that M1015, they'll buy large amounts of RAM, they'll spend $2k+ on SSDs. It's all up to the server admin to decide if they want to buy products that work with ZFS. Some will and some won't.

 

[scurrier]

cyberjock,
Once again thanks for an awesome informative post.

I plan on going with an i3-4130, so will definitely have an Intel-based motherboard. Supermicro appears to be a solid choice and I noticed the X10SL7 looks to be very good for a FreeNAS box since it has 6x SATA and 8X SAS on the LSI 3208. But I'd have to cost compare that board to a similarly-equipped board without the SAS controller and instead adding the IBM M1015 card. Maybe it's basically a wash on cost. At which point the only drawback of the M1015 would be the extra power draw, which might even be comparable to the LSI, not sure. The M1015 would have the advantage of being able to move to another system if it became necessary in the future.

I understand what you're saying about ZFS being designed without concern for cost. I'm willing to spend money on what's necessary or required to ensure a good system. But I'm not willing to waste money. I think of "waste" as cost that is not proportional to the benefit provided. That's what I'm trying to weigh and your post really is helping me with that.

re: whether or not ZFS is appropriate for me. I want to be in the 8-16TB range with this system. The reason why I chose ZFS is because I have suffered data corruption on systems before and want to ensure that my new NAS is robust with respect to corruption. As far as I know, a mirrored windows setup will not protect me against corruption. The mirrored windows setup was actually my first plan, until I realized it has really poor reporting and does not protect against corruption. It would have simplified permissions management on my mostly-windows network, but the feature is not really the serious server-grade solution that I am looking for.

If I lose any home videos or pictures, my wife will want to kill me. And I will be an even worse critic than her. That's why I want to spend the time to learn and do it right.

 

[Z300M]

cyberjock,
Once again thanks for an awesome informative post.

I plan on going with an i3-4130, so will definitely have an Intel-based motherboard. Supermicro appears to be a solid choice and I noticed the X10SL7 looks to be very good for a FreeNAS box since it has 6x SATA and 8X SAS on the LSI 3208. But I'd have to cost compare that board to a similarly-equipped board without the SAS controller and instead adding the IBM M1015 card. Maybe it's basically a wash on cost. At which point the only drawback of the M1015 would be the extra power draw, which might even be comparable to the LSI, not sure. The M1015 would have the advantage of being able to move to another system if it became necessary in the future.

From memory, I think the X10SL7-F is no more than $80 more than a far more basic Supermicro board and an even smaller price premium over some of the fancier Supermicro boards; but you are unlikely to find an M1015 for less than $100. And if you need even more SATA ports later, you can still install an M1015 in an X10SL7-F.

 

[JimBean]

cyberjock,
Once again thanks for an awesome informative post.

I plan on going with an i3-4130, so will definitely have an Intel-based motherboard. Supermicro appears to be a solid choice and I noticed the X10SL7 looks to be very good for a FreeNAS box since it has 6x SATA and 8X SAS on the LSI 3208. But I'd have to cost compare that board to a similarly-equipped board without the SAS controller and instead adding the IBM M1015 card. Maybe it's basically a wash on cost. At which point the only drawback of the M1015 would be the extra power draw, which might even be comparable to the LSI, not sure. The M1015 would have the advantage of being able to move to another system if it became necessary in the future...

From memory, I think the X10SL7-F is no more than $80 more than a far more basic Supermicro board and an even smaller price premium over some of the fancier Supermicro boards; but you are unlikely to find an M1015 for less than $100. And if you need even more SATA ports later, you can still install an M1015 in an X10SL7-F.

Just a quick inquiry about the X10SL7-F, does it have 8 SAS ports or 8 SATA ports? As in, would I need a SAS break-out cable to connect a HDD to them, or just regular SATA cables?

I am, like OP, just starting a build and interested in what sort of controller (if any) I should get. Thanks.

 

[jgreco]

SAS and SATA use a similar connector for single channel connections - you can plug a SATA drive into SAS and it should work. The SFF8087 or 8088 you are probably thinking of as a "SAS" connector is actually a multilane cable. Just to be clear.

 

[JimBean]

SAS and SATA use a similar connector for single channel connections - you can plug a SATA drive into SAS and it should work. The SFF8087 or 8088 you are probably thinking of as a "SAS" connector is actually a multilane cable. Just to be clear.

Oh okay, thank you very much.

 

[Z300M]

Just a quick inquiry about the X10SL7-F, does it have 8 SAS ports or 8 SATA ports? As in, would I need a SAS break-out cable to connect a HDD to them, or just regular SATA cables?

I am, like OP, just starting a build and interested in what sort of controller (if any) I should get. Thanks.

The X10SL7-F has the same kind of connectors for the SAS ports and the "regular" SATA ports:

http://www.supermicro.com/products/motherboard/Xeon/C220/X10SL7-F.cfm

The SAS connectors are the blue ones at the front left.

I just ordered an X10SL7-F.