M1015 doesn't see drives - then it does

[RedBear]

So I've got this new ThinkServer TS440 (YUX, the one with eight 3.5" drive bays built-in). I put an old 500GB SATA drive I had just pulled from an Apple Time Capsule in one of the bays and it was detected just fine, as displayed during the little boot-up BIOS dialog. Worked fine too, I installed Elementary OS Freya on it as a test. That was with the built-in LSI 9240-8i "RAID500" card.

Just to see if it was possible, I tried to cross-flash the built-in LSI 9240-8i card to IT mode. No DOS or EFI flashing utilities would even acknowledge that any LSI card was present in the machine with the RAID500 installed. I went ahead and replaced it with an IBM ServeRaid M1015 from eBay that was pre-flashed to Phase 19 firmware, IT mode (with the BIOS ROM still installed).

Since this will be my first FreeNAS system I had ordered 10 of the smallest, cheapest decent SATA drives I could find for use as a test-bed / playground so I can make all my initial mistakes on what are basically throwaway drives. They're 120GB Seagate SATA drives that were only about $15 apiece. I went ahead and filled up the remaining 7 hard drive bays once the requisite drive caddies arrived, leaving the 500GB drive installed. Then I booted up FreeNAS, connected to it and did the initial configuration, and tried to create a ZFS volume.

At this point I was surprised to note that the only disk FreeNAS seemed to be seeing was the 500GB drive I had initially installed. No other disks were visible in the GUI. I was all like, "WTF? Did I just waste $150 on SATA drives that are too old to be compatible with the M1015?" But then I was like, "Nah, SATA connections are all backwards compatible back to SATA-I, just like USB, right?" Anyway, FreeNAS was whining about the M1015 firmware version (19) not matching the driver version (16), so I decided to deal with that problem first and then see what's up with the drives not showing up.

Now, I hadn't been paying attention before or I would've noticed that none of the new 120GB Seagates showed up on the M1015's boot-time initialization screen either. Which would seem to explain why FreeNAS didn't see any of them. I noticed now, but went ahead and erased and reflashed the M1015 with the proper Phase 16 IT firmware, _without_ the boot rom since most of the instructions I found just leave it out. Without the boot rom there's no more initialization screen, so I couldn't see what the M1015 thought was attached, but when I booted up FreeNAS lo and behold all drives were now visible! And of course I was able to create a ZFS volume and a share and start playing around. I had even replaced the 500GB drive with another 120GB Seagate just so they would all match (yes I know that's not strictly necessary).

All working now, fantastic, right? But when I don't understand why things happen it's always worrisome. As far as I know the card was in IT mode the whole time, so why would only the 500GB former Time Capsule drive show up at the boot rom initialization screen? Was it the boot rom that was somehow interfering? If I flash the mptsas2.rom back to the card will all my drives suddenly disappear again?

I'd appreciate any input from those more experienced with these cards. Is there some reason to consider this normal behavior for an M1015?

 

[jgreco]

There's no particular reason to consider that normal.

The BIOS thing is a matter of preference, and some people have reported various issues with the BIOS. I personally prefer to see the manifest list and haven't had problems with it, and I do not like needing to boot into the OS to check the attached drives, so our cards usually have the BIOS.

First thought that comes to mind: go verify that your SAS-8087 is fully inserted and click-locked into place. Ditto drives. I mean, fine, I understand your problem doesn't *seem* like it would necessarily be that, but if something is intermittent, "check the cables."

But let me congratulate you - you sound like you're taking the time to experiment and learn about your hardware platform BEFORE going further. So absolutely experiment and identify the cause of this issue. It could be a BIOS thing. Could it be a knock-off M1015? Etc. Hard to know, but some experimentation will probably get you some results.

 

[RedBear]

There's no particular reason to consider that normal.

The BIOS thing is a matter of preference, and some people have reported various issues with the BIOS. I personally prefer to see the manifest list and haven't had problems with it, and I do not like needing to boot into the OS to check the attached drives, so our cards usually have the BIOS.

First thought that comes to mind: go verify that your SAS-8087 is fully inserted and click-locked into place. Ditto drives. I mean, fine, I understand your problem doesn't *seem* like it would necessarily be that, but if something is intermittent, "check the cables."

But let me congratulate you - you sound like you're taking the time to experiment and learn about your hardware platform BEFORE going further. So absolutely experiment and identify the cause of this issue. It could be a BIOS thing. Could it be a knock-off M1015? Etc. Hard to know, but some experimentation will probably get you some results.

The sentiments are well-received, thanks.

Yeah, ever since the days of those obnoxious Parallel ATA ribbon cables that gave me trouble I've been a stickler for making sure data and power cables are always inserted, removed and reinserted at least twice before use. With a little final wiggle to boot. I like to feel like I've given the two metal surfaces the best possible chance to rub off any microscopic defects or "tin whiskers" so they can wind up with good solid metal-to-metal contact and maximize the probability of long term reliability. Same with the SATA drives. I feel pretty confident about the physical connections. Click-locked for sure.

There's also no sign of intermittency besides "it didn't work before the reflash, and now it does". I wouldn't know exactly what to look for to determine if it's a knock-off card, but it looks virtually identical to the LSI 9240-8i that came with the TS440. All the appropriate labeling and a SAS address in the standard 500605Bxxxxxxxx range.

I actually have two of the M1015s, but stupidly I went ahead and reflashed them both at the same time, since I was successful with the first one after an extraordinary amount of hoop jumping to get sas2flash to run on this motherboard. What I should have done is boot from the reflashed one to see if it worked, then swap it for the other one that was still flashed to P19 with the boot rom. I think I will experiment with flashing back the boot rom and maybe even try to replicate the original conditions if I can track down the P19 files. But since I didn't do the original flashing I can't be sure of getting back to the original state with absolute certainty.

In short: Argh.

 

[Ericloewe]

, but it looks virtually identical to the LSI 9240-8i that came with the TS440. All the appropriate labeling and a SAS address in the standard 500605Bxxxxxxxx range.

That's normal, since they're essentially the same card.

 

[jgreco]

Where'd you acquire the M1015's from?

I want to say that the M1015 was being put in IBM servers between 2009-2010. So there are two things that you see: one, at that time, a ton of M1015 on the market, and then, two, about three years later, another ton of them. These correspond to the new servers getting cards pulled and getting upgraded with "real" RAID cards, and the decommissioning of off-lease servers. During that second window, in particular, it seems that some enterprising back alley Shenzhen operation released a crapload of knockoff LSI 9240-8i's on eBay that looked more or less like M1015's. There's even been some debate as to whether sometimes these are rejects off the actual LSI production assembly lines.

TL;DR: if your card came from the USA it is much more likely to be a legit pull from a decommissioned server. If it came from China (or HK or etc) it is more questionable.

 

[RedBear]

That's normal, since they're essentially the same card.

Which makes it all the more strange to me that sas2flash screams "There's no LSI card here!" when the original card is in the system.

Where'd you acquire the M1015's from?

I want to say that the M1015 was being put in IBM servers between 2009-2010. So there are two things that you see: one, at that time, a ton of M1015 on the market, and then, two, about three years later, another ton of them. These correspond to the new servers getting cards pulled and getting upgraded with "real" RAID cards, and the decommissioning of off-lease servers. During that second window, in particular, it seems that some enterprising back alley Shenzhen operation released a crapload of knockoff LSI 9240-8i's on eBay that looked more or less like M1015's. There's even been some debate as to whether sometimes these are rejects off the actual LSI production assembly lines.

TL;DR: if your card came from the USA it is much more likely to be a legit pull from a decommissioned server. If it came from China (or HK or etc) it is more questionable.

Came from a US seller. With a nice note attached warning me not to plug the cards into a regular PCI slot backwards as that would destroy the card. (They came without brackets attached, just as advertised.) After not finding any suitable looking full/standard height PCI brackets on Amazon, I ordered a couple off eBay that were specifically advertised as being for the M1015. Those came from Taiwan. But not the cards. I'm quite leery of buying any sort of important or expensive electronic items from outside the US. (Or from New York/New Jersey, for that matter. Same issue, "enterprising back alley".)

 

[jgreco]

Ok, just checking. The host 9240 ... not sure.

 

[Robert Trevellyan]

Since this will be my first FreeNAS system I had ordered 10 of the smallest, cheapest decent SATA drives I could find for use as a test-bed / playground so I can make all my initial mistakes on what are basically throwaway drives.

I just want to say, this fills me with joy!

 

[RedBear]

I just want to say, this fills me with joy!

LOL. You can't truly understand something until you figure out all the different ways to break it, I always say.

 

[Robert Trevellyan]

LOL. You can't truly understand something until you figure out all the different ways to break it, I always say.

I just think your whole approach is so refreshingly sound.

1. Let me buy an off-the-shelf box that will do a perfectly fine job of hosting FreeNAS for a sensible timeframe.
2. Let me not assume that I'll get it right first time.
3. Let me not expect someone on the forum to give me a recipe for optimizing simultaneously along the three orthogonal axes of cost, performance and capacity (OK, I'm putting words in your mouth with this one, let's just say I have some pet peeves).

 

[RedBear]

I just think your whole approach is so refreshingly sound.

1. Let me buy an off-the-shelf box that will do a perfectly fine job of hosting FreeNAS for a sensible timeframe.
2. Let me not assume that I'll get it right first time.
3. Let me not expect someone on the forum to give me a recipe for optimizing simultaneously along the three orthogonal axes of cost, performance and capacity (OK, I'm putting words in your mouth with this one, let's just say I have some pet peeves).

Yeah, well, that pretty much sums it up. Nice to have my singularly reductionist, entropy-paranoid, keep-it-simple, Murphy's Law approach appreciated for once. I'm looking long term here, trying to build something that will serve first as a primary file server and then as a backup for the next 10, 15, even 20 years if possible. By inexplicably finding a YUX model TS440 (with an E3-1245v3 with AES-NI, 8 hot-swap bays built in and support for 32GB of ECC RAM) for just $399, I think I really lucked out on finding a decent low-cost first FreeNAS rig. Even after adding the RAM and the M1015 card and a second redundant power supply, the total cost for the base system is around the same as a vastly inferior Synology DS1815+ (their cheapest 8-bay option).

Since just trying to identify the best/correct CPU cooler for a given CPU/RAM combination was already making my head spin, I knew attempting DIY right away was a very bad idea. Plus the motherboards I was looking at were already more expensive than the barebones ThinkServer. Which is... very weird to me. I just know I'm not looking at the right motherboards and CPUs and chassis/cases, because my rough estimates for a bare system capable of giving me ECC RAM and 8 hot-swap bays were always winding up around $1,500-$2,000. That's before hard drives, of course. I need to learn a lot more about what's available before I start bothering people with targeted questions.

By the way, even with the limitation of 32GB RAM and the bizarre contortions required to get the LSI flash utilities to run on the TS440 motherboard (don't even ask), I think I would still recommend this as a great off-the-shelf way to get a low-cost FreeNAS box with 8 hot-swap bays. For Pete's sake, it's like $300 cheaper than a 4-bay FreeNAS Mini even before you upgrade the Mini from 16GB to 32GB! The TS440 (YUX) is my answer to looking for an affordable pre-assembled system on the market in between the Mini and things like the 12/16-bay "FreeNAS certified" boxes from iXsystems.