Heads-up: Fake Intel i350 GbE adapters in the wild

[Ericloewe]

It seems the latest server product to be the subject of cloning is Intel's i350 line. Both 2-port and 4-port cards seem to be available as fakes.
Naturally, acquiring Intel cards through validated channels or known-good rebrands is the safe option.

These seem to be elaborate counterfeits, cost-reduced by replacing certain critical components (notably the PHY isolation transformers) with cheap clones. Reliability seems to be rather low, with failures being typical after few months.

More details here: https://forums.servethehome.com/index.php?threads/comparison-intel-i350-t4-genuine-vs-fake.6917/

In an unrelated note, the i350 line has been replaced with the i350v2 line, which fixes an abnormal inrush current at startup.

 

[adamjs83]

Ughhh, thanks for posting, I think I have one of these in my nas and another in my esxi host.

 

[jgreco]

Naturally, acquiring Intel cards through validated channels or known-good rebrands is the safe option.

That won't save you. About ten years ago, I had a problem while onsite at a data center and had NewEgg FedEx me some dual port Intel cards. Came back from the east coast, and couldn't get them to come up. Six months later at the next site visit, I had the cards pulled and sent back here. Fakes. NewEgg wouldn't admit it but there were other people who had reported getting fakes from them as well.

 

[Ericloewe]

That won't save you. About ten years ago, I had a problem while onsite at a data center and had NewEgg FedEx me some dual port Intel cards. Came back from the east coast, and couldn't get them to come up. Six months later at the next site visit, I had the cards pulled and sent back here. Fakes. NewEgg wouldn't admit it but there were other people who had reported getting fakes from them as well.

Phew, that's a nasty experience. I guess it's another horror story about Newegg for the archive.

 

[jgreco]

Well, NewEgg's actual answer was that it was past the window on returns. So I have some very expensive examples of pretty good counterfeit cards, with one obvious sign: the box that the cards came in, on the printed-label end, say "DAUL" rather than "DUAL". Oh yes there it is, I posted about that a few years back complete with picture.

https://forums.freenas.org/index.php?threads/new-intel-nic-no-connection-what-next.8424/#post-36646

Unfortunately it's very difficult to get esoteric stuff shipped overnight to a data center from most distributors or even most retailers, so NewEgg (with their ships-today capability) remains a pretty compelling option for on-site crisis resolution. When the data center's 800 miles away from our workshop here, that's a big thing...

 

[wreedps]

A big give away for fake cards is bad soldering. I deal with fakes with Cisco stuff everyday. I would assume Intel would be similar.

 

[jgreco]

A big give away for fake cards is bad soldering. I deal with fakes with Cisco stuff everyday. I would assume Intel would be similar.

Some of them are just fine, and the suspicion is that in some cases these may actually come off the same production lines (think: rejects). The bad soldering thing isn't as prevalent as it once was.

 

[wreedps]

Heard of any problems with Chelsio or Mellanox fakes?

 

[jgreco]

Not offhand. The usual target is high dollar name brand stuff with readily available silicon. For awhile there seemed to be issues with knockoff IBM ServeRAID M1015's (and presumably other OEM 9211's). Intel's always been problematic. But I'd expect Chelsio and Mellanox are less tempting targets as they're specialty cards, and it is less likely that counterfeit silicon would be easily acquired.

Just an educated guess though.

 

[wreedps]

Sounds accurate to me, thanks!

 

[jgreco]

Don't know about accurate, but it does help to contemplate what the motivations are in these things. Basically it comes down to what sorts of things might sell well in some Shenzhen back alleyway, and whether or not it is practical to actually acquire the stuff to make the knockoffs to begin with. For something like an ethernet chip or a HBA controller, it probably doesn't require as much to do that as it would a Xeon CPU.

So if you look at stuff like the Prolific PL2303, a common serial-to-USB chip that is widely counterfeited, it becomes pretty obvious that this is a real problem for manufacturers of silicon, especially simplistic silicon like the PL2303. Prolific attempted to cope with it by introducing checks for counterfeit hardware signs in their driver, but that's probably a losing strategy in the long run. That the counterfeiters are able to strip the packaging off the silicon, and duplicate the silicon of a trivial device like the PL2303 is bad enough, but it seems clear that the Heisenbergs of the illicit silicon business have probably moved up to the more profitable stuff.

 

[Ericloewe]

So if you look at stuff like the Prolific PL2303, a common serial-to-USB chip that is widely counterfeited, it becomes pretty obvious that this is a real problem for manufacturers of silicon, especially simplistic silicon like the PL2303. Prolific attempted to cope with it by introducing checks for counterfeit hardware signs in their driver, but that's probably a losing strategy in the long run.

Wasn't that FTDI? They even bricked counterfeit chips by screwing with their PCI configuration stuff.

 

[jgreco]

No, Prolific. FTDI probably did this too. The problem isn't limited in scope after all. The newer Prolific drivers won't work with many generic (i.e. counterfeit) Prolific adapters.

 

[Bidule0hm]

Wasn't that FTDI? They even bricked counterfeit chips by screwing with their PCI configuration stuff.

Yep, FTDI has earned a very bad reputation after doing that because a ton of products stopped working (including important industrial products).

 

[jgreco]

Sure, but the point I was making was that the low end counterfeiters are ripping off Prolific and that it apparently doesn't take a ton of talent. As perhaps THE most widely available RS232-USB chipset, I can tell you it sucked when supposedly-Prolific-based devices from fairly reputable vendors wouldn't function with the Win8 drivers, where Prolific had added driver checks. FTDI is probably a step up from that. Intel and LSI are an order of magnitude more complicated than that. Yet there are copycats even of those, presumably from those with more "leet" skills.

Getting back to the motivation thing, if you can sell Prolific's chips to people building Prolific based devices at a reduced price, you might still be able to score some reasonable profit if you're putting out sufficient quantity. You didn't have to bankroll R&D or driver development - just copy the chip and run with it.

But if you then look at the market, flooded with a cheap RS232 converter, you're making it difficult both for the ripped-off company and also for any competitors (FTDI or FTDI clones) to make much money. It is a business of slowly killing the goose who laid the golden (silicon?) egg. So I'm guessing it is one rogue fab somewhere cranking out fake Prolific like there's no tomorrow.

So the more talented, instead of seeking a smaller portion of an already low-margin pie, are chasing after the bigger fish like Intel and LSI. The cards you get through the typical HP/IBM/Dell/Supermicro sales channels will of course be legit, but there are lots of companies who order preconfigured "this week's great deal" servers and then maybe swap in a few extra parts, many of which are purchased through the retail channels, or worse, eBay.

This is why I've usually told people to avoid cards coming from the Far East (could be legit, maybe not, but a PAIN to deal with if not!) and instead try to source things that are here in the US, especially server pulls. It's all a matter of likelihood. I've gotten fake cards through the retail channel from a supposedly-reputable vendor. I've gotten lots of valid gear through the risky eBay market as well (though in fairness there I have enough longevity and credibility as a buyer that it isn't a problem getting a refund).

 

[Bidule0hm]

I was just agreeing with Eric.

Checks --> ok, no problem with that; Bricking on purpose --> not ok at all.

Now if I ever need a USB/RS232 converter I'll chose anything but a FTDI chip. Remember Kingston big fails with RAM, SSDs, ... now we just avoid Kingston products, it's as simple as that.

 

[jgreco]

Now if I ever need a USB/RS232 converter I'll chose anything but a FTDI chip. Remember Kingston big fails with RAM, SSDs, ... now we just avoid Kingston products, it's as simple as that.

So obviously you're not buying Intel CPU's either, then, so I'm just kinda wondering what's left. You running ARM CPU's? Seriously, you can't name a major company that hasn't had a misstep and hasn't sold some bit of crap hardware.

Further:

Introducing protections that brick counterfeit merchandise is probably fair play. If I send a fake Intel ethernet card to Intel for RMA, I don't get it back (bricked or otherwise). I don't see this as functionally much different than FTDI bricking counterfeits. Either way you get to complain to the vendor who in theory will make you whole (but, of course, not in practice). I'm still left with a bad taste in my mouth for NewEgg because they sold fakes, and then used their return window policy to evade responsibility for addressing the issue.

 

[Bidule0hm]

Yeah but there's fail and fail... and there's also the fact that you have plenty of other serial converters or RAM makers but only one for CPU (AMD) and it's not perfectly equivalent (ECC, etc...) so you don't have much choice with Intel.

Introducing protections that brick counterfeit merchandise is probably fair play.

Yep but the problem with the serial converter is that some product makers weren't even aware that they used fake chips, it wasn't their choice and before they even had a chance to correct the problem after discovering it the products were already bricked.

It would be barely ok if they put a big warning in the driver update that it'll brick any counterfeit chip, but they didn't.

 

[jgreco]

Okay, but this is basically irrationality on your part, in my opinion.

If product makers weren't aware that they used fake chips AND they were willing to correct the problem, that means they'd have to replace the chips, which means that they'd need to have the product RMA'd to them to have the chip desoldered and replaced. In that case, bricking the product doesn't prevent them from doing that replacement. It is probably damn inconvenient to the customer, yes. But counterfeiters are always screwing things up for someone. But this isn't as big a problem as you imply, since the RMA can still be done and remediation is accomplished.

So I'm going to go out on a limb here and say that the problem you really find offensive is that in many cases it was the end user left hanging, because many product makers had no intention of remediating the issue. The user updated a driver and POOF the hardware is toast. End user bought the device a year or two or three ago, maybe via mail order, possibly a generic label product, and there's no one (besides, perhaps, the user's cheap self) to hold responsible. That's frustrating because at least with what happened with Prolific, the user might think the "solution" is to swear at Prolific and figure out how to get the older Windows driver rigged to work. But you really don't have a right to expect to be able to use ANY of Prolific's intellectual property to support your use of counterfeit goods. That's a key point here. Just because you CAN roll back to an earlier Prolific driver does not make it ethically or morally right to do so; they created their driver and distributed it for free for the benefit of people who had purchased chipsets that they manufactured. Using it on a countefeit chip is just a form of theft.

Since the FTDI countefeit chips cannot be used without a driver from FTDI, the same thing is effectively true there. The difference is that they are actually actively bricking counterfeit devices. But it wasn't even quite that; they were simply removing their vendor ID from the counterfeit devices (which is, after all, a reversible thing).

So this really comes down to a bit of a dilemma. If you're an end user, you are understandably all pissed off and incredibly frustrated when your Prolific or FTDI device stops working on an updated driver. You're thinking, "Damn! I paid good money for this cable!" But upon discovering that it is counterfeit and that you are stealing someone else's intellectual property to make your crap hardware work, what do you do next?

Do you:

A) Insist that you have a right to continue to use your counterfeit hardware, which means that you must then steal Prolific's/FTDI's intellectual property, and become complicit in the counterfeiting crime, or

B) Decide that you need to make it right by purchasing a genuine product and then gaining legitimate access to the driver intellectual property?

I would note that there's no third path here. There is no case that says you can keep using your counterfeit hardware without being complicit in damaging the company whose gear is being counterfeit. In the specific case of FTDI, it was probably too-aggressive a move to have the driver remove their USB ID from counterfeit devices, but really it doesn't change the basic issues.

Put differently:

Our electronics shop builds servers. You can buy a server that runs FreeNAS from us. This is totally above-board.

What happens if we steal a copy of TrueNAS and put it on a server that we then sell to you, not telling you that it was not an authorized copy? That's theft, right?

So if iXsystems then releases a firmware update that is unknowingly incompatible, let's say because they dropped support for an ethernet chipset that they had never shipped in a TrueNAS box but that we had shipped in our TrueNAS-ripoff, would they be liable?

What about if iXsystems releases a firmware update that validates the chassis serial number against a list of systems that iXsystems is known to have built, and refuses to run if the chassis doesn't match?

At what point would you say iXsystems has gone wrong here?

 

[Bidule0hm]

which is, after all, a reversible thing

Yeah, but not by the usual john doe user.

A) Insist that you have a right to continue to use your counterfeit hardware, which means that you must then steal Prolific's/FTDI's intellectual property, and become complicit in the counterfeiting crime, or

B) Decide that you need to make it right by purchasing a genuine product and then gaining legitimate access to the driver intellectual property?

I'd chose B) but in the mean time I have a bricked product and if it's part of something important, like my business for example, I have downtime.

What about if iXsystems releases a firmware update that validates the chassis serial number against a list of systems that iXsystems is known to have built, and refuses to run if the chassis doesn't match?

No problem, as long as it doesn't brick my chassis if it doesn't match.