Looking to get 10 GB networking, somewhat confused on my options. Need x4 pcie

[NickF]

The other problem is that the Mellanox stuff, especially the older CX-2 and CX-3 stuff, does not implement as much offload in silicon as other cards such as the Intels or Chelsios. Getting driver support to line up with what the silicon theoretically supports has been an ongoing problem.

That's why I am saying - they have come a long way, and have surpassed Intel and Chelsio in many of those respects. Have you seen Bluefield-2 as an example?

 

[Davvo]

scale was more compatible with hardware.

Not really.

 

[jgreco]

That's why I am saying - they have come a long way, and have surpassed Intel and Chelsio in many of those respects. Have you seen Bluefield-2 as an example?

I do not consider not having a feature to be surpassing Intel and Chelsio. If we are going to compare vehicles, let's not compare a 2000 Ford F150 (gas guzzler) to a Tesla Cybertruck (not even available). An Intel X520 (Sep 2009) is arguably similar to a Mellanox ConnectX-2 (Aug 2009) but does a better job of interrupt coalescing, supports VLAN offload, and I believe there's some other offload crap that the CX-2 is really weak on.

I don't really care about Bluefield-2 as I'm not going to spend thousands of dollars on a single network card, or tens of thousands on the ethernet switch to plug it into. If I needed 100G, I'd go get an E810-CQDA2 for $800. I have no idea what the extra money spent on Nvidia would get me other than to tick me off by enriching a company that has been, in my opinion, an opportunistic money hole for the world to throw excess cash at.

 

[NickF]

Fair enough....

 

[jgreco]

at the time i didnt know that scale was more compatible with hardware.

Well, it wasn't at the time, and it isn't really now either. It's more compatible with crap hardware that you can't make good use of on a server. It's more compatible with bargain basement bin you might find useful on your desktop. So the compatibility factor of Linux really isn't all what its mad zealot enthusiast users like to make it out to be.

 

[ncc74656]

I started this project on a bit of a budget. I had looked at sysology for their network storage arrays and found the processors they had were kind of lacking, if I ever wanted to expand storage it became pretty expensive to buy their caddies and everything. So I looked at one of their solutions and took that budget to be what I would try to build this in.

So far I would say I'm fairly successful. I have a 12th gen Intel with built-in quick sync and 64 gigs of RAM, with two pools of storage that all in all came out to be cheaper than sysology. But, now I'm running into issues with obviously figuring out how to do networking stuff and I also want to expand it to more pools but, I don't have the PCI Express openings nor do I have enough ports on my LSI cards anymore. So I guess I'm trying to figure out how to use a back plane? Or a sas expander? Something like that.

Bottom line is, this has been a DIY adventure that so far has worked out in my favor financially. But it's not without its hurdles

 

[jgreco]

So I guess I'm trying to figure out how to use a back plane? Or a sas expander? Something like that.

Suggest you read the thread at

Can my current system handle my new hard drives?

A few years ago I replaced my original 8x3TB drives with the current 8x10TB drives (in RAIDZ2)... I wasn't thrilled having spent a shit load of money to end up with a pool that was already 25% full but I was hopeful it would take me a while to fill the space. I was not expecting an alert warning...

www.truenas.com

starting around maybe message #12 and then also the SAS Primer at

Don't be afraid to be SAS-sy ... a primer on basic SAS and SATA

With the introduction of SAS 12Gbps, seems like "it's time" to do a braindump on SAS. Work in progress, as usual. History By the late '90's, SCSI and PATA were the dominant technologies to attach disks. Both were parallel bus multiple drop...

www.truenas.com

 

[ncc74656]

Kool, thanks