Question about 10Gb cards

[faktorqm]

Hello! I have some questions regarding 10Gb cards because I never build before a network with this kind of SFP+ cards.

Currently I'm on a budget, and after take some time reading several forum posts and experiences regarding this cards, the ix system's reccomendations, and other blogs (with a dubious reputation) I have some questions.

My goal: Communicate a FreeNAS with an ESXi.

FreeNAS will be on (it's not installed yet): Supermicro X10SLL-F; Intel Xeon E3-1230 v3 @ 3.30GHz; 32GB ECC RAM DDR3; LSI 9207-8i IT mode
ESXi will be on (it's not installed yet): MSI A320m-a pro; AMD Ryzen 7 2700, 64gb ddr4 2400mhz, ssd 240gb for boot

- Can I mix cards from different brands? For example, a Intel X520-T2 with a MELLANOX CONNECTX-3 ? My question primarily is because I see that not always the same card is supported in freenas and in esxi. May be if I can buy two diferrent cards it will be more cheap than get the same supported model for both.
- What about this card? Chelsio 110-1088-30 b0, cc2-n320e-sr is very cheap atm. On the first posts names S320E-SR-XFP but it doesn't state if it is currently working or have issues.

Thanks for your time! Regards!

 

[Elliot Dierksen]

Yes, cards from different manufacturers can communicate (assuming they all follows the specs). It is tempting to use DAC/Twinax cables, but don't do it! That is possible if you are using the same manufacturer's cards, but can be a huge headache if you aren't. One side or the other probably won't like the DAC cable. Use optical SFP+ modules and fiber patch cords. The prices on all of those have come down a bunch, plus it is much easier to run fiber patch cords through cable management than trying to do it with DAC cables which are not flexible.

 

[faktorqm]

wow! Thanks a lot Elliot! A Twinax cable is just I was about to buy. So I need (4) SFP+ optics adapters, in that way every optic adapter will solve any compatibility issue, and then trough fiber I can connect them without problems. Sounds good! I will choose two cards and will return for more advices! I usually see that "it's supported" it's not equal to "headache free" hardware. Thanks again!

 

[jgreco]

Hello! I have some questions regarding 10Gb cards because I never build before a network with this kind of SFP+ cards.

Currently I'm on a budget, and after take some time reading several forum posts and experiences regarding this cards, the ix system's reccomendations, and other blogs (with a dubious reputation) I have some questions.

My goal: Communicate a FreeNAS with an ESXi.

FreeNAS will be on (it's not installed yet): Supermicro X10SLL-F; Intel Xeon E3-1230 v3 @ 3.30GHz; 32GB ECC RAM DDR3; LSI 9207-8i IT mode
ESXi will be on (it's not installed yet): MSI A320m-a pro; AMD Ryzen 7 2700, 64gb ddr4 2400mhz, ssd 240gb for boot

- Can I mix cards from different brands? For example, a Intel X520-T2 with a MELLANOX CONNECTX-3 ? My question primarily is because I see that not always the same card is supported in freenas and in esxi. May be if I can buy two diferrent cards it will be more cheap than get the same supported model for both.
- What about this card? Chelsio 110-1088-30 b0, cc2-n320e-sr is very cheap atm. On the first posts names S320E-SR-XFP but it doesn't state if it is currently working or have issues.

Thanks for your time! Regards!

Please head on over to the 10 Gig Networking Primer which was written to help explain a lot of Need-to-Know's.

ESXi has pretty good support for a large number of cards, but you may have to dig up drivers from a manufacturer's website and install them on your hypervisor host.

FreeNAS does best with the Intel X520 or Chelsio T520-CR cards, though other cards may work.

The way to virtually guarantee compatibility with SFP+ is to use the correct optics, which are generally best picked off eBay used, or you can sometimes find a correctly coded optic from places like FS.COM which are generics that include the correct programming to work with a device. So even if your Intel card wants Intel optics, your Cisco switch wants Cisco optics, and your Chelsio card wants Chelsio optics, it all just works when you use the correct optics. In many cases this isn't necessary, and you can use twinax, or generic uncoded optics, but this is NOT guaranteed to work and finding a combination that does work can end up being more expensive.

 

[faktorqm]

Thanks jgreco, yes I read a lot of your posts, the guide you pointed me out, and other useful posts from Chris Moore too
As currently I'm on a budget, my question was pointed to buy a T520-cr card for example but for the esxi another one cheaper. but maybe I need to wait to save more money and then buy two T520-cr. I saw too many post saying that X520 from intel get too hot if it is on a bad cooled case. (probably my build xD) Thanks a lot for your feedback!

 

[acp]

FreeNAS will be on (it's not installed yet): Supermicro X10SLL-F; Intel Xeon E3-1230 v3 @ 3.30GHz; 32GB ECC RAM DDR3; LSI 9207-8i IT mode

- Can I mix cards from different brands? For example, a Intel X520-T2 with a MELLANOX CONNECTX-3 ?

Your system is very similar to my setup (running on a ASRock board). The only problem I had with the connectx-3 was with direct connect. I was connecting to another connectx-3 board it wouldn't always detect the link was up even with up in the options. The other server was running xenserver never had that issue. The ultimate solution was using a switch and never had a problem again. Using the switch it always detect the link. Maybe it was a driver issue. It could be resolved since that was a couple of years ago. I was/am using dac cables considering the distance was short since it be was in the same rack. I'm using mikrotek switches now.

 

[faktorqm]

Hi guys, may be a silly question but I want to make sure anyway. This card Fujitsu Primergy S26361-D2755-A11-1-R791 has the same chipset that the Intel X520 however the price is lower in the used market. I'm wondering if it is trully a OEM rebrand of the same chipset (and same functions) that the Intel one. I mean, if the Intel X520 card has the same functionality and performance that Fujitsu one or I missing something. Thanks in advance! Regards!

 

[Yorick]

Given that a DAC cable is literally 10 bucks, I’d at least give it a try. If it doesn’t work, no worries, fiber it is. For example: https://www.fs.com/c/intel-10g-sfp-dac-1380

 

[faktorqm]

Thanks for your answer Yorick, but my question is about the card Fujitsu D2755 which use the same chipset (Intel 82599) that the Intel X520-DA2. I'm wondering if the Fujitsu works like the Intel or it's cutted. Intel specs Fujitsu specs Thanks!

 

[SweetAndLow]

Bought 2 Chelsio T520-cr for $100. I dunno what kind of budget you're on but it doesn't get any cheaper than that. Just search ebay. Maybe you're not in the US and have fewer options though.

Don't touch dac cables, they are not worth the effort, I got a box that I need to throw in the garbage. Tranceivers and fiber is the easiest solution.

 

[faktorqm]

Hi! yes that what I'm trying to catch!. 2 chelsio t520-cr for $100, even 150 would be great, but I didn't see anything even close. I'm writing from Spain,. Maybe that prices here are higher or the sellers with those prices does not ship to Europe. Currently all I see is $120 just for one card. I will dig better!
Currently, for that price I can get 2 Fujitsu like I posted before. If someone can confirm that will works as good as the Intel X520 (which is a proven model) I will get those. Thanks!

 

[Yorick]

List of NICs and their equivalent OEM parts

So, in the RAID controller section, there's a thread that's been really useful to me that maps all the LSI controllers to their equivalent OEM versions by Dell, HP, IBM, etc. I thought I would start a thread to do something similar for network interface controllers. I've particularly found this...

forums.servethehome.com

 

[faktorqm]

yikes! will I have to learn to search (again)? :facepalm: Thanks a lot Yorick.