SOLVED Multiple FreeNAS servers

[pirateghost]

No, I didn't unplug all but one NIC. I set them up so that they were running on different IP addresses.

Different IP addresses on the same subnet is an issue too.

 

[UdoB]

I set them up so that they were running on different IP addresses.

You probably know this, but as it is important I want to mention it: as far as I know having different IP addresses is not sufficient. The chosen addresses need also to be in different (logical) networks.

Best regards

 

[jgreco]

There's also a nice sticky on the topic.

https://forums.freenas.org/index.php?threads/multiple-network-interfaces-on-a-single-subnet.20204/

 

[tstorzuk]

Yay,

I finally got the 3 servers up and running without any IP address issues!! I'm transferring files from one server to another to offload some of the excess (way above the recommended storage capacity of 80% maximum).

Thanks for all the help. I had no idea that I needed to physically have only 1 NIC connected to the switch. But I guess that makes sense since it isn't a managed switch.



Back to my previous question about upgrading my network.....jgreco,

After reading your links and information about what's what, how would you recommend I upgrade my network? Start with a switch that has a couple of 10GbE ports (like the Dell PowerConnect 5524), and a couple of NIC's (Chelsio S320E-SR-XFP) for one server and one for one workstation......

Then at a later date, add another switch (like the Netgear ProSafe XS708E) and more NIC's (Chelsio S320E-SR-XFP) for the other server/s and a few more workstations?

Would this work? Are they all compatible with each other, or did I miss something in your network hardware link that I didn't pick up on?

 

[jgreco]

Yay,

I finally got the 3 servers up and running without any IP address issues!! I'm transferring files from one server to another to offload some of the excess (way above the recommended storage capacity of 80% maximum).

Thanks for all the help. I had no idea that I needed to physically have only 1 NIC connected to the switch. But I guess that makes sense since it isn't a managed switch.

Yeah, you get the best combination of too-smart and too-stupid network layers working together when you do that. It's a catastroph**. It's even more fun when it happened to kinda-sorta-work somewhat and then breaks when you make some apparently unrelated change. ;-)

Back to my previous question about upgrading my network.....jgreco,

After reading your links and information about what's what, how would you recommend I upgrade my network? Start with a switch that has a couple of 10GbE ports (like the Dell PowerConnect 5524), and a couple of NIC's (Chelsio S320E-SR-XFP) for one server and one for one workstation......

Then at a later date, add another switch (like the Netgear ProSafe XS708E) and more NIC's (Chelsio S320E-SR-XFP) for the other server/s and a few more workstations?

Would this work? Are they all compatible with each other, or did I miss something in your network hardware link that I didn't pick up on?

The 5524 is a nice option for a small home user, but it probably isn't suitable if you want to go further. The 5524's are in theory stackable with an HDMI cable but there are horror stories.

I have no specific comments on the Chelsio card except that XFP is not the same as SFP+; this mostly means that as a practical matter you have to do SR (10G) fiber and you have to concern yourself with finding the correct optics. I have no objections to that at ALL because it's most likely to lead to plug-and-play, but I've kept things here strictly SFP+ because I think it easier. The optics may be specific to a manufacturer (so get Chelsio XFP and Dell SFP+'s if you want least-hassle) but once you've gotten it over to SR fiber, anything is supposed to plug into anything and It Just All Works (Or The Manufacturers Suck).

The real problem is that 10G is in this phase where it all kinda sucks - technology transition. The low end 10G switches like the XS708E are copper based, but most of the switches with uplink-style 10G ports are SFP+, so if you start small with a 5524, you've acquired SFP+ which then means you have to bump up to an SFP+ switch. None of them are cheap (IMHO), even used on eBay a Dell 8024F is around $1400. There are some edge switches that have 4x 10G SFP+ and that might be an option, but they'll be probably nearly $1000.

At this point, more than a few ports of 10G is still pretty expensive. But if you're contemplating an XS708E then at least you aren't totally out of the ballpark as to cost. I don't have any specific advice except to note that my own findings for our operations here was that SFP+, especially with fiber, is a more-costly but highly effective solution. We can plug legacy 1GbE gear, either fiber or copper, into conventional 1GbE SFP's (they do make them for 1GbE copper) in a 10G switch.

But it is worth pondering that moving forward, the dominant technology is very likely to be 10GBASE-T, so when you buy that new server in two years and it has copper ports built-in and you have invested in SFP+, ****ANGER****.

I have no happy solutions. Sorry.

 

[tstorzuk]

I have no happy solutions. Sorry.

jgreco,

Thank you for your candor. And the knowledge. I didn't understand that the Chelsio cards were XFP and the other components were SFP+.

I do have time on my side, now that I have finally gotten my servers all functioning. So I guess I can play the waiting game, and see which standard becomes the dominant one. Otherwise I would have definitely gone the SFP+ route to allow incremental upgrading of all of my devices.

Again, thanks to all for all your help. I wouldn't have been able to have gotten this figured out without your help.

 

[jgreco]

It's frustrating for those of us working in the business. In 2004-2006, we did upgrades of everything to gigE as what we thought would be a short wait for 10G to become reasonably available (meaning not-insanely-priced). Here it is, 10 years later, that's fully one quarter of the average professional's career length, and 10G is still not looking particularly attractive....

 

[bernardc]

Thank you, jgreco and tstorzuk. This was my problem of long standing.

Again jgreco is the exemplar of problem solving with civility. My hat is off to you.

 

[jgreco]

Again jgreco is the exemplar of problem solving with civility. My hat is off to you.

How dare you, sir. (takes the hat and smacks @bernardc with it.)