SOLVED NIC to NIC & NIC to Network

[twf85]

Before I screw something else up, I thought I might ask for a little guidance. Similar to this thread.

The motherboard in my FreeNAS build has 3 Gbe ports, one of which I believe is meant to allow access to the BIOS via web portal.

Currently, I have one port connected to a Switch and another connected directly to a nearby Windows 10 PC. The other posts I've read (aside from the link above) seem to want to only have a direct link to their FreeNAS box via connected PC (like this thread).

I've bungled it once already and had to visit the box in person to reset Networking, so I'd like to avoid any more of that if possible (though it was good to go through the motions of "recovering" from the error I introduced).

I've manually set the IP address of the NIC in the PC to "10.0.10.1" and the mask to "255.255.255.0". I also added an "IPv4 Alias" to the `igb0` interface (10.0.10.2/24).

If I try to browse "10.0.10.2" from a File Explorer menu, after a time, it does seem like it is connected, but I take that as a false positive. I also tried "10.0.10.3", and got the same result.

I do plan to mitigate the need for this by adding a Managed Switch and a couple of multi-port NICs to both of the boxes; however, for the time being, I'd like to setup an exclusive connection between the two boxes that is independent of the network (without bringing either box off of the network in the process).

I tried reading the Sticky that was kind of up this alley, but I am more or less lost when it comes to LACP/LAGG. I only understand the broad strokes and have no firsthand experience, but I was able to gleam enough from reading it to know that I needed to make sure this new set of NICs were assigned to a different subnet. From what I can gather, things should just "work" if they are setup properly, so I know I'm not doing something right. I don't have a crossover cable, but I could create one if need be.

Apologies in advance if this is a duplicate of an already answered question...


UPDATE:

I added another Interface, this time using the `em0` NIC, and manually set the IP to "10.0.10.2" and netmask to 24. I also changed the PC's NIC to use "10.0.10.2" as the Gateway. That appears to have done the trick.

Forgot to mention, at the start, I added a Firewall Rule to allow connections to the "10.0.10.0/24" subnet.

The only strange thing was that I wasn't able to browse the available Shares. I could ping both ways, so I decided to try mapping a network drive using the alternate IP address (10.0.10.2). Although it took a minute, it did let me add it and the Share was accessible. I ran tests and confirmed that traffic was going out of the intended NIC when using either IP address to send files.

Did I do this right?

 

[Elliot Dierksen]

I would say you first need to back up, and try to describe what you are trying to accomplish. You are mentioning multiple NIC's, devices, and IP networks without describing your intended purpose. Start there, and then we can try to figure out how to get there.

 

[jgreco]

From what I can gather, things should just "work" if they are setup properly, so I know I'm not doing something right. I don't have a crossover cable, but I could create one if need be.

It isn't clear to me what you're trying to do, but even when things are set up correctly, networking can be an incredibly complicated topic and unusual topologies rarely "work" without some effort and tweaking.

Crossover cables have not been a thing for twenty years. 10Mbps and 100Mbps ethernet both had it, in part due to their use of only two of the four pairs. However, with the advent of 1Gbps ethernet (~20 years ago), auto MDI/MDI-X is part of the standard. Do not try to make a crossover cable. The only crossover cable that will work correctly is the one where you inadvertently don't cross over anything. Failure modes will include not working at all, or possibly adjusting the speed down to 10/100, or other interesting but damaged/broken things.

I do not miss crossover cables.

 

[twf85]

... try to describe what you are trying to accomplish.

Apologies, I guess the title wasn't self-explanatory enough. I do tend to get carried away...

I want a direct connection between my FreeNAS box and one Windows 10 PC that will enable each device to communicate with one another in a way that won't interrupt normal LAN traffic to each device.

Does that help answer your question?

To accomplish this, my FreeNAS box had one Gbe open, and I added a spare NIC to the Windows PC. I connected the devices together using these open ports.

It isn't clear to me what you're trying to do, but even when things are set up correctly, networking can be an incredibly complicated topic and unusual topologies rarely "work" without some effort and tweaking.

Crossover cables have not been a thing for twenty years. 10Mbps and 100Mbps ethernet both had it, in part due to their use of only two of the four pairs. However, with the advent of 1Gbps ethernet (~20 years ago), auto MDI/MDI-X is part of the standard. Do not try to make a crossover cable. The only crossover cable that will work correctly is the one where you inadvertently don't cross over anything. Failure modes will include not working at all, or possibly adjusting the speed down to 10/100, or other interesting but damaged/broken things.

I do not miss crossover cables.

A few (older) tutorials I read that covered a Windows-to-Windows connection mentioned crossover cables. I've only ever made one, and that was only because I mistakenly thought a cable was bad when investigating a communication error between an older WinXP PC and the CNC control unit it was built to communicate with (~WaterJet cutting equipment).

I'm in no hurry to create any unnecessary work for myself, so thank you for cutting that idea down early. I also appreciate the brief on that type of cabling :)

....

Here's my `ifconfig` output:

Code:

em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC>
        ether MAC:ADDRESS
        hwaddr MAC:ADDRESS
        inet 10.0.10.2 netmask 0xffffff00 broadcast 10.0.10.255
        nd6 options=9<PERFORMNUD,IFDISABLED>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=6403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether d0:50:99:d0:cc:f0
        hwaddr d0:50:99:d0:cc:f0
        inet LAN.IP.RNG.73 netmask 0xffffff00 broadcast LAN.IP.RNG.255
        inet6 REDACTED%igb0 prefixlen 64 scopeid 0x2
        inet6 REDACTED prefixlen 64 autoconf
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
        inet 127.0.0.1 netmask 0xff000000
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        groups: lo

Like I said in my Update, this does appear to work. I can see traffic going out of the intended NIC in the Windows PC when I want it to. Due to the inability to view Shares (though I could add them using their full SMB paths), I worry I've made this work the wrong way.

One of the tutorials I read suggested making each of the devices the other's opposite, such that:

Device 1 (IP): 10.0.10.1
Device 1 (Netmask): 255.255.255.0
Device 1 (Gateway) : 10.0.10.2

Device 2 (IP): 10.0.10.2
Device 2 (Netmask): 255.255.255.0
Device 2 (Gateway) : 10.0.10.1

It wasn't until I added the Interface using `em0` that I got traffic moving, so at least there's that. I wasn't able to set the Gateway in the emulated interface, so I couldn't follow the instructions in the tutorial completely.

At this point, if someone could give their stamp of approval or say, "You're doing it wrong, I would have done it *this* way", that would be great. :) If I can get there, then I will begin to test the link earnestly.

Thank you both for your replies :)

 

[Elliot Dierksen]

I want a direct connection between my FreeNAS box and one Windows 10 PC that will enable each device to communicate with one another in a way that won't interrupt normal LAN traffic to each device.

You are seriously over-complicating this. FreeNAS makes an awesome storage appliance and a passable hypervisor, but it isn't a switch/router. It sounds like that is what you are trying to make it do. Just get a switch, and connect all the devices on one IP network. With a switch, unicast traffic is filtered so traffic between host 1 & 2 doesn't impact and is not seen by any other hosts. If you feel like you need more bandwidth from FreeNAS, get a switch the supports LACP. Then you can have FreeNAS create a LAGG to increase the overall effective bandwidth.

 

[twf85]

You are seriously over-complicating this. FreeNAS makes an awesome storage appliance and a passable hypervisor, but it isn't a switch/router. It sounds like that is what you are trying to make it do. Just get a switch, and connect all the devices on one IP network. With a switch, unicast traffic is filtered so traffic between host 1 & 2 doesn't impact and is not seen by any other hosts. If you feel like you need more bandwidth from FreeNAS, get a switch the supports LACP. Then you can have FreeNAS create a LAGG to increase the overall effective bandwidth.

I don't see how. From the research I've done, FreeNAS supports direct connections without the need for a Switch. I'm also not the only person who wanted to do something like this. I don't see why it would be overly complicated to have a direct connection peacefully coexist with a connection to the local network, so long as they are not on the same subnet.

I'm not trying to make it act as a switch or router. I just don't want either of the devices' primary NICs being bogged down by backups or file transfers from one to the other.

I did state that I was planning on buying a managed switch that supported link aggregation. I spent too much on the FreeNAS build and my budget for this sort of thing is non-existent at the moment. I did have a spare NIC collecting dust, so I thought I'd give this a try as a temporary workaround.

Lastly, I've already got it working. Not sure if you caught that bit... My main question now is do I have it setup correctly, or is there a better way to configure each of these machines for an optimal connection?

 

[Redcoat]

If you are questioning your setup based on Windows 10 share browsing ability, I'll suggest that this is likely the result of MS's changes to the SMB protocol availability in W10 - see https://forums.freenas.org/index.ph...rise-1709-update-smb-share-not-visible.63025/ for relevant info and links.

 

[Elliot Dierksen]

If you were wanting to wring every possible bit out of network performance out of the connection from FreeNAS to multiple hosts that were pushing lots of volume to and from the FreeNAS, I suppose that would be a way to do it with gigabit links. You didn't say what your storage use case is, or how these devices reach the internet (I am assuming they all want to reach the internet). If the windows boxes only have the link to FreeNAS, then you are trying to use it as a switch/router, which is really not one of its strengths. If you have multiple network connections, you have given your windows boxes multiple default gateways. That means that it will distribute packets between the equal cost paths. In this case (as it is mostly), that isn't a good thing, and probably won't work (or won't work well). My day job is network infrastructure, so I understand the notion of dedicated connections. I wouldn't do it the way you are doing it without some really specific use case because I think it is needlessly complicated. I can do complex, but there has to be a good reason for it. Otherwise it just ends up being a pain in the butt to support and troubleshoot.

 

[twf85]

Point taken. Thank you all for your input...

EDIT: After watching this video, I'm going to go with 10GB nics. Seems to be the most cost effective way to achieve high throughput at the moment.