Having trouble configuring network settings for Intel NIC installed in TrueNAS server

[hllywdunddnnpnt]

Hello, apologies if I am posting in the wrong place. I have spent quite sometime here on the forum and on google, reading posts to see if I can figure it out on my own, but nothings helping me understand. I don't have much knowledge in the area of networking but I have been watching youtube videos to and learn more, but I am completely stuck and don't what I'm doing wrong and what to do

I have TrueNAS Core (TrueNAS-12.0-U8.1) installed on an old PC and I originally installed a ASUS PCE-C2500 2.5Gbe Network Card, I had ethernet running from the asus nic to my switch and another from my switch to my actual PC. I had the nic's ip set to static (192.168.0.25). With Windows Share's (SMB) setup I had no trouble accessing my data or the TrueNAS GUI and everything has worked flawlessly for the past few years. I also have a Plex server running on my TrueNAS which is used in multiple locations within my House.

A couple months ago wanting to upgrade to a 10 Gb connection I bought 2x Intel X540-T2 10G Dual RJ45 cards and I installed one nic in my TrueNas and the other in my PC. So my goal is to have my PC directly connected to my TrueNAS server via the 10gb intel nic (to get the 10gb speed), while still having my TrueNAS server connected to my switch via the 2.5gb asus nic (so the rest of the house still gets the 1gb speed).

From memory I tried to set a static ip for the intel nic to something like (192.168.0.33) but TrueNAS gave me the error "The network 192.168.0.0/24 is already in use by another interface". So I set it to DHCP but then I couldn't connect to my TrueNAS from my PC thought the intel NIC, I then switched the asus nic's ip to DHCP and the intel nic's ip to static (192.168.0.34), which I then got boxed out of the TrueNAS gui and couldnt connect to my TrueNAS at all through any PC. After an hour of mucking around I was able to log back into the TrueNAS gui and because everthing worked fine (but only at 2.5gb speed) I just left it. But randomly last night I got blocked out of the TrueNAS gui and couldnt access windows SMB shares, after playing with the ethernet cords I was able to get back into the TrueNAS using 192.168.0.41 but I can't get windows SMB shares to work. I assume this was caused by the asus nic getting assign a new ip via DHCP.

At the moment my PC and TrueNAS are connected directly via both intel nics, with the intel nic's ip address on TrueNAS set statically to 192.168.0.41, then TrueNAS is also connected to my switch via the asus nic and its ip address on TrueNAS is set to DHCP and has the ip of 192.168.0.26. Currently the only way I can connect is through 192.168.0.41, but both nics have to be connected, the moment I disconnect the ethernet cord from either nics on TrueNAS I can no longer connect to the TrueNAS and I'm only getting 2.5gb speed.

em0 is the 1gb port built into the motherboard
re0 is the port on the 2.5gb asus nic
ix0 and ix1 are the dual ports on the 10gb intel nic, for which I'm using ix0

Windows PC: ipconfig /all

re0 in TrueNAS is connected to the 2.5gb Realtek port built into my PC's motherboard
ix0 in TrueNAS is connected to the first port of the same 10gb intel nic insalled in my PC

Network Global Configuration

Every change I try to make to either the network settings on both the TrueNAS server or my PC, screws everything and I can no longer access the TrueNAS gui and I have to scramble to reverse the changes to get access again.

I am aware that nics like my 2.5gb asus nic are generally frowned upon in the community due to limited support, but it worked completely fine before I installed the intel nic, so I am pretty sure the issue lies with my network settings and lack of knowledgem rather then the card itself.

Could anyone please shed some light on what I am doing wrong or what I can do to get it to work.
Are my settings wrong?, is what I want to do even possible?

I can provider further information in need be, this is the only information I can think to provide.

Thanks

 

[Redcoat]

Your issue is that you have two NICs on the same subnet.

To fix this, put the two 10GB NICs on static addresses on a different subnet that that of the rest of your network - say 192.168.1.10 and 192.168.1.11 respectively. Put your two 2.5GB NIC's with statics on say 192.168.0.XXX and 192.168.0.YYY where XXX and YYY are not in the range where your DHCP server is allocating addresses.

 

[hllywdunddnnpnt]

thanks for the reply, I continued to play around with it since I posted and this is my current setup

Which I can now access the TrueNAS gui with both 192.168.0.41 and 192.168.1.41, but I still can't access either from within windows.
\\TRUENAS does show up under Networks, which is my TrueNAS but it wont connect either

 

[WI_Hedgehog]

Expanding on @Redcoat's answer:

In very general terms the original problem has to do with routing: if your server has 2 cards on 192.168.0, which does it pick to send data to clients? You'd think "the address the request came in on should be the address that responds to the request." That seems straightforward.

It turns out networking has several "layers" and is more complex than just an IP address, so that actually doesn't work. Yes, it may "kind of" work in some situations and then maybe not work or simply break for no reason, or keep working until it doesn't, but going down that path is full of pitfalls that aren't obvious, much like crossing street intersections without looking because "you've been doing it and haven't been run over" [yet].

The Subnet Mask of 255.255.255.0 will cause a system to only listen to addresses where the first three sets match, meaning a client on 192.168.0.xxx can't listen to a server on 192.168.1.xxx.

If that's not complex enough, if there's a network switch in the middle (the box that networking cables from different computers plug into) it only listens on the subnet it is set to listen on.

So, your original IP subnet will serve 192.168.0.xxx fine. The second card can be set to 192.168.1.xxx/24 and directly wired to the PC; the PC then gets set with an IP 192.168.1.yyy and the mask is the same 255.255.255.0 as before (on the server side it's denoted as /24... look up how that works if you like).

Do not set the subnet masks to something that's non- standard, Windows gets all stupid. First get things working, then you can break stuff.

Remember the GUI is at an IP address, and should be accessed from the same subnet. Computers and browsers have layers of caching, so when you change something do a refresh with CTRL-F5. (Look up F5 vs CTRL-F5 if you want.)

And, if you put the GUI on a different IP or switch DHCP on/off it may take the server 1-2 minutes to switch the back-end configurations around and restart the sub-systems on the new settings (at least that's my experience when mucking around). Patience is a virtue?

 

[danb35]

See also:

Resource - Multiple network interfaces on a single subnet

We've had a number of people over the years bring up the topic of multiple network interfaces on a single subnet. Often done by Windows administrators or storage admins, where it works to varying degrees, this is sadly not proper IP networking...

www.truenas.com

 

[WI_Hedgehog]

I added a bunch of stuff to my previous post to expand it further so you can hopefully get this running quickly and with little effort.

With networking, or most any computer stuff, keeping things simple is usually best. You can go way down the rabbit hole and set up a complex system...then find an update introduced a bug or removed functionality and things mysteriously stop working, or suddenly "have intermittent problems." If you want to spend a week sorting it out that's fine by me, but I have fun hobbies and would rather spend time on them than sorting out network issues....but that's me.

 

[danb35]

Yeah, I remember another recent thread where the poster had set his subnet mask to /30, and then set a default gateway that wasn't in that range (and, IIRC, the one other device was set to the broadcast address for that subnet). KISS applies as well here as anywhere else (says the guy who has at least five VLANs for his home network...).

 

[hllywdunddnnpnt]

Thanks danb35 I had a quick skim read and I'll read the first article tomorrow

I changed the subnet mask on my PC for my 10gb intel nic to 255.255.255.0

So on TrueNas
2.5gb nic (re0) has a static ip of 192.168.0.41
10gb nic (xi0) has a statip ip of 192.168.1.41

On windows PC
2.5gb (motherboard port) has DHCP enabled and assinged 192.168.0.15
10gb nic has a statip ip of 192.168.1.41
both have a subnet mask of 255.255.255.0 and a defalt gateway of 192.168.0.1

And as I mentioned I can access the TrueNAS gui in the browser using
//192.168.0.41
//192.168.1.41
//truenas.local

I was having 2 issues
1. I couldn't access the TrueNAS gui in the browser using 192.168.1.41, but now I have the IP addresses and subnet masks set correctly I can.
2. I couldn't connect to TrueNAS through Windows, to access my SMB shares in any shape or form.

These changes havent fixed the 2nd problem. Even after installing the intel nics back in Janurary, my windows SMB shares were working fine, //TRUENAS would show up under Networks, I could click on it and it would list my SMB shares, which I have mapped to network drives and I could access my files. But last night I could no longer access any of my files, when I click on any of the mapped drives or click //TRUENAS under network it says it can't connect.

Even if I disconnect the ethernet cord from both intel nic ports so TrueNAS and my PC are no longerd irectly connected and want to stick to just the original 2.5gb interface, I still can't access my drives.

Although I would like to be able to use 2 different interfaces, I have wasted hours today, purely trying to get it to work with only one interface.

I have Network Discovery enabled under Network and Sharing on my PC
I have reset both my PC and TrueNas machine
I have restarted the SMB service in the TrueNAS gui
I can still access my plex server on 192.168.0.80 which is running on my TrueNAS so the machine is running properly

But I still cant access my files. and I dont understand why I cant

 

[hllywdunddnnpnt]

Thanks everyone for your input and trying to help me out, to give a better explanation of my situation.

When I first set up TrueNAS on my old PC, TrueNas installetion worked flawlessly, I installed a 2.5gb nic which worked fine, setting up window shares to access my files worked completely fine. the 2.5gb nic had its ip address statically set to 192.168.0.26, with my windows shares mapped to network drives I could access my files on my TrueNAS like any regular local HDD. Setup was easy and simple.

Then back in January I bought and installed 2x 10gb nics in TrueNas and PC, because the 2.5gb nic's ip was set to 192.168.0.25 it wouldn't let me set the 10gb nic's ip to 192.168.0.30, which I didn't understand why, but enabling DHCP on the 10gb nic did assign it to 192.168.0.28. Everything sill worked and there was network activity on both interfaces, but I was only getting 2.5gb speed, assuming my map network drives were still connecting via the 2.5gb nic, I changed the 2.5gb nic to use DHCP and statically set the ip of the 10gb nic to something like 192.168.0.31, but when I applied the changes I was kicked out of the TrueNAS gui and I couldnt get back on using any IP.

After connecting another ethernet cord into the motherboards 1gb network port and into my switch, I was able to get back into the TrueNAS gui using the original IP and reverted the changes I made. I then disconnected the ethernet cord from the 1gb port, I didn't understand why it did this, but I had a ethernet cord connecting TrueNas and PC together directly via the 10gb intel nics, both TrueNAS and PC were connected to my switch via 2.5gb nics. With both interfaces connected I could access the TrueNAS gui in the browser and access my files, via the mapped network drives, but if disconnected one the interfaces, I would lose the connection, both interfaces had to be connected for me to be able to access the gui and my files. Even though I was still only getting 2.5gb speed, I decided to leave it, as I was atleast able to access my files.

But everything broke last night out of no where, I tried to get it working again both with and without the 10gb nic, Not sure when but DHCP allocated the 2.5gb nic a different IP, as it was different when I checked, so Im not sure if this is what broke everything.

But I coudn't connect to TrueNAS through windows to access my files. Surely regardless of what settings I have set for the 10gb nic, as long as the settings for the 2.5gb nic are correct, I should be able to access my files. But because I still couldnt connect, I assumed that incorrect settings on the 10gb nic were preventing me from connecting to TrueNAS through windows using the 2.5gb nic which is what I had originally setup. Hence playing aroudn with settings on both nics.

At this point I would settle to only have my TrueNAS connected to my switch via the 2.5gb nic, like I originally had, but I can't even get that working. I have only unplugged the ethernet cord from the 10gb intel nics, would I have to remove the 10gb intel nic from the TrueNAS machine?
,

 

[Redcoat]

So on TrueNas
2.5gb nic (re0) has a static ip of 192.168.0.41
10gb nic (xi0) has a statip ip of 192.168.1.41

On windows PC
2.5gb (motherboard port) has DHCP enabled and assinged 192.168.0.15
10gb nic has a statip ip of 192.168.1.41
both have a subnet mask of 255.255.255.0 and a defalt gateway of 192.168.0.1

You're reporting the same IP for both 10GB NICs... Copy and Paste, or ...?

 

[hllywdunddnnpnt]

You're reporting the same IP for both 10GB NICs... Copy and Paste, or ...?

Sorry stayed up last into the early hours of the morning so I was tired when sending the last message, so that was a typo.
ip address for the intel nic installed in PC is still and has always been 192.168.0.77, as reflected in the previous screenshots. I have tried changing all the ips around but still can't connect.

The fact I can access the TrueNAS gui in the browser using //192.168.0.41, //192.168.1.41 and //truenas.local, mean that there are no issues with conflicting ips and that the issue is somewhere else.
I have changed //192.168.1.41 to //192.168.1.51 so that the last numbes are all different but still nothing.

At the moment I am trying to get back to the basic setup up, of my TrueNAS only being connected to my switch via the original 2.5gb nic I installed on build. I no longer have a ethernet cord connecting both 10gb intel nics, so surely anything regarding the 10gb intel nics won't cause any problems with the 2.5gb nics.

On my PC
- I can access the gui in browser //192.168.0.41
- //192.168.0.41 pings normally in cmd via "ping 192.168.0.41"
- //TRUENAS still shows up under "Networks" (in Windows Explorer), though upon clicking on it, it takes around 30 seconds to think than says it can't connect
- enting \\192.168.0.41 into windows explorer yields the same result
- I can still access all media on my plex account via 192.168.0.80 that is running on TrueNAS

- I have reset both PC and TrueNAS machines multiple times

The only issue is that I can't access any of my files on the HDDs that are installed in my TrueNAS machine through Windows Explorer like I have all this tim, on my PC.

UPDATE:

Kicking myself for not trying this sooner, but I got my old laptop out and I can connect using \\192.168.0.41 via windows explorer, which gains access to my shares and all my files (on my old laptop). So the issue must lie with my PC and nothing to do with my TrueNAS machine.
I have just been chasing my tail, fixing and breaking the same thing over and over which was never broken in the first place.

I cannot do this on my PC. Is there any information I can provide that could help spot the issue.

I hadn't changed any settings on my PC leading up to it breaking, I accessed some files via the mapped drives back on thurdsday, but on friday night while watching plex (on PC) out of no where my PC just turned off, it didn't freeze or show any signs of freezing, it didn't come up with the Windows 10 equivelant of the bluescreen of death or anything like that, it was like the PC suddenly lost all power, like I literally unplugged it from the wall. This is when I noticed I couldnt access my files after starting my PC back up. No other electrical devices were affected including my TrueNAS machine. The CPU on my PC is running at a stable 55 degrees celcius (ranging between 50-60), as my first thought was maybe the CPU overheated and shutdown the PC, not sure if people would consider that hot, but my CPU's been at that temp ever since I built my PC back in 2021. with no issues

Thanks everyone for your input, its just been super frustrating

 

[Redcoat]

Have you checked your Windows credentials - perhaps there's some legacy crap in there?
CMOS battery?
Reset W10?
Good luck, understand the frustration.

 

[WI_Hedgehog]

@hllywdunddnnpnt :

On the Windows box:

ipconfig /flushdns​

netsh int ipv4 reset​

netsh int ipv6 reset​

netsh winsock reset​

nbtstat -R​

nbtstat -RR​

Clear your browser cache manually so it does not use the old information. (Seriously, I've seen Windows preemptively load old data even though the IP "stuff" changed dramatically and the data is no longer relevant.) Or hit CTRL-F5 5-10 times over 30 seconds so it gets the point it needs to reload the data.