Setting up Network troubles: SOLVED

[ZiggyGT]

I broke my network again. I added a 10Gb card and now I cannot get connected. I tried to simplify things and re-establish connection on a 1Gb network. Once that was working and from the GUI I planned to reconfigure. I can ping the new address 192.168.1.61 from two different computers on the network 192.168.1.62 and 192.168.1.68 but I cannot get the browser to connect at either.

I cleared all the network setups one at a time on the terminal and reentered the desired address for the server as 192.168.1.61. I used "ifconfig -a | less" to confirm which port was active. This is a X9DRD-7fln4 so there are 4x 1Gbs jacks and now 4x10Gbs ports. (I am not trying to use the 10Gbs ports now). Two of them was previously setup as a bridge at 192.168.0.61. I did an "ifconfig bridge1 destroy", but still the command line on my server thinks it is at 192.168.0.61 no matter what I reset with the terminal. How do I erase the current network configuration and add the new one without a full install? I don't want to lose the shares and other info.
I attached 2 pictures one shows the config of igb2. The other shows the nameserver. My entire network is set up on 192.168.0.X except for the network I use to manage it as 192.168.1.X. I do this so I can setup my bridge and avoid two interfaces on the same network. What does the UGRS flag mean?

Attached are the outputs of the command that were suggested here:

 

[ZiggyGT]

I removed all the 10Gbs cards in an attempt to simplify things. I cannot shake the bridge1 that has 192.168.0.61 even though all the cards in the bridge are removed. I setup addresses on 2 of the 1Gbs ports. One at 192.168.1.71 and one at 192.168.0.100. I was able to ping these ports OK but the GUI does not repond on either address. I have gone into the shell and did a "ifconfig bridge1 destroy" but it is back as soon as I leave the shell.

I looked in the manual at the reset command

1. Reset Configuration to Defaults Caution! This option deletes all of the configuration settings made in the administrative GUI and is used to reset TrueNAS® back to defaults. Before selecting this option, make a full backup of all data and make sure all encryption keys and passphrases are known! After this option is selected, the configuration is reset to defaults and the system reboots. Storage ➞ Pools ➞ Import Pool can be used to re-import pools.

If I do this it seems that i will lose my jail and access info. I dont want to do that I just want it to forget the ethernet settings.

Is there a way to completely erase the network configuration for the server console? I am stumped

 

[nKk]

I'm confused about your network topology.
Let's try to clear some points.
You use:
- 192.168.0.0/24 as network for data access
- 192.168.1.0/24 as network for TrueNAS management

After you destroy the bridge and remove 10G card what ip address you want to have on TrueNAS?
From your first picture I see that you configure "igb2" NIC with 192.168.0.100/24 IP address

...
inet 192.168.0.100 netmask 0xffffff00 broadcast 192.168.0.255
...

and you have "description" text:

...
description: 192.168.1.61/24
...
but this is only description text not configured IP address.

From all this you should be possible to access TrueNAS shares using 192.168.0.100 from PC that have IP address from 192.168.0.0/24 network.

But if you configure TrueNAS WEB interface to be accessible only from 192.168.1.0/24 network you can't access it until you set additional IP address from 192.168.0.0/24 or remove restriction of WEB interface.

 

[ZiggyGT]

I'm confused about your network topology.
Let's try to clear some points.
You use:
- 192.168.0.0/24 as network for data access
- 192.168.1.0/24 as network for TrueNAS management

Exactly 192.168.0.0/24 is my mostly 10Gbs network, 192.168.1.0/24 is for maintenance and is 1Gbs. I try to set a related address so it is easy to remember. only the third digit changed.

After you destroy the bridge and remove 10G card what ip address you want to have on TrueNAS?
From your first picture I see that you configure "igb2" NIC with 192.168.0.100/24 IP address

I can only destroy the bridge in the console shell. I have tested the gui access while still in the shell. The only address that shows on the console when I exit the shell is 192.168.0.61 which was the bridge1 address.
Doing "ifconfig -a" the bridge is still present. once I exit from shell. I can ping the 192.168.0.100 address from another machine but the web gui does not connect. (note 192.168.0.100 was create when I switched to DHCP thinking that might help)

...
inet 192.168.0.100 netmask 0xffffff00 broadcast 192.168.0.255
...

and you have "description" text:

Sorry for the confusion igb3 is set to the maintenace address I am trying to use. to try to debug I plugged in another port set to my maintenance address of 192.168.1.61. I can ping that address as well from another machine on that network range, but no response on the web interface. This is why I am stumped. I know I cannot have two devices on the same subnet with Truenas so I thought changing to the maintenance port would allow me to reconfigure. no dice. I'll post the net dumps after I run some errands. Thanks for the help.

...
description: 192.168.1.61/24
...
but this is only description text not configured IP address.

From all this you should be possible to access TrueNAS shares using 192.168.0.100 from PC that have IP address from 192.168.0.0/24 network.

But if you configure TrueNAS WEB interface to be accessible only from 192.168.1.0/24 network you can't access it until you set additional IP address from 192.168.1.0/24 or remove restriction of WEB interface.

How to configure for web access. or remove the restriction?

 

[nKk]

You can check if there is custom configuration for WEB UI at:
System -> General
"WEB interface IPv4 address" - define IP address for WEB server to bind. Default is 0.0.0.0 that means bind to all IP addresses.

Do you have access to shares on TrueNAS or SSH?

And perhaps it's a good idea to restart the server because after all this network configuration changes from console it's not guarantee that all services use the right IP address, or you can try to restart only the middleware:

Code:

service middlewared restart

 

[ZiggyGT]

You can check if there is custom configuration for WEB UI at:
System -> General
"WEB interface IPv4 address" - define IP address for WEB server to bind. Default is 0.0.0.0 that means bind to all IP addresses.

I changed the address of igb2 to be 192.168.0.61 via the console. Then restarted the system. I checked the System>general as you suggested and found I had at some time. Do not remember had set the UIF to 192.168.0.61. I reset it to 0.0.0.0. The basic system is operating properly now.

next step I'll add the maintenance address. and then try to get the bridge to work again.

The above issue is probable why I could not access via the maintenance address in the past.

Thanks loads for the help! I now understand what was happening.

Do you have access to shares on TrueNAS or SSH?

And perhaps it's a good idea to restart the server because after all this network configuration changes from console it's not guarantee that all services use the right IP address, or you can try to restart only the middleware:

Code:

service middlewared restart

 

[ZiggyGT]

Resolved my network issue with the info from @nKk. I appreciate the help.

One factor complicating this was my choice of DAC. I had purchased a couple Molex DAC cables. They worked fine with Intel, Mellanox and Solarflare SFN5152F cards I had. Because the Mellanox and SolarFlare cards I had were no longer officially supported I bought some newer SolarFlare cards. When I added the extra card to expand the bridge to my remote machine, I installed the newer SF432-1012-R3 S7120 removing the SFN515F card that was installed. These cards do not work with the MOLEX DAC cables. My switch is mounted very high and I cannot see the link lights without standing on a chair. The GUI for my switch gave me the clue. I replaced the Molex DAC with an optical cable I had laying around. And everything worked again.

1. Resolution and lesson: Don't restrict the Web interface address and then mess around with the address or bridges
2. Don't change too many things at once
3. Don't use Molex DAC with SolarFlare SF432-1012-R3 S7120 (or "don't buy a bunch of different mfg products and expect them to work"
4. Use a fixed address on a bridge, not DHCP