Register for the iXsystems Community to get an ad-free experience and exclusive discounts in our eBay Store.

Force Jails To Use Alternate NIC

Steven Wormuth

FreeNAS Experienced
Joined
May 2, 2017
Messages
120
Good day,

I purchased a second NIC and installed it into my FreeNAS system. The card is recognized, and "netstat -i" shows a new interface "em0" in the list. I currently have multiple jails working perfectly. The FreeNAS is running on xxx.xxx.1.100. Jails running on xxx.xxx.1.110, 111,112, etc... The purpose of the second NIC is to move these jails off to a different network. Ideally, I'd like the second NIC to be assigned something like xxx.xxx.2.100, and the jails to move to xxx.xxx.2.110, 111, 112, etc...

So how should one accomplish this?

Right now, if I connect a cable to the new NIC, the router will assign the IP xxx.xxx.2.100 and I can no longer access any of the jails on their original IP addresses. Even if I change the jail IP's to what I want them to be in the GUI, I can't get to the jails anymore. I really need some step-by-step help on what changes to make.

Thanks for any help you can offer!
 

sretalla

FreeNAS Expert
Joined
Jan 1, 2016
Messages
2,325
If you want your jails on a NIC that's on a new subnet, you can't network them to the old subnet with a cable to the same switch (unless you configured VLANS on the switch and use the assigned ports) without a router between the subnets.

How were you planning to connect the subnets?

Code:
                                          Other Client(s)
Lan 1 x.x.1.0/24 --|----------------------------|---------------------------|-----

                   Freenas                                                 Router

Lan 2 x.x.2.0/24  --|-----------|-------|---------|---------------------------|-----
                              Jail1   Jail2     Jail3..
 
Last edited:

Steven Wormuth

FreeNAS Experienced
Joined
May 2, 2017
Messages
120
Hi there,

I have a PFSense router with multiple interfaces. One interface is specifically for FreeNAS, with a cable from its interface in PFsense to the FreeNAS box, and a static IP for the FreeNAS NIC. FreeNAS gets it's internet on this interface.

Another interface in PFSense is for all the automation in the house. From that interface on the PFSense NIC, I have a physical cable to a wireless access point in bridge mode so IP's are assigned by PFSense to anything home automation that connects wirelessly. I need this second FreeNAS NIC to connect to THIS network. There is a port on this access point to hardwire a device, and my thought was that I could plug the new NIC in FreeNAS hardwired to this port. I was going to static IP assign an IP in the second subnet to this NIC, and tell my jails to route through that.

If that explanation makes sense?
 

sretalla

FreeNAS Expert
Joined
Jan 1, 2016
Messages
2,325
So your plan sounds great so far (more-or-less matching my diagram... if you can call it that)...

What you need to look at is the firewall rules and routing on your pfSense box to connect the two subnets together and allow the traffic you want to get through (and only that traffic, if you want the benefits of the segregation you're setting up).
 

Steven Wormuth

FreeNAS Experienced
Joined
May 2, 2017
Messages
120
That's about the best diagram I could've come up with. LOL

So when I connected the cable between the new NIC in FreeNAS and my access point, the PFSense box did assign the IP to the new NIC. At that point I couldn't access the jails anymore, and didn't want to go any further until I knew what I was doing. I assume that's because of a conflict somewhere between the new NIC being assigned IP's in the second subnet, and the FreeNAS settings still telling the jails to assign an IP in the original one.

At this point, I need to do the following (as I see it).

1. Set up the static IP for the new NIC in PFSense's DHCP.

2. Change the jails to use static IP's in that subnets range in FreeNAS.

3. Change the jails settings to use the appropriate interface.

The new NIC is "em0" in FreeNAS. But right now the jails are using "vnet0". Which settings should I change to get this to properly take effect? I have a troubleshooting rule enabled in PFSense which I can toggle to pass all traffic to rule out a firewall issue. So right now all traffic should be passing through to all interfaces internally on PFSense.

Thanks!
 
Joined
Apr 6, 2019
Messages
3
Hello, did you ever get this working? If so, wondered if you might share how you configured the jails network settings?
 

KevDog

FreeNAS Experienced
Joined
Nov 26, 2016
Messages
371
Ok guys - he has two cables coming into his FreeNAS installation. On on the x.x.1.0/24 network and the other on the x.x.2.0/24 network.
He states one nic is known as em0, however I'm going to guess the other is em1. He needs to create a bridge for the x.x.2.0/24 network (lets call it bridge 2 which corresponds to the xx.2.0 network) within FreeNAS. Bridges are like virtual switches. They need members or "virtual cables" to be plugged into it similar how you would plug networking cables into a physical switch. I this case the members would be whatever em is associated with the x.x.2.0/24 network and the other members would be the jails. Jails are created with vnets (as described above) and each interface for each jail would be associated with vnet0:bridge2. Routing traffic in between the x.x.1.0/24 and x.x.2.0/24 network needs to be done at the router level. Because the original poster was using pfsense as the router, he needs to allow traffic to pass back and forth between the networks by creating firewall rules that alllow this. In addition each network needs to be assigned its own DHCP server and DNS server.
 

amlamarra

Newbie
Joined
Feb 24, 2017
Messages
34
Thank you, @KevDog. I did get this figured out yesterday. Personally, I was confused as to whether or not I would need the jails on the second bridge to be using "vnet1" instead of "vnet0". Apparently, that doesn't matter. I was also confused because when I had my FreeNAS system setup, I never created a "bridge0" interface in the GUI, nor did I add em0 to be a member of that. Those were automatically done. I have since gone and created bridge0 and bridge1 in the GUI with em0 and em1 as members, respectively. I'm using this to put certain jails in a DMZ on my router/firewall (EdgeRouter X).

There's just one more thing that feels a bit odd for me. When I only had 1 NIC on my system, each iocage jail had the "ip4_addr" value set to vnet0|192.168.1.xx/24. Now, I have to remove the vnet0| part in order for the jails to have access to the Internet (or even the LAN).
 

KevDog

FreeNAS Experienced
Joined
Nov 26, 2016
Messages
371
Ok -- just some followup questions
Where are you determining the ip4_addr value? Are you setting this manually or are you using a command to find this value?
 

amlamarra

Newbie
Joined
Feb 24, 2017
Messages
34
Ok -- just some followup questions
Where are you determining the ip4_addr value? Are you setting this manually or are you using a command to find this value?
To get the value:
Code:
root@freenas ~ # iocage get ip4_addr blog
192.168.5.44/24


To set the value:
Code:
root@freenas ~ # iocage set ip4_addr="192.168.5.44/24" blog
ip4_addr: 192.168.5.44/24 -> 192.168.5.44/24


You can do the same in the GUI when editing a jail. Just make sure to set the "IPv4 Interface" dropdown to "------". I just prefer the CLI as clicking through the web GUI seems to take forever.
 

KevDog

FreeNAS Experienced
Joined
Nov 26, 2016
Messages
371
Ok I just wanted to confirm -- I'm honestly not sure why whatever process wants to assign your jail IP address with a vnet(x)| prefix. I'm guessing its something with your config but it's difficult to know.

I set all of my jails with static IP addresses so I haven't noticed this behavior on my system however without further testing I'm not saying it wouldn't happen to me.

I agree the command line is rather useful and quick, however I'd really encourage you to make these changes in the GUI -- if only for the simple fact that the changes will survive a reboot. If you're doing some quick and simple testing and don't care about the changes being persistent, by all means the CLI is the way to go.
 
Top