Jail via physical interface

[stranger]

This is a question that seems to have been asked many times but I can't find a suitable answer.

I've been happy with my collection of jails for personal use but I've decided to use the spare capacity to host some jails that I'll share with others. Added to that I've now got a more capable router so I want to segregate my network. The idea is that the admin of the NAS will run on 10.0.0.x, the personal jails on 10.0.1.x and the shared will run on 10.10.0.x.
I've got interfaces igb0, igb1, igb2 and igb3 available to me but let's just simplify and say that I want the
10.0.x.x on igb0
10.10.x.x on igb3

I've been quite happy using VIMAGE and so I thought that the simplest way would be to create another bridge interface, bridge1 and then delete the epair interface from bridge0 and add it to bridge1.

so three points:
Is there a better/more standard way of doing this?
How can I get the new bridge to survive reboots. I've tried with rc.conf but that failed
Can this be added as a feature request - perhaps preferable as a warden option. Alternatively perhaps specifying the physical interface's IP or name when creating the Jail. Perhaps it could do something clever like assign the physical interface based upon its subnet.

 

[cyberjock]

I'd put in a feature request (but please look to ensure that another one doesn't already exist first).

I'm not aware of any easy way to get the new bridge to survive reboots. I cringe at the idea of editing those files though. :/

 

[stranger]

Editing files is fine for me and if this were FreeBSD I'd already have done it but you never quite know how an appliance works internally.
Does anyone know how the bridge0 is configured and what scripts are used. Then I could write a little hack of my own without stomping over FreeNAS. I checked in the sqlite DB and it doesn't seem to be configured there.

Could you please point me to where enhancement requests are submitted.

Does anyone know if this feature is in 9.3?

 

[stranger]

btw I did a test by configuring two different jails on the 10.10.x.x nets, one with VIMAGE and the other without.
Without VIMAGE, I went to the host system and changed that jail's interface to the one connected to the 10.10 network.
On the VIMAGE jail I switched the bridge that it used so that it is in the same bridge as the 10.10 interface.

Both of them can ping the IP of that's assigned to the physical interface but neither of them can ping the router in that subnet (10.10.0.1).
The host system can ping it and traceroute confirms that it's using the correct interface.

Therefore there's something I'm missing - the physical interface has to be configured to allow traffic to pass thru it from the jails. Any idea where this is done?

Thanks for any help

 

[stranger]

Actually one thing that I'm not clear about is what the values for bridge mean in the jail network settings in the GUI.
I had assumed that it was to connect the jail to the physical interfaces but obviously that's not happening (unless it's a bug). the docs weren't very clear about it.

 

[stranger]

I've found a bug report/enhancement request that already exists and it looks like they've no intention of fixing it.
https://bugs.freenas.org/issues/3909

I'll have a look at it myself to see if I can fix it.
Here's the problem, I don't have the faintest idea where it's configured - which scripts etc.
To save me a month of searching where I might just give up in frustration, can someone point me to where the bridge is configured please.

Thanks

 

[dlavigne]

Did you get anywhere with this?

 

[stranger]

Well I have got an alternative bridge set up however traffic is not being passed thru it. I can ping the IP of the physical interface attached to the bridge from the jail but I can't go outside the host to the router and beyond.
My bridge looks exactly the same as the one setup by default:
I set up one bridge per interface in the start up scripts. Here the default physical interface is igb2:
bridge2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 02:51:c2:46:04:02
nd6 options=1<PERFORMNUD>
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: epair3a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 16 priority 128 path cost 2000
member: epair9a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 24 priority 128 path cost 2000
member: epair6a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 19 priority 128 path cost 2000
member: epair4a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 17 priority 128 path cost 2000
member: epair2a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 15 priority 128 path cost 2000
member: epair0a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 13 priority 128 path cost 2000
member: epair5a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 18 priority 128 path cost 2000
member: epair1a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 14 priority 128 path cost 2000
member: igb2 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 4 priority 128 path cost 20000

The bridge that I set up looks like:
bridge3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 02:51:c2:46:04:03
nd6 options=1<PERFORMNUD>
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: epair10a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 25 priority 128 path cost 2000
member: igb3 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 5 priority 128 path cost 20000

I set up the epair10a/b interface in the jail and can use it to ping the interface but not the router.


Now I could patch this myself if I could figure out how the bridge is setup by the jail scripts. Can anyone point me to this?? Please!!!

In the bug report it has been suggested that I use a netgraph interface (ngctl). Is this worth doing?
FreeBSD is the latest flavour of UNIX that I've picked up (about the 6th or more) but I'm still learning its peculiarities....

Thanks

 

[dlavigne]

Note that in the upcoming 9.3, you can specify an interface when creating a jail.

 

[stranger]

Note that in the upcoming 9.3, you can specify an interface when creating a jail.

I didn't see that in the release notes of the beta. I also see that the bug report that I mentioned that the feature is aimed at a future release.
Where did you find this out?

 

[dlavigne]

http://doc.freenas.org/9.3/freenas_intro.html#what-s-new-in-9-3 (2nd last sentence) and http://doc.freenas.org/9.3/freenas_jails.html#adding-jails.

 

[stranger]

Thanks for the info. I'll put up with it for now and install 9.3 when it's out of beta.

Thanks very much.

I guess that https://bugs.freenas.org/issues/3909 should be marked resolved for 9.3