VLAN - explain please on how to do it.

[kdragon75]

Untagged is configured on a per physical link basis. This is primarily speaking from experience with Cisco switches (a lot) and HP switches (some). A VLAN is a VLAN. Which single VLAN is the untagged one is unique to each physical link (or link aggregation). Honestly, I am not trying to be argumentative here.

I always understood it as "untagged" ports tag all untagged traffic as the specified VLAN. Then you cal also allow passing of additional tagged vlans.

 

[Elliot Dierksen]

I always understood it as "untagged" ports tag all untagged traffic as the specified VLAN. Then you cal also allow passing of additional tagged vlans.

Sorry, I am probably splitting hairs. The port/link aggregation configuration is how it decides the VLAN to which untagged traffic belongs on inbound. When it forwards the the traffic, it makes the tagged/untagged decision based on the configuration of the outgoing link(s). Sorry, getting carried away because my day job is network engineer for a Cisco partner. Please excuse my anal-retentiveness. :)

 

[kdragon75]

Thank you for the insight.

 

[Kennyvb8]

All right. Been playing slot now. Can't get it to work.
Despite all your comments.
I figured out according to UniFi forum that standard Lan is vlan1 so I created that in freenas along with vlan400.
But if I do iocage ip4_addr=vlan400 blah blah it doesn't get connected to subnet noir wan. Only vnet works.
If I then turn off vnet from new UI. I can get the connection working but from same Mac address as main em0 (removed lagg0) instaed of its own Mac address.
So basically I can't get what I want. No guides help as there is no guides

Thanks for the help anyway


Sent from my iPhone using Tapatalk

 

[short-stack]

All right. Been playing slot now. Can't get it to work.
Despite all your comments.
I figured out according to UniFi forum that standard Lan is vlan1 so I created that in freenas along with vlan400.
But if I do iocage ip4_addr=vlan400 blah blah it doesn't get connected to subnet noir wan. Only vnet works.
If I then turn off vnet from new UI. I can get the connection working but from same Mac address as main em0 (removed lagg0) instaed of its own Mac address.
So basically I can't get what I want. No guides help as there is no guides

Thanks for the help anyway


Sent from my iPhone using Tapatalk

There are some issues with using vnet enabled jails on top of a VLAN.

https://redmine.ixsystems.com/issues/30276#change-234018

They are been fixing them, and the last few should be fixed by 11.2.

If your VLAN networks work in jails with VNET off, be a little patient and they will be fixed soon.

I use all unifi network gear, with multiple VLANs, and it all works great.

Biggest thing is to make sure everything matches.

Make sure all VLANs are tagged on trunk ports and make sure that the proper VLAN is set on access ports.


Sent from my iPhone using Tapatalk

 

[short-stack]

All right. Been playing slot now. Can't get it to work.
Despite all your comments.
I figured out according to UniFi forum that standard Lan is vlan1 so I created that in freenas along with vlan400.
But if I do iocage ip4_addr=vlan400 blah blah it doesn't get connected to subnet noir wan. Only vnet works.
If I then turn off vnet from new UI. I can get the connection working but from same Mac address as main em0 (removed lagg0) instaed of its own Mac address.
So basically I can't get what I want. No guides help as there is no guides

Thanks for the help anyway


Sent from my iPhone using Tapatalk

Can you share screenshots of your latest configs? You’ve made changes but I want to make sure you’ve updated everything you need to have updated.


Sent from my iPhone using Tapatalk

 

[nathank1989]

Hey all.
I, too, have a UniFi setup, but I have a board with two NICs and I cannot for the life of me figure out how to get igb0 to port 5 on my switch with the port profile set to ALL and my igb1 interface on port 6 of my USW with profile set to VLAN 20. Everything goes through igb0 and I cannot get things to accurately route through igb1 and on vlan20

Anyone who has 2+ nics with mgmt on one, jails/vms on the other working plugged into the same USW with different port profiles and vlans and can communicate to web, and intervlan, please share how you achieved that because I'm about to kick this box out the window.

 

[short-stack]

Hey all.
I, too, have a UniFi setup, but I have a board with two NICs and I cannot for the life of me figure out how to get igb0 to port 5 on my switch with the port profile set to ALL and my igb1 interface on port 6 of my USW with profile set to VLAN 20. Everything goes through igb0 and I cannot get things to accurately route through igb1 and on vlan20

Anyone who has 2+ nics with mgmt on one, jails/vms on the other working plugged into the same USW with different port profiles and vlans and can communicate to web, and intervlan, please share how you achieved that because I'm about to kick this box out the window.

The reason for that is because you are sending a trunk will all VLAN from Port 5 to igb0, and igb0 is set as the default gateway for the device.
Create a new port profile in the Unifi Controller, with all of your VLANs except for VLAN 20 like this

And then assign only the VLAN 20 profile to port 6 and plug that in to igb1.

Once your Unifi set up is complete, go in to FreeNAS networking make sure that you don't have a 'VLAN 20' interface created for the igb0 interface, and then for Interface igb1 just set the IP you want on VLAN 20. You do not need to tell igb1 that it's on VLAN20 since it'll be the only network on that link.

 

[Kennyvb8]

Can you share screenshots of your latest configs? You’ve made changes but I want to make sure you’ve updated everything you need to have updated.


Sent from my iPhone using Tapatalk

Sadly no. Had to undo it
To get everything working again


Sent from my iPhone using Tapatalk

 

[StarkJohan]

Did you make any progress? What short-stack suggests seems pretty easy but that approach doesn't work for me as I can't see how to make a jail or VM specifically use a certain NIC.

 

[kdragon75]

Yeah FreeNAS needs a virtual network topology manager.

 

[nathank1989]

Yea, I got so frustrated trying to set this up correctly, I just said F-it and made the two ports 1 lagg interface and vlaned the jails
It isn't what I wanted, but might be a better solution while I am bittorrenting a few large projects.

 

[chipgoon]

Thought I'd chime in. After days of tinkering I finally have something that works well enough for my iocage setup. Hope it helps someone, or someone improves on it. YMMV.

My setup

• FreeNAS11.2-Beta3. Have a couple of iocage jails using VNET networking (VNET is a weird little bridge setup used by iocage, which gives jails their own network stack, apparently non-natted).
• Single physical NIC for all traffic. I access Freenas from untagged main LAN.
• I want jail to have access to a particular VLAN, running in a different subnet from main LAN.

Testing procedure
I tested all sorts of combinations

• Make VLAN interface on host and assign it directly to jail without VNET
  Could not make it work, maybe due to inexperience
• Make a bridge on host, add only host VLAN and VNET
  No networking inside jail. Weird errors issued by ping, like 'host down' or 'incorrect parameter'.
• Make bridge on host, add NIC plus VLAN plus VNET
  This one was a doozy, took a long time to troubleshoot. My VLAN networking in jail (and host) was incredibly slow and intermittent. Turns out Host VLAN chokes up when it's bridged with it's parent NIC. FreeBSD dislikes this, probably makes routing loops in kernel.

Finally found a good hint explaining you need to create a VLAN in the jail rather than the host.
Then I went through a lot more tests as I tried to isolate the jail to just use the VLAN, trying lots of permutations of jail & bridge subnetting & routing options.

In the end I realized that VNET just wants to be bridged with the NIC, and reside in the same subnet as the host. Thus Jails must be networked to main LAN in Host. However, the good news is that it's pretty easy to give jails access to VLAN at the same time.

My solution

1. Configure VNET as usual in FreeNas
   a) System->Tunables->add
   cloned_interfaces / value=bridge0 / type=rc #this creates bridge0 on host
   b) System->Tunables->add
   ifconfig_bridge0 / value=addm re0 up / type=rc
   #re0 is the name of my NIC. Modify as required. Adds NIC to bridge. VNET will add itself to bridge later when you start the jail
   
   
2. Configure jail for VNET as usual.
   Give it an IP in your main LAN or use DHCP.
   In basic properties IPv4 interface should be vnet0.
   In Network Properties interfaces should be vnet0:bridge0.
   Optionally you may allow raw_sockets in jail properties, I don't think it makes a difference
   
   
3. Edit jail's /etc/rc.conf and add the vlan. Probably easiest is to start the jail and from a shell edit /etc/rc.conf
   
   Code:

     ## Add this at end of jail's /etc/rc.conf
     ## For this example I'm using vlan 6, subnet 192.168.6.0/24, jail vlan ip 192.168.6.66
     cloned_interfaces="vlan6"
     ifconfig_vlan6="inet 192.168.6.66 netmask 255.255.255.0 vlan 6 vlandev epair0b"
     ## epair0b is bridge interface created by VNET in jail
   

   
   
   
4. Restart jail
   From jail's shell you can check out network interfaces using
   Code:

   ifconfig -a

   and check routes with
   Code:

   netstat -nr

That's it, you should be able to ping and access stuff on your VLAN from your jail and viceversa, as long as the rest of your network is set-up properly of course. Jail should start and stop cleanly.
You should not need to tinker with the routing at all. Unless your VLAN and main LAN are in same subnet, which introduces routing and interfacing and bridging issues. So don't do it.
As far as I can tell the VLAN tagging is propagating properly through the bridges.

Other ramblings

• As to why this method works so simply, I think freeBSD must create it's own special internal bridge between VLAN and Parent interface automatically. This also would explain why VLAN was stuttering and freezing when I manually bridged a VLAN and it's parent NIC in FreeNas host. This behavior is noticeably different on a network platform such as Open VSwitch, which I had a lot of 'fun' configuring in a Proxmox installation (very nice virtualization platform also based on FreeBSD).
• I did not need to configure or create VLAN on FreeNas host, only in jails.
• In this setup I have access to jail from both main (FreeNas) LAN and a VLAN. I could not confine jail to communicate over VLAN only, but for my use case this is good enough.
  
• I am not quite sure if updating the jail might affect the edits on /etc/rc.conf. I'll leave that for someone else to figure out.
• Maybe the required jail edits could be added straight from GUI, probably in Jail properties -> exec.start field. If someone knows how to do it I'd like to know
• I added a couple of extra system tunables (net.add_addr_allfibs value=0 type sysctl & net.fibs value=4 type=loader) which are supposed to configure VLANs in separate routing rables.. I don't believe this is necessary, but read it as a suggestion somewhere along the way.
• I do have MTU on my NIC set at 1503 which I believe is required (minimum) given the frame size with 802.1Q

 

[chipgoon]

I did some more research and it is possible to isolate the Jail networking to use just a VLAN defined in the host and nothing else. No VNET required, just assign the jail to the VLAN interface and specify a new IP which works as an alias.

There are some routing issues though. The jail inherits the host's networking stack so the default route (gateway) is in a different subnet from the VLAN, which prevents it from having internet access in my case.
I've fixed this by creating a separate routing table for my VLAN, in its own FIB in Freenas.
For example my VLAN is named vlan6, in its own subnet 192.168.6.0/24, gateway 192.168.6.1, FIB = 1

Code:

route add -net 192.168.6.0/24 -iface vlan6 -fib 1
route add default 192.168.6.1 -fib 1

You can add this routing permanently by using system tunables. Also note the other system tunables (net.add_addr_allfibs & net.fibs) I mentioned in my previous post, as they will be required.

For the Jail, you specify the FIB to use in the GUI under network properties, exec.fib

I also had to fix the jail's DNS resolver, as it inherited /etc/resolv.conf from FreeNas which does not work in my case. Modify in the Jail GUI under network properties, resolver. Also, select berkeley packet filter.

 

[StarkJohan]

There are some routing issues though. The jail inherits the host's networking stack so the default route (gateway) is in a different subnet from the VLAN, which prevents it from having internet access in my case.
I've fixed this by creating a separate routing table for my VLAN, in its own FIB in Freenas.
For example my VLAN is named vlan6, in its own subnet 192.168.6.0/24, gateway 192.168.6.1, FIB = 1

If this works as you described and is stable you have really made my day. I do have a stable setup for now using vnet to force a jail onto a VLAN and a separate physical NIC but avoiding vnet should really make things easier as the vnet bridge setup is messed up if I restart the jail in question.

If I understand your FIB solution correctly the VLAN could be tied to a separate physical NIC, right?

 

[chipgoon]

If I understand your FIB solution correctly the VLAN could be tied to a separate physical NIC, right?

I don't see why not. Pointing the VLAN to the proper parent interface during creation ought to be enough.

As advertised, just set your system tunables. Then in jail GUI set your vlan & IP address, set berkeley packet filter, set exec.fib, and set resolver to something like nameserver 192.168.6.1

Still working like a champ for me in FreeNAS-11.2-RELEASE-U1

 

[HolyK]

Guys be very careful what are you bridging together without separated network stacks. If your jail interface ends up in promiscuous mode you have security hole as hell if the jail gets compromised.

 

[raidflex]

Guys be very careful what are you bridging together without separated network stacks. If your jail interface ends up in promiscuous mode you have security hole as hell if the jail gets compromised.

So is using Vnet for the jail then necessary for a properly secured Vlan using a single physical LAN connection?

 

[HolyK]

@raidflex Not necessarily. Important is to know your own environment and do the homework. See the "NAS" in general is (was?) accessible only from LAN or from outside via some secured way (VPN, Citrix, whatever...). FreeNAS and the way how we (me included) are using it is not a "NAS" anymore but more like a frame with several VM environments. Meaning the extended security has to take place a moment one of the Jails has access to/from outside. So few points in general from security point of view.

- Promiscuous mode is BAD thing (excluding host system) in general especially for jails with outside world connectivity.
- VNET is ideal as each Jail gets its own network stack so controlling the data flow is much more easier than on NATted.
- Isolate your Jails. At least those with outside access from the rest of the network.
- Get an L2 switch. Seriously...
- ... and a proper router. Not some sh!tty SOHO thing which has only "Enable security" checkbox in WebUI and nothing else.
- VLANs are superb way for isolation. At least make your "safe/internal" jails on one and the "risky/open" VLAN. By default block communication between them. Also block communication between jails inside the "risky" VLAN. Then implicitly allow only those you really need.
- Drop everything coming from outside to you NAS or Jail IPs. Seriously you don't need to have anything reachable from outside by default.
- dst-nat only what is required. Then allow only established and NATted connections (so fabricated packets can go to hell). (If you have any sort of web-server then more work will be required here)
- If you have any sort of "guest" network for your friends or family visiting you time to time - isolate that completely from everything else. It is really not funny having your network security hardened from outside access then let your friend to access your network over Wifi from his laptop which has an active rootkit. Not cool!
- Restrict connections to your HOST system. Even your beloved wife which of course has serious IT education from you can do a mistake and click some dirty links on FB. Logically you can not cut her from having access to all of the TVshows and Movies but she shouldn't be able to SSH or whatever to your FreeNAS host. Getting host system compromised is just a plain disaster. Make sure you and you only can access it.

There is much more things to do but the above are the main ones. I would say security is similar to the data backup. Two kind of people... Ones which care about it seriously and the ones which did not (yet) found their personal information (financial, photos, accounts, ...) on the "Internet".

(sorry for bad grammar :] )

 

[raidflex]

Thanks this is good, I was mainly looking for Vlan setup with Freenas as I have not set this up until recently.

I actually have a PFsense firewall, Cisco managed switch, and UniFi AP with multiple Vlans setup for security. I also have OpenVPN setup on PFSense for my own secure access back to my network. I like to keep my network secure and this is why I was interested in the proper way to setup Vlans on Freenas. I know in the past there has been issues with Vlans and Freenas and that is mainly why I never looked into it much. Recently I needed to setup a InfluxDB server for graphing my IOT device usage/events and I wanted to keep the jail that InfluxDB was running in on my IOT VLAN and not on my main LAN since this needed to communicate with Hubitat Hub. Now I could add a rule to explictly allow the InfluxDB jail to communicate with my Hubitat, but I wanted to just keep it completely off my main Vlan. I figured I have all this power sitting there on my Freenas server which is on 24/7 so why not use it, instead of setting up a separate Raspberry Pi.