Questions / Issues with networking config

[maverickws]

Hi there,

I am sorry I am new to this forum, first time FreeNAS user.
I am looking to test and use FreeNAS as NFS storage for VM disks.

The server has the following specs:

• Intel Xeon E3-1246V3
• 4x RAM 8192 MB DDR3
• 1x 240GB SDD
• 2x HDD SATA 2TB Enterprise
• Realtek 8111/8168/8411 Gigabit Ethernet
• Version: FreeNAS-11.3-U4.1

The server is with other servers located at a datacenter. For all purposes it has a public IP, delivered via DHCP by the provider. There are 4 additional private VLAN's for private use that we use to connect the different servers.
After installing FreeNAS I proceeded to setup the network part to try to setup the VLAN's.
Aside from finding several bugs to finally be able to setup the vlan, the vlans can't have MTU lower than 1492 and only one interface can have DHCP. So for now I am going to focus on these two bugs: DHCP and MTU.

I have read about the DHCP issue here: https://www.ixsystems.com/community/threads/multiple-interfaces-with-dhcp.58365/
I honestly don't understand what is so hard to get about the use of DHCP in several interfaces and why is this an issue. We use pfSense as firewall which is also FreeBSD based, the network performance with a plenitude of multi-dhcp-services all kinds of network configs, links, whatever. So it's not an underlying system issue.

I managed to add the VLAN with a manual IP address. I can ping destinations within that /24 network.
I added a static route for network/24 go through gateway for that network, I lost access to the FreeNAS and had to access via console to change the default route, because the default route was changed to that of the static route?!?!?!
Is FreeNAS ignorant of Classless Inter-Domain Routing?
I have 2 VLAN's for this server, each on its /24 network. It gets an IP from that network, a gateway TO THAT NETWORK, and period. It looks like we're talking about implementing some quantic solution that has never been seen before?!?

DHCP on multiple interfaces:
We too use ARP binding to provide IP addressing. But not only. We have a redundant firewall setup using CARP, in case one firewall dies we have failover. The DHCP server provides DHCP option 26 to ensure all clients use MTU 1400 as per our spec, and provides other options of our specifications. FreeNAS doesn't even allow to select MTU under 1492, its hardcoded, I am required to add a post-init command. I find this extremely limited and definitely having an hard time to enjoy FreeNAS.

Another, even on the simpler interface config where you just have to get your addressing from DHCP (only one interface, nothing else), you can't get the IPv6 from the DHCP server. Even on console, you select DHCP for IPv4 and then you have to manually put the IPv6.

Right now I am stuck on the part that I can't send traffic to the private data network /24 and adding a static route changes the default route cutting me off. I have to go on console and add default route to the outside again. There are two private networks to assign to this server, the administration network and the data network. How can this be achieved?

I truly appreciate some comments, and if anyone knows some more fit-to-purpose open-source software fit to small company usage that doesn't come with this retrograde limitations, we would immensely appreciate.

[mod note: integrated a correction -JG]

 

[jgreco]

FreeBSD and FreeNAS are capable of relatively sophisticated networking compared to most NAS products. Some of us run entire ASN's with FreeBSD. With FreeNAS, I have noticed that there tends to be breakage in code area B when changes are made to code area A, which is probably a side effect of networking having the potential to be relatively complicated, especially to reinstantiate on the fly correctly, because you often have to undo things in the correct order or you get side effects.

As for the DHCP, I explained in that thread that you get nondeterministic behaviour when you have multiple DHCP interfaces, and you shouldn't be running DHCP on a NAS anyways because you can get connections dropping when the lease ends, which you don't want for a fileserver. I don't get why this is so hard for people to grasp.

No idea about the MTU thing. MTU's smaller than 1500 are extremely unusual in the IPv4 arena, and even in the IPv6 arena, where it was popularized due to widespread use of tunnels, it is now discouraged. If you have some actual engineering reason for doing this, what is it? Is it just that your Realtek ethernets break on the larger MTU with vlans (a known problem)? If so, why not just get a decent Intel card? Even the Intel desktop chipsets generally work very well with FreeBSD.

 

[maverickws]

@jgreco thank you for your reply.

Indeed FreeBSD is capable of handling incredibly sophisticated networking! We use FreeBSD based software as router/firewall and we never found any nondeterministic behaviour having multiple dhcp interfaces. That is what puzzles me, and honestly I don't understand the nature of that affirmation.

We have a multitude of systems with different network interfaces and dhcp enabled on them. Honestly I cannot recall ONE situation where having multiple DHCP interfaces was the issue.
The MTU is a specification of the platform we're using. Being it common or not, having hardcoded to not be able to put under 1492 is an abnormality. There is confirmation by @Samuel Tai at this topic:

Unfortunately, this limit of 1492 for option mtu appears to be hard-coded in /usr/local/lib/python3.7/site-packages/middlewared/plugins/network.py, line 708, in FreeNAS 11.3.

So someone went out of their way to do this, which I find unbelievable.

Anyway, please can you then help me in order to proceed with testing of FreeNAS, to explain how can I put networking to work? because right now I have an interface VLAN with a static IP address (the same assigned in the DHCP server). I have added the static route and by doing so the default gateway got changed to that of the static route. Can you explain this behaviour? If I add a route for network 1.2.0.0/16 I want traffic for destinations from 1.2.0.0 to 1.2.255.254 all to go through that gateway. But that is not the default gateway. I'm really having an hard time grasping why this is happening, and honestly I do not understand that of the dhcp on multiple interfaces misbehaving? Maybe that was an Intel card issue, they also have their handful of them. *shrug*

 

[maverickws]

Well,

I'd like to inform that I solved this issue by opting with Rockstar. Found the software perfectly capable of having multiple auto interfaces, and since apparently is nmcli bound configuring the vpn interface with dhcp on was no trouble either.
So I am indeed very sorry I was curious to try FreeNAS, but rockstor appears to be less bloated and more straight to the point. Kudos for this project if anyone's looking for an alternative there's a solid contender.

 

[pschatz100]

Generally speaking, I would consider this a classic FreeBSD is not Linux thread. You are having some difficulty setting up a complex network using Realtek nics. It is no secret that Realtek support in FreeBSD is not as good as is Intel support - this is not entirely surprising. I suspect you may have been able to do what you wanted if you had been using recent vintage Intel nics.

At the end of the day, you moved to a Linux based system. I'm not surprised the Realtek nics work better under Linux. Good luck.

 

[maverickws]

@pschatz100

You really should have saved your time instead of typing that "reply". First, I use more FreeBSD than linux or at least it'd go 50/50. The two machines I have in front of me right now, using them every day for decades, are FreeBSD based. Being FreeBSD based means something wonderful, not being handicapped.
I can only be sorry for all that "complexity" that Realtek cards do in your head. There are some Realtek cards that aren't over the top and some are reasonable. I have several FreeBSD based systems using Realtek cards and I never had issues with them. But yeah you know, Intel nics are bulletproof and don't fail nor have issues. We all know that.

Now, specifically about this topic, the DHCP option is related to the network implementation on FreeNAS. It has NOTHING to do with FreeBSD or not, FreeBSD is able to manage wonderfully several interfaces with different configuration sets on them on the same machine. The handicap is not on the underlying OS.

The handicap is on the developers who insist that they're choice of forcing people to only have one DHCP interface is the good choice. It's their choice and how they see it so it must stand correct above all others. People shouldn't be able to have more than one DHCP interface.

Now, moving to Rockstor, I had no issues in setting a static IP on the interface leaving the DHCP option enabled, which means the interface does go to DHCP to get a bundle of info, but it has its IP from the start. I have used this technique in many places, never fails. Only "failed" with FreeNAS. But this issue isn't FreeNAS by self, it's a developers option.