Help with 10Gb/MTU/Jails

rknaub

Explorer
Joined
Jan 31, 2017
Messages
75
FreeNAS 11.2 U5
Supermicro X8-dth
Xeon x5680 (x2)
48GB ECC RAM
Melanox 10Gb NIC

Not sure if I should post this in Networking or Jails, but this is probably more networking related.

I have been using plugins for plex, radarr, etc. Recently I have decided to start the jails over and rebuild everything from scratch as a jail instead of a plugin, following this guide here. I have been using jumbo packets with mtu 9000 in my nic options because that is the only way I am able to sustain 10Gb speeds using iperf. This creates a problem when I install a jail because I receive an error about bridge0: invalid MTU, and this causes the jail to not start upon boot. The jail will start manually just fine. If I remove the mtu 9000 from my NIC options, then the jail will start normally upon reboot, but when I test with iperf I am unable to get much more than 3-4 Gb/s. Is there anyway to create the jail with the mtu 9000 so it doesnt say invalid, or am I missing something?

Thank you.
 

rknaub

Explorer
Joined
Jan 31, 2017
Messages
75
I went in and changed one of the two jails I currently have created to DHCP, and now they are both starting automatically.
 

hadesfactor

Cadet
Joined
Aug 21, 2019
Messages
2
@rknaub I am having the exact same issue but dont want to set to dhcp for a few reasons. I spent over a week playing with tuning and different configs across my LAN then just shut off jumbo. I know there is a heated debate whether or not its needed etc but in my case as of yours I cannot sustain anything speeds higher the 2-3gbps.

my stats are:
HPdl360 Gen8 LFF 384gb of ram (i know overkill but free from work) dual 2620s
controller set to HBA mode via HP util
12x3tb SAS in 2x6 disk Z2s
intel Intel X520 on server as well as workstation
Cisco 3750 using the 10gbe X2 using sc->lc om3/4

Ive played with the networking configs and tuning to no end and can't surpass the speeds I am getting via ipref3. I am a net eng not sys eng so my skills are more prevalent on the net side of things. I have been using freenas for approx 3 yrs and it serves as my primary storage as well as my plex server.

I know my bottleneck will be the drives and thats completely understandible but I cant figure out why I cant push it past the 2-3gbps as far as the LAN goes. I was setting to everything to jumbo either 9014/9000 depending on the hardware etc but as OP stated above, bridge0 fails to auto start but starts manually. I tried setting the bridge to assign with mtu 9000 in rc but I feel as I dont have the commands right if possible even (can't remember what I tried as it was months ago) I spent a lot of time reading through the forums here and every I could find and it has helped a lot but this was 1 answer I was unable to find.

My question I guess is do I have to create the bridge 1st or just jump to cloning

# ifconfig bridge0 addm ix0 then set the MTU then set the bridge manually in the jail or am I getting this all wrong
 

rknaub

Explorer
Joined
Jan 31, 2017
Messages
75
I also tried tuning following several recommendations I found here. Eventually gave up and stuck with dhcp, it just works for me - sorry your situation is not the same. Maybe this is a bug in U5? I never had the issue in earlier versions, and have been using Jumbo for a few years.
 

hadesfactor

Cadet
Joined
Aug 21, 2019
Messages
2
I also tried tuning following several recommendations I found here. Eventually gave up and stuck with dhcp, it just works for me - sorry your situation is not the same. Maybe this is a bug in U5? I never had the issue in earlier versions, and have been using Jumbo for a few years.

I've actually been having this issue since 11.2 1st hit which was when I upgraded my hardware to a different box. I don't know if jumbo frames WILL fix the issue but I hate not being able to try it and I'm sure its just my lack of linux/unix/bsd knowledge lol
 

rknaub

Explorer
Joined
Jan 31, 2017
Messages
75
I just wanted to reply to myself here, because this problem just cropped up again for me and I totally forgot about this, but came up on a google. FYI now on 12.0-U8.1 and this problem still exists for me. Also worth noting I'm using Chelsio T520 cards now instead of Mellanox, and 96GB RAM.
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,702
You might want to look at this setting:

iocage get vnet_default_mtu jailname
 
Top