Hey there!
I have issues with the bandwidth of my jails. I did a download from github to test. Inside the jail this download had round about 300KB/s. Doing the same download outside the jail I get 5MB/s. So something seems to be not correct in my networking for the jail.
My box has four network-interfaces. Two are put into a lagg-interface to serve my needs. The other two interfaces are unused at the moment. On top of the lagg there are vlans an ontop of those are bride-interfaces. One of this interfaces hold the IP of my TrueNAS and is used to connect the jails. The jails shoud use vnet. I must admit, I don't really understand the jail-network-settings. I hope someone can light the dark a little for me. I hope to give all infos needed.
Might be a little picture helps:
The output of ifconfig looks like this:
And the whole jail-config uding iocage is this:
I have issues with the bandwidth of my jails. I did a download from github to test. Inside the jail this download had round about 300KB/s. Doing the same download outside the jail I get 5MB/s. So something seems to be not correct in my networking for the jail.
My box has four network-interfaces. Two are put into a lagg-interface to serve my needs. The other two interfaces are unused at the moment. On top of the lagg there are vlans an ontop of those are bride-interfaces. One of this interfaces hold the IP of my TrueNAS and is used to connect the jails. The jails shoud use vnet. I must admit, I don't really understand the jail-network-settings. I hope someone can light the dark a little for me. I hope to give all infos needed.
Might be a little picture helps:
Code:
IGB2 IGB3 │ │ ▼ ▼ LAGG0 │ │ ▼ ▼ VLAN10 VLAN90 │ │ ▼ ▼ BRIDGE10 BRIDGE90 │ │ │ │ │ │ │ │ ▼ │ ▼ jail2 ─► vnet │ jail1 ─►vnet ▼ TrueNAS-IP
The output of ifconfig looks like this:
Code:
# ifconfig igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6> ether b4:7a:f1:dc:52:9c media: Ethernet autoselect (1000baseT <full-duplex>) status: active nd6 options=9<PERFORMNUD,IFDISABLED> igb1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6> ether b4:7a:f1:dc:52:9d media: Ethernet autoselect status: no carrier nd6 options=9<PERFORMNUD,IFDISABLED> igb2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6> ether b4:7a:f1:dc:52:9e media: Ethernet autoselect (1000baseT <full-duplex>) status: active nd6 options=1<PERFORMNUD> igb3: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6> ether b4:7a:f1:dc:52:9e hwaddr b4:7a:f1:dc:52:9f media: Ethernet autoselect (1000baseT <full-duplex>) status: active nd6 options=1<PERFORMNUD> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6> inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5 inet 127.0.0.1 netmask 0xff000000 groups: lo nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> pflog0: flags=0<> metric 0 mtu 33160 groups: pflog lagg0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: Lagg-Interface Daten options=e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6> ether b4:7a:f1:dc:52:9e laggproto lacp lagghash l2,l3,l4 laggport: igb2 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> laggport: igb3 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> groups: lagg media: Ethernet autoselect status: active nd6 options=9<PERFORMNUD,IFDISABLED> vlan10: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: Daten options=200401<RXCSUM,LRO,RXCSUM_IPV6> ether b4:7a:f1:dc:52:9e groups: vlan vlan: 10 vlanpcp: 0 parent interface: lagg0 media: Ethernet autoselect status: active nd6 options=9<PERFORMNUD,IFDISABLED> vlan90: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: Management options=600703<RXCSUM,TXCSUM,TSO4,TSO6,LRO,RXCSUM_IPV6,TXCSUM_IPV6> ether b4:7a:f1:dc:52:9e groups: vlan vlan: 90 vlanpcp: 0 parent interface: lagg0 media: Ethernet autoselect status: active nd6 options=9<PERFORMNUD,IFDISABLED> bridge10: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: Trusted-Bridge ether 02:6a:88:d2:ce:0a inet 10.10.10.10 netmask 0xffffff00 broadcast 10.10.10.255 id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto stp-rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: vnet0.15 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 14 priority 128 path cost 2000 member: vnet0.11 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 15 priority 128 path cost 2000 member: vnet0.10 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 13 priority 128 path cost 2000 member: vnet0.9 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 12 priority 128 path cost 2000 member: vlan10 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 8 priority 128 path cost 2000000 groups: bridge nd6 options=9<PERFORMNUD,IFDISABLED> bridge90: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: Management Bridge ether 02:6a:88:d2:ce:5a id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto stp-rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: vlan90 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 9 priority 128 path cost 2000000 groups: bridge nd6 options=9<PERFORMNUD,IFDISABLED> vnet0.9: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: associated with jail: resticbackup as nic: epair0b options=8<VLAN_MTU> ether b6:7a:f1:21:d3:ca hwaddr 02:56:99:13:3d:0a groups: epair media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>) status: active nd6 options=1<PERFORMNUD> vnet0.10: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: associated with jail: syncthing as nic: epair0b options=8<VLAN_MTU> ether b6:7a:f1:5b:07:f8 hwaddr 02:1d:13:a3:e3:0a groups: epair media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>) status: active nd6 options=1<PERFORMNUD> vnet0.11: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: associated with jail: testjail as nic: epair0b options=8<VLAN_MTU> ether b6:7a:f1:83:ff:06 hwaddr 02:db:36:24:fd:0a groups: epair media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>) status: active nd6 options=1<PERFORMNUD> vnet0.15: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: associated with jail: mailbackup as nic: epair0b options=8<VLAN_MTU> ether b6:7a:f1:bd:fb:d0 hwaddr 02:3e:ba:91:cf:0a groups: epair media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>) status: active nd6 options=1<PERFORMNUD>
And the whole jail-config uding iocage is this:
Code:
# iocage get -a testjail CONFIG_VERSION:28 allow_chflags:0 allow_mlock:0 allow_mount:0 allow_mount_devfs:0 allow_mount_fusefs:0 allow_mount_nullfs:0 allow_mount_procfs:0 allow_mount_tmpfs:0 allow_mount_zfs:0 allow_quotas:0 allow_raw_sockets:0 allow_set_hostname:1 allow_socket_af:0 allow_sysvipc:0 allow_tun:0 allow_vmm:0 assign_localhost:0 available:readonly basejail:0 boot:1 bpf:0 children_max:0 cloned_release:12.2-RELEASE comment:none compression:lz4 compressratio:readonly coredumpsize:off count:1 cpuset:off cputime:off datasize:off dedup:off defaultrouter:auto defaultrouter6:auto depends:none devfs_ruleset:4 dhcp:0 enforce_statfs:2 exec_clean:1 exec_created:/usr/bin/true exec_fib:0 exec_jail_user:root exec_poststart:/usr/bin/true exec_poststop:/usr/bin/true exec_prestart:/usr/bin/true exec_prestop:/usr/bin/true exec_start:/bin/sh /etc/rc exec_stop:/bin/sh /etc/rc.shutdown exec_system_jail_user:0 exec_system_user:root exec_timeout:60 host_domainname:none host_hostname:mailbackup host_hostuuid:mailbackup host_time:1 hostid:30363150-3530-4e43-5831-31373039434b hostid_strict_check:0 interfaces:vnet0:bridge10 ip4:new ip4_addr:vnet0|10.10.10.22/24 ip4_saddrsel:1 ip6:new ip6_addr:none ip6_saddrsel:1 ip_hostname:0 jail_zfs:0 jail_zfs_dataset:iocage/jails/mailbackup/data jail_zfs_mountpoint:none last_started:2021-10-17 08:50:47 localhost_ip:none login_flags:-f root mac_prefix:b67af1 maxproc:off memorylocked:off memoryuse:off min_dyn_devfs_ruleset:1000 mount_devfs:1 mount_fdescfs:1 mount_linprocfs:0 mount_procfs:0 mountpoint:readonly msgqqueued:off msgqsize:off nat:0 nat_backend:ipfw nat_forwards:none nat_interface:none nat_prefix:172.16 nmsgq:off notes:none nsem:off nsemop:off nshm:off nthr:off openfiles:off origin:readonly owner:root pcpu:off plugin_name:none plugin_repository:none priority:99 pseudoterminals:off quota:none readbps:off readiops:off release:12.2-RELEASE-p10 reservation:none resolver:/etc/resolv.conf rlimits:off rtsold:0 securelevel:2 shmsize:off stacksize:off state:up stop_timeout:30 swapuse:off sync_state:none sync_target:none sync_tgt_zpool:none sysvmsg:new sysvsem:new sysvshm:new template:0 type:jail used:readonly vmemoryuse:off vnet:1 vnet0_mac:b67af1bdfbd0 b67af1bdfbd1 vnet0_mtu:auto vnet1_mac:none vnet1_mtu:auto vnet2_mac:none vnet2_mtu:auto vnet3_mac:none vnet3_mtu:auto vnet_default_interface:none vnet_default_mtu:1500 vnet_interfaces:none wallclock:off writebps:off writeiops:off
Last edited: