Iocage and IPv6 autoconf ?

Joined
Oct 7, 2016
Messages
29
Hi,
I have been unable to find an answer in the docs or using google.
How are you supposed to configure an autoconf IPv6 address in an iocage jail ?
I am using vnet jails and have been able to get it working manually by setting rtsold_enable=YES and starting rtsold service after the jail is booted.
But what is the magic set of options to create a jail and having IPv6 autoconf work when the jail is started. ?

I am using 11.1-U1.

Thanks,
Paul
 

alvistar

Cadet
Joined
Feb 22, 2018
Messages
2
Hi,
I am facing issues as well with IPV6 and autoconf.

Are you using VNET or not?

With VNET I can set iocage "ip6_addr:vnet0|accept_rtadv"

It actually changes configuration in the jail to start rtsold but it doesn't start at boot.
I can manually start with "service start rtsold".

Additionally this is not enough, because it's not getting link local address.

So basically I need to:

ifconfig epair0 inet6 auto_linklocal
service rtsold start

----
I had no lucky without VNET.

a.
 
Joined
Oct 7, 2016
Messages
29
I have been able to get it working manually just like you did but it needs to work automatically when the jails start for it to be acceptable for production jails.
I have tried only VNET, that's what I needed on the old warden jails I still have, those work fine with IPV6 autoconfig BTW.
As the warden jails are working it looks like the it's an iocage problem and not a jails problem.

Paul
 

alvistar

Cadet
Joined
Feb 22, 2018
Messages
2
Thanks Paul.

Let's wait from developer. I didn't succeed in automate at boot, nor rc script, no tunables...
 

Brandon Schneider

Arbiter of iocage
iXsystems
Joined
Nov 12, 2015
Messages
12
Hey guys, setting ip6_addr to 'vnet0|accept_rtadv' should work in 11.2-RC1 when it's released :)

Lemme know how it goes!
 

Patrick M. Hausen

Hall of Famer
Joined
Nov 25, 2013
Messages
7,737
Unfortunately it doesn't.

Running 11.2-RC1, created new jail with VNET and IPv6 autoconfiguration activated. I kept the two lines in /etc/rc.conf:
Code:
ipv6_activate_all_interfaces="YES"
rtsold_enable="YES"


The result:
Code:
root@mineos:~ # ifconfig
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
	options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
	inet6 ::1 prefixlen 128
	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
	inet 127.0.0.1 netmask 0xff000000
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
	groups: lo
epair0b: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=8<VLAN_MTU>
	ether 02:ff:60:c5:1d:60
	hwaddr 02:35:d0:00:0c:0b
	inet 217.29.46.103 netmask 0xffffffe0 broadcast 217.29.46.127
	nd6 options=1<PERFORMNUD>
	media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
	status: active
	groups: epair


Any ideas? Without reliable SLAAC, I still cannot migrate my bhyve VMs to iocage jails ...

Kind regards,
Patrick
 

Patrick M. Hausen

Hall of Famer
Joined
Nov 25, 2013
Messages
7,737
When I manually configure accept_rtadv and auto_linklocal, the link local address works:

Inside the jail:
Code:
epair0b: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=8<VLAN_MTU>
	ether 02:ff:60:c5:1d:60
	hwaddr 02:35:d0:00:0d:0b
	inet 217.29.46.103 netmask 0xffffffe0 broadcast 217.29.46.127
	inet6 fe80::ff:60ff:fec5:1d60%epair0b prefixlen 64 scopeid 0x2
	nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
	media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
	status: active
	groups: epair


From my Mac:
Code:
$ ping6 mineos.local
PING6(56=40+8+8 bytes) fe80::1cff:bafc:ea6c:722c%en0 --> fe80::ff:60ff:fec5:1d60%en0
16 bytes from fe80::ff:60ff:fec5:1d60%en0, icmp_seq=0 hlim=64 time=5.656 ms
16 bytes from fe80::ff:60ff:fec5:1d60%en0, icmp_seq=1 hlim=64 time=4.398 ms
16 bytes from fe80::ff:60ff:fec5:1d60%en0, icmp_seq=2 hlim=64 time=4.090 ms


Still no SLAAC, though.

BTW: why is the interface inside the jail named epair0b and not vnet0 like with all the iocage jails we run in our data centre?

Kind regards,
Patrick
 
Last edited:

Patrick M. Hausen

Hall of Famer
Joined
Nov 25, 2013
Messages
7,737
Got it working now! Did not change the autoconf settings in the UI, but changed my rc.conf from:
Code:
ipv6_activate_all_interfaces="YES"
rtsold_enable="YES"

to just:
Code:
ifconfig_epair0b_ipv6="inet6 accept_rtadv auto_linklocal"


The jail now get's an autoconfigured address and default gateway. Only question remaining: why epair0b inside the jail?

Kind regards
Patrick
 
Last edited:

Patrick M. Hausen

Hall of Famer
Joined
Nov 25, 2013
Messages
7,737
Another thing I noticed: even when I remove rtsold from rc.conf it gets started, anyway. But in a somehow inconsistent way:
Code:
root@newcloud:~ # service rtsold onestatus
rtsold is not running.
root@newcloud:~ # ps awux|grep rtsold
root	   77326  0.0  0.0   6312  2056  -  IsJ  08:49   0:00.00 /usr/sbin/rtsold -a
root	   78420  0.0  0.0   6660  2460  3  R+J  08:56   0:00.00 grep rtsold
root@newcloud:~ # ll /var/run/rtsold.pid
ls: /var/run/rtsold.pid: No such file or directory
root@newcloud:~ # grep rtsold /etc/rc.conf
root@newcloud:~ #

Looks like iocage is starting the daemon somehow during startup of the jail regardless of the setting. While I do understand and appreciate to manage IP address and hostname settings from "outside", IMHO enabling and disabling of services should be left entirely to the "inside" of a jail. Plus it keeps adding rtsold to rc.conf every time I change the settings in the UI and click on "Save". Please don't do that. That's the administrator's job. Or Ansible's ;)

It's these things that make me prefer VMs at the current state of affairs. I want a clean separation of host and guest and the host should not mess with the guests - ever.

Kind regards
Patrick
 

mjt5282

Contributor
Joined
Mar 19, 2013
Messages
139
patrick, thanks for posting the 'work-around' for ipv6 in the jails. IMHO one of the pain points for freenas and bsd virtualization is that the upstream provider of packages can end-of-life the OS version and essentially force consumers (i.e. us) to upgrade, whether or not we or the OS version is ready ... I am running 11.2RC1 and while most things are working, things like ipv6 in iocage jails needs a little more attention.
 

Patrick M. Hausen

Hall of Famer
Joined
Nov 25, 2013
Messages
7,737
Thanks, @mjt5282. In fact the OS is more than ready. We run dozens of large servers with hundreds of jails with iocage in production - on stock FreeBSD 11.2. It's the FreeNAS integration that sometimes makes things easy - would not want to build all that sharing stuff myself. Sometimes more difficult - networking is definitely one area here.

Patrick
 

IK_Pegasi

Cadet
Joined
Mar 17, 2019
Messages
1
Got it working now! Did not change the autoconf settings in the UI, but changed my rc.conf from:
Code:
ipv6_activate_all_interfaces="YES"
rtsold_enable="YES"

to just:
Code:
ifconfig_epair0b_ipv6="inet6 accept_rtadv auto_linklocal"


The jail now get's an autoconfigured address and default gateway. Only question remaining: why epair0b inside the jail?

Kind regards
Patrick
Sorry I am new to the system. After I edited the file, upon restart. The new line added got changed to
ifconfig_epair0b="DHCP"
Any idea what is wrong? Thanks.
 

Patrick M. Hausen

Hall of Famer
Joined
Nov 25, 2013
Messages
7,737
Sorry I am new to the system. After I edited the file, upon restart. The new line added got changed to
ifconfig_epair0b="DHCP"
Any idea what is wrong? Thanks.
I have static IPv4 with autoconf IPv6 ...

Patrick
 

Decade

Cadet
Joined
May 29, 2019
Messages
5
Well, I just tried static IPv4 and autoconf IPv6 on FreeNAS 11.2-U4.1, and it not only did not set up IPv6, but IPv4 stopped working and the interface had no IP address.

What’s most frustrating is that I did get IPv6 working, on one version of FreeNAS 11.2, but I didn’t write down which version nor the steps I took to get IPv6 to work.
 

bestboy

Contributor
Joined
Jun 8, 2014
Messages
198
Basic question: In order to use IPv6 in a jail, is it required that the host is IPv6 enabled?
Coz I'd like to have my FreeNAS to be IPv4 only and have the exposed services in my jails to use IPv4 as well as IPv6. Is that even possible? Will the host being able to do Neighbor Discovery Protocol and delegate Router Advertisements and such to the jails?
 

Patrick M. Hausen

Hall of Famer
Joined
Nov 25, 2013
Messages
7,737
@Decade, static IPv4 with SLAAC IPv6 works without problems for me. Could you post the output of these commands?
Code:
iocage get all myjail
iocage start myjail
iocage console myjail
ifconfig -a
cat /etc/rc.conf


@bestboy, of course not. Technically. VNET is all layer 2. In reality iocage tends to make assumptions about your environment that are not necessarily true. Like if a jail has got an IPv6 address, it must have an IPv6 default gateway, too. Or the host must have IPv4 for a jail to be able to use IPv4 ... these are all hard coded checks in iocage assuming that things are "always that way". We submitted quite a bunch of pull requests to weed them out in the last couple of days. IMHO a tool like iocage should never check for "consistency" or some such. Provide mechanism, not policy. Never put assumptions about network architecture in code. The admin will know what he is configuring.

My 2 ct.

@Brandon Schneider, are you coming to EuroBSDCon? We could have a chat about this ...

Kind regards,
Patrick
 

Brian Lucas

Cadet
Joined
Dec 31, 2016
Messages
1
I am seeing a similar if not the same issue. I have a valid autoconfigured IPv6 address on my freenas box, and even have a virtual machine running ubuntu that is getting a valid ipv6 address, but none of my jails get an address.

Code:
root@freenas:~ # iocage get all test
CONFIG_VERSION:14.1
allow_chflags:0
allow_mlock:0
allow_mount:0
allow_mount_devfs:0
allow_mount_nullfs:0
allow_mount_procfs:0
allow_mount_tmpfs:0
allow_mount_zfs:0
allow_quotas:0
allow_raw_sockets:0
allow_set_hostname:1
allow_socket_af:0
allow_sysvipc:0
allow_tun:0
available:readonly
basejail:no
boot:off
bpf:no
children_max:0
cloned_release:11.2-RELEASE
comment:none
compression:lz4
compressratio:readonly
coredumpsize:off
count:1
cpuset:off
cputime:off
datasize:off
dedup:off
defaultrouter:none
defaultrouter6:none
depends:none
devfs_ruleset:4
dhcp:off
enforce_statfs:2
exec_clean:1
exec_fib:0
exec_jail_user:root
exec_poststart:/usr/bin/true
exec_poststop:/usr/bin/true
exec_prestart:/usr/bin/true
exec_prestop:/usr/bin/true
exec_start:/bin/sh /etc/rc
exec_stop:/bin/sh /etc/rc.shutdown
exec_system_jail_user:0
exec_system_user:root
exec_timeout:60
host_domainname:none
host_hostname:test
host_hostuuid:test
host_time:yes
hostid:00000000-0000-0000-0000-0cc47aac26b2
hostid_strict_check:off
interfaces:vnet0:bridge0
ip4:new
ip4_addr:none
ip4_saddrsel:1
ip6:new
ip6_addr:vnet0|accept_rtadv
ip6_saddrsel:1
jail_zfs:off
jail_zfs_dataset:iocage/jails/test/data
jail_zfs_mountpoint:none
last_started:2019-06-03 18:29:00
login_flags:-f root
mac_prefix:02ff60
maxproc:off
memorylocked:off
memoryuse:off
mount_devfs:1
mount_fdescfs:1
mount_linprocfs:0
mount_procfs:0
mountpoint:readonly
msgqqueued:off
msgqsize:off
nmsgq:off
notes:none
nsemop:off
nshm:off
nthr:off
openfiles:off
origin:readonly
owner:root
pcpu:off
priority:99
pseudoterminals:off
quota:none
release:11.2-RELEASE
reservation:none
resolver:/etc/resolv.conf
rlimits:off
securelevel:2
shmsize:off
stacksize:off
state:up
stop_timeout:30
swapuse:off
sync_state:none
sync_target:none
sync_tgt_zpool:none
sysvmsg:new
sysvsem:new
sysvshm:new
template:no
type:jail
used:readonly
vmemoryuse:off
vnet:on
vnet0_mac:02ff60ae1b75 02ff60ae1b76
vnet1_mac:none
vnet2_mac:none
vnet3_mac:none
vnet_default_interface:auto
vnet_interfaces:none
wallclock:off

root@test:~ # ifconfig -a
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
        inet 127.0.0.1 netmask 0xff000000
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        groups: lo
epair0b: flags=8842<BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 02:ff:60:ae:1b:76
        hwaddr 02:77:d0:00:0e:0b
        nd6 options=1<PERFORMNUD>
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        groups: epair
root@test:~ # cat /etc/rc.conf
hostname="test"
cron_flags="$cron_flags -J 15"

# Disable Sendmail by default
sendmail_enable="NO"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"

# Run secure syslog
syslogd_flags="-c -ss"

# Enable IPv6
ipv6_activate_all_interfaces="YES"
rtsold_enable="YES"



Followup. Following the instructions above for the changes to rc.conf, as well as specifying a manual IP for IPV4 made my IPV6 start working. Leaving IPV4 as DHCP prevented my IPV6 from working. Is this expected behavior? It would automatically change my rc.conf entry to be DHCP for V6 as well when my IPV4 was set for DHCP. I would change it and reboot, and it would revert back to DHCP.
 
Last edited:

bestboy

Contributor
Joined
Jun 8, 2014
Messages
198
I just want to quickly post some observations about IPv6 auto configuration in jails from my side for
FreeNAS-11.2-U4.1:
  1. when DHCP is used for IPv4, then IPv6 auto configuration won't work at all
  2. in order to use a static IP for IPv4 the host interface needs to be vnet0. Other interfaces like bridge0 or igb0 won't work and will not assign a static IP
  3. when a static IPv4 is used on vnet0, IPv6 auto configuration works with the mentioned rc.conf changes (ifconfig_epair0b_ipv6="inet6 accept_rtadv auto_linklocal")
  4. it seems that IPv6 auto configuration works with any host interface (vnet0, bridge0 and igb0)
However, while I can get an IPv6 address in my jail now, it seems I still cannot receive any IPv6 traffic.
Code:
root@ipv6:~ # ping6 fdbb:dead:babe::1
PING6(56=40+8+8 bytes) 2003:xxxx:xxxx:xxx:xxxx:xxxx:xxxx:xxxx --> fdbb:dead:babe::1
^C
--- fdbb:dead:babe::1 ping6 statistics ---
21 packets transmitted, 0 packets received, 100.0% packet loss


EDIT:
Well, the IPv6 connectivity issue is probably not related to the jail configuration. I guess it is an issue of my network configuration. I don't really know what's the problem yet, but it seems to be a very specific problem to BSDs. Various IPv6 enabled hosts do work just fine when they run one of these OSes: linux (test with kali and ubuntu), windows 7, windows 10 as well as various android devices.
However, BSDs seem to not be able to receive IPv6 traffic (tested with FreeNAS jail as ITT, FreeBSD 12, OpnSense and OpenBSD 6.5).
 
Last edited:

Patrick M. Hausen

Hall of Famer
Joined
Nov 25, 2013
Messages
7,737
in order to use a static IP for IPv4 the host interface needs to be vnet0. Other interfaces like bridge0 or igb0 won't work and will not assign a static IP.

Just to clarify:

vnet0 is not the host interface. For example:
Code:
interfaces:vnet0:bridge0

This tells iocage to add the jail's vnet0 interface to the host's bridge0 interface.

Code:
ip4_addr:vnet0|217.29.46.105/26

This tells iocage to configure a static IPv4 address for that jail interface.

Code:
ip6_addr:vnet0|accept_rtadv

And finally this tells to perform SLAAC for the jail interface.

Kind regards,
Patrick
 
Last edited:

Andistorm

Cadet
Joined
Jun 18, 2019
Messages
3
Hi,
i read this thread. It helped me a lot. Thanks @Patrick M. Hausen !
I can use ping and ping6 and my jails have an ipv4-adress and an ipv6-adress, but i can't reach the web-surface of my jails with ipv6 (not local, not gloabl).
I am using nextcloud. Please help me :)

Kind regards,
Andreas
 
Top