SOLVED Update from 9.10.2-U2 to U3 boot process stops during jail boots

[Murac]

How's it going community?

So I updated last night. I have a monitor connected directly to my server for emergencies and it doesn't make it to the Console Setup Menu. It logs the boot, starts only one of my jails, and also starts my iohyve vm, and just stays like that. I can manually start my other jails though. I checked the logs in the WebGUI and there is one new difference describing something from the [smbd], which I believe was the sole reason for the update.

Log:

Code:

Apr 21 18:38:07 freenas ntpd[2329]: ntpd 4.2.8p9-a (1): Starting
Apr 21 18:38:10 freenas smbd: dnssd_clientstub ConnectToServer: connect()-> No of tries: 1
Apr 21 18:38:10 freenas root: /etc/rc: WARNING: failed precmd routine for vmware_guestd
Apr 21 18:38:11 freenas smbd: dnssd_clientstub ConnectToServer: connect()-> No of tries: 2
Apr 21 18:38:12 freenas smbd: dnssd_clientstub ConnectToServer: connect()-> No of tries: 3
Apr 21 18:38:13 freenas smbd: dnssd_clientstub ConnectToServer: connect() failed path:/var/run/mdnsd Socket:35 Err:-1 Errno:2 No such file or directory

I don't think ntpd is part of this but I included it just incase.

I checked /var/run/ and these are present:

Code:

srwxrwxrwx   1 root  wheel		0 Apr 21 18:38 mdnsd=
-rw-r--r--   1 root  wheel		5 Apr 21 18:38 mdnsd.pid

Anyone got any ideas?

EDIT: this looks related: https://bugs.freenas.org/issues/4234

 

[Ericloewe]

EDIT: this looks related: https://bugs.freenas.org/issues/4234

Serial console? I'm not sure how you arrive at any sort of connection, because I don't see one.

The devs say it's all noise: https://bugs.pcbsd.org/issues/17805

 

[Ericloewe]

Where does the boot process stop, exactly?

 

[Murac]

Yeah I just assumed that because the console wasn't showing up it coudld be that. but better idea is that the process gets stopped at this point.

It looks like when it tries to start transmission something happens.

Paraphrased:

Code:

route: writing to routing socket: File Exists
add net default ....: already in table
starting jail
transmission daemon: UDP Failed to set recieve buffer: no buffer space available (tr-udp:65)
transmission daemon: UDP Failed to set recieve buffer: requested 4194304, got 42080 (tr-udp:84)
then it starts doing ethernet stuff with epair1b/2b etc

Found this: https://forums.freenas.org/index.php?threads/sysctl-in-gui-has-no-effect.14633/

and this: https://forums.freenas.org/index.ph...iled-to-set-receive-buffer-please-help.15203/

 

[Murac]

Well look at that, I added the sysctl's referred to in that link and restarted the jail and the console menu showed up. I'll mark the thread solved but I won't know for sure until I reboot.

Thanks

 

[Murac]

False alarm, nothing's changed.

I even tried reverting to U2 and still the problem persists. this isn't a new setup. could it be an issue with my router and dhcp? I never saw these "chatty" logs before the update.

Edit: After reviewing the boot process the weird points are:

1. Generating Host.conf
unrecogonized source [mdns]

2. Something to do with not being able to add the defualt route (because it already exists)

3. Not being able to load /etc/local/samba4 (or something similiar. it's telling me to testparm but it passes the sanity check

4. The ultimate hangup seems to be when a certain jail is loading. If I just restart the jail, it boots, and the rest of the jails follow. Could a service within the jail cause something like this?

 

[Murac]

While the errors i mentioned above persist in the logs, I managed to get things working and fully booted again. The issue was starting a program in /etc/rc.d/program without the "&"

Rookie mistake