TrueNAS Upgrade Woes

[ub3r]

Greetings,

I upgraded from FreeNAS to TrueNAS some months ago and was running "relatively" OK (aside from a bug report I submitted).
Last week I attempted to log into the Web GUI and was unable to do so.

• I was presented with the login page
• I entered my credentials and attempted to Log In
• Upon doing so, the login screen just appeared to be infinitely "thinking" or "loading", without ever getting past the login page.

It should be noted that all file system functionality was working perfectly fine at this time, so I put off troubleshooting until I was on vacation. Just before my vacation, we had a power outage and the system was powered down after the UPS was drained. After powering it back up the next day, the system is not very happy.

I plugged a keyboard/monitor into the box and am looking at the following:

Code:

lo0: link state changed to UP
middlewared: loaded plugin middlewared.plugins.update_.utiolsMiddleware startup is idle for more than 240 seconds
########################################################
MIDDLEWARED FAILED TO START, SYSTEM WILL NOT BEHAVE CORRECTLY!
########################################################
Loading early kernel modules:
Syncing disks . . .
Failed to run middleware call. Daemon not running?
Failed to run middleware call. Daemon not running?
Failed to run middleware call. Daemon not running?
Failed to run middleware call. Daemon not running?
Alarm clock
Starting file system checks:
Setting hostuuid: (redacted).
Setting hostid: (redacted).
Mounting local filesystems:/
Failed to run middleware call. Daemon not running?
Loading kernel modules:
hwpmc: SOFT/16/64/0x67<INT,USR,SYS,REA,WRI> TSC/1/64/0x20<REA> IAP/4/48/0x3ff<INT,USR,SYS,EDG,THR,REA,WRI,INV,QUA,PRC> IAF/3/48/0x67<INT,USR,SYS,REA,WRI>
Setting hostname: (redacted).
Setting up harvesting: [UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,NET_ETHER,NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED
Feeding entropy:

I have a prompt at the "Feeding entropy: " line.

My setup is as follows
Processors: (2) Xeon E5-2667v2
Motherboard: Intel S2600CP2J
HBA: (3) LSI SAS9211-8i flashed in IT Mode
Drives: 20x 3TB SATA drives

Troubleshooting performed: Took it apart, cleaned it, isopropyl'd all the connections, put it back together again, powered up and same dealio.

Thanks for taking the time to read this. Any input would be greatly appreciated. Have a great holiday!

 

[Kris Moore]

Ouch! Not sure if this is a bug yet. Can you look at output of /var/log/middlewared.log and see if there's any obvious errors at the end?

 

[ub3r]

Greetings Kris,

Pardon my ignorance, however I don't believe that I'm presented with a shell, just the "Feeding entropy: " line with a blinking cursor. Is there a fancy trick I can do to get to a shell prompt?

Thanks!

 

[Kris Moore]

Oh shoot, I thought you were dumping to a shell prompt after that. It sounds much sicker than expected. Some things to try:

1. Do you have any old boot-environments you can select at the boot-loader prompt? Do they boot any farther?
2. Can you boot an install image? If so you can try installing to a new boot-environment and importing the old config.

Also, I didn't see where you mentioned, what kind of boot-device are you using? SATA disk or something else?

 

[ub3r]

I'm using a USB 3.0 thumb drive for the boot device.

1. Trying a FreeNAS 11.3 environment right now...
2. I will try to do a fresh install and see if that fixes my issues.

One thing that I also noticed just now, I'm seeing the following

Trying to mount root from zfs:freenas-boot/ROOT/11.3-U3.2 [] . . .
(da18:umass-sim0:0:0:0) : WRITE(10). CDB: 2a 00 00 f0 75 fb 00 00 80 00
(da18:umass-sim0:0:0:0) : CAM status: CCB request completed with an error
(da18:umass-sim0:0:0:0) : Retrying command
This repeats a bit.

I'm assuming umass == usb mass storage (usb stick?)
Maybe my USB stick has died?! :D That would be great if its that simple LOL. I will try to see if that fixes it also. I'll make sure to update the thread with my findings.

Thanks again Kris!

 

[ThreeDee]

Perhaps time to move to an SSD .. you could even use a USB to SATA adapter if you are out of SATA ports... USB port provides enough power for a 120'ish GB drive. Of course, using an actual SATA port would be best .. I used the USB to SATA adapter for my boot drive 120GB SSD on my old setup without issue for a long time.

..and .. FreeNAS/TrueNAS doesn't always play nice with USB 3.0 anything

 

[ub3r]

Update --

I installed the latest version of TrueNAS CORE (12.0 U1) on a spare SSD I had laying around and everything is back up and running.

Thanks again for the help Kris!