WebUI stops after a few days

[WebUIProblem]

I've got my TrueNAS server since a few months now and have been fighting with the webUI from the start. It would work for a seemingly random amount of time (2 hours to 2 days) and then hang on the "connecting to TrueNAS" screen as seen below:

Now since about a week it still does kind of the same, but instead of hanging on that message it will just continuously try to load the page with nothing ever showing up. (tried with mutliple browsers)

Difference with before is that now my SSH will not connect anymore when this happens, but my SMB share keeps working..
It used to be the other way around.

A hard reset will fix things for some time again, but is of course less than ideal.
I am running version TrueNAS-12.0-U6.
Should I just set the server up to reboot every day or does anyone know of a different (real) solution?
Thanks in advance to anyone willing to help

 

[Alecmascot]

Ryzen cpu ?

 

[Patrick M. Hausen]

Instead of rebooting the server try service middlewared restart.

 

[WebUIProblem]

Ryzen cpu ?

No it's a hp proliant server with two intel xeon CPUs

 

[WebUIProblem]

Instead of rebooting the server try service middlewared restart.

Don't really know how to do this shen SSH is not connecting

 

[WebUIProblem]

small update:
I updated the system to 12.0-U7 and instead of not showing anything and hanging on loading the site, today i did get to the login screen but instead it hangs on this:

SSH still not working
SMB share does still work
Jails and VMs work perfectly as well

 

[jgreco]

hp proliant server

With a crappy Proliant RAID controller of some sort?

What's all the noise about HBAs, and why can't I use a RAID controller?

1) An HBA is a Host Bus Adapter. This is a controller that allows SAS and SATA devices to be attached to, and communicate directly with, a server. RAID controllers typically aggregate several disks into a Virtual Disk abstraction of some sort...

www.truenas.com

 

[jgreco]

Don't really know how to do this shen SSH is not connecting

Well, before there was SSH, or even telnet, there was ... the console. ;-)

 

[WebUIProblem]

With a crappy Proliant RAID controller of some sort?

What's all the noise about HBAs, and why can't I use a RAID controller?

1) An HBA is a Host Bus Adapter. This is a controller that allows SAS and SATA devices to be attached to, and communicate directly with, a server. RAID controllers typically aggregate several disks into a Virtual Disk abstraction of some sort...

www.truenas.com

No I use a m1015 HBA (IT-mode) and have the onboard raid controller disabled

I should've thought about the console, I'm a bit embarrassed, will try that right now as it's hanging at the moment

 

[jgreco]

No I use a m1015 HBA (IT-mode) and have the onboard raid controller disabled

I should've thought about the console, I'm a bit embarrassed, will try that right now as it's hanging at the moment

Well, it was worth a shot. The RAID controller thing on big-name vendor boxes is a real problem and can sometimes lead to weird lockups.

As for the console, no worries, and no need to be embarrassed. That's why you have a community of people to bounce stuff off of.

However, I will note that it would be a good idea to post a much better description of your hardware setup. Since the average FreeNAS/TrueNAS setup doesn't just mysteriously hang for no apparent reason, you've posted a question of the "my car won't start" variety, without actually telling us much about what kind of car it is. This has a tendency to generate low-quality answers, and many posters won't even bother to comment. Most of the posters here are fine with weeding through a little too MUCH detail, but too LITTLE detail makes it kinda tempting to pass on the question.

The Proliants are also known for sometimes using crappy Broadcom ethernet chipsets. If the thing is going entirely offline, I might suggest that some ping testing from the console when you are having issues might be prudent. If it is really just the middleware going away, that's often a sign of some jail or process running away and filling the memory (causing the kernel to kill random large processes such as the middleware in an effort not to crash), in which case "top" from the console may be your friend in identifying that.

 

[WebUIProblem]

Well, it was worth a shot. The RAID controller thing on big-name vendor boxes is a real problem and can sometimes lead to weird lockups.

As for the console, no worries, and no need to be embarrassed. That's why you have a community of people to bounce stuff off of.

However, I will note that it would be a good idea to post a much better description of your hardware setup. Since the average FreeNAS/TrueNAS setup doesn't just mysteriously hang for no apparent reason, you've posted a question of the "my car won't start" variety, without actually telling us much about what kind of car it is. This has a tendency to generate low-quality answers, and many posters won't even bother to comment. Most of the posters here are fine with weeding through a little too MUCH detail, but too LITTLE detail makes it kinda tempting to pass on the question.

The Proliants are also known for sometimes using crappy Broadcom ethernet chipsets. If the thing is going entirely offline, I might suggest that some ping testing from the console when you are having issues might be prudent. If it is really just the middleware going away, that's often a sign of some jail or process running away and filling the memory (causing the kernel to kill random large processes such as the middleware in an effort not to crash), in which case "top" from the console may be your friend in identifying that.

You're asolutely right, should've started with this.

So I'm using rather old hardware, bought it from a company that was upgrading to something more recent so got it for cheap and this is my first server so didn't want to spend too much if it didn't work out.

I have an HP Proliant DL360 G7 with:
- 2x Intel(R) Xeon(R) CPU X5650
- 112 GB RAM (7x 8GB DDR3 1333 MHz per CPU) these are set up in the same banks/slots for each CPU according to the manual of the server.
- M1015 HBA card with the IT-firmware flashed to it.
- Boot drive is an SD Card currently (I believe this actually might still be run by the onboard P410 RAID controller, or am I wrong?)
- 8x 140GB SAS HDDs

I know running off of a SD card is considered bad, might this be the culprit?
I tried running it off of a SSD with a PCIe card but that card had a JMicron chipset that did not work nicely with my BIOS (during startup even with no drives in it would ask "press any key to continue" with a 3,2,1 countdown and then not respond or actually start counting down)
Should I try another way?
All my HDDs are already part of a pool and the current USB ports are controlled by the Raid controller (I believe)

As I said, I'm still very new to this all so if there's still some information missing please let me know.
I hope you now know which car it is that won't start, maybe not yet the model number or the reason why but it's a start.

 

[WebUIProblem]

I just checked the console and the problem might be a bit bigger than i thought. I thought it was something with middlewared, but i just got a message saying:
Solaris: warning boot-pool has encountered an uncorrectible I/O failure and has been suspended.
I also see a few MCA warnings telling me one of my RAM modules (bank 8 channel ??) is not working properly.
Think these might be correlated somewhat... oops.
Weird thing is, i checked with the manual, and the modules in bank 8 of both CPU's, that is/are supposed to not be working properly, are empty sockets.

I bought this off of someone who told me he did a RAM check and two RAM modules were not good anymore, so we took those out, I believe those used to be in the bank 8 slots, don't know if that has anything to do with it. Or does TrueNAS assign its own numbers to the RAM banks?

If needed I will try removing the RAM one by one (after turning the server off every time of course) and see if the error still shows up.

It could also still be the SD Card that stopped working though, but I'd imagine all the VMs and Jails would stop working then as well..

 

[Peterv]

I just checked the console and the problem might be a bit bigger than i thought. I thought it was something with middlewared, but i just got a message saying:
Solaris: warning boot-pool has encountered an uncorrectible I/O failure and has been suspended.
I also see a few MCA warnings telling me one of my RAM modules (bank 8 channel ??) is not working properly.
Think these might be correlated somewhat... oops.
Weird thing is, i checked with the manual, and the modules in bank 8 of both CPU's, that is/are supposed to not be working properly, are empty sockets.

I bought this off of someone who told me he did a RAM check and two RAM modules were not good anymore, so we took those out, I believe those used to be in the bank 8 slots, don't know if that has anything to do with it. Or does TrueNAS assign its own numbers to the RAM banks?

If needed I will try removing the RAM one by one (after turning the server off every time of course) and see if the error still shows up.

It could also still be the SD Card that stopped working though, but I'd imagine all the VMs and Jails would stop working then as well..

HI
What was the outcome of this? I have the same issue and don't want to start swapping out hardware if not needed.

Thanks
Peter