bhyve uses 10-15% idle cpu

[adrianwi]

Not sure exactly what caused it, but my bhyve VM is now reporting much lower system usage, down from ~130% to 6%

I upgraded the Ubuntu 16.4 LTS VM today using apt-get update and then upgrade. I had a few issues getting Onlyoffice Document Server running and ended up uninstalling and reinstalling that, but otherwise, that's all I changed.

The VM is running CrashPlan, Docker (with about 9 active containers) and Onlyoffice.

UPDATE

Back to almost 200% at around 06:00 this morning?

 

[andrew2005]

I'm still getting this issue (in my case 20% CPU at guest idle) on 11.2-BETA2. I removed VNC, rebooted the guest and it seems to have fixed the issue. Seems to be VNC related.

 

[jgreco]

VNC is likely polling the display for changes, and if a client is connected, compressing and transmitting them. This can be nontrivial and is affected by the settings you've selected. For example, UltraVNC offers the ability to handle screen updates via full screen polling, full screen polling upon event, system hook, or mirror driver. These affect how heavily the CPU is impacted.

 

[Kennyvb8]

VNC is likely polling the display for changes, and if a client is connected, compressing and transmitting them. This can be nontrivial and is affected by the settings you've selected. For example, UltraVNC offers the ability to handle screen updates via full screen polling, full screen polling upon event, system hook, or mirror driver. These affect how heavily the CPU is impacted.

Problem is no one is connected. Yet still high CPU idle.


Sent from my iPhone using Tapatalk

 

[jgreco]

Problem is no one is connected. Yet still high CPU idle.

FreeNAS isn't going to be able to reach into your VM, see that no one is connected, and force the VM to "be idle." If you remove VNC and it fixes the issue, that's suggesting that VNC needs to be tuned differently.

If the VNC implementation is actively polling the display and doing work, this will make the VM appear to need CPU cycles, because it actually *is* using CPU cycles. Machines consume cycles when tasks are running, not because clients are connected.

VNC servers might assume that they are running on physical hardware and that it is just fine and dandy to be doing frequent polling of the display because the VM itself is "not busy." On a physical platform, this would result in no noticeable impact. On a VM, it would result in some consumption of resources on behalf of the VM.

So, I will once again say that you will probably need to work with your VNC server's settings, or possibly change VNC servers, to identify what settings and modes result in the lowest CPU impact. If your VM is consuming CPU cycles, any hypervisor would attempt to provide those cycles, and that's what FreeNAS is likely to be doing. A hypervisor's job isn't to try to figure out whether those cycles are being used meaningfully or on behalf of a connected client.

 

[Ericloewe]

But VNC is running on the host, it's operating on the emulated VGA (SVGA, VESA, whatever) console, AFAIK. It should be plenty capable of knowing whether there's a client attached.

 

[jgreco]

But VNC is running on the host, it's operating on the emulated VGA (SVGA, VESA, whatever) console, AFAIK. It should be plenty capable of knowing whether there's a client attached.

So this is actually the VM console we're talking about, provided by bhyve itself? Then you should probably try running a VNC server on the guest. A VNC server embedded in the guest is actually in a better position to be able to coordinate with the guest OS when changes happen to the display, but is still very sensitive to the particular combination of configuration options.

It's also worth noting that reported load is an inexact science, so if you have a program that is doing a modest amount of work but doing it very frequently, this can appear to be "fairly busy" to the host. It might not actually be consuming quite as much CPU as is being reported.

 

[jgreco]

Not sure exactly what caused it, but my bhyve VM is now reporting much lower system usage, down from ~130% to 6%

I upgraded the Ubuntu 16.4 LTS VM today using apt-get update and then upgrade. I had a few issues getting Onlyoffice Document Server running and ended up uninstalling and reinstalling that, but otherwise, that's all I changed.

The VM is running CrashPlan, Docker (with about 9 active containers) and Onlyoffice.

UPDATE

Back to almost 200% at around 06:00 this morning?

Stepping back for a minute, this is quite possibly a Ubuntu thing. We see this all the time on ESXi hypervisors after someone installs Ubuntu.

Ubuntu desktop is kinda problematic. 16.04 runs Unity by default and if you log in to the VM via SSH and run top, you may find compiz going nuts. The best solution seems to be to ditch Unity and go to Xfce as documented here and elsewhere. This is apparently only an issue on systems without OpenGL hardware support, such as most VM environments.

 

[Kennyvb8]

Stepping back for a minute, this is quite possibly a Ubuntu thing. We see this all the time on ESXi hypervisors after someone installs Ubuntu.

Ubuntu desktop is kinda problematic. 16.04 runs Unity by default and if you log in to the VM via SSH and run top, you may find compiz going nuts. The best solution seems to be to ditch Unity and go to Xfce as documented here and elsewhere. This is apparently only an issue on systems without OpenGL hardware support, such as most VM environments.

I run server version of Ubuntu so no desktop

I have disabled VNC in byhve in the freenas GUI.but needed it to install the os in the VM. When the VM idle the byhe still goes crazy and sometimes toggle vnc helps. Teller me it isent a failure/fault/bug
In byhve ? Read the post here


Sent from my iPhone using Tapatalk

 

[Ericloewe]

Sounds like it's worth reporting, if you're using FreeNAS' VNC thing and there are no clients attached. Just to make sure it's on the radar.

 

[jgreco]

I run server version of Ubuntu so no desktop

I have disabled VNC in byhve in the freenas GUI.but needed it to install the os in the VM. When the VM idle the byhe still goes crazy and sometimes toggle vnc helps. Teller me it isent a failure/fault/bug
In byhve ? Read the post here

So when this happens, and you ssh into the guest to see what's going on, what does top(1) in the guest report? Does the VM think it is idle (99% idle or better)? Or is there lots of random crap running, as Linux tends to do?

 

[jgreco]

Sounds like it's worth reporting, if you're using FreeNAS' VNC thing and there are no clients attached. Just to make sure it's on the radar.

If you re-read this in the context of "my VM occasionally seems to be doing random background crap I didn't know about" it's pretty consistent with that. Toggling VNC could be unrelated, so it's probably better to see if we can identify if there's a more likely culprit here before filing potentially spurious bug reports.

 

[Ericloewe]

If you re-read this in the context of "my VM occasionally seems to be doing random background crap I didn't know about" it's pretty consistent with that. Toggling VNC could be unrelated, so it's probably better to see if we can identify if there's a more likely culprit here before filing potentially spurious bug reports.

Good point.

 

[adrianwi]

I'd forgotten about this, as at some point a VM reboot has the cpu idle use back down to 6-8%.

For me, it has to be something on ubuntu as that's restarting more regularly than FreeNAS is being updated.

 

[Kennyvb8]

So when this happens, and you ssh into the guest to see what's going on, what does top(1) in the guest report? Does the VM think it is idle (99% idle or better)? Or is there lots of random crap running, as Linux tends to do?

Yup. Ssh and it's idle doing around 1-2% while byhve reports 10-15% or higher.


Sent from my iPhone using Tapatalk

 

[Kennyvb8]

Sounds like it's worth reporting, if you're using FreeNAS' VNC thing and there are no clients attached. Just to make sure it's on the radar.

Allrdy a bug report if you re-read the thread


Sent from my iPhone using Tapatalk

 

[jgreco]

Yup. Ssh and it's idle doing around 1-2% while byhve reports 10-15% or higher.

What's consuming that 1-2%? If it is something that is consuming frequent CPU timeslices, this might actually be a situation where it is being reported to be using more CPU than it actually is due to the way CPU busy-ness is sampled and reported. But I am more curious about what's going on when:

Not sure exactly what caused it, but my bhyve VM is now reporting much lower system usage, down from ~130% to 6%

[...snip...-JG]

Back to almost 200% at around 06:00 this morning?

this is going on.

 

[ChrisFor]

I had the same problem with FreeBSD as a bhyve guest consuming upto 19% of cpu time. (The vm only runs backup cron-jobs at night).
After removing the vnc device the cpu usage of this vm is down to 0%.

Another bhyve vm running two syncthing instances on Debian 9.4 is now also down to 0% from 6-7% cpu usage after removing the vnc device.

So, thanks for the tip!

P.S. I am on FreeNAS-11.1-U6

 

[Steve Dowling]

Definitely something fishy with the VNC device. I have four Ubuntu VMs that were sitting at about 20% all of the time. After removing the VNC device they have all dropped to next to nothing most of the time. They are all server versions of Ubuntu without GUI and I was only using the VNC device to figure out what was going on when they don't respond to SSH.

 

[rostrum81]

Removing VNC device on Freenas 11.1-U4, Ubuntu 16.04 LTS VM reduced my idle CPU from ~40% down to few percents. Thank you for advice!