SOLVED Bhyve sockets, cores and thread counts

[Alex98234]

Hello FreeNAS community,

Bhyve appears to be assigning CPU cores configured in the FreeNAS VM GUI as individual CPU sockets with 1 CPU/core per socket. I have observed this problem in Windows Server 2012 R2 and Ubuntu Server 16.04.02. I am using an Intel Xeon E3-1240L v5 2.1 GHz (4 cores, 8 threads). Is this by design or is additional configuration required that is not available in the FreeNAS VM GUI?

 

[Allan Wilmath]

Here is a snippet:
To fix this the following sysctl must be set in /boot/loader.conf (and a reboot to take effect):

hw.vmm.topology.cores_per_package=4


Here is the link to the whole article.

https://www.ateamsystems.com/tech-blog/howto-windows-10-bhyve-w-freebsd-11/

 

[Alex98234]

Here is a snippet:
To fix this the following sysctl must be set in /boot/loader.conf (and a reboot to take effect):

hw.vmm.topology.cores_per_package=4


Here is the link to the whole article.

https://www.ateamsystems.com/tech-blog/howto-windows-10-bhyve-w-freebsd-11/

Hi Allan,

Thank you for your response. I followed the instructions and restarted my FreeNAS system but the issue remains.

Here is a screenshot after restart of my /boot/loader.conf

 

[dlavigne]

Was the VM created on 11 or 9.10.x? It sounds like this bug: https://bugs.freenas.org/issues/22924.

 

[Zwck]

mhh what are the consequences of this?

 

[Alex98234]

Was the VM created on 11 or 9.10.x? It sounds like this bug: https://bugs.freenas.org/issues/22924.

Yes that is the exact problem I am experiencing! I also tried subsequently adding hw.vmm.topology.threads_per_core and it was still ignored.

/edit Both VMs were created on FreeNAS 11.

mhh what are the consequences of this?

Some OS (especially non-server) don't allow for more than 2 CPU sockets e.g. Windows 10.

 

[yottabit]

In FreeNAS 11, I set this as a tunable in the Loader config. I set mine to 32, and gave the VM 16. Worked like a champ. Windows 10 only allows 2 sockets, but allows up to 256 cores per socket (x64 version). Marketing. ::eyeroll::

Sent from my Pixel XL using Tapatalk

 

[shamo316]

In FreeNAS 11, I set this as a tunable in the Loader config. I set mine to 32, and gave the VM 16. Worked like a champ. Windows 10 only allows 2 sockets, but allows up to 256 cores per socket (x64 version). Marketing. ::eyeroll::

Sent from my Pixel XL using Tapatalk

are you experiencing any issues with running that many cores?? my VM with 10 cores seems slow and laggy

 

[yottabit]

are you experiencing any issues with running that many cores?? my VM with 10 cores seems slow and laggy

Nope, running fine. I wanted the max cores available to render living photos in Lytro Desktop. It cut the time by half compared to my 4-thread i7 NUC of higher clock speed. All 16 threads ran at 100% for 2 hours. The bhyve process was running around 1600% as you'd expect, too.

Sent from my Pixel XL using Tapatalk

 

[Alex98234]

I still can't get my cores to appear on a single socket/package. Is the cores per package sysctl set at the FreeNAS level or a VM level?

 

[yottabit]

I still can't get my cores to appear on a single socket/package. Is the cores per package sysctl set at the FreeNAS level or a VM level?

Seems to be a setting for the whole host. I set mine in the tunables section of the FreeNAS 11 GUI, and rebooted just for good measure.

Sent from my Pixel XL using Tapatalk

 

[Alex98234]

Seems to be a setting for the whole host. I set mine in the tunables section of the FreeNAS 11 GUI, and rebooted just for good measure.

Sent from my Pixel XL using Tapatalk

Thank you so much! I followed your instructions and now my cores appear on a single socket/package. I was adding the tunables manually through a terminal session to /boot/loader.conf not realizing I could do it through the UI *facepalm*.

 

[Peter Cheong]

In FreeNAS 11, I set this as a tunable in the Loader config. I set mine to 32, and gave the VM 16. Worked like a champ. Windows 10 only allows 2 sockets, but allows up to 256 cores per socket (x64 version). Marketing. ::eyeroll::

Sent from my Pixel XL using Tapatalk

Hi,
It is great. would you mind to share with us, how you did it specifically? thanks

 

[yottabit]

You can add the sysctl in the FreeNAS GUI under the System menu.

 

[yottabit]

Add this in System -> Tunables, then reboot.

Variable: hw.vmm.topology.cores_per_package
Value: 32 (or number of cores to show per CPU socket)
Type: loader
Enabled: yes