AMD P-States not working below C3

[MjSmith]

Hi Guys unfortunatly i need your help again,

after some powertweaks of my new truenas build i made the decision to switch from hdds to ssds in kind of noise also psu (special case).
Besides that i also switched the mainboard from a biosstar with limited bios settings and only 1x M2 slot to an asrock one with 2x m2 slots (mirror boot pool) and in my imagination more bios tweak options.

What i got is a little bit of everything but whats annoying me is the fact that the cpu doesnt go now below C3.

OLD HW:
Board: Biostar B550T-Silver
Pool: RaidZ1 3x Western Digital WD Red Plus 4TB
PSU: be quiet! SFX-L Power 500W

NEW:
Board: ASRock B550 Phantom Gaming-ITX/ax
Pool: RaidZ1 2x Transcend SSD230S 4TB
PSU: Corsair SF Series Platinum SF600 600W SFX

I tweaked alot with BIOS Settings to get the consumption as low as possible and i reached for now 19W with running kubernetes but no active app at the moment.


What i found out:

• There is no dedicaded p-state bios setting other than the AMD Globale P-State Option

-> Set on Auto TrueNAS work down until C3 (19W)​

-> Set on Disable Powertop show no PState Data at all (25W)​

• The loaded driver ( cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_driver ) is "acpi-cpufreq". After reading that documentary Link is only c3 possible with that one.

-> Dont know what TrueNAS loads on the old configuration but could that be the problem?​

• Unpluged the SSDs to find out if these block deeper P-States than C3 - no change

Does anyone of you have an idea what i can do to reach C5 or can give me some tips what i missed in the asrock bios?


For now i know i will either buy a asrock or biostar board again ... msi makes such a better job in case of bios for me.

 

[varet]

I do see the governor driver is acpi-cpufreq and not amd-pstate! and I tried some CMDlines to enable it. but no success!

I am playing around but I think we are limited by the kernel of TrueNAS Scale, till we reach a newer version which has this switch.
Although there is a bug where c6 crashes the box... and we need also to limit up to C5 state... but till we have access to that governor, no tests.

from my dmesg

[ 0.014631] Unknown kernel command line parameters "BOOT_IMAGE=/ROOT/22.12.1@/boot/vmlinuz-5.15.79+truenas zfsforce=1 amd_pstate=passive", will be passed to user space.

kernel 5.17+ has this

 

[MjSmith]

yeah its strange ... why shall one mainboard with b450 and same cpu get the amd-pstate driver and the other one not.. so the answer is wait for it (kernel update) yes?

 

[varet]

did both run truenas scale with the same kernel?
as I know the new governor is from 5.17 maybe the old driver was different? only if you run both boxes and then run:

cpupower frequency-info

will show us if they run the same Driver or not

 

[MjSmith]

no chances to compare sadly ... i returned the other mobo because i thought this one gives me more tweaks to safe power ...

so my current one shows:

forogot answering the kernel question: So yes both run/ did run with the same latest truenas scale version.

 

[varet]

the CPU will not go lower than C3 with any board and this AMD processor.
The main issue you have is (and I have too) the kernel driver is not there.

I think a bit of patience will bring a new minor introducing this. till then, we can accept 3-4watt extra consumption.

I also pay similar rates, as in Germany (judging from geizhals) so it is ok.

I have idle 36w!!! measured at Plug which is a lot more than what you have. sometimes I see 45-50w idle.. with spinned down disks
I suspect is still the driver

An Asus Ace pro x570 with Ryzen 5 Pro 5650GE

I have:
2*NVME 256GB (boot)
2*SSD 1TB (for iscsi volumes) (mirror) and fast data
5* 14TB Red Pro (I power them down, as they are rarely running) one is hot spare. Mirror/Stripe setup
10gb SPF+ intel
and 2 ethernet onboard (for backup

No VMs or other stuff just data.
running SMB/NFS and iscsi.

then 5 k8s nodes pulling from this machine, storage

I am happy with the monster I made. maybe the CPU is overkill, if after the newer kernel is in, I do not see better numbers. I might replace the CPU with something simpler.