ASRock X470D4U

[poldi]

I have some promising results from my further testing. As I mentioned above I also set 'PSS Support' to 'Disabled' (under Advanced->CPU Configuration, to my understanding this used to be called Cool'n'Quiet). This is in addition to the other settings, i.e. 'Global C-State Control' and 'C6 Mode' both set to 'Disabled' and 'Power Supply Idle Control' set to 'Typical Current Idle' (going the overclockers route, sort of). With this my server survived two nights so far without crashing. This is a first so far, so keeping my fingers crossed that this will continue.
If this turns out to be a known good configuration I will try to activate Global C-State Control and C6 Mode again to check if I get any reduction in power consumption while keeping it stable.
In the meantime I also got a reply from ASRock. They gave me access to the latest BIOS and BMC update to test. Also they suggested to test with only one DIMM installed and with a new power supply.
So quite the homework to do. The power supply I anyhow wanted to swap as I have a modular one lying here it is just a mayor pain to swap this in my server case. BIOS and BMC update I will also take for spin although if the above settings turn out to be stable I will focus on optimizations of the settings first.

 

[MikeyG]

Interesting that PSS Support would help (if that actually makes it stable for a few days.) My understanding is that would be responsible for clock/voltage levels where as the C states are sleep states. I didn't see Cool n Quiet/PSS Support discussed in the threads that mention Ryzen crashing under freebsd, so I never tried it. I'd be curious how stable your system would be just running under windows - I've been making the assumption that we're trying to address some kind of bug with how FreeBSD handles Ryzen, and not some underlying hardware issue between MB/CPU. Your situation may of course be different.

What power supply do you have now? Why are they suggesting a replacement?

 

[poldi]

I have some updates to report on and I think I found at least one issue causing my instabilities.
However first things first.
I did quite a bit of changes to my system. I updated to latest BIOS and BMC (3.25 and 1.90 respectively). These were supplied to me by ASRock so I suppose there are still in beta. After that the system was stable at idle for 2 days. However I then suddenly encountered multiple restarts when reading from the array and doing transcodes in Plex. The stack traces always showed something related to accessing the ARC (as far as I could tell).
That got me very curious, so I proceeded testing each memory DIMM using MemTest86 and that finally got me some results. I received more than 100 correctable (single bit flips) errors during one pass for each DIMM.
I don't have any proof but my theory is that if the error rate for these DIMMs is so high, it is not at all unlikely to get a double bit flip every now and then. Which would cause a panic and trigger a restart.
So result is I order new RAM, tested it to make sure it doesn't have any issues, installed it, and since then it runs stable.
So what are the lessons learned:

1. Does the X470D4U with Ryzen 2nd gen have issues with instability when waking up from idle under FreeNAS? Not sure, but if so, there might be hopes for BIOS 3.25.
2. Is ECC correctly implemented by ASRock ... apparently yes, but don't rely on the BMC event log. Although there is a category for ECC errors, the event log never showed anything.
3. Can it cost you lots of time troubleshooting issues that could have easily been identified with a proper pre-test of the system? Absolutely yes.

 

[MikeyG]

Which power settings are you using now that seem to be stable?

I've been running with just 'Power Supply Idle Control' set to 'Typical Current Idle' for the last couple weeks without any crashes. C6 and Global C state control currently enabled for me. I did switch over to a 3600x though, so not sure if that had any effect.

Glad to hear you found a concrete problem with a solution. Sometimes its nice to get errors that point towards something like that.

 

[poldi]

Actually with the BIOS update all settings went back to default and I didn't intervene there as I wanted to test the new BIOS with stock settings.
I checked that all the settings were corrected, i.e. C6 Mode, Global C State Control, PSS Support are set to enable; Power Supply Idle Control is also set to default (just don't quite remember what that was).

 

[edge-case]

I'm hoping one of you guys can help. I don't know if I'm being dumb, ignorant [or both], but I can't get [ethernet] networking functioning and it's driving me crazy:

New X470D4U m/board with Ryzen 7 2700 and 32 GiB ECC PC
[As a pre-test, I set up the motherboard/CPU in a test case/PSU; Windows 10 Pro installed successfully to a 256 GiB NVMe SSD, and networking worked with no problems].

I already had/have a fully functioning FreeNAS [USB] set-up on other motherboard/CPU.

IPMI functions correct - gets an IP address via DHCP [ethernet] and I can happily interact with it via another computer on my network via ethernet.
[I even used the web interface to flash the BIOS from 3.10 -> 3.30].

FreeNAS - I simply can't get it to work with the Ethernet ports/network interface; it never connects to my LAN and can't be pinged or find it's own
DHCP address.

History:
I've been running FreeNAS-11.2-U7 [and previously U6] for months [on a SanDisk USB 3.0 stick] with no problems on a different, Ryzen 7 based system [ASRock B450 Pro; Ryzen 7 1700; 16 GiB RAM], and simply removed the motherboard & CPU, and replaced it with the X470D4U and Ryzen 2700 [that I had previously set up in a test case with Windows 10 and was working successfully [including ethernet]].

Things I've tried with FreeNAS:
- [doesn't work] swapping ethernet cable between the two 1 Gb/secs ports...
- [doesn't work] with and without an ethernet cable plugged into the IPMI port
- [doesn't work] booting with a freshly created FreeNAS-11.2-U7 USB installer
- [doesn't work] updated the BIOS from P3.10 to P3.30
- reset BIOS settings to default

Other OS on same machine with no other changes; ethernet networking functions flawlessly:
- with a Ubuntu 18.04.03 install USB
- Windows 10 installed on an NVMe SSD

Any thoughts? Something obvious I've missed in BIOS settings or the like?
This is my first "server" type motherboard [and first with IPMI] and I'm afraid I may have overlooked something totally obvious.

As an aside, I see that Intel has FreeBSD drivers on their website, but neither of you mentioned needing to install anything in your write-ups above.
Intel® Network Adapter Driver for 82575/6 and 82580-based Gigabit Network Connections under FreeBSD*

Any help would be appreciated. I can provide any logs / dumps as needed.

Thanks

 

[edge-case]

Here's the output of ifconfig:

 

[edge-case]

..and trying to reset the network interface [igb0]:

 

[edge-case]

So - and sorry for multi posting in this thread - I was able to manually configure the network interface [assigning a static IP address, and turning of DHCP] and everything appears to now be working fine...

I assume that the IPMI interface was interacting with the DHCP server and preventing ability for igb0 to get assigned an IP address, but I had tried turning off that in BIOS [or at least though I had]? Weird that DHCP worked in Windows and Ubuntu,but not FreeNAS...

Anyways, sorry again for the multiple posts.
If there's a way of deleting [or editing] my posts above - and this one - I'd be happy to do so...

 

[poldi]

Hi, don't worry about the posts I do understand that this is a pressing issue.
To be honest I did not have any such issues once I configured my network interfaces from the command line. I also run FreeNAS on a static IP and the IPMI on DHCP which works ok for me. I do remember that I set up DHCP for both FreeNAS and IPMI in the beginning and that worked fine as well. In general the interfaces are all separate, with each having a dedicated MAC address. As such they can all work independently and there should be no competition between IPMI NIC and others if they are configured correctly.
The IPMI interface is configured to DHCP out of the box hence this was working straight away. In FreeNAS you need to configure the interfaces first. Since in your screenshot the interface failed to start I would suspect that there was an issue with the config that you resolved afterwards by resetting the configuration.

 

[edge-case]

Thanks. :)
It was driving me crazy, and preventing me from working on real life stuff..
If I had been using a static IP address on my existing machine/config I don’t think the problem would have even come up...

My plan was/is to let this sit for a week or so (using test data) to verify stability, etc., and then I’ll start migrating my main data files to it.
(Currently, I use a 2014 MacMini with multiple DAS enclosures as my main file server; and want to retire/ re-purpose that system).

On my home network, I have everything set as DHCP, since I don’t need / set up remote access and access everything as <server-name>.local rather than IP address.

The USB / config file worked perfectly when I put it back in the old motherboard; I think it’s that my FreeNAS install won’t let IPMI and normal interface both use DHCP; once the web GUI was up and running I did some digging and when I tried to turn on DHCP it gave me an error saying I couldn’t because - paraphrasing* - “something else was already using DHCP”.
Windows and Ubuntu didn’t have this DHCP conflict - they happily assigned two IP addresses via DHCP - so maybe it was something weird about my FreeNAS install.

*I’ll get the exact message when I get home tonight...

Thanks again for your reply.

 

[MikeyG]

Anyone on this thread updated to 11.3 release? As soon as I did, all my instability problems with this board came back worse than ever. Last night it reset twice in two hours. I finally had it stable for weeks without a reset on 11.2-U7.

 

[edge-case]

Anyone on this thread updated to 11.3 release? As soon as I did, all my instability problems with this board came back worse than ever. Last night it reset twice in two hours. I finally had it stable for weeks without a reset on 11.2-U7.

I’m not seeing any problems here so far. My X470D4U system has been running 11.3 continuously with zero problems for over 5 days...

 

[MikeyG]

@marshy do you have any specific BIOS settings, or is most everything on default?

 

[edge-case]

@marshy do you have any specific BIOS settings, or is most everything on default?

I was having all sorts of random shutdowns, i think related to RAM/ECC [and RAM timing?] but possibly also power states. My tribulations are documented here.

In summary, I changed to the following settings and have been trouble free ever since:

- set the memory timing to 2666 Mhz [instead of the 2400 it defaults to]
- disabled global C states
- set Power Supply Idle Control to Typical

 

[MikeyG]

Thanks @marshy. I originally had to mess with all that for 11.2, and then when I moved to a 3600 it got stable without those settings. Went back through all the same steps now that I'm on 11.3 and then it reset twice in one hour last night.

From my own experience and those on the forums, it seems like this board with Ryzen might be fine out of the box, it might require tweaking to get stable, or you might continue to struggle with it like me. If it works, the price/performance is great with ECC support, but at this point I wish I'd stuck with Supermicro + Intel as my other system with that combo has been totally solid. Mostly posting this for anyone else like me who is considering the platform.

 

[poldi]

Anyone on this thread updated to 11.3 release? As soon as I did, all my instability problems with this board came back worse than ever. Last night it reset twice in two hours. I finally had it stable for weeks without a reset on 11.2-U7.

Hi,
i upgraded to 11.3 about a week ago and I did not experience any stability issues so far. Actually since I swapped the faulty RAM sticks everything has been rock solid. I didn’t have a single unscheduled restart since my post at the end of Nov.
I am running BiOS 3.25 (don’t see the need to upgrade to the latest one at the moment) and put all settings in BIOS back to default. So no power management/idle state tweaks applied.

 

[edge-case]

Update: After getting the RAM speed correct - per my post above - I'm now at 52 days uptime, with no problems, and no ECC errors....

 

[Chamrajnagar]

Hi all,

I have placed an order for this board to upgrade my FreeNAS box.

Trying to locate some compatible memory and I'm having difficulty finding DDR4-2666 16GB ECC modules.

It looks like this matches ASRock's tested list:

M391A2K43BB1-CTD - Samsung 1x 16GB DDR4-2666 ECC UDIMM PC4-21300V-E Dual Rank x8 Module

Get ➡️ Lifetime warranty ✅ Great savings ✅ Buy factory original M391A2K43BB1-CTD - Samsung 1x 16GB DDR4-2666 ECC UDIMM PC4-21300V-E Dual Rank x8 Modules @Memory.NET!

memory.net

However there is a cheaper part:

M391A2K43DB1-CTD - Samsung 1x 16GB DDR4-2666 ECC UDIMM PC4-21300V-E Dual Rank x8 Module

Get ➡️ Lifetime warranty ✅ Great savings ✅ Buy factory original M391A2K43DB1-CTD - Samsung 1x 16GB DDR4-2666 ECC UDIMM PC4-21300V-E Dual Rank x8 Modules @Memory.NET!

memory.net

Would anyone happen to know the difference? The part numbers are identical except one is 'BB1' and the other is 'DB1'.

Thanks!

 

[Chamrajnagar]

Update to post above. I received a reply from memory.net:

The M391A2K43DB1‐CTD is the newest version of the M391A2K43BB1‐CTD

Samsung hasn’t even started making the DB1-CTD yet, so your only option at the moment is the BB1-CTD, which is a great part and will run in that motherboard.