Will this Ryzen build FreeNAS? And should I go with SCALE over CORE?

MGone

Dabbler
Joined
Feb 16, 2022
Messages
20
So, despite still not having found any available memory this is where I'm at for my build. Thanks for those who've helped me get this far!

One thing to note is that I am fully aware that the processor is a wee-bit overkill. However, I had a chance to pick this up from a friend for pretty much the same as the Ryzen 3 3100 I was considering....so here goes.

Purpose
An alternative to an off the shelf 8 bay NAS such at the Synology DS1821+ (4 core embedded Ryzen V1500B) or QNAP TVS-872X (i3-8100T)
Smallish form factor and as quiet as possible.
File server for max 10 people (mix local & remote)
Host occasional Minecraft server for my kids and their friends
Have the capability to act as a media server in the not too distant future (Plex/Jellyfin) by possibly adding a GPU.

Components (Amazon UK links provided and surprisingly they were often cheapest!! Although if you're UK based I'd consider looking at Scan.co.uk amongst others too!)
  • Case (£78): Fractal Node 804
  • Motherboard (£255): ASRock Rack X470D4U
  • CPU (£215): AMD Ryzen 5 5600X (6 core 3.7GHz 65w Vermeer)
  • Cooler (£50): Noctua NH-D9L with NF-A9 92mm Fan
  • Memory (£200?): 2x 16Gb KSM26ED8/16ME (still searching!)
  • Storage (£480): 3x 4TB WD Red Plus, 3x 4TB Seagate Ironwolf
  • Boot Drives(£36): 2x Gigabyte 120GB 2.5" SATA III SSD
  • PSU (£75): Seasonic Focus PX 650W
  • Case Fans (£95): 3x Noctua NF-A12x25 PWM 1x Noctua NF-A14 PWM

    Total Build Cost: £1004 with just boot drives, £1484 including Storage and Metadata cache drives (although shopping around and getting the CPU from my buddy should knock off about £200)

    Future additions could be:
  • Media Metadata drives: 2x WD RED SA500 M.2 2280 SSD
  • Media Storage: 1x 8TB WD Red Plus, 1x 8TB Seagate Ironwolf
  • 10GbE card: Chelsio T420-CR Dual Port 10GbE (in slot PCIe4 3.0x8)
  • Storage Expander: LSI MegaRAID 9220-8i IT Mode (in slot PCIe5 3.0x8)
  • GPU for Hardware Encoding: NVIDIA Quadro P400 or similar lower end but 4K capable (in slot PCIe6 3.0x16 operating at x8)
Discussion
So I thought I'd share my reasoning behind these decisions and hopefully get some feedback if my assumptions and understanding is misguided!

The case, cooler and fans were all pretty much set from the start. There really isn't much choice out there for mATX cases that hold a decent number of drive bays. There's a fair few other builds kicking around with this case and so it made sense to go with a tested solution. I'll admit that the fans are a little luxury, but having used Noctua for years and never had an issue it seemed a no-brainer. Also, they now have an incredible compatibility checker which allowed me to bit a cooler to fit the CPU, MB & Case!

Having read a number of builds based upon more media workstation or high end gaming motherboards I went around the houses for quite some time flicking back and forth between Intel and AMD platforms. I was tempted by the Xeon W's but found myself soon pushing £600 for the CPU&MB. I tried looking at older boards/cpus, mainly LGA2011v3 and LGA1150 boards but felt ultimately that for what was available to me in the UK I was getting more value with the AMD....(as a side note, it was older motherboards that was the problem...especially in the mATX format as I decided that full size or rack wasn't going to wok for us at the moment!)

My motherboard choice centred around a few priorities, with ECC support and IPMI being the two that wouldn't be easy (or possible) to add in later.
  • ECC Support (I heard this is recommended :wink:)
  • IMPI (running without a screen)
  • 8+SATA 3 (and or enough PCIe slots to add expansion)
  • Intel LAN (1GbE should be fine for the time being, but allow for expansion)
  • 2 M.2 slots (for fast metadata drive)
  • On board VGA (without needing APU chip support)
I did what I think many have done and tried to find a consumer (read cheaper) board that met these criteria. One of the reasons for going down the AMD route was the hope that it would be easier to find consumer boards that supported ECC. This was not straight forward as many that suggest the did, did so by not actually supporting the ECC function, so in effect were useless. It also opened my eyes to the fact that CPU selection was important as several of the motherboards stated that many of the APU enabled Ryzens would not work with ECC. Which then prompted me to add onboard VGA support as a requirement...something that most modern boards seem to lack as they are geared up to using the chip APU.

The real nail in the coffin for consumer boards though was the lack of IPMI. Even the most basic of NAS boxes allows you to access them via a network connection to update the bios and cycle power etc...and the additional features that this would make running the system easier.

In the end it boiled down to the ASRock Rack x570D4U and it's older sibling the X470D4U (and their 10GbE cousins!)

I decided I didn't need PCIe 4.0 for now nor 10GbE, so it was the X470 for me as I think the motherboard gives enough expansion for my future needs.

Storage was far more straight forward. I analysed our current usage and felt that 8TB would be sufficient for the next few years at least, but that 10-12TB would give more headroom to accommodate new cameras and larger files. After lots of reading I decided that this would be best achieved by setting up a pool of three vdevs, each containing a 2 disk mirror. I have chosen to pair one Seagate with one WD red in each vdev to reduce the chance of a 2 disk failure in any one vdev. The two SSD's would be another mirrored vdev creating a boot pool.

At a later date I can add a couple of 8TB drives in a mirrored vdev to create a media pool and the m.2 SATA drives would then form another vdev in a separate pool for the cache for Plex.

I chose the gigabyte drives as boot drives simply because I was after the cheapest I could find...I narrowed it down to whatever is on offer and these have the best DWPD at 0.57 compared with 0.3 for most of the others.

The m.2 drives were a little more tricky because the chosen motherboard only supports m.2 SATA not NVMe, and so this limited my choice performance wise, but the WD Red seemed a solid choice, although I was disappointed to find out that the SA500 is almost an order of magnitude slower than the newer NVMEe WD Red SN700...!

So that just leaves the PSU...and 750w may seem big, but here is how I got to that size:
Component
idle Watts
peak Watts
Motherboard2525
Ryzen 5 5600X1065
Memory1212
CPU Cooler0.61.2
Fans3.06.7
3x 4TB WD Red Plus12.363
3x 4TB Seagate Ironwolf11.8564.8
2x Gigabyte 120GB 2.5" SATA SSD
0.345.22
2x WD RED M.2 SATA 500GB0.16.7
1x 8TB WD Red Plus4.121.7
1x 8TB Seagate Ironwolf3.9521.6
Chelsio T420-CR Dual Port 10GbE612
LSI MegaRAID 9220-8i811
NVIDIA® QUADRO® P400530
102.24 W idle345.92 W peak

If I apply the 1.25 multiplier suggested in the excellent 'size your PSU' guide I get a MAX power of 432.4W so in theory I could get a 450W PSU and be covered for my planned upgrades. However, there were two factors (aside from future additions) which have pushed me to look at larger PSUs. Firstly is the efficiency curves. Most of the PSUs I have looked at have a maximum efficiency at around 40-60% load, and the second is that by not working the PSU too hard the fan won't kick in and it will contribute to a quieter system. 650W may be a bit too much in the other direction, and I could consider a 550W, but the Seasonic Focus PX-650 seems a bargain at the current price.

So, I think that's the lot hardware wise...I hope my thought process made sense...please hit me up with any suggestions as I'd like to hit that buy button whilst some of these prices are still low!!

Oh, and should I be considering TrueNAS Scale rather than Core because of the poor support for NVIDIA GPUs in Core?

One final thing I thought might be useful was to link (in no particular order) the resources I have found most helpful in my decision making.
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,703
Storage Expander: LSI MegaRAID 9220-8i IT Mode (in slot PCIe5 3.0x8)
I think you need to swap this for an HBA version... I don't know that you can get IT firmware for these. Anything with MegaRAID (or generally RAID) in the title is bad.
 

MGone

Dabbler
Joined
Feb 16, 2022
Messages
20
I think you need to swap this for an HBA version... I don't know that you can get IT firmware for these. Anything with MegaRAID (or generally RAID) in the title is bad.
Ah, okay thanks for picking that up! I had assumed that since it said IT mode and the seller listed it as being FreeNAS compatible it woud be okay. I'll do a little more digging when the time comes.

Out of interest, might I be better off putting the additional SAS/SATA HBA in now and utilising that rather than the motherboard SATA connections? Is there any advantages to one over the other?
 

LarsR

Guru
Joined
Oct 23, 2020
Messages
719
When using Core with ryzen cpu there might be some additional bios tweeks you have to do.
With first and second gen Ryzen you had to disable amd cool&quit, erp-ready and global c-states for the system to become stable. Otherwise there could be hard lock-ups due to core not liking the advanced power saving modes.
I dont know about 3000 and 5000 series cpu's because i was annoyed by that and switched to scale while it was still in beta. Since then i never looked back to core... and since scale is debian based, the cpu temps for ryzen cpu's are also recognized correctly
 

MGone

Dabbler
Joined
Feb 16, 2022
Messages
20
When using Core with ryzen cpu there might be some additional bios tweeks you have to do.
With first and second gen Ryzen you had to disable amd cool&quit, erp-ready and global c-states for the system to become stable. Otherwise there could be hard lock-ups due to core not liking the advanced power saving modes.
I dont know about 3000 and 5000 series cpu's because i was annoyed by that and switched to scale while it was still in beta. Since then i never looked back to core... and since scale is debian based, the cpu temps for ryzen cpu's are also recognized correctly

I'm still a little fuzzy on the differences between Core and Scale and any pros and cons when it comes to actual setup and management. Are things such as Jails, VMs and Plex available in Scale, or is it more tailored to just enterprise use? It's not likely to suddenly disappear behind a paywall for example?
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,703
Out of interest, might I be better off putting the additional SAS/SATA HBA in now and utilising that rather than the motherboard SATA connections? Is there any advantages to one over the other?
If your onboard SATA is decent and has sufficient ports with the right speeds (I guess is the case), no real advantage.

I had assumed that since it said IT mode and the seller listed it as being FreeNAS compatible it woud be okay. I'll do a little more digging when the time comes.
It may be one of the models where that is available, by all means do the research. Maybe better to find one that's only ever an HBA if you can.

I'm still a little fuzzy on the differences between Core and Scale and any pros and cons when it comes to actual setup and management. Are things such as Jails, VMs and Plex available in Scale, or is it more tailored to just enterprise use? It's not likely to suddenly disappear behind a paywall for example?
CORE is on FreeBSD, SCALE is Linux (Debian-based).

Jails are a FreeBSD thing, so not an option in SCALE, only CORE.

Since there's no Docker (or other container engine compatible with Kubernetes in popular circulation) on FreeBSD, that's a SCALE thing... unless you run a Linux VM in CORE to get that... makes accessing the storage from containers a litle more janky, but certainly possible.

VMs are probably going to be "better" on SCALE since the KVM engine has wider use and support and the BHYVE engine of CORE isn't so mature, nor well adopted in general.

SCALE is "new" so lots of things still have sharp edges and (some) functionality which will make it very compelling isn't yet there (clustering of Kubernetes/Apps).

Plex is an App in SCALE and available as a Plugin Jail in CORE... plenty of other ways to have it on both platforms.

Plex and file sharing in general are pretty easy to transfer from one to the other, so there's not a lot of harm in going one way to decide later to go another.
 

LarsR

Guru
Joined
Oct 23, 2020
Messages
719
Both Core and Scale are open source. If you purchase a system configured by IxSystem you pay a licence for enterprise support. That Licence allows you to get support direcly from IxSystems rather then asking for support on the community Forums. You can't purchase that licence on its own. And Ix clearly statet that both scale and core will stay free and open source. They use the thousands of users as testers to get feedback on new releases and reports on bugs.
 

anodos

Sambassador
iXsystems
Joined
Mar 6, 2014
Messages
9,554
Some users have reported issues with timers on newer ryzen CPUs / mobos on SCALE (probably due to 5.10 kernel). https://jira.ixsystems.com/browse/NAS-114976 We'll begin BlueFin nightlies soonish, and will eventually have the 5.15 kernel there. This is just FYI, and is not to say that Ryzen support on Core will be better or that you'll have problems with this particular mobo + CPU combination.
 

MGone

Dabbler
Joined
Feb 16, 2022
Messages
20
If your onboard SATA is decent and has sufficient ports with the right speeds (I guess is the case), no real advantage.

It may be one of the models where that is available, by all means do the research. Maybe better to find one that's only ever an HBA if you can.
Yeah, I'll keep an eye out for a straight HBA one...I'm assuming that a used Dell or HP with the LSI/Broadcom chipset would be just as good?

Plex and file sharing in general are pretty easy to transfer from one to the other, so there's not a lot of harm in going one way to decide later to go another.
That's good to know....I might start loitering around the scale pages and see how things progress whilst I'm getting the build under way.
Both Core and Scale are open source. If you purchase a system configured by IxSystem you pay a licence for enterprise support. That Licence allows you to get support direcly from IxSystems rather then asking for support on the community Forums. You can't purchase that licence on its own. And Ix clearly statet that both scale and core will stay free and open source. They use the thousands of users as testers to get feedback on new releases and reports on bugs.
I'm pretty new to all this, but it seems like an awesome community of dedicated experts....IxS are getting a great deal by the looks of it :)
Some users have reported issues with timers on newer ryzen CPUs / mobos on SCALE (probably due to 5.10 kernel). https://jira.ixsystems.com/browse/NAS-114976 We'll begin BlueFin nightlies soonish, and will eventually have the 5.15 kernel there. This is just FYI, and is not to say that Ryzen support on Core will be better or that you'll have problems with this particular mobo + CPU combination.
Thanks for the heads up! I'll keep an eye on what's going on....is it the Vermeer chip in particular or are the issues with Matisse too?
 

MGone

Dabbler
Joined
Feb 16, 2022
Messages
20
Is anyone able to help me understand the function of SMBus? I'm going through the motherboard manual checking I haven't missed anything and there appears to be a PSU SMB header as well as IPMB and BMC SMB headers? Do I need to make sure my PSU can connect to the SMB header?

Also, do I require a separate TPM board to plug in? I know I have these on my windows workstations, but I'm unsure if that is something used by FreeBDS or Debian.

Thanks
 

Etorix

Wizard
Joined
Dec 30, 2020
Messages
2,134
1/ Only if you want to monitor the PSU by software.
2/ No.
 

MGone

Dabbler
Joined
Feb 16, 2022
Messages
20
I've only gone ahead and hit buy! In the end I went with these Integral memory modules that say they are a direct replacement for the QVL Samsung 32GB ones: M391A4G43MB1-CTD-IN

So with a total of 64GB I should have more than enough for my needs for some time.

Thanks for the help folks! I'll update once I have had a chance to put it together as I'll probably need some advice on a suitable testing regime to burn in disks and check memory is functioning.
 

artlessknave

Wizard
Joined
Oct 29, 2016
Messages
1,506
IT mode and the seller listed it as being FreeNAS compatible it woud be okay
megaraids that can in fact be converted to IT mode exist, however, there are a ton of fake cards, so you need to find a reputable seller of crossflashed cards.
one I would point to is
who has youtube channel with videos on things like how to crossflash and how to spot fakes. his cards are on the pricier side though.
 

MGone

Dabbler
Joined
Feb 16, 2022
Messages
20
megaraids that can in fact be converted to IT mode exist, however, there are a ton of fake cards, so you need to find a reputable seller of crossflashed cards.
one I would point to is
who has youtube channel with videos on things like how to crossflash and how to spot fakes. his cards are on the pricier side though.
Thanks for that...I'll check out the channel for some more info and dig around a bit....no rush as I've got plenty of TB at the moment....although I did stumble onto r/datahorder the other day soooooo....:wink:
 

artlessknave

Wizard
Joined
Oct 29, 2016
Messages
1,506
here is where we ask for help with technical stuff.
r/datahorder is where we just ask for "help!"
 

allanonmage

Dabbler
Joined
Aug 20, 2023
Messages
31
After reading the hardware recs, a few other posts, and posting my hardware, I came to the conclusion that BSD and/or TrueNAS Core did not support commonly available consumer hardware, unless you are a BSD driver developer and could rewrite half the OS to make it work (so, basically, not at all). So I just installed Scale instead. The read/write performance different seems to be 10% or less, so my intent was to save myself a headache from hardware issues.

Did I misunderstand the hardware compatibility and support?
 

ChrisRJ

Wizard
Joined
Oct 23, 2020
Messages
1,919
If possible, via a board with a sufficient number of SATA ports, I would try to avoid a separate HBA. Speed will usually not be better for HDDs, in fact there may even some additional latency. The latter is really not a problem, but I want to make it as clear as possible that in and off itself an HBA is not superior in terms of speed compared to an on-board SATA port.

In addition, HBAs consume a decent amount of power and require extra cooling (unless they are placed into a server case with a lot of properly funneled air flow).

Is there a particular reason for not using simply 2 large HDDs? The idle power consumption is not exactly low, after all.
 

Etorix

Wizard
Joined
Dec 30, 2020
Messages
2,134
Did I misunderstand the hardware compatibility and support?
Probably, if you're under the impression that SCALE has "better" or "wider" hardware support than CORE…
SCALE might have marginally more support for consumer-grade hardware in the sense that it may "work" for some values of "work" which are not too stringent. The compatibility list for hardware which is "validated to work as intended even under heavy workloads" is actually the same for CORE and SCALE.
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,703
The compatibility list for hardware which is "validated to work as intended even under heavy workloads" is actually the same for CORE and SCALE.
Which is a true statement... but the FreeBSD drivers for that hardware have millions more hours of running time/testing and even input from iX to tweak their performance and reliability, whereas the Linux drivers do not (at least not yet).
 
Top