Ryzen server

[Raevyn]

Hello,

I have been studing ZFS for a FreeNAS system I am looking at building, and I was hoping to get a few questions answered and some thoughts on this build I have in mind.

First, I would be using AMD Ryzen, probably 7 3700X although if its considered a better idea, I could go to Ryzen 9 3900X instead. Im a bit confused which is best as I hear ZFS is resource intensive, but what that actually means in hardware gets a bit subjective I think. I did read in one article it is better to use Intel, though I dont know why, and if it really is I do have an Xeon 1650-v4 sitting around I could use instead. Its performance is not on par with Ryzen 7 or 9 but, if it will do the job its an option.

So what I have looking at is a MicroATX board. I know not to use RAID controllers so I would need to get some SATA drive to PCI adapters as I want to have 10 drives. That brings my first question; according to the information in the FreeNAS guide, once I add drives to a pool, thats it. I can remove a drive with a larger one without data loss but I wont gain any additional space until all the drives in that pool are the same size is that correct? I am also seeing that I can setup multiple pools so rather than installing all 10 drives in a machine, I could do 5, and when I purchase the next 5, I can create another pool which then my question is, would it still appear on the network as a singular volume or as 2? Also that would be a bit of redundancy if I made the 2nd set a mirror wouldn it?

As for the rest of the configuration, I read it is best to boot from USB, and having 2 USB mirrored is best. I would like to use encryption but its only able to work on BSD? I use Linux so that may be an issue. That brings me to the ZIL. I was thinking of getting a couple of SSD and mirroring them but in reading the guide I found on the forum, it says RAM can do that. I would be getting a UPS on it, and I would have 64GB of ECC.. is that more than enough where I dont need the SSDs? I am also reading that L2ARC is probably not needed the more RAM you have, which it happens to mention 64GB of RAM so I am thinking getting SSDs for read or write caching would be uncessary and yield no advantages?

I read that I can make another system that freenas can mirror so I have my data in two different locations, but I was wondering if anyone would choose to use ext4 or some other file system in case ZFS has an issue and gets corrupted?

 

[Tigersharke]

I believe the primary resource of that "ZFS is resource intensive" you mention is RAM.

As for intel vs AMD, that is not a discussion to start. I suspect part of the reason some point to intel is due to iX Systems preference or exclusive intel use in their products. I am an AMD consumer.

 

[Heracles]

Hey Raevyn,

Personally, I am using Intel only but I have no evidence to justify this reality. About ZFS being resource intensive, Tigersharke is right that RAM is the first resource to focus on. For CPU, you can kill your server if you push compression to its highest level. Your pool will drop to such a slow speed, it may even turn to almost unusable. But here, the point is more to adjust compression to a proper level more than boosting the CPU for a compression level that is too high and does not provide much benefit.

Most people here recommend to use an SSD as a boot drive, more than USB. Here, I am using mirrored USB sticks. I backup my config but in all cases, I do not have much config in FreeNAS itself. I will re-do it from scratch should I need. Not a big deal.

As for encryption, just don't do it. If done perfectly right, encryption will not provide you with much. If anything is wrong, it will easily destroy your entire pool. Remember than whenever your server is running, encryption is of no benefit at all. The only case where encryption will be of any use is if you remove one drive from your pool. Only then the cryptogram (the encrypted drive) will be separated from the key (the system). So only then encryption will be of any use. As long as a cryptogram sits next to its key, encryption is useless.

Don't lock yourself out of your own data with a self-inflicted ransomware in the name of an illusion of security.

 

[Tigersharke]

HDD encryption only protects against direct access to data on the drive, not data in transit, not data viewed by unauthorized access to your server should anyone gain entry to your network. Although I do not run FreeNAS, my personal FreeBSD box has been on ZFS since the first FreeBSD release that had ZFS. The only HDD encryption I have used is for swap because when swap is used a lot of temporary things get put there which may be best kept inaccessible. I have't thought about how this adds any security to my system but there have been enough horror stories about encryption to avoid using it for any other parts of my HDD/pool.

 

[Raevyn]

So what I am hearing is not to use encryption, and resource intensive refers more to RAM than CPU, assuming my compression level is average which is fine for me.

I would use a pair of SSD for boot as the size I need is cheap but before I decide that, the ZIL and L2ARC.. is it best to use SSD for them as well? Only so many ports and all.

I decided I will use my Xeon 1650v4 as it is a full ASUS ATX and seems as though it will work just fine for this. Plu I can expand its memory if I need to easily.

 

[Tigersharke]

I believe that you could use a HDD for ZIL or L2ARC but only if your storage is HDD, ie, similar speed or slower. The cache devices would be accessed fairly frequently and the point in having them is to speed up the rest of the disk/ZFS operations. As I understand it, you can use multiple devices for ZIL and/or L2ARC much the same way ZFS handles multiple drives as one entity (logical drive?).

 

[Raevyn]

I believe that you could use a HDD for ZIL or L2ARC but only if your storage is HDD, ie, similar speed or slower. The cache devices would be accessed fairly frequently and the point in having them is to speed up the rest of the disk/ZFS operations. As I understand it, you can use multiple devices for ZIL and/or L2ARC much the same way ZFS handles multiple drives as one entity (logical drive?).

Hm wouldnt using a regular HDD for the ZIL and L2ARC be pointless though? If the idea is to be speedy, it seems like on a HDD the read/write is going to be many times slower than on SSD, or would it still speed things up?

 

[Tigersharke]

Hm wouldnt using a regular HDD for the ZIL and L2ARC be pointless though? If the idea is to be speedy, it seems like on a HDD the read/write is going to be many times slower than on SSD, or would it still speed things up?

Yes of course it would have less impact but I believe that having them (even if all storage including cache were HDD) would cause some improvement. A worse-case scenario would be if the storage were SSD and the cache were HDD because in that situation the cache would very likely slow the whole.

Other ZFS/FreeNAS experts may have much greater opposition to such a setup and will also surely have more educated responses.

 

[Raevyn]

Yes of course it would have less impact but I believe that having them (even if all storage including cache were HDD) would cause some improvement. A worse-case scenario would be if the storage were SSD and the cache were HDD because in that situation the cache would very likely slow the whole.

Other ZFS/FreeNAS experts may have much greater opposition to such a setup and will also surely have more educated responses.

Oh okay I understand. Hm.. I have to think about that