TrueNAS 12 Core build, with a couple small SSD's on top of normal disk storage questions..

[WB3FFV]

I have been doing a lot of reading here while putting together a TrueNAS 12 server for use here in my home lab. That said, the more I read about the various devs like cache, log, metadata and dedup, the more unsure I become of the best layout.

First off the server is a Dell R720xd, with a pair of Xeon E5-2697v2 processors, along with 192G of ECC RAM installed, plus typical things like redundant power supplies, so the box is pretty much loaded up. It's the LFF chassis, so I have a dozen 3.5" (LFF) 7200 rpm 3TB SAS drives in the front, which I know isn't the largest storage going, but it's way more than I used to have on my Synology, and it should suit my needs. I also wanted to get my NAS on to the 10GE LAN, and I have done that with this new server, in fact I used both 10G ports in an LACP group for redundancy there as well to the network.

Here is where it gets confusing. I loaded the OS on to the Dell internal Dual-SD card module (I have two Kingston 32G high-endurance cards in a mirror), and then I setup all the drives in the front of the chassis as a large Raid-Z2 storage. Now this server has 2x of the 2.5" (SFF) SAS slots in the rear of the chassis, and I figured I should put them to use. So I took and loaded in two 200G SAS SSD's into the slots. Now the question is, what to do with the two SSD's?

My initial thought was, with 192G RAM, I probably have plenty of ARC, so take the two SSD's in a mirror and use them for SLOG. Looks almost like overkill for SLOG from my reading, but I am not sure. I also see that it can be L2ARC, but again with 192G of RAM, it doesn't seem like I should need additional caching. Finally I see that I can use them as a metadev (fusion?) filesystem, but as my main disk is Raid-Z2, it barks about wanting to use an SSD mirror for fusion, so I was afraid to force it.

So I set out with Google, and started reading, and I of course have found many articles, but the more I seem to read various threads, the more confused I end up becoming on how to best use the couple small SAS SSD's I have in the chassis. I don't want to start a holy war over anything, but I am all for some input on this configuration I have, and what might best take advantage of the hardware I have to use.

Just as an FYI, I have a couple jails running with stuff like Plex, Logitech Media Server, and I have been playing with NextCloud, though I really haven't done diddly with it, just learning what it's about. I also do SMB and NFS shares in general, and setup a small iSCSI drive to use with Adobe Lightroom on my workstation as it will not let you keep it's catalogues on a mapped drive, but it sees very light use.

On a side note, where is the best place to keep the system dataset, if it matters?

Regards...

 

[sretalla]

Finally I see that I can use them as a metadev (fusion?) filesystem, but as my main disk is Raid-Z2, it barks about wanting to use an SSD mirror for fusion, so I was afraid to force it.

That was the right outcome... a special VDEV should have the same redundancy level as your pool... a simple mirror of two SSDs isn't a match of redundancy level to RAIDZ2 since a simple mirror can only lose 1 disk and remain intact, whereas RAIDZ2 can lose 2 disks and remain intact.

The right way to add a special VDEV for a RAIDZ2 pool is a 3-way mirror.

With the amount of RAM you have, it's doubtful that you will have need for L2ARC (unless you have a use case where you know you will be re-accessing a very large volume of data a lot).

With a RAIDZ2 pool, you only have the IOPS of one disk (perhaps a little more), so adding SLOG (although without knowing the specs of your SSDs, could be a terrible idea anyway if they don't have good write endurance and PLP) is unlikely to help much unless you have a few lightly used VMs connected for block storage.

IMHO, the best thing to do with those SSDs is create a pool with them in a mirror and use that pool for your jails and/or VMs, potentially storing app data and config on them for good performance.

 

[sretalla]

On a side note, where is the best place to keep the system dataset, if it matters?

Not on your boot pool... that will significantly reduce the life of your SD cards.

If you go ahead with the Mirrored pool I suggested, this would be a good location for it.