L2ARC Help Needed

[ejn1111]

Hi, I'm working on my parts list for first build centered around a x11SSM-F board. I have purchased already a Supermicro AOC with 2 NVME slots. My pool will be 6x4TB in Raid Z2. I read that I don't need a huge card for SLOG (if I read correctly) but PLP is important so purchased a Kingston DC1000B 240GB card for the SLOG drive. For L2ARC do I read correctly that bigger is always better and as such maybe shoot for a larger drive? I'm leaning towards a Samsung 970 Evo 500GB for $90 but wanted to check the forums first for input. For another $50, I can get another Kingston DC100B server grade NVME at 480GB capacity but not sure if its worth it.

Appreciate any insight around drive selection that the forum may have. Also, is there any issue installing and setting up the SLOG and L2ARC drives after the initial setup and Pool configuration or does it have to be done at the same time? Many thanks!

 

[Elliot Dierksen]

SLOG matters if you are doing synch writes like NFS. You are better off investing in more RAM first prior to doing a dedicated L2ARC. RAM is faster than anything. It is definitely more expensive, but I have an Optane 900P as my SLOG, and it made a huge difference for me since I present the storage to ESXi via NFS.

 

[HoneyBadger]

The DC1000B is designed as a "boot drive" and isn't really the ideal candidate for an SLOG device due to low endurance - but if your environment will remain small then it's viable due to the large size. Check the link in my signature to the "SLOG thread" to run a quick benchmark on it as far as performance; it does have PLP so it should hopefully perform well enough.

As far as your questions, maxing RAM first is encouraged first as it's much faster than SSD, but if you have reached the limit of slots on your board or what's financially reasonable for DIMM size or total capacity then adding an L2ARC is fine.

No issue with adding SLOG and L2ARC after the pool is already set up - just ensure that you add them as "log" or "cache" devices and not as additional member vdevs to your pool!

 

[ejn1111]

The DC1000B is designed as a "boot drive" and isn't really the ideal candidate for an SLOG device due to low endurance - but if your environment will remain small then it's viable due to the large size. Check the link in my signature to the "SLOG thread" to run a quick benchmark on it as far as performance; it does have PLP so it should hopefully perform well enough.

As far as your questions, maxing RAM first is encouraged first as it's much faster than SSD, but if you have reached the limit of slots on your board or what's financially reasonable for DIMM size or total capacity then adding an L2ARC is fine.

No issue with adding SLOG and L2ARC after the pool is already set up - just ensure that you add them as "log" or "cache" devices and not as additional member vdevs to your pool!

Thanks, I saw the boot drive moniker but also said it can be used as a general drive also so thought it would be ok. Can you elaborate on what you mean by environment remains small? eg number of users or overall pool size?

On memory, I have 4 slots and can go 64GB but the memory can drive the budget up fast. I have 16gb with 1 stick, I have another 8GB stick that i was going to try and add so will have 24GB (but wont be interweaved). Each 16GB stick is $150-$200 so was thinking a L2ARC NVME was a economical speed upgrade (albeit not as much as RAM)... Is it better to have nothing and just wait until RAM is maxed? I ask as it appears there are many systems with a read cache and RAM not maxed out on the M/B. Many thanks!!!

 

[HoneyBadger]

Thanks, I saw the boot drive moniker but also said it can be used as a general drive also so thought it would be ok. Can you elaborate on what you mean by environment remains small? eg number of users or overall pool size?

"Small" as in the total amount of data written to the array - the DC1000B is rated for 0.5 DWPD which means 120G/day on the 240G drive.

I definitely understand on the cost of larger RAM sticks.

But ultimately let's go back to "what is the use for the array?" to determine if you will actually benefit from L2ARC and SLOG. If you aren't doing any sync writes, then you don't need an SLOG device. If you're doing media storage, then L2ARC might only be useful for metadata to have fast file browsing, since you're unlikely to be re-watching the same movie/TV show over and over.

 

[ejn1111]

"Small" as in the total amount of data written to the array - the DC1000B is rated for 0.5 DWPD which means 120G/day on the 240G drive.

I definitely understand on the cost of larger RAM sticks.

But ultimately let's go back to "what is the use for the array?" to determine if you will actually benefit from L2ARC and SLOG. If you aren't doing any sync writes, then you don't need an SLOG device. If you're doing media storage, then L2ARC might only be useful for metadata to have fast file browsing, since you're unlikely to be re-watching the same movie/TV show over and over.

Thanks... My use case today is mainly a backup device (PCs, Phone's of normal home backup items) and media storage (mainly musics files streaming and some videos). Assuming large backups are synchronous writes?

Also, I would like to get the NAS quick enough that I can access files with the same feel as if they were on my PC directly. Eg Lightroom edits and library access remains on NAS and accessed from the network instead of moving groups of files, working on them on a PC and then moving them to storage on the NAS.

 

[HoneyBadger]

Virtual machines and databases would be synchronous writes - that's where you are demanding that the writes go to stable storage and wouldn't be lost even in the case of a power failure right when the copy finishes. Regular consumer file storage generally doesn't require that level of protection, and is fine with being "cached" in RAM until the disk flush happens.

An L2ARC for metadata (and even some data) might be useful in your case, but bear in mind that local SSD will be faster than a remote disk over 1Gbps. 10Gbps will make it closer, but there's always the latency penalty of having to go across the wire/TCP overhead.

I'd suggest taking the 240G drive you've already purchased and use that for L2ARC, since you don't have a workload that requires SLOG.

 

[ejn1111]

Virtual machines and databases would be synchronous writes - that's where you are demanding that the writes go to stable storage and wouldn't be lost even in the case of a power failure right when the copy finishes. Regular consumer file storage generally doesn't require that level of protection, and is fine with being "cached" in RAM until the disk flush happens.

An L2ARC for metadata (and even some data) might be useful in your case, but bear in mind that local SSD will be faster than a remote disk over 1Gbps. 10Gbps will make it closer, but there's always the latency penalty of having to go across the wire/TCP overhead.

I'd suggest taking the 240G drive you've already purchased and use that for L2ARC, since you don't have a workload that requires SLOG.

Thanks for the awesome advice and helping a first build! FYI, I am adding a 10G SFP+ supermicro PCI card to the build and plan on running 10G when i make some other home network modifications (running some fiber). I ordered another 16Gb Ram stick basis the above advise as well. Much appreciated!