Bad DD performance on all the pools of my system

[PK1048]

I know the L2ARC (cache) was mentioned before, but a little detail on why adding an L2ARC may actually hurt performance.

The ARC resides in main memory, which is 2nd only to onboard CPU cache memory (L1,L2,L3) in terms of speed.

The L2ARC resides on a drive and will be slower than ARC.

For each entry in the L2ARC, some space is allocated in the ARC to point to it. This reduces the effective size of the ARC.

Also note that the L2ARC is empty at boot and is populated as the system reads data out of the zpools. So until the L2ARC 'heats up' it will be providing almost no benefit. Large systems where the L2ARC provides substantial read acceleration have been known to script reading all the data the L2ARC is intended to cache, this script is run at boot time and pre-loads the L2ARC.

 

[flotueur]

Thanks, that makes sense.
Initially I thought that the system would always go to the cache drive before the HDDs which is why I thought it was a good idea to add one.
Now that I understand that ZFS uses ram as cache (ARC) and uses the cache drive ONLY if they amount that needs to be caches is less than the ram available, we can safely assume that with my workload and 64gb of ram (32 effectively allocated to ZFS), a cache drive is not required.

Thanks for the explanation about the references. In a weird way, it almost feels to me like the L2ARC is a way to compress the ARC to get more caching capacity! Yet that reinforce the fact I would really need to have either more users to rip the benefit off from caching...

What about the other vdevs proposed by TrueNas in the case of a SOHO? Are any useful to consider? I briefly read about them but it doesn't seem any would help in my use case...