It was designed to either improve performance or do nothing, so there isn't anything that should be bad. To explain what I mean by do nothing - if you use the L2ARC for a streaming or sequential workload, then the L2ARC will mostly ignore it and not cache it. This is because the default L2ARC settings assume you are using current SSD devices, where caching random read workloads is most favourable; with future SSDs (or other storage technology), we can use the L2ARC for streaming workloads as well."
Yeah, what anodos said. That's spot on.
BIG, BIG difference between the example you are giving and what this forum regularly sees (and what this thread is about). Only a fraction of 1% of users will even 32GB of RAM.
8GB of RAM with an L2ARC on FreeNAS is shown, time and time again, to hurt performance. While people can argue until they are blue in the face that "it was designed to either improve performance or do nothing" I can tell you with
certainty that it does NOT work that way on FreeNAS. As we've seen in dozens and dozens of threads from just this year that is just not the case.
The problem is virtually nobody here can afford to build their system with hardware that is roughly appropriate for the workload. People drop 8GB in a FreeNAS box and then freak out when they can't do 1GB/sec throughput on their 50TB 14 disk pool. Unfortunately we don't perform miracles with FreeNAS (the miracles plugin costs extra).
One thing you'll learn Linda is that far more people will want to do it wrong (almost always in the name of saving money or saving watts, which they usually end up getting wrong) and watch it stab themselves in the eye before they realize they should have just done it per our recommendations to start with. If you look at slide 15 of my noob presentation I mention specifically that L2ARCs can and will hurt performance. If you don't have the RAM resources to support an L2ARC you *will* hurt performance. FreeBSD may be different with 8GB of RAM. I don't know as I'm not going to argue how FreeBSD behaves. But in FreeNAS-land adding an L2ARC when you don't already have a respectable amount of RAM will actually hurt performance.
The problem is people that don't have ZFS and/or FreeNAS experience read stuff like you just quoted, immediate equate that to "it can't hurt me then" and then look at their "soon-to-be-FreeNAS" setup. They decide they'll go with 8GB of RAM and instead of buying another stick of RAM they decide that a 240GB SSD would give them 240GB of cache, far more than the 8GB of RAM. Since 8GB < 240GB clearly it's the smarter way to go, right? So they drop their money and then are woefully disappointed when it doesn't work like they think. The reality is that they shouldn't be eyeballing a 240GB L2ARC until they have something like 50GB of RAM. And probably like 0.5% of users in this forum have hardware that will even support >32GB of RAM.
You also have things to consider that you will not see anywhere else in FreeBSD-land. For example, FreeNAS allocates 1-2GB of RAM for RAM disks, it has to run services and is an appliance-like setup, and it has "decided" on many settings for you with the "slimmed-down" WebGUI. Even when you look at FreeBSD it recommends 1GB of RAM minimum for ZFS but strongly urges more.
Then you read stuff like the
ZFS evil tuning guide, which says you can use ZFS on FreeBSD with 1GB of RAM but then goes on to say that 64-bit is recommended but 32-bit is supported with sufficient tuning. That would seem to virtually imply that 4GB of RAM minimum is a good place to start. People also read that kind of stuff and then say to themselves "well, gee, FreeNAS doesn't need 8GB of RAM" and then we see broken pools. I hate poorly written documentation like the evil tuning guide that says one thing but then implies a whole different ballgame. Is 1GB workable or do you really need >4GB? There's a 4-fold+ increase in RAM needs based on how you want to argue it.
Much of FreeBSD knowledge applies to FreeNAS, but not in the same way as many expect and definitely is detrimental to people that look at FreeBSD's system requirements, then FreeNAS' requirements, and then say that the FreeNAS documentation must be wrong. Similar code, but totally different design and so they share much but not all. If you don't know the difference you can quickly find yourself with bad advice and giving bad ideas that backfire on you. Most of our recommendations revolve around the experience of seeing what does and doesn't work for various users.
