ZIL and L2ARC on same disk

[juhokki]

I'm looking for advice how I should utilize my 2 80GB Intel 320 SSDs for FreeNAS.

I was thinking that I mirror the disks and create two partitions on them; one for ZIL and one for L2ARC. Is this a good solution? And if it is, how big should the ZIL partition be?

The box currently only has 8GB of ECC RAM, but I plan on upgrading it in the near future.

 

[anodos]

I'm looking for advice how I should utilize my 2 80GB Intel 320 SSDs for FreeNAS.

I was thinking that I mirror the disks and create two partitions on them; one for ZIL and one for L2ARC. Is this a good solution? And if it is, how big should the ZIL partition be?

The box currently only has 8GB of ECC RAM, but I plan on upgrading it in the near future.

You almost certainly need neither an L2ARC nor a ZIL. I recommend reading this: http://forums.freenas.org/threads/slideshow-explaining-vdev-zpool-zil-and-l2arc-for-noobs.7775/

Some more specifics: as a rule of thumb L2ARC is only really useful if you have lots of RAM (64GB+) and a ZIL is only useful if you're performing lots of synchronous writes.

 

[L]

What are you serving on the box? how much disk? and what are your reliability concerns??

I actually have recommended that for a number of people. With ssd you don't have rotational delay or head movement that you would have with hdd.. The only thing I don't know is how to partition those ssd's with freenas. If I only had 2 ssd's, I would carve off like 1GB for zil/slog on both drives and mirror those partitions, then stripe out the rest for cache.

 

[anodos]

What are you serving on the box? how much disk? and what are your reliability concerns??

I actually have recommended that for a number of people. With ssd you don't have rotational delay or head movement that you would have with hdd.. The only thing I don't know is how to partition those ssd's with freenas. If I only had 2 ssd's, I would carve off like 1GB for zil/slog on both drives and mirror those partitions, then stripe out the rest for cache.

A system with only 8GB RAM shouldn't have an L2ARC. It will only hurt performance.

 

[juhokki]

What are you serving on the box? how much disk? and what are your reliability concerns??.

I will use it to share storage with ESXi servers. I am currently using local storage with the ESXi servers, but plan on moving to shared storage with the FreeNAS box. I have a total of 4 1TB HDD disks.

Reliability concerns? It's a home/lab environment so possible downtime is not a big concern, but I would like to keep the data safe (I currently back up the VMs to USB disk and CrashPlan using the free version of Veeam).

 

[L]

Whether or not arc will help depends on workload. If this is serving out like workgroup small files arc will absolutely help. If this is a media server, no arc, no zil. Small files, or DB, yes arc, yes zil.

If it's esxi, then with nfs or iscsi, yes arc yes zil..If this is a lab, nice test bed to see what kind of zil you might need. I recently found zilstat in /usr/local/bin. It should show how much zil you are currently using without dedicated drives for logging.. that can help you size the partition for zil.

 

[juhokki]

You almost certainly need neither an L2ARC nor a ZIL. I recommend reading this: http://forums.freenas.org/threads/slideshow-explaining-vdev-zpool-zil-and-l2arc-for-noobs.7775/

Some more specifics: as a rule of thumb L2ARC is only really useful if you have lots of RAM (64GB+) and a ZIL is only useful if you're performing lots of synchronous writes.

Thanks for the link. I will give it a read.

Would L2ARC make sense with 32GB of RAM?

I will be using the storage for ESXi VMs so all of the writes will be synchronous(?). In this case it would make sense to use ZIL?

 

[anodos]

Thanks for the link. I will give it a read.

Would L2ARC make sense with 32GB of RAM?

I will be using the storage for ESXi VMs so all of the writes will be synchronous(?). In this case it would make sense to use ZIL?

That is one of the cases where a ZIL helps. Size of your ZIL should not exceed 1/2 the size of your physical memory. See here: http://docs.oracle.com/cd/E19253-01/819-5461/gfgaa/index.html In reality you probably won't want to go above 1-2GB.

I probably still wouldn't add at 32GB RAM. I think that with ESXi storage you may want to set up mirrors rather than RAIDZ vdevs. Maxing out ARC (RAM) is recommended before trying to add an L2ARC device.

 

[jgreco]

I actually have recommended that for a number of people. With ssd you don't have rotational delay or head movement that you would have with hdd.. The only thing I don't know is how to partition those ssd's with freenas. If I only had 2 ssd's, I would carve off like 1GB for zil/slog on both drives and mirror those partitions, then stripe out the rest for cache.

It's a terrible recommendation. It seems logical on one hand, but what you're really doing is creating competition/contention for the resource. The things that make for a good SLOG device (write performance, write endurance) are not the same things that make for good L2ARC (cheap! fast! big!). The cost per gig of an appropriate SLOG device is much higher than the cost per gig of an appropriate L2ARC device, so the smart move is to purchase a small SLOG device and a much larger inexpensive L2ARC and not have reads and writes competing for the resource.

 

[L]

Yea, but if all you have is all you have then use it..Not everyone has the luxury to buy a bunch of gear for a test platform. Since this is a test platform, then I would test it.

 

[jgreco]

No, you don't use it just because it's all you have. In order to be successful, a ZFS system must be built thoughtfully and correctly. Failure to do so can result in all sorts of problems, including performing poorly, but extending out into the realm of total pool loss.

In this case, advocating the addition of L2ARC is pure idiocy because the system in question only has 8GB. The first response actually indicated that this was a problem, yet you felt compelled to go and suggest something that is very bad for multiple reasons anyways.

Do not add L2ARC to an 8GB system. Period. Probably not to 16 either. Maybe at 24 or 32GB, but you have to remember that L2ARC stresses out the ARC and that you shouldn't exceed maybe a 4:1 ratio of L2ARC:ARC.

 

[juhokki]

That is one of the cases where a ZIL helps. Size of your ZIL should not exceed 1/2 the size of your physical memory. See here: http://docs.oracle.com/cd/E19253-01/819-5461/gfgaa/index.html In reality you probably won't want to go above 1-2GB.

In the FreeNAS Guide PDF it says that:
"If you use a 60GB SSD it is recommended you use only 1-2GB max for the ZIL. This allows the SSD to use its builtin wear leveling mechanism to maximize reliability and performance(Intel only)."

Does that mean that if I use the 80GB Intel 320 SSDs for ZIL with 1-2 GB size only I wouldn't "waste" the extra space as it would help with wearing of the drives?

Edit: What I meant to ask if I should return these drives and order some smaller SSD drives instead?

 

[jgreco]

In the FreeNAS Guide PDF it says that:
"If you use a 60GB SSD it is recommended you use only 1-2GB max for the ZIL. This allows the SSD to use its builtin wear leveling mechanism to maximize reliability and performance(Intel only)."

Does that mean that if I use the 80GB Intel 320 SSDs for ZIL with 1-2 GB size only I wouldn't "waste" the extra space as it would help with wearing of the drives?

That's correct. That is why the manual says that. I've asked for a more formal method to actually use a small partition because that helps in a more subtle way...

 

[jgreco]

Reference previous post: http://forums.freenas.org/index.php...ssd-drive-to-underprovision.11824/#post-54426

Also bug tracker: https://bugs.freenas.org/issues/2365

 

[L]

On a test platform it would really depend on how write is taking place. Quite often with vm's you can see numbers like 90% read. If you have a low number of vm's(4-10) you may be able to have almost all of the working set in arc, assuming these are primarily running os

 

[jgreco]

Really? On 8GB? No, you're not going to get much of anything in the ARC and VM performance without a lot more RAM will suck.

 

[L]

The performance with l2 should be better than without

 

[jgreco]

Wrong. See post 11 in thread.

 

[wintermute000]

If it helps, I'm running more or less vanilla iscsi using a zvol on the rig in my sig, just for Esxi labbing as well. The iSCSI is entirely on a single sandisk Ultra plus 256Gb SSD. It basically appears to behave the same as a normal spinning disk in terms of user perception when you're in the VM, I get around 600Mbit speeds down my iSCSI dedicated NIC. I have done no special freenas config, fancy separate ZIL etc. in fact its running off USB (like the ESXis lol).

 

[jgreco]

Not to burst your bubble but 600Mbit/s is in the realm of 60MB/s which is kinda poor for SSD.

You'd kinda think that SSD would be awesome as pool components but it actually turns out to be not-quite true. The problem of sync writes still occurs; if ESXi pushes out a block it has to be committed. With RAIDZ or even mirrors that involves multiple device writes. If nonredundancy is fine then the single SSD is a good compromise in some ways. You're not running sync=always I'm guessing...?