About the FreeNAS storage and what to put in which location...

[Neosphere]

Hello there,

so let's assume I use 4x 2TB WD Reds (2x RAID I) for storing documents and media I want to keep safely.

- Where do I install FreeNAS when I additionally want ZFS cache? On a second SSD? So 1. SSD = Cache (Write + Read) and 2. SSD = FreeNAS?
- Where do I install plug-ins like Plex Media Server (or Apache with Wordpress) for best performance? Same SSD as FreeNAS?
- Is the idea to run Plex Media Server form the FreeNAS' SSD and access media on the WD Reds then?
- I was reading something about installing FreeNAS on USB Sticks. Isn't that slow as hell, even with USB 3.0? And where do I put the plug-ins then?

 

[danb35]

Where do I install FreeNAS when I additionally want ZFS cache?

You probably don't want ZFS cache, and you clearly don't understand it (there is no write cache to SSD, for example). But the answer to "where do I install FreeNAS" is the same whether you're going to use a cache or log device or not: to a USB stick, CF card, or small-ish SSD (which would include a SATA DOM). If you want cache or log devices, those would be on separate SSDs.

Where do I install plug-ins like Plex Media Server (or Apache with Wordpress) for best performance? Same SSD as FreeNAS?

No, nothing goes on the boot device other than the OS itself. Most users install their plugins and/or jails onto their data pool. Whatever's being frequently accessed is going to live in RAM anyway due to ZFS caching. You can set up a separate SSD pool for your jails if you want, but there's rarely a real benefit in doing so.

Is the idea to run Plex Media Server form the FreeNAS' SSD and access media on the WD Reds then?

No, see above.

I was reading something about installing FreeNAS on USB Sticks. Isn't that slow as hell, even with USB 3.0?

No; read speed on USB sticks is reasonable, it's writes that will kill you. But FreeNAS doesn't write to the boot device very much. And again, due to ZFS caching, most of the OS is going to live in RAM most of the time anyway.

And where do I put the plug-ins then?

As above--in most cases, onto the data pool.

 

[Neosphere]

You probably don't want ZFS cache...
and you clearly don't understand it...

Yes, I don't understand it fully. Explain it to me or point my to an article, please ;)
- So ZFS cache is s.th. else than ZFS log?
- Why do you think I don't want log and/or cache? Wouldn't it speed up my box, especially if I want to run VMs?
- You can choose to add write and read cache when buying the mini NAS from iX systems. Which caches do they use?

From the HW guide:
USB sticks:
"Unfortunately, reliability can be very bad and random write performance (which gets exercised during updates and other boot environment manipulations) is almost as bad."

About SLOG:
"SLOG devices SLOG devices should be high-end PCIe NVMe SSDs, such as the Intel P3700. The latency benefits of the NVMe specification have rendered SATA SSDs obsolete as SLOG devices, with the additional bandwidth being a nice bonus."

 

[danb35]

So ZFS cache is s.th. else than ZFS log?

Yes. Cache is a read cache (L2ARC), log (SLOG) provides persistent storage for synchronous writes to be committed before they can be written to spinning rust. Sync writes with a proper SLOG will be faster than sync writes without one, but will still be slower than async writes. Lots of material on this; here's an article from iX covering them in more detail: http://www.freenas.org/blog/zfs-zil-and-slog-demystified/

Why do you think I don't want log and/or cache? Wouldn't it speed up my box, especially if I want to run VMs?

A L2ARC can speed things up, but it also consumes RAM, and that's RAM that could (and would) otherwise be used for primary ARC. Unless you've maxed out your RAM (say, around 64 GB), the loss of RAM from ARC is probably going to outweigh the performance benefit from L2ARC. A proper SLOG can help with a VM-heavy workload.

You can choose to add write and read cache when buying the mini NAS from iX systems. Which caches do they use?

I have no idea what devices they use; maybe someone else can answer that. But if they're referring to the SLOG as a "write cache", that's kind of misleading--ZFS just doesn't work that way. A proper SLOG device has some unique characteristics, and I think you'll find threads in the Hardware forum addressing them and suggesting suitable devices.

 

[Ericloewe]

I have no idea what devices they use; maybe someone else can answer that.

Some variety of SATA SSDs.

I think you'll find threads in the Hardware forum addressing them and suggesting suitable devices.

That's covered in the Hardware Recommendations Guide (see my sig).

 

[Ericloewe]

From the HW guide:
USB sticks:
"Unfortunately, reliability can be very bad and random write performance (which gets exercised during updates and other boot environment manipulations) is almost as bad."

SSDs are better for boot, but they're not "spend 150 bucks on extra hardware" better. If you have a spare SATA port, SSD for boot is the way to go.