Help choosing a configuration for vdevs and zpools.

[Miguel Nunes]

Hi all

I have been using FreeNAS in test mode only since April and I am convinced I want to keep using it.
I have started with old HDDs just to have a feeling on how to operate and manage the system using the web UI and some shell commands.

Now I want to go into production mode and there are some question about the best choice for vdevs and zpools.

My situation is this. I have 8 available slots for hot swapable hdds and 3 extra for 2.5" drives.

Currentlty I have 2 Pools. One uses one vdev with 4 500GB drives and RAID-Z2 and they other one uses a vdev with 3 2TB disks in RAID-Z1 (I remind you that this is just for testing and I am aware that with disks of this size RAID-Z2 would be the right setup. I just didn't have enough old hdds arround to implement).

I am planning to upgrade to 4 or 6TB NAS drives. What would be the trades of choosing the following configurations:

First - 2 pools each having 1 vdev of 4 discs.
Second - 1 pool with 1 vdev using 8 hdds in RAID-Z2 or even RAID-Z3.

What would be the implications in terms of data security and performance?
Would it be worse to have a flat disk spacde without silos?
I plan to use some iSCSI volumes and I read one should keep 50% freespace at least for performance.

I have 32GB of ECC RAM with an 8core avoton cpu (similar board that is shipped with the FreeNAS appliance).

What options would you recomend?
Are there others that would work better?

If there is the need for more information about my setup please feel free to ask.

Meanwhile thanks in advance for your atention.

Miguel

 

[Spearfoot]

It's hard for us to give you good advice without knowing how you intend to use your system... That said:

My main FreeNAS system has a single pool made up of 7 disks in a RAID-Z2 vdev. I use it for iSCSI and general-purpose file sharing. I'm the only user, and performance is very good.

However, if you're planning on providing block storage for several users, performance considerations do arise, especially if they are running their virtual machines hard. Mirrors are the best topology for iSCSI storage because they deliver the most IOPS. If you were to configure your 8 drives into a pool made up of 4 mirrored pairs, this would give you 4 times as many IOPS as the same 8 drives configured using RAID-Z2.

Also, if you plan on providing block storage to VMware ESXi (or any other hypervisor) you will either need a SLOG device or else turn off synchronous writes for the dataset. While turning synchronous writes off is okay for a home lab, it is not recommended in any kind of production environment.

In terms of security: most users here on the forum would probably agree that 8 drives in a RAID-Z2 configuration offers the best compromise between redundancy and storage efficiency.

Good luck!

 

[Miguel Nunes]

Thank you for your feedback Spearfoot.

I made a small pause to rethink the system use, and I think I can clarify things a bit more.

Purpose: General propose NAS with some production level features like iSCSI access.

Motherboard: Avoton (8cores) based motherboard similar to the one in FreeNAS Appliance.
RAM: 32GB ECC RAM (maybe soon expanding to 48GB if the mem controller permits)
HDD connectivity: 12 Sata Ports (Mix of Sata 2 and Sata 3 Ports)
NIC's: 3 x 1Gigabit Links (Whiling to find an affordable Intel dual NIC card to replace the current extra PCI-X card and be able to take advantage of a 4 Links topology)

Shares: File shares for OSX
TimeMachine Share.
File shares using NFS for Linux and FreeBSD
File shares using SMB for Windows

Rsync Targets: Used to backup data from an existing Synology DiskStation
Synology: It will be comissioned for development support and experimentation using iSCSI for ESXI, Citrix and KVM.
It has the 4 NIC links available and will run some volumes using btrfs. One volume will be EXT4.
Accepts a second target for TimeMachine.
Synology also serves as a NFS server
Provides DHCP/DNS roles for the home network.
It provides GitLab Service.
It Provides SVN.
Other Services on a need basis.

iSCSI: One VMWare ESXI datastore for not want to loose vms. (Starts with 1TB)
One Citrix Storage Repository for the same purpose (1TB)

Windows volume (250 to 500GB) to run a stable Windows machine.
Linux volume (250-500GB) to run a stable Linux Distribution
Willing to try bhyve if passthrough works and allows nvidia drivers to work under Windows and Linux

Jails: Plex to access media on tv.

bhyve VMs: The needed ones not using more than 16GB of the total 32GB. The permanent machines will use at most 4GB with low load.


To sum up: I will be replicating critical data between the Synology and the FreeNAS. Synology will be more dedicate to development/test purposes.

I will take into account your hint about IOPS and perform a test with my current configuration to see how it handles the different loads.

Since I will be using compression, I don't have to be limited to the 7 disks rule on RAID-Z2.

What about 2 vdevs using RAID-Z2 striped on a zpool? Would this be advantageous for increased IOPS with less risk than 4 vdevs of simple mirror? The way I see it, if I have 2 failures on the same vdev I will loose the entire pool. The option of RAID-Z2 or even RAID-Z3 could work for me to have a better balance between capacity and security on a 8 drives vdev.

About the log system, my intention was ti use the internal drives with ZIL approved drives if I find any lack of performance. But that would mean the need of a mirrored vdev to avoid loosing the pool.

To finish up,

How would you suggest me to benchmark the FreeNAS?

Thanks in advance.

 

[Stux]

Basically each vdev provides the IOPS of a single disk.

So 4 striped mirrors is 4x the IOPSs of a single Raidz2 vdev.

You have conflicting requirements ;)

For the NAS stuff 8 way RAIDZ2 is the right choice. For VM block storage 4 mirrors is the right choice.

An alternative would be to make a mirror of 2 SSDs for the VMs in a separate pool. And use the Raidz2 for mass storage.