Best ZFS config for specific uses

[davegibson]

Hello all!

I have a few 48TB systems (16x3TB, LSI SAS HBAs), which I expect no more than 4gbps out of (bind 4 GigaNICs). I've been studying ZFS RAIDZ, and was looking for some insight on the best ways to set up my ZFS pools for two specific but separate uses (meaning, the iSCSI box will not do NFS, and vice-versa):

1. As iSCSI extents for VMware.
2. As NFS/CIFS shares

My main issue is that I want to ensure 4gbps performance, and am unsure that a single, 16-drive RAIDZ (1 or 2) would be the fastest method, since RAIDZ performs at single-spindle speeds. I see many possibilities, from making multiple vdevs, each at RAIDZ, to increase speeds.

The problem I'm having is that I cannot see how to spread a zvol over multiple volumes. Am I missing something in the interface or manual? When I create a zvol, I can only select one ZFS volume, so cannot span, so cannot therefore get performance boosts above single-vdev RAIDZ. How are people getting faster-than-single-spindle performance from RAIDZ?

And on the more generic subject, how would you set up my 16 3TB drives to get the best in performance (VMware) and/or space (NFS)?

Thanks!
-Dave

 

[William Grzybowski]

I think there might be some misunderstanding here...

Doing a multiple vdev is on the same Volume/pool, so there is no "spreading zvol over multiple volumes", because there is only one, but with 2 raidz groups...

I think you could try 2 setups:
2 raidz vdevs (8 disk each) and 3 raidz vdevs (5 disks each)...
Doing a single vdev of 16 disks is not recommended, SUN doesn't recommend a vdev with more than 10 disks...

Also is worth to mention a raidz2 is slower than raidz in FreeBSD (possible due to a bug)