Configuration for 12 drives

[eroji]

I am finally buying the various parts for the build and I am looking to add 4 more 4TB drives to the setup, with a total of 12 to fill the chassis. The question is, what would be the best configuration for this?

Option 1
Intel 750 400GB ZIL
2 x IBM 5015 HBA IT mode
12 x 4TB raidz2

Option 2
Intel 750 400GB ZIL
2 x IBM 5015 HBA IT mode
6 x 4TB raidz2
6 x 4TB raidz2

Option 2 would give less useable space but more redundancy and easier for future upgrade as I can do 1 pool at a time while copying stuff over to the one that will remain untouched. However, will a single ZIL device work for 2 pools in the system?

Option 1 would net the most space, but I am unsure about the reliability of 12 drives in raidz2, not to mention, difficulties with upgrading to larger capacity drives.

 

[Robert Trevellyan]

Option 2 would give less useable space but more redundancy and easier for future upgrade as I can do 1 pool at a time while copying stuff over to the one that will remain untouched. However, will a single ZIL device work for 2 pools in the system?

Option 1 would net the most space, but I am unsure about the reliability of 12 drives in raidz2, not to mention, difficulties with upgrading to larger capacity drives.

You've correctly identified the reasons why Option 2 is probably best, and you probably don't need and won't benefit from a dedicated SLOG device anyway. The only reason I see to pick Option 1 is if maximizing storage capacity is your #1 priority.

 

[eroji]

I will be using the pool(s) for ESXi datastore so sync write performance will be important. So the question now is, will 1 SLOG/ZIL device work for 2 pools off separate HBAs?

 

[gpsguy]

For an ESXi datastore, striped mirrors would be a better option.

 

[eroji]

What about for the ZIL/SLOG question?

 

[anodos]

What about for the ZIL/SLOG question?

The pools can't share a slog. For your workload striped mirrors are probably best. Make sure you backup your data.

 

[Robert Trevellyan]

I will be using the pool(s) for ESXi datastore so sync write performance will be important. So the question now is, will 1 SLOG/ZIL device work for 2 pools off separate HBAs?

I searched the documentation for SLOG, and it referred me to this forum post by @jgreco. According to my understanding of that post, a dedicated SLOG device is per pool, so if your vdevs are all in one pool (which seems likely if you go with striped mirrors) then you would only need one SLOG device. I don't think it makes any difference whether the vdevs are connected to the same HBA.

 

[jgreco]

I searched the documentation for SLOG, and it referred me to this forum post by @jgreco. According to my understanding of that post, a dedicated SLOG device is per pool, so if your vdevs are all in one pool (which seems likely if you go with striped mirrors) then you would only need one SLOG device. I don't think it makes any difference whether the vdevs are connected to the same HBA.

Nicely said.

The idea of having drives all connected to a single controller is a peculiarity of hardware-based RAID controllers. For ZFS, it is an irrelevancy, except to the extent that different controllers might be more poorly supported or something like that.

 

[eroji]

So if for a mirror+stripe setup, if I want to upgrade the drives in the pool (swap out for larger drives), is this at all possible without a complete rebuild of the pool?

 

[jgreco]

Yes. Ideally you want to upgrade one vdev at a time, with the autoexpand property configured to on.

 

[eroji]

Well then, guess it's decided. Thanks to everyone for all the info.