Help Choosing Block Size for 12 Wide raidz2

[homerjr43]

What is the optimal block size for a 12 Wide Raidz2 filled with 3TB drives.

Based on the advice detailed in...
http://blog.delphix.com/matt/2014/06/06/zfs-stripe-width/ and https://docs.google.com/spreadsheet...TjHv6CGVElrPqTA0w_ZY/edit?pli=1#gid=930519344

It appears that I have the least amount of overhead with 256k block size. Will this significantly decrease my performance? Currently I only have gigabit network, so I am hoping that i will still be able to saturate that connection.

Any suggestions?

 

[zambanini]

raidz2 will be slow anyway. without telling more about your expected workload, you will need a crystal ball.

 

[homerjr43]

This will be for homeuse: download manager, data backup, media serving/streaming, and possibly iSCSI drives in the future for a few front ends (5 Max). My frustration, is that to one man 10Gbps is slow, while to another 100MB/s is fast. Just trying to understand the drawbacks of 64 vs 128 vs 256 block sizes.

Thanks for the help!

 

[zambanini]

iscsi...tell more about your hw.

the block size depends in io speed vs throughoutput. but cyberjocks guide explains that great.

 

[homerjr43]

I have a Supermicro 12 Bay 2U server Dual Xeon (currently L5520) with 24GB ECC ram. Using 8 sata ports on M1015 and 4 on the integrated intel motherboard. At this point do not worry about the iSCSI issue, that may be used in the future for XBMC/Kodi frontends. I am just trying to figure out the overhead issue as noted in the articles above vs speed. It seems that even with a raidz2, the speed should be plenty fast with 12 drives, even at 128 or 256 block size.

 

[SirMaster]

That table is showing the size of your data after compression in blocks that you are storing to the pool and how much space will be taken by parity+padding

It's not talking about selecting a block size for your pool. ZFS uses a 128K block size and only recently did OpenZFS gain the ability to use larger block sizes.

They added this feature in FreeNAS 9.3 but it's not an option in the GUI. You would have to create your pool in the CLI with the larger block size which they don't support doing.

 

[homerjr43]

Ok, so from my understanding, you could select the max block size and that ZFS will dynamically choose the best block size (it is not static like my old intel raid). I know you can do up to 1MB blocks with ZFS on oracle equipment, but this is not supported yet in FreeBSD. So, if what you are saying is true, my overhead with a 12 wide raidz2 is variable from 18 to 67%?? I am new to this, I have ran an intel raid on my desktop, with great sucess for the past 6+ years, but just purchased this server with the plan to utilize Freenas and the benefits of ZFS.

If 256 or 512 is capable in CLI, what does it mean that it is not supported?

 

[cyberjock]

Honestly, if you are trying to do a 12 disk RAIDZ2, that choice alone is going to trump any kind of other optimizations you think you are going to do. It's like an ant trying to push a car in a different direction than the car wants to go. Doesn't matter how much the ant tries, the car is gonna win, ever time.

If you want performance don't do a 12 disk RAIDZ2. Do two vdevs (or more).

 

[homerjr43]

Again, I do not need "performance." I appreciate the responses, but what I am trying to determine is if the Raidz2 will be able to read and write at least 250MB/s or at least 2Gbps. This is not a high performance request, and high performance is not my need. Maximum storage with double redundancy is my need. I am just trying to determine if this setup will accomplish this. Also, it appears that for my needs, the larger the stripe size the better. So, before I start dumping data on to the array, I want to make sure I am choosing the best option.

Based on this article, https://calomel.org/zfs_raid_speed_capacity.html, performance looks great on a "12x 2TB raid6, raidz2 17 terabytes ( w=507MB/s , rw=256MB/s , r=660MB/s )"

Should I expect similar performance?

 

[cyberjock]

Again, I do not need "performance." I appreciate the responses, but what I am trying to determine is if the Raidz2 will be able to read and write at least 250MB/s or at least 2Gbps. This is not a high performance request, and high performance is not my need.

You tell us?

I have a RAIDZ2 of 10 disks. I can get speeds that cap out at 4MB/sec (that is not a typo). I can also do over 800MB/sec. Same pool and same hardware. The workload type matters, along with a crapload of other things.

So you can't set some arbitrary limit on what you want, then expect to get that at all times. ZFS doesn't work that way.

Should I expect similar performance?

That's why I said what I said above.

 

[homerjr43]

It is odd, that Freenas has an option in the GUI for 256, 512, and 1024 block size when creating a zvol, but if you select higher than 128 it says "'volblocksize' must be power of 2 from 512B to 128KB" I assume that is why Sirmaster stated that it is not supported. So, lets assume I am sticking with my 12 wide raid z2, what are the benefits of this higher blocksizes? If they are not supported now, can i set them manually, and assume that they will be supported in the future?

 

[SirMaster]

It is odd, that Freenas has an option in the GUI for 256, 512, and 1024 block size when creating a zvol, but if you select higher than 128 it says "'volblocksize' must be power of 2 from 512B to 128KB" I assume that is why Sirmaster stated that it is not supported. So, lets assume I am sticking with my 12 wide raid z2, what are the benefits of this higher blocksizes? If they are not supported now, can i set them manually, and assume that they will be supported in the future?

zvol blocksize and zpool blocksize are 2 very different things.