Pool disk Vdevs advise

[Fred974]

Hi,

I have a dedicated server for FreeNAS that is going to be use as iSCSI taget for xenserver storage.
The server has 6x 600gb sas HDD.
Could anyone please tell me what is the best layout to create the pool for block storage?
My initial tough is creating 1pool with 3 vdevs with 2 disk each

Any advise is welcome.

Thank you

 

[l@e]

That is the optimal for hight IO but not so good for storage amount.
It will have a total of 1.3-1.4 tb of pool (there is a good calculator around) and still you might want to have the pool not fully used, keeping a min 30-40% free. (@20 performace will go down)
If that is enough for your vm storage?
Do you have still empty bays there for future expansions?

 

[HoneyBadger]

Mirrors is definitely correct for block storage for VMs - I would argue "the only correct option, if you value performance."

Can you please post the rest of the system specifications (CPU, motherboard, memory, network connection type)? You will also want to consider the requirement for an SLOG device if you want to ensure data safety in case of crash/powerloss.

 

[Fred974]

@HoneyBadger Badger
I have the following Hardware
1x Dell PowerEdge C6220 II 4 x Node Ultra-Dense 2U Rack Server with 4 x Node servers installed and populated.

Each of the 4 x Nodes are configured as below:
2 x Intel E5-2650 Xeon Eight-Core 2.0GHz CPUs
64GB (8x8GB) DDR3 RAM
6 x 600GB SAS 10k 6Gbps Hard Drives installed
LSI SAS-9210-8i Controller
Embedded Intel i350 Network Controller (2 x 1Gb Ports, 1 x 100Mb Dedicated
Management Port)
Intel X520 Dual-Port SFP+ 10Gb Mezz Network Card

@I@e I have no empty bays left butr for future expansions, I have Dell PowerVault MD1200 that I can use later with the dell H800 in jbod

Mirrors is definitely correct for block storage for VMs - I would argue "the only correct option, if you value performance."

What would you advise for better performence?

Also If i create a mirror, will it still be using zfs?

 

[Fred974]

anyone?

 

[Matute]

Hi Fred, let me tell you some stuff out of my (no so big) experience:
- Avoid any kind of RAID from your server. Your controller should be a plain HBA. Don't know much about the one you mention but just in case I tell you that (which anyway is written all over the forum)
- AFAIK ZFS will be used regardless of the type of vdev, you will be using ZFS not RaidZ#. I think ZFS is the File System that handles the way blocks are written, snapshots, etc. And that is the same regardless of parity vs mirroring.
- For VMWare (I read you were using xen, but I have only experience with VMWare) it's very important to have a ZIL (or SLOG, it's the same) because of the way the Hypervisor writes to disk (it needs confirmation of data being physically written) and if you don't have a SLOG then it get's painfully slow. Now the SSD used as a SLOG is a full chapter on it's own. I use (recommended all over the forum) the premium Intel line, one that has a capacitor powered protection not to loose data in case of a power outage (also note that a power outage sometimes is not really a power outage but for example the need to do a hard reset on the server, so the typical answer of "I have a UPS" is not enough) The size doesn't matter much because of the way the SLOG works (read about it) and it's only related to amount of memory you have (if my memory doesn't fail)
- I also think that for better perfomance you would be better going the "mirror" way.

 

[Fred974]

@Matute
Thank you for the feedback. I don't use raid controller so I'm good on that point. I didn't know about need ssd for the ZIL so I need to research that.
What about raidz2? Looking at this blog entry https://calomel.org/zfs_raid_speed_capacity.html it seem to be the way forward

 

[Fred974]

I found this video along the way that could help someone else.

 

[Chris Moore]

I found this video along the way that could help someone else.

Don't. That is not a good video for anything because he had poor hardware for testing. Not enough disks and low powered processor. The results are not indicative of anything except his specific build performance.

 

[HoneyBadger]

I'm going to have to circle back later on this but my immediate thought is that the C6220 is not the ideal candidate for being a FreeNAS server - it's far better suited to be a dense compute solution, since there's no easy way to attach all 24 drives to a single node. And FreeNAS isn't robust enough to be used as a hypervisor with bhyve yet, at least not compared to the competition. (Sorry if I hurt anyone's feelings here. ;) )

You'll want to make sure you flash the IT firmware to those LSI cards, and I don't believe the H800 is a true HBA.

But as far as performance, your initial instinct of mirror vdevs is correct (3 vdevs of 2-way mirrors) which would yield about 1.8T of space.

 

[Matute]

I still feel that the best option for what you need is a mirror rather than a Z2, even more if performance during a resilver is something you need. Of course there are many many aspects to take into account:
- your need for space
- if the NAS can go offline or have a degraded performance in case of a resilver
- etc.
Regarding the ZIL/SLOG I did some tests and then some more, and using VMWare the difference was HUGE.
Try to read about SLOG devices in the forums and guides, there's a lot of info. My recommendation is not to use a standard SSD. Think about the SLOG as the place where data that is in memory waiting to be written is stored just in case. That's why it need not be too big. Please anyone correct me if I'm wrong.

 

[Fred974]

I'm going to have to circle back later on this but my immediate thought is that the C6220 is not the ideal candidate for being a FreeNAS server

Umm, I think I'm starting to come to the same conclusion.
I have another C2100 with 24 disk but they are old 146GB sas and the cost of having another serv in data center is not cheap either :(
Maybe I will try to achive HA in xenserver via haproxy master/slave on 2 node in xcp-ng

 

[Chris Moore]

Hi,

I have a dedicated server for FreeNAS that is going to be use as iSCSI taget for xenserver storage.
The server has 6x 600gb sas HDD.
Could anyone please tell me what is the best layout to create the pool for block storage?
My initial tough is creating 1pool with 3 vdevs with 2 disk each

Any advise is welcome.

Thank you

You are not wrong, but I think the hardware you have posted is NOT ideal for FreeNAS. FreeNAS is for mass storage, but what you posted is a system that is intended for high density compute. Do you have a storage server that you can put FreeNAS on?

 

[Chris Moore]

Umm, I think I'm starting to come to the same conclusion.
I have another C2100 with 24 disk but they are old 146GB sas and the cost of having another serv in data center is not cheap either :(
Maybe I will try to achive HA in xenserver via haproxy master/slave on 2 node in xcp-ng

How many virtual machines are you looking to run? What kind of performance are you looking to get out of this?

 

[Fred974]

Initially looking at 3x web server, 3x database cluster 2x haproxy 1xunifi controller, 1 mailserv so around 10-15 VM
I would like the best performance but I haven't got experience in it so I don't rely know how to quantify good/bad performance.
Is the Dell PowerEdge C2100 more suitable?

 

[Chris Moore]

Is the Dell PowerEdge C2100 more suitable?

To hold the storage, yes, but not for compute capacity to run 15 VMs and we have web servers and database servers in the mix. Those can be quite demanding. How much traffic do you expect? I am guessing this is something you are looking to do on the cheap as a startup?

 

[Chris Moore]

Initially looking at 3x web server, 3x database cluster 2x haproxy 1xunifi controller, 1 mailserv so around 10-15 VM
I would like the best performance but I haven't got experience in it so I don't rely know how to quantify good/bad performance.

@Johnnie Black do you have a moment to look at this?

 

[Fred974]

the C2100 will be dedicated to FreeNAS. The VM will be on the C6220 nodes and that where the computing occur.
Yes, we are trying to create a small hosting business. To start we will only have our own website in there as I am a web developer and we will host only our clients. Once we gain more experience we looking at reselling hosting/vps but this is far down the line.

Would you say that at this stage, I use the 4 nodes with local storage and forget HA + FreeNAS SR? and go the classic way with HAproxmaster on 1 node and slave on another. same for webserver? 1 on each node

 

[HoneyBadger]

Umm, I think I'm starting to come to the same conclusion.
I have another C2100 with 24 disk but they are old 146GB sas and the cost of having another serv in data center is not cheap either :(
Maybe I will try to achive HA in xenserver via haproxy master/slave on 2 node in xcp-ng

A C2100 is pretty much an ideal option (although it doesn't have as many PCIe slots as the R510) - I would strongly suggest that the C2100 be your FreeNAS server (swap the disks so that you're using the 600G models of course) and the C6220 be diskless compute nodes; although that means paying for an additional 2U in the datacenter.