Not entirley sure of how best to configure 16 drive system

[mintra]

Hi I have just built a 15 drive system as follows:

i7 920 18Gb RAM 2x IBM M1015 (Will IT firmware) in the end I decided on 16 3Tb 7200rpm Seagate drives, for performance.

The user I am building it for has a couple of Equalogics and a load of Nexentas (which I am phasing out for licence reasons).

He has said he want two drives as hot spares (so that leaves 14) and the rest in one array. Noting a comment about not having one vdev that means I would be best to group this as two vdevs.

So I have read as much documentation as possible and I am not entirely clear what is the best way to go. He will be using this for a lot of virtual Esxi machines viewed as iSCSI from esxi servers.

I could set up two RaidZ3 groups of 7 (is this two vdevs) and then somehow join them together, seems a bit awkward.

Also they are software developers logging lots of packets of small data, so I am not sure this is optimal performance. My feeling is that we should do two different configurations on the system one a group of drives configured for performance and another group for speed. Then they could chose where to put the vm based on which was best.

Thanks in advance.

 

[cyberjock]

First, if he plans to go with ESXi datastores, you are talking LOTS more RAM than 18GB.

Second, ZFS should be used with ECC. Your 920 doesn't support ECC.

Third, you are possibly talking about l2arc and possibly a ZIL too. Your l2arc must be "right sized" for your system, so be sure you do all your homework.

Fourth, you are going to potentially have issues with just 2 vdevs. Normally the recommended config is many mirror vdevs for adequate performance with ESXi.

Nothing personal, but it sounds like he's paying you for this. You're going to need to do a lot more homework and earn that paycheck. ;)

 

[mintra]

I have seen the comments on ECC RAM, but this is a board they already had, have suggested more RAM to them but we can only go to 24Gb. Thing is we already have about seven servers
with non ecc ram and Nexenta and have not had too many issues. I appreciate this has more drives. This is only in a testing stage I do actually see this posting as doing my homework and will pass on your comments, about RAM.

I will look at the l2arc and zil as my next study item. Thanks for advice.

 

[cyberjock]

I will tell you that an l2arc isn't going to be particularly useful with so little RAM. Usually we recommend people have 64GB of RAM before the l2arc option should be considered. In the manual it says you shouldn't add an l2arc until you have maxed out the motherboard's RAM. The reason being there's a relationship between the arc and l2arc, and you can't cheat the system and expect to win. If you don't have enough the l2arc will actually hurt performance.

As for the ECC vs non-ECC, you are the perfect example of the problem with this argument. It's one of those things that works until it doesn't. And the day that the non-ECC RAM has problems you can kiss the pool goodbye. That's why I spent something like 8 hours writing up that sticky in the forum. It's one of the ways that you can trash your pool without warning and without a chance for recovery(except from backup, but again, that's not likely). We used to see weekly losses with pools related to RAM alone. Not one user recovered any data. :(

 

[mintra]

Hi

I have read your power point presentation, I do have the Intel quad nic in, interestingly all the other machines are Supermicros boards which may be able to support EEC RAM so what I will be likely to suggest is that we use this machine as a backup area for all the other servers (I dont think they backup their development machines at present). Then go through them one by one upgrading the RAM to EEC and move them from Nexenta to FreeNAS at the same time for the prodcution machines.

The problem I had was I was blind to the add extra device button, now I see it I can see how you knit together vdevs into zpools.

So taking your advice I should create one zpool made up of 7 mirror vdevs. This would allow for failure of more than one drive but not when they were on the same mirror, in which case everything would go.

Thanks for your help, the support from this forum seems very good.

 

[gpsguy]

You may want to consider talking with iXsystems about your project. They are the FreeNAS developers and also have a commercial offering - TrueNAS.


Sent from my phone

 

[mintra]

OK changed motherboard for supermicro one RAM for ECC and CPU to Xeon, luckily all the nexenta boxes we may move to free nas have supermicro boards. Thanks for the help.