SLOG or L2ARC first for iSCSI and which SSD

What the single SSD should be used for to speed up my iSCSI service ?

  • Use it as an SLOG

    Votes: 0 0.0%
  • Use it as a L2ARC

    Votes: 0 0.0%

  • Total voters
    0
  • Poll closed .

Heracles

Wizard
Joined
Feb 2, 2018
Messages
1,401
Hi,

Recently, I bought a used server (R820 ; 32 Core ; 256G RAM) as an upgrade for my ESXi server. Thanks to that, I moved my main FreeNAS from its T-130 to the T-330 I freed. The 8 drives are paired in 4 mirrors and a single pool. The server is maxed out on RAM (64G) and there is only a last PCI slot in which I can put a PCIe SSD card. More info in my sig.

The server is used for storage for my private cloud service (NFS storage presented to the web frontend) as well as for iSCSI for the ESXi. iSCSI is configured with jumbo frames over 4 dedicated 1G Ethernet ports. They are all connected peer-to-peer, no switch. Round-Robin is also in place after 1 request.

The cloud does not need any significant performance and was already more than happy with the previous 5 HDD RaidZ-2 and 16G of RAM I had. The question is more about the performance for iSCSI and the ESXi. These two can never have enough... Not that I have any specific problem as of now; I am just looking on how to achieve the most and highest performance with what I have.

From what I found in the forums, the ideal would be to add 2 SSDs, one as SLOG and one as L2ARC. Because I can fit only 1 card, I am wondering which of these two is the most important to provide first. My understanding is that the SLOG should go in first.

As of now, I have about 12 VMs. I intend to grow that to about 20 soon enough and maybe more. The R820 has its own local storage as Raid-10, also with 8 drives. FreeNAS does not need to do it all. Also, because the iSCSI link is over 4x 1G, no need to go above and beyond that limit. I have no plan getting 10G Ethernet anytime soon.

Any advice on the model of PCIe SSD I should go for ? Because there will be a single one, should it be SLOG or L2ARC ?

Budget is up to 500$ but is elastic...

Thanks in advance for sharing your own experience and knowledge,
 
Joined
Oct 18, 2018
Messages
969
The server is maxed out on RAM (64G) and there is only a last PCI slot in which I can put a PCIe SSD card.
Perhaps I missed it but what number of lanes is the slot that is available? And does your board support port bifurcation? If you have an 8x slot and it supports bifurcation you could get an adapter and put two drives in. If the slot is an x16 and support x4x4x4x4 you're in an even better position potentially.

From what I found in the forums, the ideal would be to add 2 SSDs, one as SLOG and one as L2ARC.
Some might argue ideally you'd add 3, 2 for SLOG to mirror them and 1 for L2ARC. :) I suppose that depends on your system, budget, etc.
 

Heracles

Wizard
Joined
Feb 2, 2018
Messages
1,401
Hey PhiloEpisteme,

Perhaps I missed it but what number of lanes is the slot that is available? And does your board support port bifurcation?

You clearly not missed it : I did not mentioned a word about it because I completely forgot about that option. For sure I will investigate that. The slot is indeed twice too large : 8 when the card needs 4. Not sure for the rest but will find out for sure.

Thanks for the tip,
 
Joined
Oct 18, 2018
Messages
969
For sure I will investigate that. The slot is indeed twice too large : 8 when the card needs 4. Not sure for the rest but will find out for sure.
Ah, this could be a great thing for you! If your board supports port bifurcation on that slot (or if you can rearrange cards) you can pick up a Supermicro AOC-SLG3-2M2 and then you can fit 2 M.2 cards in the 1 x8 slot benefitting from the full x4 speed! Or course, it does limit you to the M.2 form factor.
 

HoneyBadger

actually does care
Administrator
Moderator
iXsystems
Joined
Feb 6, 2014
Messages
5,110
Ah, this could be a great thing for you! If your board supports port bifurcation on that slot (or if you can rearrange cards) you can pick up a Supermicro AOC-SLG3-2M2 and then you can fit 2 M.2 cards in the 1 x8 slot benefitting from the full x4 speed! Or course, it does limit you to the M.2 form factor.

Bear in mind that the Optane P4801X comes in M.2 (22110) form factor, so it's isn't terribly limiting that way, since the Supermicro card supports the "extra-long" 110mm units. That's likely your ideal SLOG in M.2 format.

If bifurcation isn't an option, then the single SSD you likely need for this is an SLOG (since hosting VMs safely requires sync writes) which would again likely be best served by an Optane card. You could use one of the 900p or 905p cards though (with the understanding that Intel won't warranty them in a "server environment") for a significant cost savings vs. the P4800X/P4801X, or pick up a used P3700.
 
Joined
Oct 18, 2018
Messages
969
Bear in mind that the Optane P4801X comes in M.2 (22110) form factor, so it's isn't terribly limiting that way, since the Supermicro card supports the "extra-long" 110mm units. That's likely your ideal SLOG in M.2 format.
This is the exact drive I plan to use in my system and why I purchased the card in the first place so I would 100% agree with @HoneyBadger here.
 

Heracles

Wizard
Joined
Feb 2, 2018
Messages
1,401
Thanks a lot for the info. I did a first search and unfortunately, it does not look good. Nowhere in the owner manual or official documentation do I find a single thing around the word "bifurcation". I will keep searching if this is referred to under a different name, but no luck so far.

I will keep you informed...
 
Joined
Oct 18, 2018
Messages
969
Thanks a lot for the info. I did a first search and unfortunately, it does not look good. Nowhere in the owner manual or official documentation do I find a single thing around the word "bifurcation". I will keep searching if this is referred to under a different name, but no luck so far.
My board didn't have anything in the documentation either but I checked with the manufacturer and in fact it was supported, buried in a setting in the bios.
 

Heracles

Wizard
Joined
Feb 2, 2018
Messages
1,401
Hi again,

So after lot of research and reviewing every bios and firmware options many times, I did not found a single sign that my PCI would do bifurcation... It looks like it is not supported in my Dell T-330...
 
Top