Register for the iXsystems Community to get an ad-free experience and exclusive discounts in our eBay Store.

PCI-e to NVMe adapter cards

ChrisReeve

Member
Joined
Feb 21, 2019
Messages
89
Good morning

I am considering upgrading my server with SSD cache, with the ultimate goal of being able to saturate my 10GbE connection. I dont strictly need those kind of speeds, but, I'd like to try.

HW:
MB: Supermicro X9SRL-F
CPU: E5-2650 v2 (8C/16T)
RAM: 64GB DDR3 1600MHz ECC (4x16GB DIMMs)
Boot drive: Sandisk 16GB USB2
Main pool: 10x10TB ZFS2

Right now, my performance is fairly good. Transfering hundreds of GBs see sustained writes surpassing 300MB/s. I havent really tested read speeds properly, but pulling random files not accessed in weeks, which I doubt are cached in RAM, gives me 500 MB/s if i remember correctly. Again, not bad, and I dont really require much more than this.

Still, I want to explore different options for caching. Some initial research have given me three options:

1. Upgrade RAM. I can add an additional 64GB (128GB total) for about 100$. Would be a good option, but transfering up to 600-700GB at a time from an NVMe drive on my desktop, will quickly fill up 128GB. I dont believe this to be the best option, even though increasing RAM often is the best first step to increase performance.

2. Upgrade with 2 SATA SSDs in RAID0. 2x500GB of the right type would give theoretical sustained sequential writes of around 1GB/s, until SLC cache has been used up. This seems like a fairly good option for me. Price would be about 120-150$ depending on the type of SSD i buy.

3. NVMe. My motherboard doesn't support NVMe, so this would have to be done via a PCI-E SSD, or a PCI-E to NVMe adapter, paired with an NVMe drive. But I'm not able to find any compatibility lists (if any exists). Does anyone know if there are any specific PCI-E to NVMe adapters that have been successfully used in freeNAS? If not, what about PCI-e cards in the price range between 100-150$?
 

sretalla

Dedicated Sage
Joined
Jan 1, 2016
Messages
2,473
Does anyone know if there are any specific PCI-E to NVMe adapters that have been successfully used in freeNAS? If not, what about PCI-e cards in the price range between 100-150$?
As I understand it, they pretty much all work, but it's the bifurcation/port splitting which is problematic and relies on your motherboard/BIOS to do the right thing for additional NVME drives to be seen in the one slot.

I have this one:

But my MoBo doesn't support bifurcation, so it uses an x16 slot to have only the first NVME slot of 4 working (which does work just fine).

They have plenty of single M.2 cards which I'm sure work fine too.
 

ChrisReeve

Member
Joined
Feb 21, 2019
Messages
89
I am considering the following adapter card: http://www.akasa.com.tw/search.php?seed=AK-PCCM2P-01

And pair it with the following SSD: https://www.tomshardware.com/reviews/adata-xpg-sx8200-pro-ssd,5955.html

Total price would be about 180$. If it doesn't work, I'll just use the SSD in my desktop instead.

Also, this is a completely different question. I am considering virtualizing freeNAS, as I have other uses for my server. I will use ESXi and HW Passthrough both my HBAs.

If I end up virtualizing freenas through ESXi. Could I either HW Passthrough this NVMe/PCIe adapter directly to freeNAS, or would I have to create a virtual drive, and then use the virtual drive/partition to use as SSD Cache?
 

sretalla

Dedicated Sage
Joined
Jan 1, 2016
Messages
2,473
I don't see anything that would indicate it won't work.

Because it's a Pci-E card, you will probably be able to use passthrough as long as you don't boot ESXi from it.
 

ChrisReeve

Member
Joined
Feb 21, 2019
Messages
89
Thank you. I guess I will find out. I am planning on booting from a Samsung 850 Evo that I have laying around.

Edit: Those PCIe to NVMe cards. Do they have any kind of logic, or is it basically an adapter?
 

Johnnie Black

Neophyte Sage
Joined
May 10, 2017
Messages
798
It's just and adapter, transparent to the OS.
 
Top