Register for the iXsystems Community to get an ad-free experience and exclusive discounts in our eBay Store.

PCI-e to NVMe adapter cards

ChrisReeve

Newbie
Joined
Feb 21, 2019
Messages
64
Good morning

I am considering upgrading my server with SSD cache, with the ultimate goal of being able to saturate my 10GbE connection. I don't strictly need those kind of speeds, but, I'd like to try.

HW:
MB: Supermicro X9SRL-F
CPU: E5-2650 v2 (8C/16T)
RAM: 64GB DDR3 1600MHz ECC (4x16GB DIMMs)
Boot drive: Sandisk 16GB USB2
Main pool: 10x10TB ZFS2

Right now, my performance is fairly good. Transfering hundreds of GBs see sustained writes surpassing 300MB/s. I havent really tested read speeds properly, but pulling random files not accessed in weeks, which I doubt are cached in RAM, gives me 500 MB/s if i remember correctly. Again, not bad, and I don't really require much more than this.

Still, I want to explore different options for caching. Some initial research have given me three options:

1. Upgrade RAM. I can add an additional 64GB (128GB total) for about 100$. Would be a good option, but transfering up to 600-700GB at a time from an NVMe drive on my desktop, will quickly fill up 128GB. I don't believe this to be the best option, even though increasing RAM often is the best first step to increase performance.

2. Upgrade with 2 SATA SSDs in RAID0. 2x500GB of the right type would give theoretical sustained sequential writes of around 1GB/s, until SLC cache has been used up. This seems like a fairly good option for me. Price would be about 120-150$ depending on the type of SSD i buy.

3. NVMe. My motherboard doesn't support NVMe, so this would have to be done via a PCI-E SSD, or a PCI-E to NVMe adapter, paired with an NVMe drive. But I'm not able to find any compatibility lists (if any exists). Does anyone know if there are any specific PCI-E to NVMe adapters that have been successfully used in freeNAS? If not, what about PCI-e cards in the price range between 100-150$?
 

sretalla

FreeNAS Expert
Joined
Jan 1, 2016
Messages
1,457
Does anyone know if there are any specific PCI-E to NVMe adapters that have been successfully used in freeNAS? If not, what about PCI-e cards in the price range between 100-150$?
As I understand it, they pretty much all work, but it's the bifurcation/port splitting which is problematic and relies on your motherboard/BIOS to do the right thing for additional NVME drives to be seen in the one slot.

I have this one:

But my MoBo doesn't support bifurcation, so it uses an x16 slot to have only the first NVME slot of 4 working (which does work just fine).

They have plenty of single M.2 cards which I'm sure work fine too.
 

ChrisReeve

Newbie
Joined
Feb 21, 2019
Messages
64
I am considering the following adapter card: http://www.akasa.com.tw/search.php?seed=AK-PCCM2P-01

And pair it with the following SSD: https://www.tomshardware.com/reviews/adata-xpg-sx8200-pro-ssd,5955.html

Total price would be about 180$. If it doesn't work, I'll just use the SSD in my desktop instead.

Also, this is a completely different question. I am considering virtualizing freeNAS, as I have other uses for my server. I will use ESXi and HW Passthrough both my HBAs.

If I end up virtualizing freenas through ESXi. Could I either HW Passthrough this NVMe/PCIe adapter directly to freeNAS, or would I have to create a virtual drive, and then use the virtual drive/partition to use as SSD Cache?
 

sretalla

FreeNAS Expert
Joined
Jan 1, 2016
Messages
1,457
I don't see anything that would indicate it won't work.

Because it's a Pci-E card, you will probably be able to use passthrough as long as you don't boot ESXi from it.
 

ChrisReeve

Newbie
Joined
Feb 21, 2019
Messages
64
Thank you. I guess I will find out. I am planning on booting from a Samsung 850 Evo that I have laying around.

Edit: Those PCIe to NVMe cards. Do they have any kind of logic, or is it basically an adapter?
 
Top