Register for the iXsystems Community to get an ad-free experience and exclusive discounts in our eBay Store.

NVDIMMS and TrueNAS 12

Joined
Apr 14, 2020
Messages
7
So I saw on the feature list that NVDIMMs willl be support in TrueNas 12 and I was wondering if anyone had played around with them yet. I have a Gigabyte MH71-HB0 and it supports 4 Optane DC Memory DIMMs and I am really interested in experimenting with these, especially now that the price for them has become easily obtainable. In fact the price for a 256GB module ($300) is almost competitive with SSD drives. I was wondering if it would we possible to use these as our ZIL device. Instead of buying a an expensive PCIe Optane Drive, we just slap a couple of these 256GB modules in there in AP mode and boom 512GB ZIL device. Would this work in TrueNAS? On a side note, is TrueNAS coded to take advantage of the speed of the modules? I have read a lot about them lately and with the falling prices I would really like to pick a couple up. Thank you for the help guys.
 

sretalla

Dedicated Sage
Joined
Jan 1, 2016
Messages
2,469
I was wondering if it would we possible to use these as our ZIL device. Instead of buying a an expensive PCIe Optane Drive, we just slap a couple of these 256GB modules in there in AP mode and boom 512GB ZIL device
You would want to be fairly sure you're battery backed for those as the amount of data written in just a few seconds could be a lot to lose.

Would this work in TrueNAS?
Maybe...
 

HoneyBadger

Mushroom! Mushroom!
Joined
Feb 6, 2014
Messages
2,463
You would want to be fairly sure you're battery backed for those
It's Optane DC - 3D XPoint on a DIMM interface. No battery backing is needed there.

In fact the price for a 256GB module ($300) is almost competitive with SSD drives. I was wondering if it would we possible to use these as our ZIL device. Instead of buying a an expensive PCIe Optane Drive, we just slap a couple of these 256GB modules in there in AP mode and boom 512GB ZIL device.
Definitely cheaper than the PCIe devices but I believe it requires special drivers for the Optane DC modules; I know Intel was working towards implementing support for persistent memory in FreeBSD but I don't know if it was completed yet.
 
Joined
Apr 14, 2020
Messages
7
Well I noticed on the feature list that it specifically stated support for nvdimms, now where or not the includes the opt and modules is anyone’s guess. Could someone on the dev team weigh in here?
 
Joined
Apr 14, 2020
Messages
7
And yes just to reiterate in ap mode the Optane modules are persistent just like their ssd cousins, however in memory mode they are not persistent, hence the rebranding.
 

danb35

Wizened Sage
Joined
Aug 16, 2011
Messages
11,318
we just slap a couple of these 256GB modules in there in AP mode and boom 512GB ZIL device.
...roughly 480GB of which would be wasted. SLOG isn't a write cache and doesn't work like one.
 
Joined
Apr 14, 2020
Messages
7
...roughly 480GB of which would be wasted. SLOG isn't a write cache and doesn't work like one.
This statement confuses me. Aren’t optane drives pretty much the default standard for a zil/slog device? If we had access to the same technology but on the memory bus, wouldn’t that make it faster? I know I currently have a 905 optane as my zil and while I don’t have that large of a pool at the moment I plan to have upwards of 100 tb eventually.
 

danb35

Wizened Sage
Joined
Aug 16, 2011
Messages
11,318
This statement confuses me.
...because you don't understand how a SLOG device is used. At any given time, it will have no more than two transaction groups' (each of which would be five seconds) worth of data. See:
 

Ericloewe

Not-very-passive-but-aggressive
Moderator
Joined
Feb 15, 2014
Messages
16,259
I think the question is more along the lines of "what's the smallest suitable SSD?" - say the P3700, for instance, that thing starts at 240 GB if I'm not mistaken. If you're really looking for speed, way oversized SSD capacities are probably inevitable.
 

HoneyBadger

Mushroom! Mushroom!
Joined
Feb 6, 2014
Messages
2,463
...because you don't understand how a SLOG device is used. At any given time, it will have no more than two transaction groups' (each of which would be five seconds) worth of data. See:
https://www.ixsystems.com/community/threads/proper-sizing-of-slog-and-transaction-group.35470/
https://www.ixsystems.com/blog/why-zil-size-matters-or-doesnt/
Unfortunately both of those links are 5 years old and refer to the "legacy write throttle" - the short version now is that with the default tunables, you'll never use more than 4G of a given SLOG device. Like most things in ZFS this can be adjusted, but you'll consume an equal amount of RAM to whatever you allow your "maximum SLOG usage" to be, which robs you of some ARC space. Obviously it's up to you to find the balance depending on your read/write usage, the "burstiness" of the workload, and how much RAM is installed.

I think the question is more along the lines of "what's the smallest suitable SSD?" - say the P3700, for instance, that thing starts at 240 GB if I'm not mistaken. If you're really looking for speed, way oversized SSD capacities are probably inevitable.
400G for that one specifically, but it's kind of moot as pointed out - to get the kind of speed and responsiveness you want out of an SLOG, you either end up with a grossly oversized SSD or a specialised (and usually extremely expensive) device. Oversized devices do help with wear-levelling to increase endurance though.
 
Top