Storage hardware for my boot pool on Dell r730xd

[jgreco]

Hmm, yeah, the Dells may have some extra weirdness going on. I'm your experience, did sas2flash just refuse to work or did the cards not work after flashing IR without jumping through hoops?

I believe it refused to flash IR P16 on top of IT P16 firmware, referring to the card as a different product or something along those lines.

 

[CheeryFlame]

Do I need DRAM on the drives or it doesn't matter?

I would buy 2x of these cheap Silicon Power SATA for the Raid-1 and 1x Samsung PCIE for the passthrough in case of a failure on the cheap drives.

This is the only way I found to save money since I'm on a budget and keeping some kind of redundancy.

Any better recommendation?

 

[Ericloewe]

Do I need DRAM on the drives or it doesn't matter?

For the boot disks? Doesn't matter a bit. The important part is getting something reputable and reasonably reliable. I say "reasonably" because failures modes that manifest after intensive use are unlikely to be triggered.

 

[CheeryFlame]

For the boot disks? Doesn't matter a bit. The important part is getting something reputable and reasonably reliable. I say "reasonably" because failures modes that manifest after intensive use are unlikely to be triggered.

Would you say that cutting on the quality for the Raid-1 drive isn't a bad idea to save on the budget since I have a reliable Samsung as a passthrough?

 

[jgreco]

Do I need DRAM on the drives or it doesn't matter?

For boot drives, only a little bit. Drives without DRAM tend to suffer more wear, and because they're designed as low cost drives, those two factors work against you a bit. You can reduce the write load (and therefore the wear) on these if you move the system dataset to your main pool.

 

[CheeryFlame]

For boot drives, only a little bit. Drives without DRAM tend to suffer more wear, and because they're designed as low cost drives, those two factors work against you a bit. You can reduce the write load (and therefore the wear) on these if you move the system dataset to your main pool.

I think I'll go for theses cheap drives and once they break I'll update them with something better. In the meantime it'll be less rough budget-wise!

Thank you all for your replies!

 

[CheeryFlame]

I've received the card, the M.2 and the breakout cable. I'm still waiting for the IR HBA that's taking longer since it's from China.

I'm afraid I haven't understood correctly the setup described in the resource page for this setup since there's no SATA connection as expected as you can see on the second picture. Although this makes sense since in the product description it was saying that the 3rd one is a PCIe M.2 only whereas the first 2x are SATA M.2 only.

I know @jgreco got the other brand card and haven't tested the Startech one, so this one might not work.

OR, I don't understand how the HBA will pick the passthrough NVME.

Thank you for helping!

 

[jgreco]

there's no SATA connection as expected as you can see on the second picture

The 980 is connected to the PCIe connector and uses NVMe. The two front cards are SATA and are merely powered by the PCIe. You will use the breakout cable to connect the two SATA connections on the card to the IR HBA. This is one of those weird freaky servery things where you are not connecting cables to the mainboard. Once you connect the IR HBA to the two SATA M.2's, you make a RAID1 out of them and that gives you a reliable boot device. The 980 is (I'm presuming you're using it for) ZFS redundancy, since ZFS cannot get a failed block on SATA#1 by asking for it from SATA#2; the IR RAID precludes that option. Instead you get it from the 980 and then just write it to the IR RAID, which writes it to both SATA#1 and SATA#2. This gives you both ZFS checksumming/redundancy and also reliable (RAID1) boot.

I know @jgreco got the other brand card and haven't tested the Startech one, so this one might not work.

I assure you they came off the same manufacturing line. Absent some manufacturing defects, they should be identical.

 

[CheeryFlame]

The 980 is (I'm presuming you're using it for) ZFS redundancy, since ZFS cannot get a failed block on SATA#1 by asking for it from SATA#2; the IR RAID precludes that option. Instead you get it from the 980 and then just write it to the IR RAID, which writes it to both SATA#1 and SATA#2. This gives you both ZFS checksumming/redundancy and also reliable (RAID1) boot.

Thank you for your reply! And how do I setup the 980 for ZFS redundancy?

 

[jgreco]

Thank you for your reply! And how do I setup the 980 for ZFS redundancy?

When you create the boot pool, you will want to make it on the device that shows up as an LSI logical volume.

Code:

root@nas0[~]# camcontrol devlist
<ATA WDC WD120EMFZ-11 0A81>        at scbus0 target 10 lun 0 (pass0,da0)
<ATA WDC WD120EDAZ-11 0A81>        at scbus0 target 11 lun 0 (pass1,da1)
<ATA WDC WD120EDAZ-11 0A81>        at scbus0 target 12 lun 0 (pass2,da2)
<LSI Logical Volume 3000>          at scbus1 target 0 lun 0 (pass3,da3)
root@nas0[~]#

So it should say something like "LSI Logical Volume 3000" in the device list when you choose the install target. Make sure you set that up as a RAID1 volume in the LSI firmware BIOS, and make sure it is the preferred boot device for your system. It is supermegaimportant to bear in mind that your TrueNAS system sees this as a single SSD device.

Once installed, you will go in and follow the guidance in the user manual to attach the 980 as a mirror.

Mirroring the Boot Pool

Provides information on how to mirror the boot pool on TrueNAS CORE.

www.truenas.com

 

[Ericloewe]

Once installed, you will go in and follow the guidance in the user manual to attach the 980 as a mirror.

It's probably a better option to install as a mirror from the start. Just select both the virtual volume and the NVMe SSD when choosing the device to install to.

 

[jgreco]

It's probably a better option to install as a mirror from the start. Just select both the virtual volume and the NVMe SSD when choosing the device to install to.

Can you do that now? It didn't used to let you.

 

[Ericloewe]

Can you do that now? It didn't used to let you.

Always has been an option, since 9.3. Just need to mark two devices instead of one, which I guess is the reason for the installer to insist that you mark an entry rather than merely selecting it and continuing with Enter.

 

[CheeryFlame]

Awesome, this seems really straightforward. Thank you and will post my results once the card gets in!

 

[CheeryFlame]

I either got the wrong breakout cable or the wrong LSI card. Since getting the card took long enough, anyone could tell me if that one will fit?

https://www.amazon.ca/gp/product/B010CMVVD0/ref=ox_sc_act_title_1?smid=A2SBTRGJKT6RTQ

 

[Ericloewe]

Your cable is SFF-8087, the connectors on the board are SFF-8643. Broadly equivalent, but a different form factor.

 

[NickF]

Cant speak to this seller, but this is what you need

Mini SAS HD Breakout SFF-8643 to 4X SATA Right Angle Server Raid Cable 0.5/1M | eBay

4x7Pin SATA is the target, connect to the hard drives with SATA port. If the Mini SAS (SFF-8643) on your backplane, this cable will not work with them. Side 1: Mini SAS (SFF-8643) Male with Latch. ▶Serial Attached SCSI (SAS) is a high-speed data storage interface designed for high-throughput and...

www.ebay.com

 

[CheeryFlame]

Just installed both PCIe cards and I haven't tried yet to access the HBA but I would like to reduce the fan noise that is totally unbearable since I installed this card inside. Is this a normal behaviour? Honestly I can't justify this noise since it's in a closet in my bedroom. the noise before was okay but now it's too much.

 

[Davvo]

Take a look at this resource.

How To: Change IPMI Sensor Thresholds using ipmitool

Many server motherboards, including the Supermicro X10 series, have their fans monitored by the BMC, which automatically attempts to recover stalling (of the motor, not in the aerodynamic sense) fans by spinning them up to maximum (PWM set to...

www.truenas.com

 

[Ericloewe]

I'm not sure that will help - some Dell-specific hacking may be necessary due to Dell doing Dell stuff and ramping up the fans in the presence of certain PCIe devices it does not get a warm, fuzzy feeling from. So it gives them the cold shoulder.