Best practices when dealing with JBOD disk shelves?

[HarambeLives]

I've searched the forums but I don't see much related to disk shelves/JBOD's, and I'm wondering if someone can give me some insight

I am decommissioning my NAS02 virtual TrueNAS install because the ESXi host it is on will be going away, and instead of building a second NAS again, I have moved my backups elsewhere so I can just connect all these drives to my main NAS01 and eliminate the need for an extra system

So, I have an LSI 9207-8e that I will add to NAS01, and then I am going to turn the 12 bay chassis that is currently the ESXi host into a JBOD. For the life of me I can't find one of the Supermicro JBOD controllers that are not shipped from China, so I'm just going to hotwire the 24 Pin connector so its on all the time. The power buttons won't be able to function, but that's fine as it will be on 24/7 anyway

* Is it okay to hot-plug the SAS connector with TrueNAS? In past systems its always been fine to plug in the external SAS connector with both the disk shelf powered on as well as the JBOD. Still the case with TrueNAS, or should I start with the JBOD powered off?

* If I need to power down the JBOD for some reason, what is the procedure in TrueNAS? How do I tell it to expect that 12 disks in a second pool will go offline?

 

[HoneyBadger]

SAS hot-plug is supported just fine in TrueNAS. But since you'll be powering down to install your 9207-8e, you could just wait until that point to install the HBA and then power on everything - but the recommended order would be to have the JBOD powered on first before the server, so that you can be confident that all drives are up and spinning.

For a power-down scenario, I'd recommend exporting the pool from the GUI first before powering off. While ZFS is designed to survive sudden interruptions such as power-outages, if you have the option of a controlled shutdown, that's definitely preferred.

 

[HarambeLives]

Sounds good, thanks!

 

[ZiggyGT]

Hot wire the supply in the 12 drive JBOD chassis sound like a good solution, Here are a couple of toys I used to power a JBOD chassis. Both chassis turn on with the power button on the main chassis. This stuff is far more cost effective than the Supermicro JBOD board.

Computer Case Miner Fan Speed Controller 8-Channel Fan Hub PWM High-Power Violent Fan Controller Fan Speed Governor
This device runs 8 fans based on a potentiometer. I plugged all the fans into that controller and set the fan speed for all with the potentiometer. Obviously this is not a professional solution but easy to hookup and may result in a quieter solution in the JBOD. These crazy miner solutions have to be good for something. Not a solution for everyone but works well for me.

From <https://www.aliexpress.com/item/325...416611490922583561e1a5e!12000027402251722!rec
https://www.aliexpress.com/item/325...416611490922583561e1a5e!12000027402251722!rec

I then connected a molex drive connector from the main chassis to the JBOD chassis to the second board the JBOD power supply plugs into that board. WHen the main chassis is turned on the relay is energized and the power on button is asserted and the JBOD power will come on.

Tishric Dual Psu Pc Block Synchronizer Power Supply Adapter Add2psu Atx 24pin To 4pin Molex Synchronous Riser Card Mining Btc - Pc Hardware Cables & Adapters - AliExpress

 

[HarambeLives]

That's a cool product, I didn't know they existed. I'll have to look into those

Turned the chassis into a JBOD today, plugged it in and all the disks showed up no problem, imported the pool without issue

 

[jgreco]

I've searched the forums but I don't see much related to disk shelves/JBOD's, and I'm wondering if someone can give me some insight

Feel free to make sure you're familiar with the stuff in the SAS Primer.

Don't be afraid to be SAS-sy ... a primer on basic SAS and SATA

With the introduction of SAS 12Gbps, seems like "it's time" to do a braindump on SAS. Work in progress, as usual. History By the late '90's, SCSI and PATA were the dominant technologies to attach disks. Both were parallel bus multiple drop topologies and this kind of sucked. SATA and Serial...

www.truenas.com

 

[ZiggyGT]

Hot wire the supply in the 12 drive JBOD chassis sound like a good solution, Here are a couple of toys I used to power a JBOD chassis. Both chassis turn on with the power button on the main chassis. This stuff is far more cost effective than the Supermicro JBOD board.

Computer Case Miner Fan Speed Controller 8-Channel Fan Hub PWM High-Power Violent Fan Controller Fan Speed Governor
This device runs 8 fans based on a potentiometer. I plugged all the fans into that controller and set the fan speed for all with the potentiometer. Obviously this is not a professional solution but easy to hookup and may result in a quieter solution in the JBOD. These crazy miner solutions have to be good for something. Not a solution for everyone but works well for me.

From <https://www.aliexpress.com/item/3256803492609715.html?spm=a2g0o.detail.1000014.1.283f7001JzHa9w&gps-id=pcDetailBottomMoreOtherSeller&scm=1007.40050.281175.0&scm_id=1007.40050.281175.0&scm-url=1007.40050.281175.0&pvid=4cc7f84b-9982-4f24-8998-e22a4dfb27d9&_t=gps-id:pcDetailBottomMoreOtherSeller,scm-url:1007.40050.281175.0,pvid:4cc7f84b-9982-4f24-8998-e22a4dfb27d9,tpp_buckets:668#2846#8115#2000&pdp_ext_f={"sku_id":"12000027402251722","sceneId":"30050"}&pdp_npi=2@dis!USD!4.61!2.95!!!!!@2101f6b416611490922583561e1a5e!12000027402251722!rec
https://www.aliexpress.com/item/3256803492609715.html?spm=a2g0o.detail.1000014.1.283f7001JzHa9w&gps-id=pcDetailBottomMoreOtherSeller&scm=1007.40050.281175.0&scm_id=1007.40050.281175.0&scm-url=1007.40050.281175.0&pvid=4cc7f84b-9982-4f24-8998-e22a4dfb27d9&_t=gps-id:pcDetailBottomMoreOtherSeller,scm-url:1007.40050.281175.0,pvid:4cc7f84b-9982-4f24-8998-e22a4dfb27d9,tpp_buckets:668#2846#8115#2000&pdp_ext_f={"sku_id":"12000027402251722","sceneId":"30050"}&pdp_npi=2@dis!USD!4.61!2.95!!!!!@2101f6b416611490922583561e1a5e!12000027402251722!rec
View attachment 63112
I then connected a molex drive connector from the main chassis to the JBOD chassis to the second board the JBOD power supply plugs into that board. WHen the main chassis is turned on the relay is energized and the power on button is asserted and the JBOD power will come on.
Tishric Dual Psu Pc Block Synchronizer Power Supply Adapter Add2psu Atx 24pin To 4pin Molex Synchronous Riser Card Mining Btc - Pc Hardware Cables & Adapters - AliExpress
View attachment 63114

I found the relay will not trigger on 5V. I will use custom 12V cable between the main chassis and the JBOD. Probably use a drive molex with 2 wires of some odd color.

I purchased a couple of these bulkhead connectors to use an external cable between chassis. They are $20 each and the cables another $25. Cheaper than a busted HBA card or backplane. They were not on my original plan because I did not know they existed. In the main chassis I will have both mini-SAS ports cabled to it. One for the JBOD and the other for an external backup device. (easy to turn off independently).

 

[HarambeLives]

The 10GTek Branded one I got on Amazon for $27 has the low profile bracket produced backwards, pretty annoying! I had to sandwich it in there to get a screw through the hole

 

[ZiggyGT]

Dang it. The ones I ordered but not recieved yet are also wrong. Hope they got the pinout right. I should have looked harder. Evidence was there. I’ll rework then with my dremel tool. Most of these different vendors sell the same parts or a clone of a clone.

I installed this in my JBOD just fine as there is no MB. I could not install in any system with a MB. If you review the half height bracket compared to the full size bracket in the photo in full size, you can see that it is poorly designed. The mounting tab on the half height is too close to the bottom of the bracket to be mounted in a system with a motherboard. there will be interference. All the ebay half height brackets I reviewed have this issue.

Mini SAS Internal SFF-8087 SAS 36P to 2 Port External HD SFF-8088 Card Adapter | eBay

Find many great new & used options and get the best deals for Mini SAS Internal SFF-8087 SAS 36P to 2 Port External HD SFF-8088 Card Adapter at the best online prices at eBay! Free shipping for many products!

www.ebay.com

and this has the same issue

other style bracket

this is a different mounting scheme

 

[HarambeLives]

Dang it. The ones I ordered but not recieved yet are also wrong. Hope they got the pinout right. I should have looked harder. Evidence was there. I’ll rework then with my dremel tool. Most of these different vendors sell the same parts or a clone of a clone.

My pinouts must be fine, seems to work alright

How hard is it to get a darn bracket right??

 

[jgreco]

Dang it. The ones I ordered but not recieved yet are also wrong.

They're not actually wrong. The expansion slots for a modern PC are descendants of AT-style ISA cards, which were "flipped" such that the PCB was on the other side of the slot -- i.e. if you installed an ISA card and a PCIe card next to each other, they would either be "back to back" or facing each other.

A SFF-8088 backplate card doesn't actually attach to the system mainboard, so there is no "correct" side for it to be on. An inspection of your picture suggests that you screwed it in using the screw to the left, which is correct for an ISA card bracket. It looks like you've "shared" the screw with a PCIe-style vented blankplate.

Obviously this would be a stupid design if designed from scratch today, but we are talking SAS 3Gbps and 6Gbps stuff from fifteen years ago. The SFF-8088 was present on stuff like the LSI SAS3801X, and TTBOMK there are no ISA SAS cards, but it seems completely believable that someone might have had a crapton of tooling or existing blanks punched for ISA that they sold cheaply to 10GTek, since it really shouldn't matter for the normal use case of converting a rack chassis to a JBOD shelf.

If you think it's a bit annoying, yes, it probably is.

 

[ZiggyGT]

The 10GTek Branded one I got on Amazon for $27 has the low profile bracket produced backwards, pretty annoying! I had to sandwich it in there to get a screw through the hole

I just received a motherboard I ordered, and it came with an 8-Port SFF-8087 to SFF-8088 miniSAS Adapter
board that had the properly built half height mounting. The vendor was https://www.pc-pitstop.com/AD8788-2. They are a little pricey but a properly built part. They have lots of other bulkhead goodies that would be handy for JBOD builds. I have all I need but thought this was a good PSA.

 

[ZiggyGT]

Here are a couple of toys I used to power a JBOD chassis. Both chassis turn on with the power button on the main chassis. This stuff is far more cost effective than the Supermicro JBOD board.

Here the dtails of my final JBOD build.
1) Repositioned the MB mounting studs to match the size of the Radio Shack perf board I had laying around for the last 20years
2) Made mounting holes in the perf board for the fan speed control and the power on relay board
3) The Power relay board in the second chassis is driven by the 12V power from a drive molex connector strung from the first chassis. Not pretty but functional.
4) I noticed that I could not tell if the 2nd chassis was on because the power light was not on. (that would plug into the MB that is not there) so I added a header to the perf board to plug in the power LED. I added the required pull up resisters for the LEDS. I plugged a mini power connector driven by the 2nd chassis supply.
4) Mounted the pot that controls the fan speed on the back of the second chassis
5) I wanted to know the speed of the fans in the 2nd chassis, so I ran a ground and tach signal from the fan control board to a fan output of the MB in the primary chassis. This now shows up as FAN4 in the IPMI. Again, it is not pretty but works. All fans in the second chassis are driven by the same PWM so it is an indicator of all the fans. Of course, I have a connector in the middle of the cable so the chassis can be separated.
6) So between the two chassis I have 3 cables, the Mini-SAS SFF-8088 to SFF-8088, the Power Molex and the tach signal line.

Cost = Mini-SAS cable $10, 2X Bulkheads $20/each, all other cables from the junk drawer. Fan speed control $5, Power relay $3.
The Fan control and Power relay card is between $80-$170. It lacks fan tach feedback and it does not ensure the JBOD chassis will power down when the UPS signals a power loss to the primary.

Probably not a best practice but works.

 

[ZiggyGT]

I made a few mods to the Mini SAS Internal SFF-8087 SAS 36P to 2 Port External HD SFF-8088 Card Adapter I bought on eBAY.

There are many vendors but all the same design. The design works OK in the JBOD but I need to send the signals out from my HBA. The problem is shown below. The half height bracket as supplied cannot be installed in a chassis with a motherboard. It works fine in my JBOD. In the planned mods photo below you can see that I have to reduce the size of the card and remake the mounting bracket. Not for the faint of heart. Fortunately, there are no signal lines near the edges, just the ground plane. I planned out the mods with a marker and nibbled away at the card. After 46 years of nibbling my Radio Shack nibbler is getting pretty dull. Trying to keep metal filings away from the connectors is a challenge. Below is the comparison of the full and half height brackets supplied and the modified full height that is now a workable half height. Final fitment and cable fitment are shown below. If I was doing it all over again, I might have gone with this https://www.thingiverse.com/thing:4392850 for the bracket. For those nervous about hacking the circuit card, here is the one that will work out of the box. https://www.pc-pitstop.com/AD8788-2 but a lot more money. I nearly stopped to return this one and buy the PC-Pitstop one, but once you start, you need to finish.