Is it possible to re-purpose these components for FreeNAS usage?

[wblock]

As we are going to be using this for virtual machine storage, I would think that a SLOG would help. Could we dual purpose the 64GB SSDs in the system for the OS and SLOG?

Not really. In the situations where a SLOG would help, using a relatively slow consumer SSD and sacrificing some of its bandwidth by sharing it as a system drive is counterproductive.

 

[Eric Schrauth]

The basics, use a single 8 port SAS HBA like this:

Dell H310 6Gbps SAS HBA LSI 9211-8i P20 IT Mode
https://www.ebay.com/itm/Dell-H310-...D-High-Air-Flow/162834671120?epid=19006955695

On what I call the front edge of the card are two SFF-8087 connectors that you would plug into a pair of cables like this:

Mini SAS 36 Pin to Mini SAS SFF-8087 Line Data Cable
https://www.ebay.com/itm/Mini-SAS-3...87-Braided-Line-Data-Cable-50cm-/252851546122

Those cables would run to the two SFF-8087 port on what I call the top (they are marked "In") of the SAS expander card:

IBM 46M0997 ServeRAID Expansion Adapter 16-Port SAS Expander
https://www.ebay.com/itm/IBM-46M0997-ServeRAID-Expansion-Adapter-16-Port-SAS-Expander-/172519142938

On the front of the SAS expander card are four more SFF-8087 port that would connect to cables like this:

Mini SAS SFF-8087 36Pin to 4 SATA 7Pin HDD Hard Drive Splitter Cable
https://www.ebay.com/itm/Mini-12Gbp...D-Hard-Drive-Splitter-Cable-50cm/311655731018

These cables would go to the individual hard drives.

The Dell H310 is the card that actually talks to the system bus, the SAS expander receives and transmits all the data through the SFF-8087 ports and the card only receives power through the PCIe slot, so you could use a device like this to provide power to the card if you are short on PCIe slots in the system.

https://www.ebay.com/itm/M-2-to-PCI...Riser-Card-VGA-Extension-Cable-1/322923171579

You would only need the power cord and PCIe slot, not the other components. It would only be to apply power to the SAS expander.

Chris,
I was wondering why this configuration over two SAS HBA that will each support 8 drives.
Thanks
Eric

 

[Chris Moore]

Chris,
I was wondering why this configuration over two SAS HBA that will each support 8 drives.
Thanks
Eric

I have built with multiple SAS controllers, but it doesn't save any money and it really does not provide any better performance.
Why do you ask?

 

[Eric Schrauth]

I have built with multiple SAS controllers, but it doesn't save any money and it really does not provide any better performance.
Why do you ask?

Was just wondering which configuration would be better from performance and cost. As an aside, is there any problem with mixing drive types such as Desktop SATA3 with NAS SATA3 drives. Currently have 10-HGST Deskstar 2TB and want to add 6 more 2TB drives and was thinking of using WD Red. Planning on upping the memory to 32GB total.

 

[Chris Moore]

Was just wondering which configuration would be better from performance and cost.

The SAS expander, being less expensive than buying a second SAS controller is slightly less expensive. The SAS expander basically works like a network switch in that it allows the 8 lanes of the SAS controller to be shared to more drives. Since the drives themselves are the slowest link in the chain, you would have to put a tremendous number of drives on a single controller before the SAS controller would be a slow point.
If I recall, it was the second FreeNAS system that I built, I used 12 drives in it and in that group of drives there were three different brands and they all had different performance characteristics. FreeNAS can handle it, just remember that each vdev will operate at the speed of the slowest drive in the vdev. I have two vdevs of six drives each on my main NAS (both vdevs in one pool) but vdev-0 is made of 2TB 7200 RPM drives where vdev-1 is 4TB 5400 RPM drives and it all works perfectly.

 

[Eric Schrauth]

No. The OS drive can't be used for SLOG. You can put the system dataset on the OS drive (I do) but that is about all.
The SLOG device needs to be a low latency device (fast) with a lot of wear endurance and power loss protection. One of the suggested options for it is a Intel Optane SSD 900P:
https://www.newegg.com/Product/Product.aspx?item=N82E16820167437

On the Optane, do we need to do anything special to use it as a SLOG or will the system just recognize it and use it accordingly?

 

[Chris Moore]

On the Optane, do we need to do anything special to use it as a SLOG or will the system just recognize it and use it accordingly?

It isn't automatic. You have to add it to the pool through the GUI. Here is a nice post with lots of photos that talks about how to configure a SLOG. It should be very similar even though they are using an Intel DC P3700.

https://forums.freenas.org/index.ph...sdv-tln4f-esxi-freenas-aio.57116/#post-401289

 

[Eric Schrauth]

It isn't automatic. You have to add it to the pool through the GUI. Here is a nice post with lots of photos that talks about how to configure a SLOG. It should be very similar even though they are using an Intel DC P3700.

https://forums.freenas.org/index.ph...sdv-tln4f-esxi-freenas-aio.57116/#post-401289

The Intel card is PCI 3.0 x4 and the Motherboard is PCI 2.0 x16. Does this cause a problem?

 

[Chris Moore]

We are still talking about the same ASUS P8B WS from the original post?

If yes, it has all 16x PCIe slots, but they are not all 16x electrically. According to the documentation I am looking at on the Asus website, the blue slot closest to the CPU is 16x only if there is nothing installed in the black slot. If you install a card in the black slot, they both become 8x slots. The two white slots are 4x electrically all the time. The best option would likely be to have the SAS controller in the blue slot and the Optane card in the black slot. Those two slots are theoretically the fastest, because they are direct to the CPU. The other two slots go through the chipset and will be slightly slower.
As for the PCIe 2.0 vs PCIe 3.0, there is a big speed disparity between the two but I don't think that will impact this because (If I recall correctly) PCIe 2.0 is fast enough to handle the speed of the Optane drive.

 

[Eric Schrauth]

Chris,
The controller that you mention is no longer available. Would https://www.ebay.com/itm/162955405818 do?

 

[Chris Moore]

Chris,
The controller that you mention is no longer available. Would https://www.ebay.com/itm/162955405818 do?

They are fabricated by different plants, and look slightly different, but the chip that runs them is the same.
The system wouldn't know the difference.

If it were me, I would still verify that the the firmware is the correct version when I get it.

PS. Not a bad price. You can get an equivalent card for less, but you would need to flash the firmware yourself. That is what I have done in the past.

 

[Eric Schrauth]

If it were me, I would still verify that the the firmware is the correct version when I get it.

How would I know about the firmware version being correct?

 

[Chris Moore]

I am going to give you this snip from the reference material and I am going to give you the link to the full reference that has been carefully compiled by one of the member here.

Code:

Useful sas2flash commands:

sas2flash -listall	 - lists all LSI 2008 controllers found. This is the controller in the 9211. Ideally equivalent to listing all 9211 variant cards found, but if it finds additional controllers DO NOT IGNORE THIS! See above - occasionally extra LSI 2008 controllers may be hidden in SSDs and on the motherboard.
sas2flash -c CARD_ID -list	 - provides full details about each card (current firmware, bios, versions, sas ID, manufacturer, whether IR/IT, etc.)

Detailed newcomers' guide to crossflashing LSI 9211 HBA and variants
https://forums.freenas.org/index.ph...o-crossflashing-lsi-9211-hba-and-variants.54/

If it is in IT mode, then you don't need to crossflash but the instructions would also allow you to load the latest firmware if you need to.
Now, if the card comes with the latest firmware already installed, there is nothing to do.
The most recent firmware I am aware of is: 20.00.07.00

 

[Eric Schrauth]

It isn't automatic. You have to add it to the pool through the GUI. Here is a nice post with lots of photos that talks about how to configure a SLOG. It should be very similar even though they are using an Intel DC P3700.

https://forums.freenas.org/index.ph...sdv-tln4f-esxi-freenas-aio.57116/#post-401289

In this post, he mentions using the SLOG device for L2ARC and swap also. With having a machine with 32GB of memory is this needed L2ARC wise? What about swap? Have you seen any benefit in doing it the way he does?

 

[Linkman]

I would be very hesitant to add an L2ARC to a server with only 32GB RAM. You could always try it and benchmark both, but if it were my box, I wouldn't bother.

 

[Eric Schrauth]

I would be very hesitant to add an L2ARC to a server with only 32GB RAM. You could always try it and benchmark both, but if it were my box, I wouldn't bother.

Trying to understand the relationship between L2ARC and 32GB RAM. Can you clarify why?

 

[Linkman]

FreeNAS and ZFS use main memory for cache, ARC. If you add L2ARC, which is basically external ARC, then you lose space in main memory and the faster ARC for the pointer table into the L2ARC. So if you only have 32GB RAM, you could actually lose performance.

That's why the recommendation is always to max out system RAM before adding L2ARC. If you are already at the max RAM your system supports, then test to see if L2ARC helps your workload or not.

I may have been a bit cavalier in my first response, I know L2ARC won't help my workloads and I am at the maximum (32GB) RAM that my system supports.

 

[Eric Schrauth]

We are maxing out the RAM in our unit (32 GB) and it sounds like your config is similar to ours (SSDs for the OS mirrored/16x2TB drives) so I would think it wouldn't help in our instance.

 

[Chris Moore]

Trying to understand the relationship between L2ARC and 32GB RAM. Can you clarify why?

Here is a demo video to help understand L2ARC. The person that made the video did a lot of testing and the system only has 32GB of memory and just 4 hard disk drives.

https://www.youtube.com/watch?v=oDbGj4YJXDw

 

[Eric Schrauth]

I was wondering if there is a resource available on how to best configure zvol/zpool. I would like to run 8TB iSCSI, 4TB Windows sharing and 4TB NFS. This will be on 16x2TB drives. Was planning on using stripping/mirroring to get the best performance. Is that a good idea?