Is it possible to re-purpose these components for FreeNAS usage?

[Eric Schrauth]

Hi all,
I have checked the various guides and wanted opinions on reusing the following list of hardware to run FreeNAS. It is currently running openfiler (I know, I know) and want to replace it with FreeNAS. I was thinking that the only thing that was really needing to be replaced that didn't seem to fit the standard FreeNAS specs was the disk controller. Any opinions?

Here is what I have (2 sets to be exact)

1x - SUPERMICRO CSE-836TQ-R800B Black 3U Rackmount Server Case 800W (1 + 1) Redundant AC-DC high-efficiency power supply w/ PFC
1x - LSI MegaRAID SAS LSI00210 (9280-16i4e) SATA/SAS 6Gb/s PCIe 2.0 w/ 512MB onboard memory controller card, Single
1x - ASUS P8B WS LGA 1155 Intel C206 ATX Intel Xeon E3 Server/Workstation Motherboard
1x - Intel Xeon E3-1230 Sandy Bridge 3.2GHz 4 x 256KB L2 Cache 8MB L3 Cache LGA 1155 80W Quad-Core Server Processor BX80623E31230
2x - Kingston 8GB 240-Pin DDR3 SDRAM ECC Unbuffered DDR3 1333 Server Memory Server Hynix M Model KVR13E9/8HM
2x - Crucial M4 CT064M4SSD2 2.5" 64GB SATA III MLC Internal Solid State Drive (SSD)
10x - HGST Deskstar 3.5-Inch 2TB 7200RPM SATA III 6 Gbps 64 MB Cache Internal Hard Drive (0F12115)
1x - Intel E1G44HTBLK Server Adapter I340-T4 (Bulk Pack) 10/ 100/ 1000Mbps PCI-Express 2.0 4 x RJ45

The SSDs are connected to the motherboard via the SATA 6.0 Gb/s connectors and are mirrored. The MegaRAID will be replaced by a standard controller instead of RAID.

Any thoughts? Thanks in advance for your input!

 

[saikee]

I believe ZFS dislike raid controllers as it does their work. Thus I have not experienced any problem using the stock Sata controller off the mobo.

 

[Eric Schrauth]

Unfortunately I need to support at least 10 drives so there won't be enough ports on the motherboard. That's why I am replacing the RAID controller with a standard one.

 

[saikee]

I just get the normal Sata III cards off Amazon. My latest FreeNAS PC has only 6 Sata ports in the mobo and I fitted two Sata controllers; one 8 ports and the other 4 ports. They are plug-and-play. Have acquired a Coolmaster Cosmos that has 16 hdd bays.

 

[wblock]

I'd say yes, it will run FreeNAS, and is pretty capable. Replace the LSI card with a LSI 9211-8i or equivalent. Or maybe the firmware can be flashed on that card, but I suspect not. I would recommend the 9211 over commodity cheap SATA cards with varying levels of support in FreeBSD. I'm not personally fond of Asus motherboards, but it's probably all right. The model number on the Ethernet card is not familiar, but most Intel cards are supported by FreeNAS.

As to whether the storage and speed are adequate, or what to use the SSDs for, it depends on how the system will be used.

 

[Eric Schrauth]

The card doesn't seem to support JBOD at all. I plan on using it for general purpose NFS, iSCSI service for Oracle VM storage and Windows file service. Thanks for the input!

 

[loch_nas]

Haha, funny! I have exact the same mainboard, processor and nearly the same RAM as a home workstation. It's still running without a single problem.

 

[joeinaz]

If you are going to do 10 disks it might be a good idea to get more memory. Rule of thumb for FreeNAS has been 1GB RAM for every 1TB of disk. If you have ten 2TB disk, 16GB may the bare minimum to use. Let's see what others recommend. As for the 10 disks, you could use an M1015 card in "IT" mode and an expander card or you can use multiple SATA cards. I recently found a 4 port card for less than $50. You could do two 4 port cards and one 2 port for your 10 ports if you go that route.

 

[Eric Schrauth]

Was wondering about the memory also. Thought that it was close enough but agree that I will wait to hear what others say. Have no problem doubling the memory that is in it.

Was looking at two of the SAS9211-8I 8PORT Int 6GB Sata+sas Pcie 2.0 card for about 80 each. Well within my budget.

 

[Chris Moore]

2x - Kingston 8GB 240-Pin DDR3 SDRAM ECC Unbuffered DDR3 1333 Server Memory Server Hynix M Model KVR13E9/8HM

You have 16GB of memory and that is probably fine but if you feel the system is being slugish, you might upgrade to 32GB. It is not a hard requirement to have 1GB of memory per TB of storage. That is one of the "thumb" rules that is meant to give rough guidance. There are use cases where you would need more memory. Just keep in mind that ZFS uses RAM for cache of read and write with ZIL (ZFS Intent Log) for writes and ARC (Adaptive Replacement Cache) for reads. It is true that more memory can make the system more performant, but in low activity (home) use situations it is not really needed. I run 32GB in my NAS, but that is only because I used to do some light virtualization on it, and may do that again if they get the new hypervisor to where it is easy to configure.

Was looking at two of the SAS9211-8I 8PORT Int 6GB Sata+sas Pcie 2.0 card for about 80 each.

I use one SAS controller like this one with the firmware flashed to IT mode:
https://www.ebay.com/itm/Dell-H310-...D-High-Air-Flow/162834671120?epid=19006955695
and a SAS expander like this:
https://www.ebay.com/itm/IBM-46M0997-ServeRAID-Expansion-Adapter-16-Port-SAS-Expander-/172519142938
The SAS expander doesn't actually need to be in a card slot, it just uses the slot for power. Are you familiar with how this would be connected?

2x - Crucial M4 CT064M4SSD2 2.5" 64GB SATA III MLC Internal Solid State Drive (SSD)

Is this for the boot drives or did you have something else in mind for them?

The SSDs are connected to the motherboard via the SATA 6.0 Gb/s connectors and are mirrored.

If you plan to use them for the boot device, you don't want to use any built-in RAID ish function to mirror them. FreeNAS can make the two disks into a zpool and boot from them so the mirror is handled by the OS instead of the hardware. It makes the installation hardware agnostic. You can take a FreeNAS installation and move it to a different hardware platform and it will continue to work. I moved mine from a socket 1366 system to my current build without needing to make a single change.

 

[Chris Moore]

Any thoughts? Thanks in advance for your input!

Did you have some questions for us?

10x - HGST Deskstar 3.5-Inch 2TB 7200RPM SATA III 6 Gbps 64 MB Cache Internal Hard Drive (0F12115)

How were you planning to lay these disks out, what RAID configuration?

 

[Chris Moore]

iSCSI service for Oracle VM storage

ZFS has some more demanding requirements if you want iSCSI (block storage) to perform well. Have you looked into that at all?
We have had a couple discussions about it just today.

SUPERMICRO CSE-836TQ-R800B

PS. I love that chassis. I had one like it for a while.

 

[saikee]

You can take a FreeNAS installation and move it to a different hardware platform and it will continue to work. I moved mine from a socket 1366 system to my current build without needing to make a single change.

Hi Chris, I have been wondering if I should move my FreeNAS system from an old PC to a new PC instead of rebuilding everything again. Your statement gives me a lot of comfort. I did this with the Linux systems and most of the time it works.

Latest
I went ahead and did just that by removing 5x8TB hdd and the USB boot drive from an existing PC and put them in my new rig with a different CPU, mobo and ram. It booted up exactly as before and every my Plex friend could use Plex right away without me lifting a finger. Magic!

 

[Eric Schrauth]

Chris. So much to chew on! Thanks for all of the fine input.

As far as the SSDs. Don't know if they should be used for the OS or ZIL/ARC purposes and replace the OS with a couple of USB Keys.

As far as the hard drives, I was looking to do RAIDZ3

As far as the controller, I am open to suggestions. Have not purchased anything yet and any recommendations/how tos would be appreciated.

Could you point me to the iSCSI discussions?

Thanks again for all of the input!!!

 

[Chris Moore]

Could you point me to the iSCSI discussions?

There is some great advice in this thread and references to other good threads:
https://forums.freenas.org/index.php?threads/freenas-for-iscsi-vmware-esxi-storage.61651/

This user was also asking about iSCSI for virtualization, but they are trying to do it on a tight budget.
https://forums.freenas.org/index.php?threads/backing-to-freenas-help-with-topology.61659

Then this one is one of my favorites because he goes into so much detail with photos and documentation. @Stux is very knowledgeable.
Build Report: Node 304 + X10SDV-TLN4F [ESXi/FreeNAS AIO]
https://forums.freenas.org/index.ph...node-304-x10sdv-tln4f-esxi-freenas-aio.57116/

 

[wblock]

As far as the SSDs. Don't know if they should be used for the OS or ZIL/ARC purposes and replace the OS with a couple of USB Keys.

ZIL and ARC are built in. SLOG and L2ARC are the add-ons. SLOG is not needed unless the system does lots of large synchronous writes, usually just a situation when acting as storage for virtual machines. L2ARC is usually not recommended unless the RAM in the machine has been maxed out.

Going from an SSD to a USB boot drive is a step backwards in reliability. A 64G SSD (or two) is pretty much ideal as a FreeNAS boot device.

 

[Eric Schrauth]

As we are going to be using this for virtual machine storage, I would think that a SLOG would help. Could we dual purpose the 64GB SSDs in the system for the OS and SLOG?

 

[Eric Schrauth]

I use one SAS controller like this one with the firmware flashed to IT mode:
https://www.ebay.com/itm/Dell-H310-...D-High-Air-Flow/162834671120?epid=19006955695
and a SAS expander like this:
https://www.ebay.com/itm/IBM-46M0997-ServeRAID-Expansion-Adapter-16-Port-SAS-Expander-/172519142938
The SAS expander doesn't actually need to be in a card slot, it just uses the slot for power. Are you familiar with how this would be connected?

If you could expand on how those devices would be connected it would help greatly! Thanks.

 

[Chris Moore]

As we are going to be using this for virtual machine storage, I would think that a SLOG would help. Could we dual purpose the 64GB SSDs in the system for the OS and SLOG?

No. The OS drive can't be used for SLOG. You can put the system dataset on the OS drive (I do) but that is about all.
The SLOG device needs to be a low latency device (fast) with a lot of wear endurance and power loss protection. One of the suggested options for it is a Intel Optane SSD 900P:
https://www.newegg.com/Product/Product.aspx?item=N82E16820167437

 

[Chris Moore]

If you could expand on how those devices would be connected it would help greatly! Thanks.

The basics, use a single 8 port SAS HBA like this:

Dell H310 6Gbps SAS HBA LSI 9211-8i P20 IT Mode
https://www.ebay.com/itm/Dell-H310-...D-High-Air-Flow/162834671120?epid=19006955695

On what I call the front edge of the card are two SFF-8087 connectors that you would plug into a pair of cables like this:

Mini SAS 36 Pin to Mini SAS SFF-8087 Line Data Cable
https://www.ebay.com/itm/Mini-SAS-3...87-Braided-Line-Data-Cable-50cm-/252851546122

Those cables would run to the two SFF-8087 port on what I call the top (they are marked "In") of the SAS expander card:

IBM 46M0997 ServeRAID Expansion Adapter 16-Port SAS Expander
https://www.ebay.com/itm/IBM-46M0997-ServeRAID-Expansion-Adapter-16-Port-SAS-Expander-/172519142938

On the front of the SAS expander card are four more SFF-8087 port that would connect to cables like this:

Mini SAS SFF-8087 36Pin to 4 SATA 7Pin HDD Hard Drive Splitter Cable
https://www.ebay.com/itm/Mini-12Gbp...D-Hard-Drive-Splitter-Cable-50cm/311655731018

These cables would go to the individual hard drives.

The Dell H310 is the card that actually talks to the system bus, the SAS expander receives and transmits all the data through the SFF-8087 ports and the card only receives power through the PCIe slot, so you could use a device like this to provide power to the card if you are short on PCIe slots in the system.

https://www.ebay.com/itm/M-2-to-PCI...Riser-Card-VGA-Extension-Cable-1/322923171579

You would only need the power cord and PCIe slot, not the other components. It would only be to apply power to the SAS expander.