Will it FreeNAS better with a little bit of SAS?

[FishNChips]

Hi,

I've been getting ready to try build my first FreeNAS box / first server. I've owned and built lots of PCs over the years but now have been reading posts and articles here and elsewhere for days so hopefully won't ask too many dumb questions. (Meaning, ones that have been answered over and over.)

This FreeNAS box will eventually replace a QNAP NAS that has served well for 7 years, and run at least one instance of Windows as a virtual machine. It will hold the files for 50+ bookkeeping clients and serve various other purposes. So it needs to be fairly decent, last for years, and be way more than just a hobby.

*** The one thing I can't get a decent handle on is the remaining value these days of making use of a SAS controller, such as the LSI 3008 built into some of the Supermicro X10 motherboards. (Properly cross flashed, and in non-raid mode obviously.) ***

I know that years ago *real* servers wouldn't use anything less, but now the SATA controllers on many motherboards are cheap but effective, and the newer technologies such as SSDs, M.2 and NVMe over PCIe have really brought a lot of changes to the equation.

In a case like mine, where I will be running around 8 drives max (maybe in a Fractal Design Define R5), and I don't really need a backplane...

--- does a SAS controller and cabling bring additional value, beyond what's available if I just used the SATA ports on the motherboard. For this question assume I would use SATA drives either way. So can the SAS interface and cabling handle heavier workloads or speed things up? Or still provide data integrity through the SAS controller / cabling over and beyond what FreeNAS will be doing in any ways that SATA is not doing?

--- I know there is the issue of extra power, thus heat, going with faster SAS drives (the spinning metal, I mean!) But do they still bring substantial value when the workload warrants it? Or is the smart decision nowadays to invest in SSDs and such - the solid state options.

Thank you for reading this long post. If I knew all the correct terminology it could have been way shorter, sorry...

Albert

 

[Arwen]

Can't answer all your questions. But here goes some:

• SATA disks attached to a SAS host port don't get any extra cable length or reliability. Basically the SAS port turns into a SATA port when attached to a SATA disk.
  
• Some SATA controllers are pretty reliable, like Intel. They may even support hot swapping disks.
  
• Some SATA controllers are notoriously bad, like certain Marvell chips, (I don't have the list handy).
• SATA disks attached to a SAS host port may transfer data faster as a burst, depending on how the SAS controller chip is wired to the system board.

To clarify the last item. Let's assume a quad port SATA chip is wired to 2 lanes of PCIe 3, (8Gbps). Thus, maximum transfer for all 4 SATA ports is 16Gbps. Except that
SATA III standard specifies 6Gbps, so 4 disks, (like some SSDs), would need 24Gbps. Now take an 8 port SAS chip wired to 8 lanes of PCIe 3, which is 64Gbps raw bus
transfer speed. More than enough for 8 disks x 6Gbps = 48Gbps. But, that requires all disks to be active at the same time, which is pretty rare.

So, performance is dependant on how a device is wired up, and at what speeds. Plus, how many disks are active on a specific controller chip, at the same time.

 

[darkwarrior]

Hi there,

yeah, some mainboards are including a SAS controller like you stated and that is a good thing. It's a very good way to have additional SATA ports available to use with FreeNAS.
there is no pratically almost no difference in reliability, speed or functionality between the onboard Intel ports and the onboard SAS controller, when using basic spinning rust (if we stick with the Supermicro X10SL7-F example = I have the same board, is running rock stable since over 4 years )
On the other hand that picture might change a bit if you use only big SSDs

You can connect SATA drives to SAS gear (ports, cables, backplane, controller, etc), but not the other way around.
One of the great things is that you can connect many drives on a backplane using a single cable. that helps to keep good airflow and the things clean and tidy

You will find a overview on all this on the link below:
Don't be afraid to be SAS-sy ... a primer on basic SAS and SATA

Concerning the SAS drives:
It is really not worthwhile to spend so much extra cash on SAS drives.
Stick to good well known NAS drives like the WD REDs and you will be golden.

Edit: Ah Arwen already answered a part of your questions :p

 

[FishNChips]

Hi Arwen and darkwarrior,

Thank you for the detailed answers. I had forgotten that the cabling to SATA drives was more limited in maximum lengths. And hadn't considered the maximum throughput of the controllers themselves. Yet in having to write it down and think it through, I can see where I should go...

I've tended over the years to buy motherboards with features I *might* use in the future but if I looked back honestly, it's seldom I used them. So many technologies had their moment in the sun, real or just as awesome potential, and I've always liked to keep my options open, but in reality it just meant spending money uselessly.

I guess I will go with the NAS type of SATA drives that are readily available and save my money. If the motherboard has a SAS controller, fine, but not spend extra money for it. The future of solid state seems to be with M.2 or on cards in PCIe slots, not with SAS, so if anything look for more M.2 or PCIe expansion potential.

Any thing I am overlooking here?

 

[darkwarrior]

Hi Arwen and darkwarrior,

Thank you for the detailed answers. I had forgotten that the cabling to SATA drives was more limited in maximum lengths. And hadn't considered the maximum throughput of the controllers themselves. Yet in having to write it down and think it through, I can see where I should go...

I've tended over the years to buy motherboards with features I *might* use in the future but if I looked back honestly, it's seldom I used them. So many technologies had their moment in the sun, real or just as awesome potential, and I've always liked to keep my options open, but in reality it just meant spending money uselessly.

I guess I will go with the NAS type of SATA drives that are readily available and save my money. If the motherboard has a SAS controller, fine, but not spend extra money for it. The future of solid state seems to be with M.2 or on cards in PCIe slots, not with SAS, so if anything look for more M.2 or PCIe expansion potential.

Any thing I am overlooking here?

Currently, you're more easily limited by the number M.2 (1x) or PCI-E (3x-5x ports on average) on the usual rigs.
Additionally seeing the prices of SSDs (with or without M.2 or PCIe connections) still being very far away from TB/$ ratios of spinning rust, I would not overthink that too much... ;)

Can be nice to have a small SSD on an M.2 slot (port ?) for VMs or L2ARC for example, but that's all.
At the end of the day your data will still be stored on big heavy and a bit slowish SATA drives :p

 

[Arwen]

One note about SAS. It's a standard that is actively being improved. Meaning SATA came out first
with 1.5Gbps. Then SATA II, at 3Gbps. SAS then followed with 3Gbps, (and SATA compatibility).
Later, both standards jumped to support 6Gbps.

But, today, SAS supports 12Gbps, while SATA has stopped, (for the moment?), at 6Gbps. Plus, SAS
22.5Gbps is just around the corner. (Note that SAS 4 is twice as fast as SAS 3, it just uses less overhead
to achive the same result.)

Last, you can buy SAS interfaced SSDs with tremendous speed, (1200Mega bytes speed), today. They
are of course outrageouslly expensive, but they can be bought. Unlike SATA that tops out at 550MBps.

 

[ewhac]

Also, if ever you decide to backup your pools to tape, there (apparently) is no such thing as a SATA tape drive. They're all SCSI or SAS.