SOLVED Interested in some final thoughts on my new NAS build

[Obi-Wan]

Hi everyone! :)

I'm currently working on my very first NAS and would be very interesting in some final thoughts from the experts! I love building computers, but it has been all about gaming in my previous builds. NAS and everything that goes along with it is new to me. I've struggled these last few days because every google search gave me ten FreeNAS forum links, but none of them would work! So I'm glad the forums are back up.

I'm building this to be a long-term storage solution for my home. It will also serve as a Plex server, but I don't expect to do much transcoding. Though I do want to be able to do a single 1080p transcode. I also expect to be running Radarr, and as I like to tinker with these things I expect that I might find some other use as well, though I don't currently know what that would be!

As for the drives I've been convinced to run mirrored drives and add more as I need them.

The current build:

• Intel Pentium G4560 3.5 GHz, 2 Cores, 4 Threads
• Supermicro X11SSM-F
• 2x Samsung DDR4 2400 MHz ECC 8 GB
• 2x WD Red WD40EFRX 4 TB
• 2x SanDisk USB 3.1 Ultra Fit 32 GB
• Crucial BX500 120 GB
• Corsair RM650X V2 650 W
• Fractal Design Node 804

Some of these are chosen based on availability. I've read on here that you ususally recommend going for the maximum amount of memory for instance, i.e., one 16 GB stick, but I struggle to find them available where I'm from.

I do wonder a bit about the CPU. Based on the Passmark score it should handle a single 1080p transcode fine, but I don't know how much everything else taxes the CPU. Any thoughts?

Anything else I need to think about before I get started?

 

[Ericloewe]

Some of these are chosen based on availability. I've read on here that you ususally recommend going for the maximum amount of memory for instance, i.e., one 16 GB stick, but I struggle to find them available where I'm from.

That's odd, but not something to really worry about for now.

2x SanDisk USB 3.1 Ultra Fit 32 GB

You're better off booting from a cheap SSD. Just avoid the cheapest SanDisk model and the WD Green line, since they have a bit of a controller firmware bug that causes trouble with FreeNAS.

Corsair RM650X V2 650 W

That seems fairly large. Are you planning on adding drives in the future? If so, how many? A 550W unit could easily handle 10 disks.

 

[Obi-Wan]

You're better off booting from a cheap SSD. Just avoid the cheapest SanDisk model and the WD Green line, since they have a bit of a controller firmware bug that causes trouble with FreeNAS.

Thanks! I originaly had a decently cheap SSD there, I can get a Kingston SSDNow A400 SA400S37 120GB for about the same price as the two USBs, but the counter arguments were:

• An SSD will use up one of the SATA slots I have available
• Having two USBs gives me redundancy
• The performance difference only matters the few times I'm rebooting the system

Any thoughts on this?

That seems fairly large. Are you planning on adding drives in the future? If so, how many? A 550W unit could easily handle 10 disks.

I am planning on adding drives. But again, there are few smaller ones available, and this specific one is on sale, which is part of the reason it's there. Is it in any way bad to have a too hefty PSU if I can get it cheap?

What I really did to find that PSU was to filter on the number of SATA connections on the PSU (8 at least) and remove everything below Gold. The site I'm using is a Norwegian one, and I do want to buy from a Norwegian store. That made this specific one the cheapest available to me.

 

[rvassar]

Thanks! I originaly had a decently cheap SSD there, I can get a Kingston SSDNow A400 SA400S37 120GB for about the same price as the two USBs, but the counter arguments were:

• An SSD will use up one of the SATA slots I have available
• Having two USBs gives me redundancy
• The performance difference only matters the few times I'm rebooting the system

Any thoughts on this?

I had string of bad luck with USB sticks. I don't know if I was just buying junk, or reusing stuff cooked in Raspberry Pi experiments, or if there was actually a problem with the chassis I was using at the time. Make sure you configure & test notifications, and keep spares on hand. I ended up picking up a couple cheap 60Gb Patriot Burst SSD's. They're pretty much EOL, but I can still get them from Amazon here in the US for $20 each. At one point I was running with one 60Gb SSD mirrored to a 64 Gb FIT USB stick with a metal case. That was actually the only USB config I ran that never caused me trouble. I abandoned it after testing the other SSD as a SLOG, and realizing it didn't have the needed power failure protection. I do a lot of NFS, so I swapped it in to the boot pool to avoid the temptation. ;)

Amazon - 60Gb SSD

On the SATA slot consumption. My previous builds were starved for SATA ports, and I picked up a LSI SAS HBA & SATA fanout cables off eBay, and have been carrying it forward. It's now in it's third chassis since I started using FreeNAS. Only you know how much storage you're going to need, or how many IOPS you need to handle. My main pool fits on 4 mirrored disks at the moment, and mostly meets my IOPS requirement. I'm planning on moving up to 8Tb disks before adding a third vdev mirror. The nice thing about adding an HBA is you can stagger the disks in the vdev's such that you have no single point of failure, or split things up along the lines of main pool & backup pool. At the moment I have my main pool on the HBA, and the boot pool on the motherboard SATA ports. I have a 5-1/4 inch to 3.5 SATA hot swap tray connected to the motherboard SATA controller as well. This allows me to drop in a 8Tb ZFS formatted single disk, do a backup, and then pull the disk and stick it in a safe drawer.

 

[Obi-Wan]

Thank you! :)

At one point I was running with one 60Gb SSD mirrored to a 64 Gb FIT USB stick with a metal case. That was actually the only USB config I ran that never caused me trouble. I abandoned it after testing the other SSD as a SLOG, and realizing it didn't have the needed power failure protection. I do a lot of NFS, so I swapped it in to the boot pool to avoid the temptation. ;)

I was thinking that "Ahh... well, maybe I could get an SSD and use a USB for redundancy just in case!" And you say that you did something like that, but then there is a "but" in there, but I don't quite know what that "but" is. What is a SLOG? And would you not recommend doing that? SSD + USB that is.

I do want to be able to take advantage of all 8 SATA ports, but realistically that is not anytime soon. The cheapest SSD I can I can find is this one:

Amazon: PNY CS900 120GB 2.5” Sata III

But I'm thinking maybe I'll get this next one since I don't know anything about PNY:

Amazon: Crucial BX500 120 GB

 

[rvassar]

Thank you! :)



I was thinking that "Ahh... well, maybe I could get an SSD and use a USB for redundancy just in case!" And you say that you did something like that, but then there is a "but" in there, but I don't quite know what that "but" is. What is a SLOG? And would you not recommend doing that? SSD + USB that is.

It's basically my insistence that there be no single storage failure point. The previous build had chewed thru a string of USB thumb drives in a matter of weeks. I had the SSD sitting around unused, but with past use on a Linux netbook, so I used it for the boot device, and since I didn't fully trust it, I needed to pair it with a mirror. The "but" is I moved from a non-ECC Dell chassis where the failures were occurring, to my current Supermicro build. I suspect the USB controller on the Dell was flakey, or not fully supported by BSD. I made it a point to use a brand new USB device, and ran with a "split mirror" for several months before the floor fell out of the small SSD market, and I picked up another 60Gb drive for $18 on sale. I should also note that the Unix command line is my native environment, and I'm comfortable with the ZFS CLI.

The trick is you have to install on the smaller device first, and then add the USB sick after the fact. You then face the issue that you will not be able to re-silver the 60Gb device should it drop out of the pool. It will complain that the 64Gb USB device is larger. There's probably some partitioning magic to be performed, but I just opt'ed for the second SSD.

A SLOG is a type of "intent log". Basically a fast write only scratchpad for the NAS to use to keep notes on. It stems from some operating system internals with regards to how files are opened and written to. In a POSIX derived system like BSD or Linux you open a filehandle and set some options flags on it, depending on how you value to data being written. This allow the OS to optimize how it performs the work. One of those flags is called "O_SYNC", which when set requires a synchronous write be performed. That means the call to the OS writing the data will not return to the program until the OS has committed to write to secure storage. That might involve moving the disk heads, waiting for a sector to rotate in to position, etc... Long... Slow... But absolutely required for transactional databases, bank records, etc... Without O_SYNC, the OS can just copy the write to a kernel buffer and say "yea, I'll get to it" and return to the program. If a crash then occurs, the write sitting in memory is lost, corrupting the database, etc...

So the thing about NFS is it's in the protocol RFC that every write is treated as O_SYNC by the NFS server. That's because it has no way of knowing if the remote client has crashed or not. This was actually the basis for the original Sun "Prestoserve" NFS accelerator in the early 90's. So the ZFS SLOG device allows the NAS to use an SSD as a non-volitile write only scratchpad, so it can get that network acknowledgement sent back to the client as fast as possible, and sort out the competing seek/write sequences between multiple clients as it best sees fit.

I do want to be able to take advantage of all 8 SATA ports, but realistically that is not anytime soon. The cheapest SSD I can I can find is this one:

Amazon: PNY CS900 120GB 2.5” Sata III

But I'm thinking maybe I'll get this next one since I don't know anything about PNY:

Amazon: Crucial BX500 120 GB

I would use the smallest one I could source. I'm also told we should avoid SSD's with Marvell controllers. I do not have a list of devices with those however.

 

[Ericloewe]

The performance difference only matters the few times I'm rebooting the system

It's more complicated than that. Also, SSD reliability easily beats USB flash drives, probably even a pair of flash drives.

Amazon: PNY CS900 120GB 2.5” Sata III

But I'm thinking maybe I'll get this next one since I don't know anything about PNY:

Amazon: Crucial BX500 120 GB

The Crucial is a better choice, and apparently cheaper now.

I do want to be able to take advantage of all 8 SATA ports

You can always add an HBA later. That's what I did on my backup server to be able to boot from SSD.

 

[Obi-Wan]

Thanks guys, you have me convinced about the SSD. The build in my original post now looks to be the final one!

Snip

I will need to read through that a few times before I understand what all that means. I see that there is actually a point in getting an even smaller SSD though, but 120 GB is the smallest SATA disk I can find. I could always order something smaller from Amazon.co.uk, but I like to order "locally".