Dell T5500 build and shutting down overnight

[pfft]

Hi everyone,

Apologies in advance for the long post.

I'm contemplating a FreeNAS build and was hoping I could get a sanity check as well as opinions/answers on a couple of specific questions.

This will be for a lightly used home NAS (and perhaps media server, offsite syncing, etc), with a few terabytes of capacity. FreeNAS seems to be overkill in many ways, for my needs, but I want protection against bitrot (I've had some experience of this in the past). I would also like disk failures to be relatively pain-free but off-site backups on removable drives are still my main protection. An alternative would be btrfs on Linux (which is particularly tempting as I use Linux widely on my "real computers") but I'm concerned that btrfs doesn't seem nearly as mature as ZFS on FreeBSD.

I have a workstation that is spare which I can use for this project: Dell T5500 with one Xeon E5645 and 24GB of ECC RAM. I have removed the Dell PCIe SAS RAID controller and verified that all 5 on-board SATA ports work (for boot and under Linux). It has two spots for 3.5in HDDs and I should be able to mount up to two more in the 5.25in bays with adapters. There is also an internal 3.5in bay which should be fine for an SSD boot drive but I don't think I can mount a spinning drive there without excessive vibration (at least, not without some special bracket or part that I don't have). PSU is the Dell OEM one, I believe rated at 875w.

I appreciate that this may not be the ideal machine for the task but I'm not likely to buy an entire new machine for this project while this one is unused, and I believe it satisfies the main requirements for FreeNAS.

I'll be encrypting my data (the CPU purports to have AES-NI) - I don't need properly high security (anti-tamper/etc) but I don't want a random hardware thief to get my data.

One of my concerns in duty cycle - I know standard advice is to run the NAS 24/7 but running overnight (in normal operation, at least) isn't an option for me. I've read this thread, but I'm still not sure how much of an issue this might be:

https://forums.freenas.org/index.ph...s-based-set-up-where-it-is-not-on-24-7.16523/

I expect I'll be running for ~12h per day on a regular schedule (automated boot and shutdown), so I should be able to run scheduled tasks reliably, however I'm concerned about tasks that might not complete in that time, in particular scrubbing. I've seen references to FreeNAS/ZFS automatically resuming scrubbing after a shutdown, but I'm not completely sure if this really means "continue from where it left off" or "start again". If the latter then I'm concerned it might never complete, or that it might block on booting until it completes or similar. Can anyone confirm the behaviour here? Anything else I should watch out for, other than scheduling tasks for when it'll be running? I'm prepared to accept a moderately higher incidence of failure/lower lifespan if that results from the hardware shutting down and spinning up daily.

The CPU has hyperthreading, though at the moment I think this is switched off in BIOS as I only see 6 cores in Linux. Is hyperthreading likely to be beneficial here? I know the Right Way is likely to be to benchmark under my particular workload but I'm unlikely to bother unless performance is an issue so it would be good to know if there's a rule of thumb on this.

I'm likely to buy three or four (probably three) new 4GB "NAS grade" HDDs to run in RAIDZ1, and a cheap second-hand SSD for boot. I'd like to get different models (and ideally manufacturers, since different models from the same maker seem to have commonalities) of drives to avoid common mode failures (I got burned by a batch of Deskstars some years ago), but I'm open to opinions on this. I understand that performance will be suboptimal compared to matched drives, but I am not expecting it to be too badly affected (?)

From a quick look on Amazon UK (I'm in the UK) the main NAS drives available seem to be Toshiba, WD and Seagate. If I go for three drives I'm likely to go for one of each (per the above) - not sure what I would buy for a fourth if I wanted to avoid two the same. Any suggestions on NAS drives available in the UK, or specific retailers, would be appreciated. I'm wary of drives from eBay, even "new" ones...

Any thoughts on my PSU and the load from the extra drives? (I generally don't stuff this many drives in a box). I'd rather avoid replacing the PSU so this may mean sticking to three drives and perhaps booting from USB and removing the video card if the power budget might be an issue.

(Also I'll likely get a UPS at somepoint though my power seems fairly stable and reliable.)

I think that's it - thanks for reading all this, apologies for any stupid questions I should already have found the answers to, and thanks in advance for any advice!

 

[m0nkey_]

I'll be encrypting my data (the CPU purports to have AES-NI) - I don't need properly high security (anti-tamper/etc) but I don't want a random hardware thief to get my data.

Make sure you backup your keys and test them! We've seen far too many issues where people have not backed up the keys, or tested them. Also note that encryption makes replacing drives more difficult. So keep that in mind. Frankly, if you're new to FreeNAS, I'd advise against using encryption and use something like VeraCrypt containers if you have anything particularly important.

One of my concerns in duty cycle - I know standard advice is to run the NAS 24/7 but running overnight (in normal operation, at least) isn't an option for me.

FreeNAS may not be you in this case. It's designed to be an always-on system allowing it to run scheduled scrubs, SMART tests and more. By powering on/off you're going to miss some of these tasks. If you run the tasks during the time you're accessing your FreeNAS server, you're going to run into slow downs.

(Also I'll likely get a UPS at somepoint though my power seems fairly stable and reliable.)

That should be one of the first items on your list.

 

[pfft]

Hi m0nkey_,

Thanks for your reply.

Re encryption I'm pretty familiar with key and certificate management and already have a robust system for managing secrets. I'm really looking for encryption of all data without support on (SMB, etc) clients. My expectation is that encryption shouldn't get in the way of ZFS recovering from the failure cases it's designed for, but I understand that in more unexpected scenarios it may get in the way of data recovery (for me this is covered by my - encrypted - off-sites).

I appreciate FreeNAS is likely overkill for me, but as I mentioned, the other alternatives which provide assurance against bitrot don't seem as mature, so while that's an argument against using FreeNAS there also seem significant arguments against the alternatives. (Edit: also btrfs only has mirroring as a stable feature rather than parity so offers less space efficiency.) I would expect that slow-down due to running maintenance tasks during the day should be acceptable given my light usage expectation, low performance requirements and the fact that FreeNAS is generally considered to be a 24/7 available solution so should be able to run acceptably during maintenance. I'm happy to test this for myself but I'm more concerned that if scrubbing doesn't complete then I may lose bitrot detection/protection. Of course, I could test this for myself too but if it's a predictable issue (if scrubbing won't resume from where it was after shutdown) then I'd rather identify that now.

To be clear on the UPS - I meant likely after build/stress-testing but before putting live data on it.

Again, thanks!

 

[pfft]

Come to think of it, I suppose I could set up a single data drive test system (since ZFS checksumming/corruption detection should still work on one disk) with a large spare HDD (yeah, I bought a WD Green too :( ), stick some data on it and test the results of interrupting scrubbing with a shutdown myself.

But if someone just knows then that's a lot easier :)

 

[danb35]

An alternative would be btrfs on Linux (which is particularly tempting as I use Linux widely on my "real computers") but I'm concerned that btrfs doesn't seem nearly as mature as ZFS on FreeBSD.

What about ZFS on Linux? It's quite mature, and would let you stick with an OS you're more comfortable with--though I didn't find the Linux -> BSD transition to be too painful, at least not so far.

I'm not completely sure if this really means "continue from where it left off" or "start again".

It resumes where it left off.

 

[pfft]

What about ZFS on Linux? It's quite mature, and would let you stick with an OS you're more comfortable with--though I didn't find the Linux -> BSD transition to be too painful, at least not so far.

Oh, it should be fine. I've actually done a fair bit of work on FreeBSD systems in the past (quite a few years ago) but I never went into it deeply. ZFS on Linux seems more "niche" and my laziness leads me to avoid less widely used technologies - Googling for error messages is less effective :)

It resumes where it left off.

Excellent, thanks.

I think I'll give the FreeNAS build a try. I can always migrate to something else - and I promise I won't blame FreeNAS for me using it for the wrong thing :)