BUILD High level of data security on a Low budget

[DGX]

As the title states: My aim is to build system that will be reliable without breaking the bank.

I live in Australia so I will likely be importing parts from the states… because technology in Australia is either overpriced, unavailable or both.


I am not really concerned about performance, provided it is not agonisingly slow. The system is for creating secure, reliable backups from my workstation and backing up my media library.

Questions:
1. Could someone that knows, verify my choices on this system build:
2. Is there any way I can drop the cost without sacrificing data-integrity?
3. If I use a Raidz2 vdev with 8 drives would the system run into issues?
…And if so of what sort (performance, reliability etc…) and how severe would they be?

Proposed Build:
CPU: Intel Box Pentium G3220 3.0GHz 2C 2T 3M S1150 (purchase planned)
RAM: Crucial 8GB PC3-12800 1600MHZ DDR3 240-Pin DIMM ECC Unbuffered CL11
Mfg Part# CT102472BD160B (purchase planned)
MOBO: Supermicro Intel Haswell Up Motherboard X10SL7-F (purchase planned)
PSU: Seasonic Power Supply SSR-360GP ATX 12 V.2.31 360W 80+ Gold PFC Retail
(purchase planned)
Hdd: WD 3.5" Red NAS 3TB WD30EFRX 64M SATA3 HDD x 6 (OR x 8) (already have 4 of these)
Case: Old Coolermaster ATX case fitted with new fans

I look-forward to your responses,
Cheers

 

[L]

You can always verify your own choices on the freebsd compatiblity list

https://www.freebsd.org/doc/faq/hardware.html

 

[DGX]

Thanks Linda,

There is reams of information in those links tho most of it seems directed around older technology. I would be happy using older technology with a storage card that is proven to have no issues with exceeding the 2.2TB limit on hard drives (mainly a drivers thing I imagine).


But I was under the impression that newer CPUs and chipsets had better power saving technologies. What’s important to me is that the system has been tested, known to be reliable and supported by the community… Last thing I want is random hardware compatibility issues

 

[mjws00]

Heh. You made it to the hardware forum. Trust the sticky. Cyberjocks post: Hardware recommendations (read this first). is definitive! :)

You are gonna see a bunch of supermicro options, and @joeschmuck 's AMD build. It is years worth of hardware arguing and advice distilled to one simple and modern post with nice links. Bottom line is it is more expensive than desktop junk, less expensive than "prosumer" gear. BSD doesn't have the widest or newest hardware support, so you are dead on targeting your hardware for this OS. Haswell and Avoton are power friendly.

The choices you made are fine though the powersupply is on the small side 550w might be better with 8 drives. The mobo is probably the best choice for an 1150 system. No gotchas, no extra junk, just proven to work. The cheaper options don't really sacrifice reliability as zfs handles that. Avoton basically has no vt-d and a whack of slowish cores. The 2550 will even get you there if you aren't transcoding. AMD is AMD, not my thing, but I believe you can trust joe's build.

Scrimping on platform/mobo is a false economy, imho. The lowend cpu is fine for most. Onboard HBA can save you buying one. The real cost is memory, and HDD's and there is no way out on those. Stick with the list and you won't have issues. Roll your own and you never know ;).

 

[Pharfar]

1. I'm sad to say, but I wouldn't recommend running a system with only 8 gb of ram. Considering how big a storage pool you're planning, you will need more (or rather ZFS will need more).
2. If you only need 6 drives (like I did), go for the x10slm-f. And use the money saved, to buy more ram.
3. 8 drives in raidz2 isnt optimal. Do some google-fu
http://christopher-technicalmusings.blogspot.dk/2012/05/zfs-raidz-make-sure-to-select-t.html

 

[L]

anecdotally I do hear people say they have problems with the supermicro servers, but what else is out there?? I sometimes buy quanta servers or dell with supermicro jbods. But I sell products that run all distros, zol, the freebsd distros and illumos and those hit the sweet spot.

As far as drives in a raidzX config, I was always skeptical in those raidz configs numbers. Knowing some zfs internals it didn't make sense. Matt Arhens blog describes why you don't need to worry about numbers of drives in a particular config.

http://blog.delphix.com/matt/2014/06/06/zfs-stripe-width/

 

[Pharfar]

Interesting read, Linda. He does recommend a minimum # disks / zfs level.

The strongest valid recommendation based on exact fitting of blocks into stripes is the following: If you are using RAID-Z with 512-byte sector devices with recordsize=4K or 8K and compression=off (but you probably want compression=lz4): use at least 5 disks with RAIDZ1; use at least 6 disks with RAIDZ2; and use at least 11 disks with RAIDZ3.

 

[mjws00]

Yep. That post should be quoted every single time someone mentions optimal disk numbers, imho. It's in the manual as well:

• Some older ZFS documentation recommends that a certain number of disks is needed for each type of RAIDZ in order to achieve optimal performance. On systems using LZ4 compression, which is the default for FreeNAS® 9.2.1 and higher, this is no longer true. See ZFS RAIDZ stripe width, or: How I Learned to Stop Worrying and Love RAIDZ for details.

I'm comfortable with Intel server boards. But they aren't nearly the value Supermicro is, and may or may not have their quirks. They are very reliable and implement to 'spec', features always lag. Used to love Tyan server boards, but haven't bought one in a decade. Asus makes server boards unfortunately I can't think of a reason to buy one over a SM, so won't know how great they are. AsRock Rack is getting pretty innovative in the space... with an expanding user base. So that competition is awesome. I'm pretty sure they will get a 'taste' of my dough when I am satisfied the guinea pigs have been slaughtered. ;)

For FreeNAS, running with the pack affords the highest level of support and security. Guess they got me drinkin the coolaid.

 

[L]

That's the reason to have a "pack"

 

[Pharfar]

If you did an upgrade from pre 9.2.1, you will not have compression on by default.

Code:

[root@papfar] ~# zfs get all /mnt/tank/delt/personligt/mads/ | grep compression
tank/delt/personligt/mads  compression           off                             default

But I salute Linda for sharing that link. It was a good read

 

[L]

Pharfar I just saw that in the 9.2.1.8 release notes

 

[DGX]

Thanks for the replies:
mjws00: That was really informative cheers;) I am still checking out the links

My thoughts (and inner augment) on PSU:
http://www.extreme.outervision.com/PSUEngine calculated this with my setup:
25% capacitor aging: Min= 318w, Recommended 368w
50% capacitor aging: Min= 381w, Recommended 431w

My basic calculation of amps on 12 v rail
HDD: 8 * 1.8 = 14.4
CPU: 55W / 12V / 0.8 (if the MOBO transformer about 80% efficient) = 5.73A
Fans: 4 * 0.125A = 0.5A
= 20.63 amps on 12volt rails at max load

The PSU (SSR-360GP ATX 12 V.2.31 360W 80+ Gold) specifies 30 as it’s max on 12v rail.
Most of the time the system will be in idle and wont anywhere near max…
A bigger PSU will surely cost more in electricity… don’t want to overload it tho as it has no redundancy.
However it will be fed clean power from a 3kva UPS… however it may crapout as the caps start to age
and 5 year warranty won’t get my data back.

I will up-size the PSU... to be on the safe side but I might need to drop to silver or bronze.

Pharfar: have you tested your system with 8GBs of ram and compared the performance to 16GBs of ram?

 

[DGX]

Pharfar: I am aware of the recommendations re number of drives for optimal performance, Hence I asked this question in my first post:

3. If I use a Raidz2 vdev with 8 drives would the system run into issues?
…And if so of what sort (performance, reliability etc…) and how severe would they be?

I am not sure if I am comfortable running 10 drives on raidz2 because the probability of multiple drive failure is significantly increased. but 12TB is not enough headroom.
Has anyone done any A B testing on performance problem associated with non-optimal drive configurations?

 

[DGX]

Thanks Linda, Matt's Blog was great

 

[mjws00]

Your math on the power supply is fine. If you are comfortable with the headroom, rock and roll. I like to have gear that I may be able to re-purpose for something else later. Good power supplies tend to stick around, imho. Everyone has their own priorities... a 450 may even be a good target. Meh ;) I've definitely gotten away with much much less (~200w) with 8-10 drives. But that is only because it was ancient gear that was just "there". Wouldn't ever risk that with someone else's drives or data.

The reality on performance with 6+ drives, is that you will be limited by your network. AB testing has to make assumptions about workloads and file sizes. Very few of which will apply to your intended use. In a nutshell the optimal math assumptions were based on fixed blocks. Real files especially with compression don't fit that model well. It is even less important on large sequential reads like a media file. So there isn't much downside to "run what ya brung".

I did split testing on zpool configs. i.e mirrored vs z2 but not 6 vs 8 vs 10. I'm down with the new school idea, but frankly 6 drives is the perfect width for anything I'd run in z2. I'd be on 2 * 6 drive vdevs before I'd run 8 or 10 wide. But with only 8 drives... and no plan to hit 12, hook em up. :) It is still 2 more spindles.

So even if you can produce a difference on a benchmark. If your pool throughput is 6x faster than your network... who cares? And if we are looking for iops the game changes. Grab the disks and test, it's a good excuse to grab 2 more drives. Unfortunately I've never seen applicable AB testing on this. Make sure you post if you find some.

 

[Pharfar]

You say that you want a high level of data security, but you dont follow recommendations. Maybe it will work, maybe it will work for 99% of the time, but you cant be sure. Some ppl have experimented with "low" ram setups:
http://forums.freenas.org/index.php?threads/running-zfs-with-8gb-ram-on-9-2-does-work-well.17343/
But if your data is important, then follow recommendations. Dont try to save money, it will not benefit you.

 

[DGX]

Your link shows me that people run freenas on 4GB of ram without problems and that others have had problems. But I can not find examples where where people are losing DATA because they are running 8GB of ram. if I find the speed frustrating I can purchase more. Are there any specific situations that 8GB of ram will cause system instability?

 

[Pharfar]

The system I linked to, had 6 TB of storage with 4 GB ram. You intend to have 18 TB of storage with 8 GB of ram. There is no way you will get any positive confirmation that "this will work". Knowing how active the forum crew is, I think it's a hint-hint that no one has written a reply. But it's your build, you decide.

 

[cyberjock]

The system I linked to, had 6 TB of storage with 4 GB ram. You intend to have 18 TB of storage with 8 GB of ram. There is no way you will get any positive confirmation that "this will work". Knowing how active the forum crew is, I think it's a hint-hint that no one has written a reply. But it's your build, you decide.

You should take his hint seriously. That was your clue that "there be dragons".

I have read this thread and watched it progress, and I'm deliberately avoiding it for good reasons.... if you want to do what you want, "good luck" is all you're going to get from me. We have tried to make it abundantly clear how much RAM you need for various activities. There is no reason why I should have to repeat myself for the 100th time. So take our advice, or ignore it. It's your data, your time, and your money on the line.

Just realize that if you ignore our warnings and recommendations and you are suddenly begging for help later because your data has mysteriously vanished nobody here will answer you. That's just a fact. We don't volunteer our time to people that didn't try to help themselves first.

 

[DGX]

I'm Sorry didn't mean to be ‘ignoring warnings’. The freenas guide says that 8GB is the min and that for home use “you can soften the rule of thumb”. I have been searching the forum for instance where 8GB has led to data loss and have not found any yet. I have, however found a lot people running systems on 8GBs without drama’s.

Please post a link to where I might find out more? cheers