mini-ITX build, 5/6 disk for home camera, media stuff, VMs and probably more

BFred

Dabbler
Joined
Sep 5, 2019
Messages
15
Hi!
I've currently been using an entry level Synology NAS with 4x4TB disks set up as RAID5 and am happy with it for the backups I do. Unfortunately after 1.5 years I am starting to do more with my NAS and realize that the Synology can't do it (or shouldn't).

So I'm looking at FreeNAS with similar size/power efficiency/noise levels to run DLNA, p2p stuff, ZoneMinder (currently 2 HD cameras), VMs (not so many), encryption maybe and probably more. I found the "NAS killer website" with some sample builds but the mini-ITX build uses non-ECC ram and it is beyond my level to start tweaking mainboads, CPUs, RAM and fans...

Storage wise I will probably get 4TB (or 6TB) disks again, maybe 5 this time for better redundancy (ZFS Raid6 equivalent then?). Definitely interested in suggestions about the disk setup I should go for.

I'll keep the Synology for backups and use the FreeNAS box to "run things" (which can break). I'm thinking of 16GB of RAM, 2x 1GB LAN ports (I don't think I need 10GB). Maybe SSD to boot and/or for cache and that's about it.

Could you kindly recommend a list of parts I should acquire within a reasonable budget ($300 to $500 without the disks)?

Thank you very much.

Fred
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,702
RAIDZ2 is not known for its high IOPS, so using it as storage for VMs is usually not a good idea. With camera recording 2 HD streams and P2P sharing using most of your available IOs, you're set up for being a very unhappy VM user.

With 16 GB of RAM, an SSD for cache is a terrible idea and you would be better off using it for the VM storage. 16GB is also pushing it for VMs together with the performance of the file hosting.

You need to think about as much RAM as you can put in the box for good performance. You need to have 64 GB of RAM before you think about a cache (L2ARC) SSD.

Do some reading in the resources section, such as the ZFS primer and hardware guides as it seems you're expecting ZFS to be the same as other RAID systems... it isn't. It's very reliable, but it can go very wrong if you don't do it the right way.

You will probably not find the gear you want as brand new for the prices you're talking about, but 2nd hand, you may be able to find a used server which will better meet the need.
 

BFred

Dabbler
Joined
Sep 5, 2019
Messages
15
Thank you very much for the tips! So maybe I lacked clarity on those VMs. Most will be temporary Linux file servers using 2GB of RAM at most when I use them. No plan to keep them running 24x7. Though you might be right in the sense that one year down the road (time flies doesn't it!) I might want to be running something else full time.

p2p runs night time or eventually when no one uses the network, and cam recording is off during night time - if that helps.

Now back to the disk setup would the equivalent of RAID 10 be a better choice? So no cache, RAID 10 and lots of RAM? Couldn't edit my post but I actually meant 5/6 disk in total like the subject says (not just 5). So I could start with either 4 or 6 I guess?

Indeed I've read a bit about second hand hardware (CPU/Mainboard mainly) and it seems to be a good option. So now that everything is better define, is it possible to come up with something in the mini-ITX format, quietness (maybe we can skip power efficiency) and pricing?

Thank you very much.
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,702
would the equivalent of RAID 10 be a better choice? So no cache, RAID 10 and lots of RAM?
If you can afford the capacity hit of going for a pool of mirrored VDEVs (RAID 10 equivalent) and that matches with your required level of fault tolerance (2 disks in 1 mirror lost means the whole show is over), then yes.

Couldn't edit my post but I actually meant 5/6 disk in total like the subject says (not just 5). So I could start with either 4 or 6 I guess?
In the Mirrored VDEV scenario, you can add Mirrored VDEVs to the limit of your hardware (disk controller and chassis) in the same pool, growing the storage and potential IOPS for the pool with each (be aware that a later addition of a VDEV to a pool means that the new VDEV will get almost all of the writes until the VDEVs are balanced with content again, so this is not great for IOPS... to avoid that you need to clear your pool and restore from backup with the final number of VDEVs in it... not as flexible as most people would like).

@Chris Moore loves shopping for old hardware... perhaps he will chip in.
 

BFred

Dabbler
Joined
Sep 5, 2019
Messages
15
mmmm... I feel I have "hard choices" to make as Mirrored VDEVs is not that convenient (I'm aware of the capacity hit). Do you have a different and maybe better option then? Would RAIDZ1 be better in that regard (IOPS)?

Thank you for asking Chris to join in ... hopefully I can get everything clear and start to go shopping ;-)

ps: note that I don't really plan to update the machine at least before 3+ years I believe. I don't think I'll need more disk space, a bit like my Synology which is doing fine and has enough power and space for what it does. So I'd rather eventually pay more to start with and be done with it for a few years (or get a 3rd NAS?)
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,702
Would RAIDZ1 be better in that regard (IOPS)?
No, RAIDZ1 has the same IOPS as 1 disk. (which is the same as RAIDZ2)

Do you have a different and maybe better option then?
No miracles, I'm afraid... what works well is to have a mirror of SSDs (or multiple mirrors depending on the performance need) to run your VMs and spinning disks to host static data and a backup of the VMs.

Also consider running all your services in jails instead of VMs, which will reduce the performance hit in terms of memory and IOPS requirement.
 

Chris Moore

Hall of Famer
Joined
May 2, 2015
Messages
10,080
I don't really have any input on mini-ITX systems. I think that board size is too small to be useful for anythnig due to lack of expandability. No expansion slots. I bought one exactly once and used it for about a year as a firewall / router. Then the board failed and I needed to buy a new one because it was out of warranty. I don't think they are fit for any purpose at all other than the very most basic of desktop PC use (not as a server at all) and even as a desktop computer, they are only good for the most basic user, like children in a school setting.
 

BFred

Dabbler
Joined
Sep 5, 2019
Messages
15
what works well is to have a mirror of SSDs (or multiple mirrors depending on the performance need) to run your VMs
So that SSD would be a separate partition from the ZFS disks in order to efficiently run the VMs? So I would be at 4/6 Disks under mirrored VDEVs and then 1 SSD for VMs. Boot is installed on the ZFS partition or elsewhere?

I think that board size is too small to be useful for anything due to lack of expandability.
Well I'm not thinking to expand much but I understand this is not what you're familiar with. Would mini-ATX be better then? I really need a small enclosure and don't plan on going more than 6 disks. I also read that mainboard prices for mini-ITX were high and you'd get much better offers for something bigger. Alas I am definitely not an expert in the matter and definitely open to "see the light" :smile: (I also understand that what I am planning now, and 12 months later can be very different).

Thank you guys for your patience and your help.

Fred
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,702
Boot is installed on the ZFS partition or elsewhere?
The boot drive must be a physically separate drive. USB is an option, but SSD is now recommended/preferred. Can't be used for storage.

So that SSD would be a separate partition from the ZFS disks in order to efficiently run the VMs?
Separate pool. If your VMs aren't going to be mission critical, a single SSD with a replication job to back them up would probably be OK... I would personally mirror them with a 2nd SSD, but that's a personal choice.
 

BFred

Dabbler
Joined
Sep 5, 2019
Messages
15
Great. Thank you very much for all your replies, it's really helping (I'm reading 28 threads per days for a week and there's a bit of everything in them - hard to be sure of anything at this stage).
I'm still looking for the possible hardware. You mentioned I would probably go over my budget with new hardware. Do you have some sample configuration in the "small box" kind of set up and maybe in the lower spec range I was planning for then?
Thanks again.
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,702
Here's a good indication of a mini-ITX build (from a year or two back)
https://www.ixsystems.com/community...node-304-x10sdv-tln4f-esxi-freenas-aio.57116/

You will get a bit of an idea from that of how messy it will be in a small chassis.

Going for a bigger box like a Fractal Node 804 and a m-ATX board gives you a lot more options for expansion (but takes up more room on your bench).
I expect you could find cheaper options at that size though.
 

BFred

Dabbler
Joined
Sep 5, 2019
Messages
15
Here's a good indication of a mini-ITX build (from a year or two back)
https://www.ixsystems.com/community...node-304-x10sdv-tln4f-esxi-freenas-aio.57116/
That looks very nice and without the SLOG (which I have no idea what it is), and a mainboard which seems to come with the CPU I found on Ebay new comes at $1200. Well at least I have some directions and maybe can fiddle around a bit to adapt it to either a bigger case (and cheaper mainboard?) or similar mainboard albeit cheaper.

Thank you. I need to do some research now, with the elements I have on hands.

Fred
 

Inxsible

Guru
Joined
Aug 14, 2017
Messages
1,123
I think that board size is too small to be useful for anythnig due to lack of expandability. No expansion slots.
Totally agree with this. If expansion is something you want in the future, stay away from mini-ITX boards. Also stay away because the mini-ITX server grade boards cost a lot more than comparable or better micro-ATX boards.
I don't think they are fit for any purpose at all other than the very most basic of desktop PC use (not as a server at all) and even as a desktop computer, they are only good for the most basic user, like children in a school setting.
I don't quite agree with this completely.

I do have a Tyan S5533 server grade mini-ITX board which is performing very well since the day I installed FreeNAS on it. More than 4-5 years ago now.

Would I buy a mini-ITX board again? No, not for servers(see expandability). But certainly cannot generalize that all mini-ITX boards are bad.
 

Inxsible

Guru
Joined
Aug 14, 2017
Messages
1,123
Definitely interested in suggestions about the disk setup I should go for.
You would be better served by keeping separate pools for your ZoneMinder and for your regular file server/media server.

The Zoneminder pool can be a mirrored pool for higher IOPS - which it would need. and your file server/media server can be a RAIDZ2 pool which has better redundancy. I would suggest getting a bigger case/rack mount case with say 8 or 12 drives capacity at least, and give 6 drives to the Zoneminder pool and 6 drives to your file server pool.

Depending on how much your 2 cameras record and how long you store that data -- you can choose to have smaller capacity drives (say 2 TB) and have 6 of them in 3 mirrors.

For your file server-- again depending on how much data you have, you could go for 4TB or 6TB or 8TB drives in RAIDZ2
 

BFred

Dabbler
Joined
Sep 5, 2019
Messages
15
Thank you for the additional information. I'm indeed searching for Micro-ATX boards now and shopping do take time (not my thing on top of that).

Likewise, thank you for the disks feedback. As I mentioned at the beginning I will be keeping my Synology for backups and using that new box for other purposes. 8 to 12 drives is a big step up from my initial plan o_O . So more considerations to add to my searches, but obviously I need several pools between VMs, Zoneminder and the other things I could be running, or at least keep the door open when building the box so it becomes possible 6 months down the road.
 

Inxsible

Guru
Joined
Aug 14, 2017
Messages
1,123
or at least keep the door open when building the box so it becomes possible 6 months down the road.
That's the right way to go about it !
 

BFred

Dabbler
Joined
Sep 5, 2019
Messages
15
Thank you for the additional information. I'm indeed searching for Micro-ATX boards now and shopping do take time (not my thing on top of that).

Likewise, thank you for the disks feedback. As I mentioned at the beginning I will be keeping my Synology for backups and using that new box for other purposes. 8 to 12 drives is a big step up from my initial plan o_O . So more considerations to add to my searches, but obviously I need several pools between VMs, Zoneminder and the other things I could be running, or at least keep the door open when building the box so it becomes possible 6 months down the road.
 

BFred

Dabbler
Joined
Sep 5, 2019
Messages
15
Ok great, I've found a thread talking about Supermicro x10 and found that one on Ebay: Supermicro motherboard X10SLM+-F Intel Xeon E3-1270v3 3.50GHz 16GB (4x4GB) for $180. I looked at the X10SLH-F too but it seems the main difference from my perspective is the SATA ports speed which is 4x6Gbps + 2x3Gpbs or 6xGbps for the later.
Either way that's only 6 ports, so how would I connect the extra SSD I'd like to fit in the box? What about extra disks I'd eventually would add later as well?
Thank you.
 

IQless

Contributor
Joined
Feb 13, 2017
Messages
142

BFred

Dabbler
Joined
Sep 5, 2019
Messages
15
Wow that was a fast answer! Great, new it seems to only cost about $55 so definitely affordable and all the internal SATA ports are at 6Gbps. So now from a rough estimate and assuming I replace the RAM by 4x8GB *extra $100* I am at:
  • X10SLM+-F mainboard + Intel Xeon E3 CPU: ~$180
  • RAM 32GB unregistered ECC: $105 (Hynix brand. Is it bad?) - I found Samsung ones at $156 otherwise
  • HBA card SAS 9207-8i: $60
  • Case: Fractal 804: $110
  • SeaSonic G Series SSR-550RM 550W Modular Plus Gold: $60-70
  • Built-in fans seems ok from what I read here
I'm at $515 total without the SSDs. I'd need at least 2 (one for boot, one for VMs), that's less than $100 depending on size. Do I need anything else to connect them to the HBA card? Anything you think won't fit or is missing?

Seems I'm getting somewhere now! (I'm going to run and check if the case fits where I want to place it)
Thank you for the help.
Fred
ps: shelves max height is 253mm and Fractal 804 height is 307... I could either put it on top or think more
 
Last edited:
Top