cloudycelt
Dabbler
- Joined
- Dec 30, 2019
- Messages
- 16
Hello all, I've been reading up on FreeNAS for some time and I'm finally at the stage where I'm ready to commit to buying and building a system. I've read up on some threads regarding particular initial areas of interest so I hope my questions come out making sense.
Firstly, my use-case:
I currently have no NAS in my home but I want to start to self-host a number of things such as the usual photos, media, movies, music, but also run a number of services that will need some backing persistent storage such as some type of git, NextCloud, Bitwarden RS, Kubernetes clusters and more, so databases are certainly in the equation.
Some requirements I made for myself:
- Physical space usage: I live in an apartment, so I don't have a basement to stash my "noisy equipment" in, so I've iterated through a few designs and figured that buying a cabinet (9U) and stashing it in a particular spot is my best bet, as that will allow me to rackmount most things for this home-lab-in-progress and keep the space usage confined to that cabinet including UPS and so on
- Noise: For the same reason as above, I've been selecting certain things to help with reducing noise, such as quieter fans and a taller chassis (2U vs 1U)
- Growth: I want to have room to expand the available storage in future without having to replace all the disks I am soon to buy
This is what I have come up with so far, I'm providing Amazon links where possible as requested in Josh's sticky post as well as a comment for each component.
I have yet to properly choose a boot disk as I have some questions/observations around utilisation of the remaining slots.
- I would ideally like to keep the remaining 4 SATA ports free to add more data disks down the road.
- I've heard it recommended that an SSD is preferable as a boot volume rather than a USB device and have looked over https://www.servethehome.com/buyers...as-nas-servers/top-picks-freenas-boot-drives/
- I've read https://www.ixsystems.com/blog/zfs-zil-and-slog-demystified/ and it sounds like I should at the very least have an SSD for use as a SLOG to make things feel speedy, which I could provide via an M.2 form factor SSD, my understanding is on a 1GbE network I can only write 625MB every 5 seconds so my SLOG does not need to be larger than that unless I use link aggregation or go to 10GbE, I see from https://www.servethehome.com/buyers...as-servers/top-picks-freenas-zil-slog-drives/ that ideally I should have a SLOG device with Power Loss Protection but that feature appears to be reserved for datacenter versions of SSDs that cost $400+ which is a little prohibitive as well as them being much larger in size than I'd actually need which feels like a waste.
- I've read https://www.ixsystems.com/community/threads/how-large-l2arc-for-64gb.45021/ which indicates I shouldn't have an L2ARC unless I have at least 64GB of RAM, which I won't initially but could down the road, so it seems preferable to leave room to add one
- The motherboard's two M.2 slots are on the PCIe 3.0 x4 bus, there is no slot for that
- The motherboard has a PCIe x 8 and PCIe x 16 slot so I could conceivably add an expansion card such as https://www.amazon.com/Mailiya-M-2-PCIe-3-0-Adapter/dp/B01N78XZCH/ to give myself another M.2 slot
So the question from these observations comes down to - what arrangement should I go for, given that I think I want an SSD boot device, a SLOG plus room for a future L2ARC?
1. Add an expansion card in the x8 slot to get another M.2 slot then use an M.2 each for boot, L2ARC, SLOG? How would they be arranged given the two existing M.2 slots share the PCIe x4 bus
2. Add an expansion card for the boot and then use the two M.2 slots in a pair for both L2ARC and SLOG as in https://clinta.github.io/freenas-multipurpose-ssd/ ? Feels like I would gain some redundancy with the pair but not against power loss which the UPS could help with, plus would putting both on the same disk impact performance, or would it not matter since the two slots are on the same bus anyway?
3. Some other arrangement? It seems like there's quite a few ways to arrange this
Thoughts on the overall build as well as the specific questions welcome! I know it maybe seems like overkill for an initial build but as mentioned I'm looking to host a decent number of things as well as running this for quite a few years rather than outgrowing it quickly.
Thanks in advance!
Firstly, my use-case:
I currently have no NAS in my home but I want to start to self-host a number of things such as the usual photos, media, movies, music, but also run a number of services that will need some backing persistent storage such as some type of git, NextCloud, Bitwarden RS, Kubernetes clusters and more, so databases are certainly in the equation.
Some requirements I made for myself:
- Physical space usage: I live in an apartment, so I don't have a basement to stash my "noisy equipment" in, so I've iterated through a few designs and figured that buying a cabinet (9U) and stashing it in a particular spot is my best bet, as that will allow me to rackmount most things for this home-lab-in-progress and keep the space usage confined to that cabinet including UPS and so on
- Noise: For the same reason as above, I've been selecting certain things to help with reducing noise, such as quieter fans and a taller chassis (2U vs 1U)
- Growth: I want to have room to expand the available storage in future without having to replace all the disks I am soon to buy
This is what I have come up with so far, I'm providing Amazon links where possible as requested in Josh's sticky post as well as a comment for each component.
Component | Part Name & Link | Comment |
---|---|---|
Cabinet | Tripp Lite 9U Wall Mount Rack Enclosure Server Cabinet | Chosen as it has a 20.5" mountable depth, so it will fit the server chassis and comes already assembled for convenience |
Chassis | SilverStone Technology 2U Rackmount RM21-308 | Chosen as it is a relatively short depth chassis but has the nice hot-swap bays and 3 80mm fans - many others I found only had 1 or used the squealer 40mm fans |
Motherboard | Supermicro X11SCH-F Micro-ATX | Chosen as it is Micro-ATX fitting in the chassis, friends highly recommend Supermicro, has 8 SATA3 ports and two M.2 slots, IPMI and uses the more recent Intel C246 chipset |
CPU | Intel Core i3-9100 Coffee Lake 4-Core 3.6GHz | Chosen partly due to the hardware recommendations guide plus this being the latest gen the board supports and the price seeming alright |
RAM | Crucial 16GB DDR4-2666 ECC Unbuffered x2 | Chosen as I wanted ECC UDIMMs and at least 32GB to start with, the board supports max 128GB in 4 slots but I couldn't find a 32GB DIMM of this sadly. Also the CPU actually only supports DDR4-2400 not 2666 but I think that just means the RAM will run at 2400, right? I didn't see any 2400 frequency for a similar price so went with this. |
PSU | Seasonic FOCUS PX-650 650W Platinum Modular | Chosen to have plenty of available wattage for when the system expands, as well as its physical dimensions (less than 160mm) for the chassis. Additionally my understanding is it will run quieter since it will be further from any kind of high load scenario. |
HDD Cable | Silverstone 36 Pin Minisas SFF-8087 to SATA cable | Chosen as it is apparently needed by this chassis to connect the backplane to the SATA ports on the motherboard |
Rail | Silverstone 20"" Ball Bearing Sliding Rail Kit RMS04-20 | Chosen as it should allow me to slide out the chassis from the cabinet rack if desired, which just seems like a nice-to-have. |
Chassis fan replacements | Noctua NF-A8 PWM 4-Pin 80mm x3 | Chosen as these came *highly* recommended by friends to reduce noise |
CPU fan replacement | Noctua NH-L9i Low-profile CPU Cooler for LGA115x | Chosen for the same reason as above. |
Data disks | Western Digital Red NAS 10TB 5400rpm x 4 | Chosen from the WD site as the Amazon page is full of reviewers saying they purchased from Amazon and WD refused to honour the full warranty saying the drives had been sold to Amazon years ago, so it seems best to order new drives direct from the manufacturer, is that correct? I chose 10TB each but wondering how much louder it would be to use the Pro 7200RPM drives and get them at a smaller capacity due to price. |
PDU | CyberPower CPS1215RM Basic PDU, 120V/15A | Chosen as it'll fit neatly into the cabinet rack and doesn't also have surge protection (which the UPS will provide) |
UPS | CyberPower OR1500LCDRM1U Smart App LCD UPS | Chosen as it seems to be a nice rackmount design with plenty of sufficient power. I saw only the 3 major brands APC, CyberPower and Tripp Lite around for UPS function but many of the reviews on the Tripp Lite of the same size says it makes noise all the time even when just recharging and I certainly don't want that. |
I have yet to properly choose a boot disk as I have some questions/observations around utilisation of the remaining slots.
- I would ideally like to keep the remaining 4 SATA ports free to add more data disks down the road.
- I've heard it recommended that an SSD is preferable as a boot volume rather than a USB device and have looked over https://www.servethehome.com/buyers...as-nas-servers/top-picks-freenas-boot-drives/
- I've read https://www.ixsystems.com/blog/zfs-zil-and-slog-demystified/ and it sounds like I should at the very least have an SSD for use as a SLOG to make things feel speedy, which I could provide via an M.2 form factor SSD, my understanding is on a 1GbE network I can only write 625MB every 5 seconds so my SLOG does not need to be larger than that unless I use link aggregation or go to 10GbE, I see from https://www.servethehome.com/buyers...as-servers/top-picks-freenas-zil-slog-drives/ that ideally I should have a SLOG device with Power Loss Protection but that feature appears to be reserved for datacenter versions of SSDs that cost $400+ which is a little prohibitive as well as them being much larger in size than I'd actually need which feels like a waste.
- I've read https://www.ixsystems.com/community/threads/how-large-l2arc-for-64gb.45021/ which indicates I shouldn't have an L2ARC unless I have at least 64GB of RAM, which I won't initially but could down the road, so it seems preferable to leave room to add one
- The motherboard's two M.2 slots are on the PCIe 3.0 x4 bus, there is no slot for that
- The motherboard has a PCIe x 8 and PCIe x 16 slot so I could conceivably add an expansion card such as https://www.amazon.com/Mailiya-M-2-PCIe-3-0-Adapter/dp/B01N78XZCH/ to give myself another M.2 slot
So the question from these observations comes down to - what arrangement should I go for, given that I think I want an SSD boot device, a SLOG plus room for a future L2ARC?
1. Add an expansion card in the x8 slot to get another M.2 slot then use an M.2 each for boot, L2ARC, SLOG? How would they be arranged given the two existing M.2 slots share the PCIe x4 bus
2. Add an expansion card for the boot and then use the two M.2 slots in a pair for both L2ARC and SLOG as in https://clinta.github.io/freenas-multipurpose-ssd/ ? Feels like I would gain some redundancy with the pair but not against power loss which the UPS could help with, plus would putting both on the same disk impact performance, or would it not matter since the two slots are on the same bus anyway?
3. Some other arrangement? It seems like there's quite a few ways to arrange this
Thoughts on the overall build as well as the specific questions welcome! I know it maybe seems like overkill for an initial build but as mentioned I'm looking to host a decent number of things as well as running this for quite a few years rather than outgrowing it quickly.
Thanks in advance!