- Joined
- Dec 11, 2015
- Messages
- 1,410
Hello Forum!
Writing this post has been a WIP throughout my research on doing a “done right” setup, while using my current hardware. This text is me sharing my thoughts & muddling through into FreeNAS world while managing a migration from an existing windows based solution without the ability to “build an extra NAS” but, rather – converting present hardware in conjunction with new vital parts into a “done right” FreeNAS box. So this is intended for fun reading/sharing-the-newbie-experience, an extended introduction to questions at the end. Enjoy it all or "TLDR;;questions at the bottom"
The short story to why I ended up here:
-NTFS has gotten the best of me.
So while the official FreeNAS videos on YouTube where anything but appealing to me, I set out to watch numerous other contributors. After a while I realized that a crap load of “internet experts” where giving quite contradictory advices on hardware requirements (in particular based around the appeal of – using scrap parts building a cheap fileserver box). This lead me to checklist the rollercoaster ride to current level of insights of freenas:
Ambition: Building a "done right" setup with smart priorities to fit my available recyclable resources and use.
User skill: Advanced home windows user (aka no sysadmin experience). No FreeBSD/Linux Experience.
Use case:
-The data is more or less “put it in the tank, don’t modify it, read it occasionally”.
- 2 windows machines/users (workstation / utility server).
- Windows backups
- Primarily CIFS/samba
-1 Gbit network will be used.
-Jails: I’d like to try out some of the plugins – owncloud, syncthing or something like that. I’m aiming to get OpenVPN to work in a jail too.
Case: Will be DIY, using MDF laying around, including a bunch of fans (I’ll share the results & pics when it is all completed. If my ideas work out as intended, there might be some others interested in this DIY solution!)
The storage configuration.
This is a bit tricky since most users appear to have enough number of drives in coherent sizes available (or ready to purchase). Far from the mishmash that I have.
I’ve been going over different raidz2 configurations using Bidule0hms calculator, to get a taste of what usable storage that comes with different solutions. I found this very helpful, since ZFS+raidz2 indeed consumes quite a lot more storage from each purchased drive than I’m used to, from NTFS\jbod.
Doing some research and fiddling around with numbers I actually got a lot of my initial questions regarding storage strategy answered. I’ll share this for anyone interested. Here’s how that reasoning turned out in numbers:
I’m willing to purchase additional drives in order to not lock myself into a upgrade frenzy shortly.
Here’s how I’ve approached the problem.
I first set out to explore the number of drives in each typical vdev and the resulting usable space.
A brief overview from Bidule0hms calculator gives:
1x 6drive Raidz2 on 6tbs: 18,8TB usable.
1x 7drive RaidZ2: on 6tbs = 23.62TB
I’ve envisioned myself having around 20TB of usable space (which also complies to a cyberjock recommendation, and well beyond "3x what you think you need the next couple of years" :D
How to migrate into the FreeNAS box and expand?
Since I’ve no place to dump all my current data – I must carefully think through the processes. Two main ideas – either start with the smaller drives, or start with the larger drives:
I’ve a couple of questions regarding CPU/RAM/Motherboard:
CPU: Here’s where I think it is wise to save money in my scenario. A E3-1230v5 is 4.5x the price of a G4400. I’m definitely leaning towards the G4400, if it is not instantly deemed to be completely out question for any reason I’ve not found yet? (encryption have been indicated as not being part of FreeNAS 10, which means the AES support won’t matter?)
RAM: I would like to start with 16gb of ram. I guess it is probably on the thin side, but remember the use case of the box. Please chime in, since I’ve no experience what so ever here.
I like the X11SSL-F-B. It seems to be a cost efficient board while featuring IPMI, Dual i210 GbE, and a good portion of expandability. Any comments on that choice?
Edit: Bonus question:
- I've read all over the place that 80% is about as much you'd want to fill a pool before ZFS goes nuts.
I guess that part of that recommendation has to do with the COW functionality. Now in my usecase, I wont have a lot of edits, but mostly reads of the data. Does the 80% "rule" apply as 'firmly' to this type of "last location storage tank" use?
Cheers
Writing this post has been a WIP throughout my research on doing a “done right” setup, while using my current hardware. This text is me sharing my thoughts & muddling through into FreeNAS world while managing a migration from an existing windows based solution without the ability to “build an extra NAS” but, rather – converting present hardware in conjunction with new vital parts into a “done right” FreeNAS box. So this is intended for fun reading/sharing-the-newbie-experience, an extended introduction to questions at the end. Enjoy it all or "TLDR;;questions at the bottom"
The short story to why I ended up here:
-NTFS has gotten the best of me.
So while the official FreeNAS videos on YouTube where anything but appealing to me, I set out to watch numerous other contributors. After a while I realized that a crap load of “internet experts” where giving quite contradictory advices on hardware requirements (in particular based around the appeal of – using scrap parts building a cheap fileserver box). This lead me to checklist the rollercoaster ride to current level of insights of freenas:
1. Getting bored by official FreeNAS videos. Waay too sleepy and dense for a complete newbie/home user.
2. Realizing most youtubers advice cannot be trusted what so ever. Everyone is an expert.
3 Changing approach to watching given speeches on ZFS. Way more interesting and appealing.
4. Finding out ECC memory is a must. That put my old fileserver gear out the window.
5. Reading the manual cover to cover. Twice. Do not understand most of it. Yet hoping for some information to get stuck in my memory for further reference.
6. Realizing the significant differences to how storage can be expanded/reduced compared to solutions I’ve used.
7. Oh shit – my windows logics do not apply here what so ever.
8 Mini-ITX Asrock C2550D4I is available at local store – WIIHAA!
9. Realizing I just bought a LSI 9201-16i that would then be useless :’(
10. Only at this stage I really get into the forum starting to read posts. (no idea how I missed the forums)
11. I find the awesome helpful posts on hardware recommendations. Cannot believe it took this long down the line. So much research in erroneous directions could’ve been avoided.
12. Needing further expandability PCIe slots (extra HBA/NIC) – Looking for X10 Motherboards. Turns out cost for low end cpu+board is roughly comparable to the C2550D4I solution.
13. Finding out X11 motherboards are the most recent. A few key posts explain the issues of skylake. Realizing this forces another SSD or 2 to be used as boot drives due to XHCI lack of drivers in current FreeNAS version.
14. Doing PSU-calculations and research on efficiency of PSUs. Very interesting.
15. Realizes a migration really must be planned in conjunction with a pool expansion strategy over the next years. Getting to work crunching numbers.
16. Feeling confident enough to make a first post.
2. Realizing most youtubers advice cannot be trusted what so ever. Everyone is an expert.
3 Changing approach to watching given speeches on ZFS. Way more interesting and appealing.
4. Finding out ECC memory is a must. That put my old fileserver gear out the window.
5. Reading the manual cover to cover. Twice. Do not understand most of it. Yet hoping for some information to get stuck in my memory for further reference.
6. Realizing the significant differences to how storage can be expanded/reduced compared to solutions I’ve used.
7. Oh shit – my windows logics do not apply here what so ever.
8 Mini-ITX Asrock C2550D4I is available at local store – WIIHAA!
9. Realizing I just bought a LSI 9201-16i that would then be useless :’(
10. Only at this stage I really get into the forum starting to read posts. (no idea how I missed the forums)
11. I find the awesome helpful posts on hardware recommendations. Cannot believe it took this long down the line. So much research in erroneous directions could’ve been avoided.
12. Needing further expandability PCIe slots (extra HBA/NIC) – Looking for X10 Motherboards. Turns out cost for low end cpu+board is roughly comparable to the C2550D4I solution.
13. Finding out X11 motherboards are the most recent. A few key posts explain the issues of skylake. Realizing this forces another SSD or 2 to be used as boot drives due to XHCI lack of drivers in current FreeNAS version.
14. Doing PSU-calculations and research on efficiency of PSUs. Very interesting.
15. Realizes a migration really must be planned in conjunction with a pool expansion strategy over the next years. Getting to work crunching numbers.
16. Feeling confident enough to make a first post.
Ambition: Building a "done right" setup with smart priorities to fit my available recyclable resources and use.
User skill: Advanced home windows user (aka no sysadmin experience). No FreeBSD/Linux Experience.
Use case:
-The data is more or less “put it in the tank, don’t modify it, read it occasionally”.
- 2 windows machines/users (workstation / utility server).
- Windows backups
- Primarily CIFS/samba
-1 Gbit network will be used.
-Jails: I’d like to try out some of the plugins – owncloud, syncthing or something like that. I’m aiming to get OpenVPN to work in a jail too.
Case: Will be DIY, using MDF laying around, including a bunch of fans (I’ll share the results & pics when it is all completed. If my ideas work out as intended, there might be some others interested in this DIY solution!)
The storage configuration.
This is a bit tricky since most users appear to have enough number of drives in coherent sizes available (or ready to purchase). Far from the mishmash that I have.
Some points to take into account that serves as a checklist/decision making reminder:
-vdevs cannot change number of drives. Thus think far ahead.
-zpools can stripe several vdevs, with equal number of drives in each vdev.
-different size drives can be used within each vdev. The smallest and or slowest drive determine size/performance of the vdev.
-once a vdev is added it cannot be removed from zpool.
-zpool can grow in 2 ways: replacing smaller disks with larger, adding (preferrably) equal number of hdds in a new vdev, added to the pool.
-multiple zpools can be used.
- Bidule0hm’s calculator https://jsfiddle.net/Biduleohm/paq5u7z5/1/embedded/result/ to get an idea of usable space.
-vdevs cannot change number of drives. Thus think far ahead.
-zpools can stripe several vdevs, with equal number of drives in each vdev.
-different size drives can be used within each vdev. The smallest and or slowest drive determine size/performance of the vdev.
-once a vdev is added it cannot be removed from zpool.
-zpool can grow in 2 ways: replacing smaller disks with larger, adding (preferrably) equal number of hdds in a new vdev, added to the pool.
-multiple zpools can be used.
- Bidule0hm’s calculator https://jsfiddle.net/Biduleohm/paq5u7z5/1/embedded/result/ to get an idea of usable space.
I’ve been going over different raidz2 configurations using Bidule0hms calculator, to get a taste of what usable storage that comes with different solutions. I found this very helpful, since ZFS+raidz2 indeed consumes quite a lot more storage from each purchased drive than I’m used to, from NTFS\jbod.
Doing some research and fiddling around with numbers I actually got a lot of my initial questions regarding storage strategy answered. I’ll share this for anyone interested. Here’s how that reasoning turned out in numbers:
I’m willing to purchase additional drives in order to not lock myself into a upgrade frenzy shortly.
Here’s how I’ve approached the problem.
I first set out to explore the number of drives in each typical vdev and the resulting usable space.
A brief overview from Bidule0hms calculator gives:
1x 6drive Raidz2 on 6tbs: 18,8TB usable.
1x 7drive RaidZ2: on 6tbs = 23.62TB
I’ve envisioned myself having around 20TB of usable space (which also complies to a cyberjock recommendation, and well beyond "3x what you think you need the next couple of years" :D
How to migrate into the FreeNAS box and expand?
Since I’ve no place to dump all my current data – I must carefully think through the processes. Two main ideas – either start with the smaller drives, or start with the larger drives:
I’ve a couple of questions regarding CPU/RAM/Motherboard:
CPU: Here’s where I think it is wise to save money in my scenario. A E3-1230v5 is 4.5x the price of a G4400. I’m definitely leaning towards the G4400, if it is not instantly deemed to be completely out question for any reason I’ve not found yet? (encryption have been indicated as not being part of FreeNAS 10, which means the AES support won’t matter?)
RAM: I would like to start with 16gb of ram. I guess it is probably on the thin side, but remember the use case of the box. Please chime in, since I’ve no experience what so ever here.
I like the X11SSL-F-B. It seems to be a cost efficient board while featuring IPMI, Dual i210 GbE, and a good portion of expandability. Any comments on that choice?
Edit: Bonus question:
- I've read all over the place that 80% is about as much you'd want to fill a pool before ZFS goes nuts.
I guess that part of that recommendation has to do with the COW functionality. Now in my usecase, I wont have a lot of edits, but mostly reads of the data. Does the 80% "rule" apply as 'firmly' to this type of "last location storage tank" use?
Cheers
Last edited: