First FreeNAS Build

[Kenfolk]

Hello all, after months and months of holding off, it is finally time for me to build my first NAS. I've been doing a lot of research on the subject, but there is a lot of different options out there.
I currently have 5 external drives plugged into my PC between 4-5TB, and they are filling up very quickly. Instead of continuing to plug more drives in, which I don't want to do because my PC performance is suffering especially on startup with all these drives, I would like to move to a box where everything is centralized instead of being run through my computer.

My budget is $3,000.

I plan on using this build to store files, more specifically large video files. I also want to be able to use a torrent client to seed some of the content I have. I doubt I'll use plex, instead I would like to be able to just stream the data, play the video files, to my computer and let my computer do all the heavy lifting.

My very first question is, is FreeNAS the right platform for me to build this, or would another NAS OS be better served for what I'm trying to do with the storage size (44TB usable) I'm wanting.

I would like 44TB of usable storage, so I was thinking of using some 4TB WD Reds. This brings me to my first question, what Raid level is most feasible? I will have everything backed up on externals in case the worst happens. I've been reading that for larger arrays RaidZ2 and 3 might not be the best, so I'd like some input on what others might think I should do.

I'm also unsure on what chasis I should get. I've been looking at the different supermicro chasis, but to be honest I'm not really sure which one(s) would suit my needs best.
Unless anyone thinks differently, I'm guessing 64GB of ECC Ram is what I should be aiming for.

 

[CraigD]

FreeNAS is all about data security, it will do everything you need it to.

For 44TB of usable storage at a minimum I would get 16 drives and create two 8x5TB devs in RAIDz2, this will give you 48TB of usable space, a single 8x10TB RAIDz2 vdev gets the same result, however rebuild times are way longer

This means a case that can house this many drives is needed, and you will need at least one HBA

Max your RAM out if you can, the more you get the better

Have Fun

 

[Jailer]

There is no way you are going to get 44TB of usable storage and build a system for under 3 grand. Drives alone will blow your budget.

 

[Mirfster]

I've been reading that for larger arrays RaidZ2 and 3 might not be the best, so I'd like some input on what others might think I should do.

Either RaidZ2 or RaidZ3 would be fine for this. The amount of vDevs you have would have some impact of the IOPS. So you would want to consider how many actual drives and vDevs you would need to result in the desired 44TB of usable storage.

Of course there is consideration as to how many drives your case can hold, which would be a factor as well.

Try using @Bidule0hm 's "ZFS RAID size and reliability calculator" to get some numbers.

 

[Mirfster]

There is no way you are going to get 44TB of usable storage and build a system for under 3 grand. Drives alone will blow your budget.

If you are a cheap old fart like me and buy used/old (along with crafty shopping) it can be done. But would still be pretty close to the allocated budget. ;)

 

[nojohnny101]

welcome to the forums!

Hello all, after months and months of holding off, it is finally time for me to build my first NAS. I've been doing a lot of research on the subject, but there is a lot of different options out there.
I currently have 5 external drives plugged into my PC between 4-5TB, and they are filling up very quickly. Instead of continuing to plug more drives in, which I don't want to do because my PC performance is suffering especially on startup with all these drives, I would like to move to a box where everything is centralized instead of being run through my computer.

My very first question is, is FreeNAS the right platform for me to build this, or would another NAS OS be better served for what I'm trying to do with the storage size (44TB usable) I'm wanting.

First it sounds like a NAS is exactly what you need. The larger question of which one to use depends on a lot of factors including by not limited to:
- desired stability
- desire for integrity and protection of your data
- community support in case of problems
- cost
- technological know-how
- what the NAS' primary tasks will be

Now I can't speak for other NAS OSes because I only have experience with FreeNAS but what I can do is list a couple of reasons why others normally pick FreeNAS over others.
- data integrity. FreeNAS is based off of FreeBSD and Open ZFS. Both of these structures are incredibly mature and stable in their own regards (when use with proper hardware). Open ZFS originated from ZFS which was developed by Sun Microsystems as an enterprise level file system. features such as snapshots, check-summing, COW, are all advantages that work very well to keep data safe.
- active development. there is a solid pace to development with FreeNAS in regards to added features and bug fixes. you'll have strong confidence you won't be left out in the cold.
- strong community. these forums are very active and the people here are incredibly knowledgable.
- user friendly. i have found the GUI for FreeNAS fairly straightforward and easy to use. there is also a very well written manual that provides details on everything that in the GUI as well as additional helpful information.

the above list is not exhaustive (obviously) but are some popular reasons why.

It sounds like you have done a good amount of research before posting which puts you way ahead most people who post as first-timers on here (including myself!). Seems like you have the drive and skills to handle FreeNAS just fine.

I would like 44TB of usable storage, so I was thinking of using some 4TB WD Reds. This brings me to my first question, what Raid level is most feasible? I will have everything backed up on externals in case the worst happens. I've been reading that for larger arrays RaidZ2 and 3 might not be the best, so I'd like some input on what others might think I should do.

as others have said, you probably will want to go with raidz2 or raidz3. In choose your pool setup, it is always as a balance between performance and data redundancy. mirrors give you greatest throughput while sacrificing your bang for your buck (only 50% usable storage in a 2 way mirror, 33% in a 3 way mirror, etc.) while raidz3 gives you incredible drive failure tolerance but not very good speed (although raidz3 is acceptable in circumstances where you care very much about data protection and the performance hit won't affect you). Thera are advantages/disadvantages to both but I would say raidz2 would probably be more appropriate in your case. you have backups which is good (raidzx is never a substitute for a proper backup plan). As others have said, if you are looking at 44TB usable (this is quite large) then you'l want to strip multiple vdevs together to get the data protection of losing 2 drives while painting reasonable resilvering times and IOPS. I don't know what you have already read but here are some great thread that you should read:
ZFS Primer
Cyberjock's Presentation explaining pool, ecc ram, vdevs, etc.

I don't know much about the supermicro hardware so i can't comment there but I will echo what others have said, the more RAM you can have the better!

Good luck!

 

[SweetAndLow]

You could possibly do 10x8TB disks in raidz2. You will be pushing the rebuild times. Other wise you should go with 2vdevs with 8 disks in raidz2.

Sent from my Nexus 5X using Tapatalk

 

[Mirfster]

(only 50% usable storage in a 2 way mirror, 66% 33% in a 3 way mirror, etc.)

FTFY. Otherwise I am going with 4-way mirrors to get 75% usable storage... ;)

 

[gpsguy]

FTFY

Otherwise I am going with 4-way mirrors to get 75% 25% usable storage... ;)

 

[Mirfster]

FTFY

Ha.. Punk... :P

 

[joeschmuck]

@Kenfolk
With respect to the storage requirement of 44TB, how much do you need now and then when would you need more?

I ask because if you wanted to, you could first create a vdev of 8x5TB drives in a RAIDZ2 which will give you approximately 26TB of usable storage. This will ease the initial pain in your wallet. You can then add an additional set of drives in a second vdev to obtain more storage, so you could easily double that storage when you could afford more hard drives. So if you take baby steps, you could get there with less financial pain.

The hard drive quantity and sizes are examples, you could go a few different directions but some directions are just a little more sound.

And someone will correct me if I have this info wrong.

 

[Kenfolk]

If I have to go above the $3000 limit, so be it, but I didn't realize I could do multiple vdevs which could save me. I was thinking I would have to do this all at once.

If I created multiple vdevs, would they have to be the same size? I only ask because right now because I'm using 20TB, and would like to jump up to maybe 9X5TB drives and then later do something less.

I was only thinking about supermicro equipment because I saw that on the recommended hardware page. Would someone recommend something else?

 

[joeschmuck]

You do not need to use vdevs of the same size. You could start with a vdev of 20TB and later add a 10TB vdev, then a 25TB vdev, then a 7TB vdev. All this is for a single pool.

I'm not the vdev expert so I'm not sure how you would remove a vdev once it's been added, if that is even possible. So lowering the storage size isn't a typical thing and as far as I know, the only way to do that is to destroy the pool and recreate it.

An option could be to create two pools and then you could just delete one when you don't need all that storage.

So try to be specific in exactly what you would like to do, tell us how many drives you want in your first build and then how much storage you want for your second build/upgrade and if you want the ability to remove that storage without affecting the other one.

 

[Nick2253]

I'm not the vdev expert so I'm not sure how you would remove a vdev once it's been added, if that is even possible. So lowering the storage size isn't a typical thing and as far as I know, the only way to do that is to destroy the pool and recreate it.

Data in a pool is striped across vdevs. To the best of my knowledge, once you add a VDEV to a pool, you can't get rid of it without losing the pool.

 

[danb35]

Data in a pool is striped across vdevs. To the best of my knowledge, once you add a VDEV to a pool, you can't get rid of it without losing the pool.

Correct. I remember hearing there's some movement among the OpenZFS devs to implement vdev removal, but it certainly isn't implemented in FreeNAS at this time.

 

[Kenfolk]

I'm trying to store files, mainly large video files, that I will be able to access on my home network from any computer (I'll let the computers play the files, i.e. I don't want to run a Plex server or anything like that). My plans don't include removing or deleting any of the storage, I will only be upgrading.

It it sounding like RaidZ2 will be my best best. It looks like the price right now on newegg for 5TB WD Reds are over $100 more than the 4TB, and I often see sales on the 4TB. I'm thinking for first build 10x4TB in RaidZ2 will give me 8 usable which should be around 29TB usable. If I can see the 5TB WD Reds go on sale for a fair price, I'd probably more likely prefer 9x5TB in RaidZ2 which should give me a little over 31TB usable space. Then I can save up to add on as needed, though I'm hoping this will buy me at least 1 year if not 2.

 

[joeschmuck]

Well 29TB is a lot of storage. If this is something like BluRay rips then I'd consider ripping only the main movie and using an MP4 container. This can drastically save you a ton of space, dropping a normal movie down to about 4 to 5GB with high quality settings. If you haven't done this before, I highly recommend you at least try it and compare the video quality at a normal sitting distance. DVD rips are in the 1 to 2GB range for a typical movie.

 

[Mirfster]

On thing to keep in mind is that pretty soon there will be "Black Friday" and "Cyber Monday". If you can wait that long, then you will get more bang for your buck...

 

[joeschmuck]

There are always sales if you are willing to wait for one but you also need to figure out what your threshold of pain is for the cost of a drive.

 

[Kenfolk]

Yea, I'm definitely going to wait for a sale. I get e-mails from newegg about sales so I'll probably wait until I see something from them. I am honestly hoping to not drop more than $2000 on drives, but am willing to go up a couple hundred if that is absolutely necessary.

I agree it is a lot of storage. I would say maybe almost half is blu-ray rips of movies, but I also have blu-ray rips of TV shows, and a not small amount of anime. Plus I've recently purchased a 4K camcorder that eats up storage and figure to see more 4K content rolling out sooner than later that I would like to have a little infrastructure in place to handle.