Please Critique my Setup for Improvement

[n00b-beginnings]

I have a possible drive change coming due to a possible failure (different post...) and I am trying to take this time to correct *any* issues I may have with my admittedly hurried first-ever FreeNAS build. I've attached some screenshots.

One of the questions I have is, If I replace ADA5, what size drive would be "best" - what if I replaced it with a 4TB vs a 2 or 3 TB or even with an 8TB etc
Another is - am I "Wrong" for mixing and matching brands / sizes etc

ADA0 and ADA6 are 2.5-inch drives installed on PCI > SSD/2.5 HDD adapters- thoughts? should they be a separate pool of their own?

Please point out anything that *should be* done differently / best practices etc

 

[Pitfrr]

For the replacement of the drive, the size does not matter as long as it is equal or bigger to the replaced drive.
So then which drive you choose is depending on what you want to do... If you plan to expand your volume, then I would advise to go for a bigger one, with a 8TB drive you have a nice capacity/price ratio and then slowly exchange each drive until all are changed and you can expand your volume.

Mixing brand is not wrong. But mixing size... in a pool all drives should have the same size (unless you plan to expand that pool and exchange the drives one by one as said earlier).

When choosing a drive, just be careful to stay away from the SMR drives (that is independently of the brand).

From your screenshot I have the impression that you are mixing drive capacity in your pool: 4x 1TB together with 1x 3TB and 1x 256GB if I see right.
That means you probably don't have any redundancy in your pool and that is rather bad because if any drive fails, you'll loose all your pool. I hope you're aware of that...

 

[n00b-beginnings]

For the replacement of the drive, the size does not matter as long as it is equal or bigger to the replaced drive.
So then which drive you choose is depending on what you want to do... If you plan to expand your volume, then I would advise to go for a bigger one, with a 8TB drive you have a nice capacity/price ratio and then slowly exchange each drive until all are changed and you can expand your volume.

Mixing brand is not wrong. But mixing size... in a pool all drives should have the same size (unless you plan to expand that pool and exchange the drives one by one as said earlier).

When choosing a drive, just be careful to stay away from the SMR drives (that is independently of the brand).

From your screenshot I have the impression that you are mixing drive capacity in your pool: 4x 1TB together with 1x 3TB and 1x 256GB if I see right.
That means you probably don't have any redundancy in your pool and that is rather bad because if any drive fails, you'll loose all your pool. I hope you're aware of that...

Correct. This is just a striped pool with no redundancy. The 256GB is actually a cache. (SSD)

I did want to take the opportunity to possibly add redundancy and increase storage capacity.

I only have 3x 3.5 bays available.

2x 2.5 via PCI card adapters

(Additional 3.5 to 2x 2.5 adapter installed for boot and cache SSD)

With that in mind. I don't have terabytes of "mission critical" data. Maybe only around 1TB of stuff I really don't wanna lose. Of course this could (would) grow with time.

What would be the best approach if I wanted to use 3x 4TB or 8TB (or 10+)?

Is there any hidden issue mixing it up with one pool including the 2 2.5 inch HDDs on the PCI card?

Ive heard it can default to the "smallest drive" so if I had 3 10TB drives and 2 1TB drives in a pool with redundancy what could happen lol

 

[Pitfrr]

To get one thing clear: once you created a pool you can not change it (so you can't add redundancy or change a stripe to a RAIDz2 for example).
So you have to evaluate your needs, anticipate on the growth of your pool and decide what you want to have and also depending on your use case.

I assume this is for a home use, so nothing fancy.
Then the best thing to do is to keep it simple.

I'd go for a RAIDz2 pool, it offers good redundancy. Also a sweet spot is about 6 to 8 drives in RAIDz2, you have a nice ratio between usable capacity/space lost in redundancy.
You could go for 6x 8TB. You said your critical data is about 1TB, so you would think that you've more than enough space here, sure... but you might also consider setting snapshots and depending on the retention period, they can take some space. This capacity gives you room to plan for the future.
But sure, with 4TB drives you'll be good as well.

And your actual drives would be still useful (as long as they are not failing) for.... backups! (and this should not be underestimated)

I only have 3x 3.5 bays available.

You shouldn't plan your storage on how many bays you have available... I know... But the other way around actually: what kind of storage do you want and look for how many bays you're gonna need.

PCI card adapters

You mean PCI extension card with SATA controller? Well, you might want to be careful here since often the drivers for those cards a not well supported by FreeNAS. The advice in the forum is rather to use HBA cards instead.


If you create a pool with mixed drives, let's say 3x 1TB, 2x 3TB and 1x 10TB then the pool will be equivalent 6x 1TB drives. It takes the smallest drive size.

Also about the SSD for cache: do you need it?
Typically in a home setup most probably you don't. A good practice is also to maximize the RAM before adding a cache.
So you could reuse your SSD as booting device or eventually for VM storage if you use VM/plug-ins.

 

[n00b-beginnings]

You shouldn't plan your storage on how many bays you have available... I know... But the other way around actually: what kind of storage do you want and look for how many bays you're gonna need.

Agreed but this was a budget build and I plan to have "off site" backup or at least "out of the (FreeNAS) Box redundancy for any critical files. Even considering rclone crypt to cloud.

You mean PCI extension card with SATA controller? Well, you might want to be careful here since often the drivers for those cards a not well supported by FreeNAS. The advice in the forum is rather to use HBA cards instead.

Here is the product i'm using so far without issues ... so far
Sedna PCI Express (PCIe) SATA III (6G) SSD Adapter with 1 SATA III Port (with Built in Power Circuit, no Need SATA Power Connector, Best for Mac), SSD not Included

Also about the SSD for cache: do you need it?
Typically in a home setup most probably you don't. A good practice is also to maximize the RAM before adding a cache.
So you could reuse your SSD as booting device or eventually for VM storage if you use VM/plug-ins.

Well My 16GB of Non-ECC ram shows 0.5GB free from time to time. This is the maximum supported by the system I believe?
(System is an old ASUS CM1745)


I only have 3x 3.5 bays available

***These bays are all full with ada1, 4 & 5
So ada5 is failing. This is the one you suggested going up to 8TB on. Which would leave two bays "upgradable"

ada4 is 2TB I believe
ada1 is 3TB I believe


so if I replace ada5 with 4TB and then rclone / rsync / sync thing for "critical - must save" files does this work?

 

[Pitfrr]

I understand the budget constraints... and it's not always easy to find a good balance...

I am not familiar at all with the PCIe to SATA adapter you're using. I would say, it depends on the controller used on these boards if their driver works well with FreeNAS... There isn't much in the description about which controller they are using.
For example I know from experience that some Marvell controllers don't work properly with FreeNAS.

About the cache:
it is to be expected that all your RAM would be used. It is not because only 0,5GB is free that you need more cache.
Since the RAM is used as cache, the more you give to FreeNAS, the more it is going to use. For example I have a system with 32GB of RAM and only 2GB are free.
I would say with 16GB of RAM, for a home use, you should be fine. You could confirm that in having a look at the ARC hit ratio in the reporting tab. The higher the hit ratio, the better (that means the data in the RAM was used to serve the data request). So it could be quite high but that's also depending on your usage of course. I'd say for a home usage with one user it should be pretty high (unless you do lots of random accesses).
There is an old thread about this topic and some numbers.
--> That's when I realized my ARC hit ratio is quite "bad", at 60%!! I don't explain it because I'm not complaining, I'm very happy with the performance so... it's a bit odd.

Replacing:
Yes you should be able to replace your failing drive with a 4TB one.
But where I'm still not at ease here is that your pool doesn't have any redundancy. So you might first want to backup all your data and recreate a pool with redundancy.

Which would leave two bays "upgradable"

You might be able to upgrade the remaining bays with larger drives but to be able to expand the existing pool you need to change all drives from that pool with bigger ones...
Hence my previous recommendation to recreate your pool to add redundancy before otherwise you won't have any and I guess you want to avoid that.

ada4 is 2TB I believe
ada1 is 3TB I believe

In the screenshots you posted; ada4 is 1TB...