FreeNAS Memory Requirement and Multiple ZPools

[JTT0]

Hey all,

I have read CyberJock's PDF presentation and a good portion of the manuals, but am still looking for an answer to the following questions:

1) Can I create two separate ZPools? (i.e. one for a 7 drive RaidZ3 and 1 for a 1 drive striped)

2) With the following system, is my memory setup fine? My motherboard does not allow me to hit the 1GB per 1TB of Memory as suggested.

OS: FreeNAS (Latest)
Case: U-NAS NSC-800
Motherboard: ASRock E3C224D2I - http://www.newegg.com/Product/Product.aspx?Item=N82E16813157467
CPU: Intel Core i3-4130T - http://www.newegg.com/Product/Product.aspx?Item=N82E16819116947
RAM: Kingston 16GB (2 x 8GB) 240-Pin DDR3 1333MHZ) - http://www.newegg.com/Product/Product.aspx?Item=N82E16820239117
NOTE: My RAM is Not Intel Certified, this is the "Sister" module, seems the other is out right now. I bought the "G" Standard Model
PSU: Seasonic SS-350MU with Flex to 1U bracket
Raid Controller: IBM M1015
HDD: Eight 3TB WD Red (7 in a RaidZ3 and 1 as a standalone)

 

[ser_rhaegar]

1. Yes, just be careful that you don't accidentally stripe the extra drive into the pool with the RaidZ3 vdev or that'll be a single point of failure for the pool. You might try this in VirtualBox for testing just to make sure you perform everything correctly.

2. You should be well off with 16GB of RAM if this is a home system, it all depends on the workload of the system though, so if it is hosting files for many users or running VMs, things might look different performance wise.

 

[JTT0]

1. Yes, just be careful that you don't accidentally stripe the extra drive into the pool with the RaidZ3 vdev or that'll be a single point of failure for the pool. You might try this in VirtualBox for testing just to make sure you perform everything correctly.

2. You should be well off with 16GB of RAM if this is a home system, it all depends on the workload of the system though, so if it is hosting files for many users or running VMs, things might look different performance wise.

Thanks for the reply! This NAS is for home use only, basically a small family of 2 adults and 2 children. I have 2, maybe 1 more coming, HTPC's setup on the network and a gaming server, which is currently doing some minor transcoding using PS3 Media Server (Might bring in-house as a Jail). On the server, I am in the process of setting up a Git Jail as well as a OwnCloud Jail. Does any of this change the outcome of 16GB being enough? I hope not.

As for the multiple ZPools, that is awesome. I have been playing with it in a VirtualBox install and have used the ZFS Volume Manager to create two volumes mimicking my setup, so they show up SEPARATELY in the GUI, is this considered two separate ZPools? I REALLY do not want these two "sets" linked for avoiding failure issues since the 7 are used as the "Optimal Setup" for RaidZ# and the excess drive will merely be used as a backup drive and can easily be tossed or used as a hot swap in the event of a failure (It is basically Nice-To-Have data but not a priority).

Thanks again!

 

[ser_rhaegar]

Yes, 16GB is plenty for your usage. Definitely a good choice.

If done correctly, yes you will see two pools in the Storage -> Active Volumes section. If done incorrectly, you will only see one pool listed and it's space will have increased by the size of the drive you striped.

 

[JTT0]

Yes, 16GB is plenty for your usage. Definitely a good choice.

If done correctly, yes you will see two pools in the Storage -> Active Volumes section. If done incorrectly, you will only see one pool listed and it's space will have increased by the size of the drive you striped.

Awesome! Now, the difference between the two is simply Add new Volume vs Extend a Volume, correct?

 

[ser_rhaegar]

Should be, yes. Just leave the "Extend volume" field to the default "------" if using the GUI. And if using the CLI, use the GUI :)

 

[JTT0]

Also, I want to break up my Backups HDD/Volume into sub-sections for Time Machine on multiple machines to have a set amount of space. Is this best done through Datasets?

Volume: Backup (1 HDD Striped)
-- Dataset: Person A
-- Dataset: Person B
etc...?

 

[JTT0]

Should be, yes. Just leave the "Extend volume" field to the default "------" if using the GUI. And if using the CLI, use the GUI :)

Thanks! You're awesome!

 

[ser_rhaegar]

I have not configured a second machine to backup with TM on my system, yet. However my plan is to do exactly that.

Dataset: TimeMachine
Sub Dataset: Laptop1
Sub Dataset: Laptop2
Sub Dataset: iMac

With an AFP share named after each computer and pointing to that computer's dataset. Then you can set a quota on each dataset if you wish to limit the space allowed for backups.

 

[JTT0]

I have not configured a second machine to backup with TM on my system, yet. However my plan is to do exactly that.

Dataset: TimeMachine
Sub Dataset: Laptop1
Sub Dataset: Laptop2
Sub Dataset: iMac

With an AFP share named after each computer and pointing to that computer's dataset. Then you can set a quota on each dataset if you wish to limit the space allowed for backups.

That's exactly what the goal is :) I will be trying to configure all of this, this weekend on my live box. I have been waiting for one final ATX Power Extension Cable to arrive to finally close it up and run tests.

 

[David Dyer-Bennet]

The FreeNAS community seems to want a LOT more memory in these things than I ever ran with ZFS on Solaris.

In test mode, I'm doing fine with 4GB of RAM and a 6x4TB RAIDZ pool (20TB usable) on FreeNAS 9.2.1. I've never seen swap space used at all.

I've been running a much smaller system, now about 3TB, on 4GB of RAM under Solaris since 2006, with all my files (not just copies, I access nearly everything except software via CIFS).

(Despite this experience, this server is intended to have 16GB of RAM when it goes into production; we had an ordering glitch resulting in running it with a small bit of RAM stolen from elsewhere, and I'm pleased to see it doing okay. But the real server will have considerably higher peak usage, and the FreeNAS docs are so consistent about asking for LOTS of RAM that we're doing that.)

(Using deduplication does require horrendous amounts of RAM, but there's no point in deduplication in most home environments.)

 

[gpsguy]

Please don't advocate running ZFS on FreeNAS with anything less than 8Gb of RAM. That's the minimum amount specified in the manual. While 4Gb might be okay with Solaris or NAS4Free. With FreeNAS, running with insufficient memory could lead to a damaged and unmountable pool.

You'll probably need at least 16Gb for your pool. Hopefully, you bought 8Gb sticks of ECC RAM, since the board supports 32Gb.

 

[JTT0]

Please don't advocate running ZFS on FreeNAS with anything less than 8Gb of RAM. That's the minimum amount specified in the manual. While 4Gb might be okay with Solaris or NAS4Free, with FreeNAS. Running with insufficient memory could lead to a damaged and unmountable pool.

You'll probably need at least 16Gb for your pool. Hopefully, you bought 8Gb sticks of ECC RAM, since the board supports 32Gb.

I did buy 8GB sticks of Kingston ECC RAM as specified in my previous post. Unfortunately, my main reason asking this question is that I wanted to verify that 16GB would be enough since my setup is based on Mini ITX and the board is 1 of 2 that supports ECC (The other being it's sister board, the C226), but neither supported 32GB of RAM. I have maxed out my board with all the RAM I can give it until I upgrade in 3 or 4 years. Over time, probably 1 per year starting next year, I plan to slowly upgrade my HDD to 4TB to eventually increase the pool size. I went with 3TB as this currently doubles my current usage from 6TB (CURRENT) to 12TB (USABLE via RaidZ3 with optimal 7 HDD) with a 3TB set for Backup.

 

[gpsguy]

Sorry for the confusion JTT0 - I was responding to David's reply. His Asus mobo will accept 32Gb.


Sent from my phone

 

[JTT0]

Sorry for the confusion JTT0 - I was responding to David's reply. His Asus mobo will accept 32Gb.


Sent from my phone

Ah, thanks for the clarification! I have been sweating it since I JUST bought this system and spent literally three months doing solid research about hardware and Jails, etc. to find out that I missed the Memory requirements. After dropping 2 grand, you kinda want your system to work flawlessly :P

 

[HoneyBadger]

The FreeNAS community seems to want a LOT more memory in these things than I ever ran with ZFS on Solaris.

Solaris user here. While ZFS has a lot of commonality between OSes, the underlying OS can always have its own different level of requirements. But one thing that is consistent is that ZFS will perform a lot better with more RAM.

 

[David Dyer-Bennet]

Yes, I believed the documentation / forum and we bought two 8GB ECC sticks, even though that felt wrong based on my experience. I'm aware that FreeBSD isn't Solaris, and I don't know FreeBSD hardly at all. And I'm quite able to believe that pool size makes a difference to memory requirements even without dedup. So I told my gut to shut up about things it doesn't really understand (while continuing to question memory requirements when I could). And yeah, one consideration was that we could put in two more if really necessary.

I've been running with 4GB only because we made a sordid ordering screwup (which will be fixed before the box goes into production). I figured it was worth trying, so long as I remembered not to blame any flakiness that might possibly be memory exhaustion on anything else. In fact, there's been no flakiness (haven't had multiple users accessing different big files all at once or things like that so far).

I'm very impressed with the stability and performance of FreeNAS so far. I benchmarked it against my 2006-vintage Solaris fileserver and actually got significantly better sequential read throughput than the Solaris box. (Different pool layouts may well explain most of it; my next test is to configure the FreeNAS box to match my Solaris pool layout and see how they perform then.)
And this was all with memory that's inadequate by the manual (so bad results wouldn't really mean anything; good results, however, mean that with a proper configuration it should be at least that good).

(Yes, at some point I'm going to boot the Solaris box with a FreeNAS USB drive and do some gentle experimenting. It won't updgrade ZFS on-disk structures without my direct command, or at least without asking me, right? If I could end up running that box with FreeNAS also it would make my life simpler in a couple of ways.)