New to FreeNAS - Please critique my setup.

[SomeRandomBS]

Hello, I am new to FreeNAS but I feel like I have done my research. I would very much appreciate more advanced users opinions and any recommendations of my current configuration. I understand that running FreeNAS from a VM is not the most desirable configuration but I feel that I have done it in the best possible way, significantly reducing my risk of data loss.

Hardware:
ESXi run from 32GB USB
SuperMicro X10SRL-F
E5-4620 V3
128GB DDR4-2133 (Runs at 1867)
4x4TB LSI RAID 10 (Main ESXi Datastore)
4x6TB Non-RAID (Separate controller from LSI RAID)

FreeNAS is run as a VM with the following specs. These can be increased if necessary, this was just my starting point.
4 vCPU
32GB RAM
40GB OS Drive on the (ESXi DataStore)
4x6TB for ZFS via DirectPath I/O PT


So I have the 4x6TB setup in Mirror (RAID10) and I have setup the following DataSets.

Archive_Dedupe (lz4 with ZFS Dedupe enabled, atime enabled)
WindowsBackups_Dedupe (lz4 with ZFS Dedupe enabled)
Downloads (No compression, atime enabled)
Storage (lz4)

I am on the fence about using Dedupe after doing research however I really wanted to take advantage of this for Archive data and Windows Backups. Thoughts?

Any other recommendations?

 

[Nick2253]

On the whole, your build looks really good. Thank you for doing your research :D. I have a couple comments for you:

• I would say that 4 vCPUs is a good spot. 4vCPUs may be overkill, depending on what you're doing with FreeNAS, so you might be able to get away with two, but I would do some benchmarking before.
  
• 32GB of memory is good, unless you enable dedup. The rule of thumb is 5GB RAM to 1TB deduped storage, so in that case, you'd need to up your memory significantly.
• I would enable compression on the Downloads directory. Generally speaking, modern CPUs handle compression very efficiently, which means less data to read from/write to the disk; in other words, enabling compression can make it faster than without. Don't go overboard; the default compression is pretty good, unless you know you're working with highly compressible data.
• Depending on how you're doing your backups, dedup may not get you much space. Unless you are backing up directly to the FreeNAS server at a file level, you won't see any dedup benefit. I would probably recommend not doing dedup, unless you're talking about 10+ machines.

At this point, I should back up a bit: what is your use case for this server? How many users? What kind of data?

 

[SomeRandomBS]

Thank you for your reply. I will answer your questions in a 1, 2, etc format to make things easier.

1. I am doing a large transfer right now, roughly 4TB of data from my DAS to FreeNAS. I am utilizing roughly 35% of allocated CPU. So perhaps you are right that 2 vCPU is sufficient, especially on 1GBe. However I have CPU to spare so I will leave it as is.

2. So I did some research on FreeNAS dedupe and perhaps it is not for me. I will have to experiment with this more but I am starting to think the benefits I am used to with dedupe are not applicable to my specific application here with FreeNAS.

3. So the reason why I didn't enable compression for this DataSet is because this particular DataSet will be accessed and read from constantly 24/7/365 from another host and I feared performance issues using compression.

4. Same as 2, I am going to ditch dedupe for these backups. I did some research and it looks like I wouldn't benefit form dedupe at all.


I have one follow up question which will answer a lot of my other questions.

1. For data I rarely access (which I would label Archive), could I use maximum compression? I assume FreeNAS has to uncompress the data when I request it?

So for example lets say I have maximum compression enabled (gzip) and it compresses a 1GB datafile to 800MB, then I request that data, do I have to wait for FreeNAS to uncompress the data before I can access it? I assume that's how it works? This is why I did not want compression enabled for #3 above.

 

[Mirfster]

4x4TB LSI RAID 10 (Main ESXi Datastore)
4x6TB Non-RAID (Separate controller from LSI RAID)

Is there a particular reason you want to have a separate large DataStore (presuming for VMs) outside of FreeNAS?

While I can understand not wanting other VMs to be dependent on the FreeNAS VM, you are missing out on the benefits of ZFS as well as "ease of expandability". Personally, that is one of the main reasons I did my AiO. No knocking your design, just curios.

Couple other pointers for the FreeNAS VM:

• 4 vCPU
  • You will want to set the VM CPUs to "High"
• 32GB RAM
  • Not thinking that is enough since you want dedup
    
  • You want to "Lock" the VM RAM
• What is the card you are going to Pass-Through?

 

[SomeRandomBS]

I did not want all my VM's to be dependent on FreeNAS. Separating the arrays makes the most sense to me for many reasons, specifically data protection, ease of use, and most importantly performance. Although I do not have much experience with FreeNAS I would assume that my hardware RAID (in RAID 10) is superior to FreeNAS in terms of performance and I really wanted performance for my ESXi DataStore.

For the Pass-Through I am not using any hardware RAID, it is just single-drives to the FreeNAS VM. I read that using hardware RAID along with FreeNAS ZFS is not recommended.

I am ditching dedupe as I mentioned in my post above.

 

[Mirfster]

Although I do not have much experience with FreeNAS I would assume that my hardware RAID (in RAID 10) is superior to FreeNAS in terms of performance and I really wanted performance for my ESXi DataStore.

You know that "some around these parts may see them as fightin' words"... ;)

I guess all in all you are not actually building a FreeNAS system, you are building an ESXi system. While I could open the debate on performance, reliability, redundancy, bit rot, etc...

I will defer and ask just one question (out of curiosity and not in a mean way..):
What is it that you are looking to gain from FreeNAS?

 

[SomeRandomBS]

That is why I said "I assume" as I have not done significant research or testing. For argument sake lets take performance out of it, having ALL my VM's dependent on a single VM doesn't make any sense to me. Also I have plans to implement SSD's into my DataStore, weather they are Cache or Primary I am not sure, but that was another reason I wanted things separate.

Yes the primary role is an ESXi system and really all I need is a file server but I figured give FreeNAS a try. I have a very large DAS attached to my primary PC which is overkill for my needs. The plan is to retire that device, so setting up FreeNAS is sort of a test to see if it needs my needs.

 

[Mirfster]

No worries, to each their own. I actually am in the process of moving all of my VMs from a Hyper-V Server and an ESXi Server to FreeNAS. I ran them all separately for years as well, but have since changed my designs.

Best of luck and Welcome to FreeNAS.

 

[Robert Trevellyan]

do I have to wait for FreeNAS to uncompress the data before I can access it?

No, it's done 'on-the-fly'.

 

[SomeRandomBS]

No, it's done 'on-the-fly'.

Thank you.


Since this is still the help forum I figured I can post this here. I ran into a small problem and could use some help.

When I go to Plugins I have none available. Once or twice I saw an error about JSON. I decided to download Plex plexmediaserver-0.9.11.16.958-amd64.pbi and install it manually. Upon installation it fails on Step 3/3 with error "Unable to find template". Google search was not very helpful, said to delete the Jails and retry which I did but was unsuccessful.

Any ideas?

 

[Nick2253]

For the Pass-Through I am not using any hardware RAID, it is just single-drives to the FreeNAS VM. I read that using hardware RAID along with FreeNAS ZFS is not recommended.

You are correct about the hardware RAID.

However, you can't pass drives through directly, which is what you seem to be implying. You have to pass a controller through. The question is which controller are you passing through?

 

[SomeRandomBS]

You are correct about the hardware RAID.

However, you can't pass drives through directly, which is what you seem to be implying. You have to pass a controller through. The question is which controller are you passing through?

The drives are connected to a separate controller and I am passing that through to the FreeNAS VM. Right now I am using the onboard controller however I will likely change this at some point.


Anyone have any advice regarding my Plex plugin issue?

 

[Nick2253]

If you're running ESXi, why are you running plugins in FreeNAS? When you virtualize, I would say that you don't want anything else running on your FreeNAS VM. Everything should be done using other VMs (or containers, if you want to go that route).

 

[Sean Coston]

Are you running FreeNAS 9.10?
The plugin and jail system seems to be a bit quirky in 9.10. I recently had to upgrade my 9.3.x to 9.10 using media manually because the gui upgrade process failed. Then I started getting errors about jail and plugin templates not being found when trying to rebuild one of my jails.
The system would not let me remove the old jail template datasets because some of my older jails had dependencies to them (I guess).
Eventually, my work around was to construct new jail templates for 9.10 by adding a template and inserting the download url from freeNAS.org for the proper version.

Now I have new and old template jails humming happily on the same system.



Sent from my Nexus 6P using Tapatalk