Setup recommendation

bonne

Cadet
Joined
May 6, 2020
Messages
9
Hi. I am updating my homelab, and I want to try out FreeNAS virutalized inside a ESX7 whitebox (Supermicro X11SCA, 64 GB RAM, 4 traditional 2 TB disk, two NVMe disks (one Samsung 970 512 GB and one 256 GB Kingston A2000).

I am used to traditional SANs (or NAS) and with these, I would have setup a RAID10 (normally with more spindles than my 4 homelab disks), but what is up on down on a virtualized SAN like this? I do not need more than 4 TB total storage.

All disks and one NVMe are set to passthrough, so I would like to dedicate these to FreeNAS and use one NVMe for cache. Would that be a good ide?

I am booting up on a USB-stick and using one of the NVMe's (the one not set to passthrough) as VM storage for the FreeNAS VM

Regards, Lars.
 

hescominsoon

Patron
Joined
Jul 27, 2016
Messages
449
running freenas inside a vm is not recommended...
 

Pitfrr

Wizard
Joined
Feb 10, 2014
Messages
1,523
If you want to go that road, you can read this post if you haven't already.

Basically make sure that you can pass through the drives so FreeNAS can access them directly.
Using a SSD as cache is probably not useful in your situation (I suppose a home usage. Maximize the RAM first and then eventually think about a cache.).
You also have to reserve the memory for FreeNAS (so 8GB minimum and I suppose you won't have any plug-in running in FreeNAS).

I would use the biggest SSD as VM storage (for FreeNAS or other VMs).
 

bonne

Cadet
Joined
May 6, 2020
Messages
9
I am further than that point. Drives has been passed through. Dedicated 32 GB RAM for the FreeNAS. System is up and running. Just need advise on best setup with that hardware. ZFS? Raid10? Cache/no cache
 

Pitfrr

Wizard
Joined
Feb 10, 2014
Messages
1,523
What do you mean here? Do I understand the question correctly: use of ZFS or not?
If no, then what's the point of FreeNAS? (note: question not meant to be rude or such, just to understand what you want to do)
Sorry, I assumed (probably wrongly) that you were already using FreeNAS (at least bare metal) with RAIDz1 (or 2) on those 4x 2TB drives...


Home lab, ok... That means probably a limited number of users and I assume no plugins (because you could run them in separate VMs).

So I'd say:
- 4x 2TB used in a RAIDz1 or RAIDz2 (depending on the type of data on it. RAIDz1 could still be acceptable for 2TB drives...)
- 512GB SSD as VM storage
- no cache
- if you don't use any plugin or so you can reduce eventually the memory usage to 16 or 8GB (to have more memory for other VMs if needed), and you can see also if this changes the performances but it shouldn't be that noticeable.

If you want to use FreeNAS for your data storage then I'd go for a RAIDz2 pool with 6 to 8 disks, it's a nice sweet spot (but 4 works of course).


And as @hescominsoon noticed, it is not advised to run FreeNAS in a VM because FreeNAS is designed to run bare metal. But if you take some precautions it is possible (and you probably read the warning in the post linked).
 

bonne

Cadet
Joined
May 6, 2020
Messages
9
To your question about why using FreeNAS? The point of using FreeNAS is to try it out.

Yes, homelab and limited number of users.

The type of data is only for data storage of VMFS files from the ESX server.

Regards, Lars.
 

Pitfrr

Wizard
Joined
Feb 10, 2014
Messages
1,523
Try out, ok...
Well if you are looking for reliability in storing your data then FreeNAS is a good way to go, indeed.
Still make sure you run regular maintenance (scrub, SMART test, ...) and it should be fine.


Are you saying that you plan to store the VMs on the FreeNAS pool?
Then you might want to look for a more specific setup (use iSCSI instead of NFS share) to have better performances. But I'm not at all knowledgeable on that topic! :tongue: So I won't be able to tell you more about it but have a look in the forum this has been discussed.
And the configuration of your pool can make a difference in performances here as well (RAIDz2 vs 2 mirrors).

I also have FreeNAS virtualized but I'm not using it to store the VMs on it. The VMs are stored on a separate disk (SSD) and I eventually share some folders from the FreeNAS pool over NFS to the VMs.

For try out: make sure the data you put on it is not important...
If you use it in production: make sure to have backups, that is even more important if you virtualize FreeNAS (better safe than sorry... well depending on your data of course).
 

bonne

Cadet
Joined
May 6, 2020
Messages
9
Been trying both raidz2 and Raid10 (or stribed VDev mirror as I understand it is called in FreeNAS)

When tested from a Linux VM on the FreeNAS datastore speed seems to be the same: I get around 14000 MB/sec when doing hdparm -T /dev/sda1 (on Raid10 as well as raidz2) and around 800 MB/sec doing hdparm -t /dev/sda1

dd if=/dev/zero of=/tmp/output.img bs=8k count=8192k gives me 668 MB/s - this is only tested on the RAID-10 though

Now, I don't know if this is a bug, but is seems I can't delete the old targets that I have made. There are no delete menu and when unfolding the targets, it seems like an UI bug - can't read anything than the first line of pixels

1590257874498.png


Or maybe I have to do it in another way?

Regards, Lars.
 

Pitfrr

Wizard
Joined
Feb 10, 2014
Messages
1,523
I can't delete the old targets
No idea here... and I'm still under 9.10.x so won't be of much help! :tongue:
And I also never played with iSCSI.
 

Samuel Tai

Never underestimate your own stupidity
Moderator
Joined
Apr 24, 2020
Messages
5,398
Or maybe I have to do it in another way?

Yes. To delete a target, it has to be not in use, so any target consumers have to be disconnected. You then have to first delete any Associated Targets to which a target is bound. It's also sometimes necessary to delete the Extent also linked in the Associated Target.
 
Top