Esxi 6.7 and freenas installation using 2 raid controllers

[Crossix]

Hello guys, id like to get your recommendations for the project that i am working on, i have a server:
Dell poweredge T340
Cpu: Intel(R) Xeon(R) E-2136 CPU @ 3.30GHz
Ram:32GB
Raid Controllers: Perc H330 (Came with the server) and LSI 6Gbps SAS HBA LSI 9211-8i IT Mode(Bought for the Freenas Vm passthrough)
Storage : 4X4TB NAS DRIVE (For freenas), 2x500GB SSD (raid1 volume using perc h330 controller) and Dual Dell sd card 14GB

server has 8 3.5 bays, so my idea is to use the perc H330 for raid 1 ssd 500gb, and use the LSI hba (passthrough to Freenas VM)card for the 4X4TB Nas (4 bays handle by lsi card) drive then install the freenas on a dual ssd card, i would like to get any recommendation or feedback, like networking, should i dedicated a NIC to the freenas (server hsa 2 NICS )?, planning for production enviroment.

Please let me know. thanks

 

[dir_d]

This is similar to my setup, I have a Dell R720xd with 12x 3.5in bays. I pass through my LSI which has the 12x bays and a dedicated NIC to Freenas. In the back of the R720xd i have 2x 2.5in bays that have 2x Samsung PM863 960GB in Raid1 with a Perc H710. The SSDs in Raid 1 i use to house my VMs for ESXi, Freenas included. All storage goes to the VM's via SMB or NFS which is controlled by Freenas.

 

[Spearfoot]

Just wondering why you want to virtualize FreeNAS on this server: why not just run it on the bare metal?

 

[Crossix]

Just wondering why you want to virtualize FreeNAS on this server: why not just run it on the bare metal?

budget, besides freenas works fine with esxi with passthrough controller flashed IT mode.

 

[Spearfoot]

budget, besides freenas works fine with esxi with passthrough controller flashed IT mode.

Do you mean that budget constrains mean that you have to run virtual machines on this server, in addition to FreeNAS? I understand.

FreeNAS does indeed work very well with ESXi -- I own three such systems, see my 'My Systems'.

The problem you'll run into is memory constraints -- 32GB isn't much memory for FreeNAS plus add'l VMs. Perhaps you only plan a very few VMs with small memory requirements?

 

[Crossix]

Do you mean that budget constrains mean that you have to run virtual machines on this server, in addition to FreeNAS? I understand.

FreeNAS does indeed work very well with ESXi -- I own three such systems, see my 'My Systems'.

The problem you'll run into is memory constraints -- 32GB isn't much memory for FreeNAS plus add'l VMs. Perhaps you only plan a very few VMs with small memory requirements?

yes, only one extra vm, more of the ram will go to the Freenas Vm, besides it will use only for file servers backup and hyper-v Vms backup, my only doubt is if i have to passthrough a NIC card to the Freenas Vm or not, i am not clear about this. cheers

 

[Spearfoot]

yes, only one extra vm, more of the ram will go to the Freenas Vm, besides it will use only for file servers backup and hyper-v Vms backup, my only doubt is if i have to passthrough a NIC card to the Freenas Vm or not, i am not clear about this. cheers

For what it's worth, I've never had to passthrough a NIC to FreeNAS. But I have 10G NICs in my servers.

 

[Crossix]

For what it's worth, I've never had to passthrough a NIC to FreeNAS. But I have 10G NICs in my servers.

you right, however some people create vmswitch for every nic to do iscsi ?

 

[Spearfoot]

you right, however some people create vmswitch for every nic to do iscsi ?

That may be. I dunno.

But if you're planning to provide block storage via iSCSI, you'll be pushing the memory limits. The User Guide states:

"For iSCSI, install at least 16 GiB of RAM if performance is not critical, or at least 32 GiB of RAM if good performance is a requirement."

May not be an issue since you don't plan to run a large number of VMs.

I just use NFS for my virtual machine data store. In fact, if you plan to run the VMs on the same machine as FreeNAS, then you don't need a NIC at all. I use the virtual storage network described in this article by Ben Bryan:

FreeNAS 9.10 on VMware ESXi 6.0 Guide

This is a guide which will install FreeNAS 9.10 under VMware ESXi and then using ZFS share the storage back to VMware. This is roughly based on Napp-It’s All-In-One design, except that it uses FreeNAS instead of OminOS. This post has had over 160,000 visitors, thousands of people have used this...

b3n.org

 

[dhenzler]

Just wondering why you want to virtualize FreeNAS on this server: why not just run it on the bare metal?

After a few hours of playing with TrueNAS on ESXi 6.7 I'm of the opinion that it's a huge waste of time for me. I certainly don't need whatever it gives. I like ESXi and I like TrueNAS, but not together. ESXi v 7.0 doesn't work with my DL380pG8 system. I've got a HP riser board LSI-2008-IT controller and it is not recognized by v 7.0. It's set to work passthrough for NAS, yet ESXi thinks not. When I make an adjustment to allow pass through none of my drives are seen. Oh bother...! Works great in TrueNAS by itself... why do I do this to myself...

It may have some benefit, but all the benefits are lost in the confusion to me... One Reporters Opinion.

 

[ChrisRJ]

Yeah, it's always a trade-off. For a while now I have been moving away from let's call them "tweaked" systems. And while it adds some level of inefficiency, the amount of time gained is worth it (I start getting old )

 

[dhenzler]

Yeah, it's always a trade-off. For a while now I have been moving away from let's call them "tweaked" systems. And while it adds some level of inefficiency, the amount of time gained is worth it (I start getting old )

I always read about a technology before trying it. My experience with ESXi has been a good one so far. I wanted to see if their was a clear benefit in putting TrueNAS under ESXi. Not only did I NOT see a benefit for my smallish needs, but the flippin thing was horrible to set up. The HP 684896-001 HP B320i Riser w/SAS Controller is a great way to get an LSI-2008-IT for $18. And doesn't use up any real estate. My DL380pG8 has a 420i controller as well, but I've moved the cables to the B320i and for NAS it can't be beat.
I really found no redeeming value in this pairing. ESXi recognized the controller but indicated that pass through was disabled. When I enabled pass through the drives disappeared. In addition changes such as this required both TrueNAS and ESXi to be rebooted... double delay whammy!
I'm 74 and enjoy technology, but hate stupid. Although NAS may provide benefits for some huge project, it's value for me was negative.

 

[dir_d]

The reason i went to ESXi and switched from bare metal was for VM support. I have windows machines and other linux machines in VMs, some i use for tests and some i use for everyday stuff. Trying to have all the VMs run in FreeNas at the time was less than ideal and had problems at the time i made the switch. If you dont run multiple VMs i would stay bare metal.

 

[dhenzler]

The reason i went to ESXi and switched from bare metal was for VM support. I have windows machines and other linux machines in VMs, some i use for tests and some i use for everyday stuff. Trying to have all the VMs run in FreeNas at the time was less than ideal and had problems at the time i made the switch. If you dont run multiple VMs i would stay bare metal.

So you run VM's in FreeNAS under ESXi?
I don't get why you would do that. You can run everything your talking about in ESXi alone.
TrueNAS has it's purposes, and honestly most are Nextcloud and Plex for the average user.

 

[HoneyBadger]

So you run VM's in FreeNAS under ESXi?

I believe you've misinterpreted what @dir_d is doing, emphasis added below:

The reason i went to ESXi and switched from bare metal was for VM support.

The VM support under ESXi/vmkernel is still significantly more mature than bhyve.

The most significant challenge for most users doing TrueNAS under ESXi is the requirement of two separate PCIe devices for storage (eg: one onboard SATA and a separate SAS controller, or two SAS controllers) and understanding the impact of that choice. With a single SAS controller there is no way to run a "nested" solution like this; with a single controller, once the VM grabs the passthrough device the host OS (ESXi) loses access to it.

 

[dhenzler]

I believe you've misinterpreted what @dir_d is doing, emphasis added below:



The VM support under ESXi/vmkernel is still significantly more mature than bhyve.

The most significant challenge for most users doing TrueNAS under ESXi is the requirement of two separate PCIe devices for storage (eg: one onboard SATA and a separate SAS controller, or two SAS controllers) and understanding the impact of that choice. With a single SAS controller there is no way to run a "nested" solution like this; with a single controller, once the VM grabs the passthrough device the host OS (ESXi) loses access to it.

If you read my system description you would have noted that I do in fact have TWO controllers in play. I'm booting from a removable SSD drive, not a thumb drive, and run the storage on a B320i-IT mode

 

[HoneyBadger]

If you read my system description you would have noted that I do in fact have TWO controllers in play. I'm booting from a removable SSD drive, not a thumb drive, and run the storage on a B320i-IT mode

Based on this comment, you've simply swapped one controller for another:

My DL380pG8 has a 420i controller as well, but I've moved the cables to the B320i and for NAS it can't be beat.

With that description, you've simply relocated "all drives are on P420i" to "all drives are on B320i" and as such there's only one controller actually handling drives, unless there's another cable that isn't described here.

 

[dhenzler]

Based on this comment, you've simply swapped one controller for another:



With that description, you've simply relocated "all drives are on P420i" to "all drives are on B320i" and as such there's only one controller actually handling drives, unless there's another cable that isn't described here.

The boot controller is SATA. You can't boot a SATA drive from a B320i silly! Don't bother responding... I'm so done with this !

 

[HoneyBadger]

You can't boot a SATA drive from a B320i silly!

SAS controllers can handle SATA drives perfectly well, including booting from them if the proper option ROM is present. There may be some edge cases based on vendor lock-in or where legacy SATA1 or first-generation SAS is involved, but modern SAS2 and SATA2 should have no issues.

Don't bother responding... I'm so done with this !

Fair enough, I'll stop trying to help you. Hope your day is as pleasant as you are.