DL380e - better install freeNAS on baremetal or put ESXi in-between.

qhash

Dabbler
Joined
May 17, 2018
Messages
18
I want to build a rack box that will serve as backup platform only. I chose a DL380e platform as it comes w/ 14 x 3,5 bays, is cheap and still pretty modern. I will use flashed H310 adapter. I would like to achieve three goals: reliability, ease of management and easy of recovery in case of failure.
Which of the two would better suite my needs:
- direct bare metal Freenas installation
- Freenas over ESXi installation
 

artlessknave

Wizard
Joined
Oct 29, 2016
Messages
1,506
depends. do you want to run VM's, or devote the hardware to freenas.
freenas over esxi will be dependant on esxi also running, so you kind of have 2 points of failure chained, plus for freenas to be in a VM your board/CPU needs to support vt-d to pass the controller through to freenas, which adds complexity. if you want to update esx or need to reboot it the vm would have to go down, though if its a backup that may be less of a concern.
dl380e isnt very specific, is it g4/5/6/7/8/9/10?
 
Joined
Dec 29, 2014
Messages
1,135
It does matter what generation of DL380 you are using, but bare metal would certainly be easier. There are a number of guides on how to virtualize FreeNAS, but it isn't the easiest thing in the world. Bare metal NAS and then separate VM hosts with the hypervisor of your choice will simplest choice.
 

Evertb1

Guru
Joined
May 31, 2016
Messages
700
Depending a bit on what you want to backup I would say that FreeNAS on bare metal is your best choice. I have a FreeNAS backup server as well and it's just that: backup. To me the backup of my data is very important, so that thing will never do anything else. And looking at it that way I did not want or need to complicate things with virtualization. And if you ever decide to do other things with that machine it's possible to transfer an existing FreeNAS installation to a VM (with a fresh install and some help of a saved configuration). I have done so when it was needed to run FreeNAS for some time on my lab system which is running ESXi. So starting out on bare metal will not close the door for virtualization if you ever change your mind.
 

qhash

Dabbler
Joined
May 17, 2018
Messages
18
Thanks for your input guys.

Its Gen8 with 8 core low-power CPU and 16GB of RAM - should be enough for ZFS backup machine.
Baremetal is going to be an easier approach for sure. I see several pros to put a FreeNAS on an ESXi VM though:
- no problems with the drivers (this might not be an issue as I see pople using DL380e Gen8 as their ZFS boxes)
- faster system recovery from a backup (ESXi host backup + VM bacup = environment can be recovered in 1-2 hours - but.. not really needed that fast)

What I am unsure is -
if LSI IT-mode controller is in a passthrough mode and presented to the FreeNAS VM will the OS recieve the same amount of information from the drives as it would compared to baremetal installation?
is recovering data and vdevs easier on baremetal installation compared to virtualized installation? I think a year ago I did a test on a virtualized environment and it seemed that vdevs on a new FreeNAS installation were restored completly from an existing "old" configuration on the drives. No single HW element was replaced, so this was not a perfect test.
 

Evertb1

Guru
Joined
May 31, 2016
Messages
700
If FreeNAS is able to run on your platform I still would run it on bare metal. I would not run ESXi, hosting a FreeNAS VM, with less then 32 GB. And I know of several forum members that run EXSi with 64 GB or more. But then again if your FreeNAS is the only VM running, 16 GB might be sufficient. A CPU with 8 cores is plenty.

As far as I know FreeNAS has full control of the LSI controller when it is passed through. FreeNAS should in no way be "aware" that it is running on a VM and interacts with the controller in the same way as it would do on bare metal. In fact I think that's the only reason that you should even think about running FreeNAS on a VM. FreeNAS and ZFS need that control.

I limit my answer about recovery to system recovery of FreeNAS when it runs on bare metal. You should take the precaution of downloading your config db after first instal and with every system change, being it hardware or software. In case of a broken boot device for example, you should be able to recover your system within 30 minutes or so. That includes a fresh install on a new boot device and upload of your config db. I have done this several times (never again USB sticks as boot device) and it never took more then 25 minutes or so.
 

qhash

Dabbler
Joined
May 17, 2018
Messages
18
Thank you. Your information is valuable to me as I was worried about baremetal recovery times and complexity. ESXi or not, I am planning to have two SSDs in a mirror as a boot device.
 

artlessknave

Wizard
Joined
Oct 29, 2016
Messages
1,506
there isnt much point virtualizing freenas just to host 1 freenas VM, you're just adding complexity for no reason, particularly since many of the things you would use a hypervisor like esx for can just be done from freenas with jails/plugins or bhybe VMs
 

qhash

Dabbler
Joined
May 17, 2018
Messages
18
one more question - lets say I have 2-3 two-way mirror VDEVS in a single pool. Can I add a spare disk so in case any drive from any VDEC fails, its replaced by hot spare?
 

artlessknave

Wizard
Joined
Oct 29, 2016
Messages
1,506
yes, a spare will replace any disk that it is greater than or equal to in any pool it is assigned to.
 

Evertb1

Guru
Joined
May 31, 2016
Messages
700

artlessknave

Wizard
Joined
Oct 29, 2016
Messages
1,506
any disk that the spare is greater than or equal too...so yes, greater, not smaller.
if we switch perspectives to make the subject the disk being replaced....then yes, the disk being replace would have to be smaller than or equal to the spare.
admit, i must, that worded directly, i did not, confusing, it may be...myself, slightly confused, even am.
 

Evertb1

Guru
Joined
May 31, 2016
Messages
700
My bad. How much of a difference that a small word can make. My teacher English on the highschool would be mad at me. Back to English 101 with you, he would say. LOL.
 

artlessknave

Wizard
Joined
Oct 29, 2016
Messages
1,506
ha, its fine, I bring in odd speach pattens from anything I've read or heard and so can sometimes sound...strange. Ive been reading translated japanese light novels and some of the translations are....clearly not by native english speakers, and I'm pretty sure that's affecting my wording....
 

qhash

Dabbler
Joined
May 17, 2018
Messages
18
:) at least here logic can kick in so no mistakes are made (smaller,greater)

it appeares I need to make another decision. I am thinking about using a P822 controller that came with the server. I can switch it to HBA mode and use 6TB HP SAS drives so I will have an environment as it was designed by the manufacturer, at least I think so ;) .
The questions are:
1. server will not boot from any disk that is connected to the P822 in HBA mode. How to use two back cages for 2x2,5 SSD (with adapters) for OS install?
2. maybe its still better to just replace P822 with H200/H310 in IT-mode?

anyone had some experience here?
 
Joined
Dec 29, 2014
Messages
1,135
I am thinking about using a P822 controller that came with the server.
That might make the manufacturer happy, but it will most certainly NOT make FreeNAS happy. Maybe it is easier in the G8 family, but in the G6 or G7 family it a HUGE FLAMING PITA to get a P822 changed into HBA mode. Even once you get there, that card will be handled by CISS driver which isn't really good for FreeNAS. I did a post some time back about the hoops I had to jump through to get that done on a G7 server. I you search the forums for references to the "ciss" driver, you will see that none of them are positive.
maybe its still better to just replace P822 with H200/H310 in IT-mode?
Yes, that is definitely the way to go.
 

Evertb1

Guru
Joined
May 31, 2016
Messages
700
2. maybe its still better to just replace P822 with H200/H310 in IT-mode?
As @Elliot Dierksen recommended, use a H200/H310. A year or so ago I bought two brandnew H310's on Marktplaats (a local ebay like site) for around 50 euro's (total, not each). Flashing them in IT mode was easy and they work great. For flashing them I looked at this webpages because depending on the motherboard you have, you need to do a little extra with the H310. But the article is very clear. And in the comments you can also find some usefull information.
 

qhash

Dabbler
Joined
May 17, 2018
Messages
18
thanks for the info.
I have three H310 and one H200, so this is not a problem. I was wondering what will happen with the fans (their noise), with the hot-swap ability and general drive information as well as HPE server behaviour with H200 and non-HP drives. Guess I will have to test, but any feedback is much appreciated.
 

qhash

Dabbler
Joined
May 17, 2018
Messages
18
coming back with some info - all is good, H200 sees all 14 bays. I am not sure abotu hotswap yet, but I have a problem with fan noise. After swaping P822 for H200 I have a jet. I have "kind of" a server room. The noise inside the room is barely standable, you can hear it outside with the door closed. With two door seperation, I still hear it.
Anyone had some success with making those fans to stay at 10-20% of maximum speed? Now they spin 90% all the time.
 
Top