ESXi Question

[LIGISTX]

Motherboard raid is actually software raid supported by drivers in windows/Linux etc and will probably not work in ESXi

I never knew that. Thanks.


Sent from my iPhone using Tapatalk

 

[LIGISTX]

Ordered another 8GB of RAM and the SAS expander. I’m fairly confident my FreeNAS needs will work fine on 12-16 GB leaving a good 12-16 free for ESXi itself and some VM’s. Entirely possible my i3 ain’t be enough, but let’s see how the CPU usage and performance looks before I jump on a new CPU.


Sent from my iPhone using Tapatalk

 

[LIGISTX]

I have for now decided against iSCSI boot for my VM’s. The cpu and RAM overhead is likely more than I require for the benefits of such an implementation, and o firstly just want to get this up and running; easier is better.

Because of this, my SLOG requirements are lower. I have a spare SSD, would it hurt to use this as a SLOG? I understand this is not the best device for this, it will wear fast (I will over provision it) and doesn’t have stellar performance, but it has to be better than no dedicated “fast” SLOG device. Going to the Z2 array imparts a lot of extra writes to the array from my understanding, and the latency of those writes does hurt write speeds. If I understand correctly even a consumer SSD can’t hurt. It may not be the best thing ever, but it can’t hurt, correct?


Sent from my iPhone using Tapatalk

 

[joeschmuck]

Motherboard raid is actually software raid supported by drivers in windows/Linux etc and will probably not work in ESXi

Yup, very true, I found out as well.

@LIGISTX You could add a true RAID card if you really want to run in a Mirror but I tried that and unless you buy good quality and compatible hardware, I'd just run a single SSD as your boot drive. Also 120GB will disappear quickly! Just letting you know from personal experience. You can have two datastores using those two SSDs and that will help out. And to be honest, just put your system together and start using it. Even with planning this project out to cover every little thing you can think of, well you will change it. Odds are you will change it several times. So long as you are aware that you will make changes, life will be good.

 

[LIGISTX]

Yup, very true, I found out as well.

@LIGISTX You could add a true RAID card if you really want to run in a Mirror but I tried that and unless you buy good quality and compatible hardware, I'd just run a single SSD as your boot drive. Also 120GB will disappear quickly! Just letting you know from personal experience. You can have two datastores using those two SSDs and that will help out. And to be honest, just put your system together and start using it. Even with planning this project out to cover every little thing you can think of, well you will change it. Odds are you will change it several times. So long as you are aware that you will make changes, life will be good.

Thanks, and I know I will make changes. I’m a tech junky, I am used to it. Doesn’t mean I don’t attempt to plan it all out from the start tho ;).


Sent from my iPhone using Tapatalk

 

[nojohnny101]

Following this thread. Once you get it up and running, please share and document your experiences as best you can!

 

[LIGISTX]

Following this thread. Once you get it up and running, please share and document your experiences as best you can!

Hopefully I can live up to the expectations haha.

Hopefully I’ll be able to start working on it soon, a few of my other hobbies are in the way. Was on vacation last weekend and have a track day this weekend. Never enough time!

But, on the subject to keep people enthused and living up, I guess I can share a few examples of what I use FreeNAS to store and sync between computers at different locations.

Few shots from the past few weekends.

FX2_5079_Focus_Stack by David, on Flickr

FX2_4791-Pano by David, on Flickr

FX1_4689_Focus_Stack by David, on Flickr

FX2_5166 by David, on Flickr

FX2_3884-HDR by David, on Flickr

Not "mission critical" data, but this is a good part of the reason I built this FreeNAS box in the first place :)

Sent from my iPhone using Tapatalk

 

[LIGISTX]

Not sure where the RAM is yet, but we are getting there! I need to order a sas to 4x sata, and then we will be able to at least get it all running off the HBA/expander.

 

[Stux]

And you know that that SAS expander doesn't necessarily need to be installed in a PCIe slot right?

 

[LIGISTX]

And you know that that SAS expander doesn't necessarily need to be installed in a PCIe slot right?

I didn’t know that at all. Why is that, there is no actual data being transferred to anything except direct to HBA? I guess I inherently know this, but then why is there even pci fingers?

I see it has a molex plug, does it in fact need that power, or if I plug it into the board, would I not need the molex power? I would rather power it via PCIe than molex, power plugs are a premium on my rm650i :/


Sent from my iPhone using Tapatalk

 

[Stux]

If you plug it into a PCIe slot it helps to locate it in your chassis and provides power.

Alternatively you can mount it wherever you want and use the molex to provide power, thus saving a PCIe slot.

 

[LIGISTX]

Ok. RAM is here, so next weekend I am going to play around with all of this.

2 main questions.

Once ESXi is set up and FreeNAS is installed with HBA passed through, should I be able to load my FreeNAS backup and it will just work? Or do I have to start from scratch?

Is using an SSD for SLOG fine? I won’t be using iSCSI, and currently don’t have a SLOG at all. It should just make things slightly faster correct? Nothing world shattering, but a bit faster due to less RAID 6 array writes. Again, this is mostly a media CONSUMPTION machine. I have no database things going on. I put movies and tv on it to the tune of a few GB a week, and when I take pictures that is about 30-50 GB maybe once to twice a month. I’m not going to kill the SSD with insane writes all the time since not much is ever actually written lol.


Sent from my iPhone using Tapatalk

 

[LIGISTX]

Well, I guess it was to good to be true. After a few headaches with my BIOS deciding to turn off its SATA controller all on its own................

I am in freenas with my backup loaded. All settings are there, but it doesn't seem to see my drives. I am not sure if the PCIe pass through is the issue or something else. How would I go about seeing what the cause could be? In ESXi I have "Reserve all guest memory" and passed through the HBA. I don't see the SAS Exapnder as available to pass through, but from my understanding that is basically invisable to the OS anyways, its just breaking out SAS's to more SAS's, correct?

Note: The SAS Expander worked fine before I started with ESXi.

 

[LIGISTX]

I booted from my FreeNAS bare metal SSD and everything is fine. I just need some advice on how to go forward, I’m really not sure what is wrong. It’s all pretty simple in the webUI. I believe I am on 6.5U2, not sure if that has an affect or not. I did see the newest version was 6.7, but I figured I would stick on 6.5 so I could follow @Stux tutorial which was extremely well done.

Hopefully we can figure this out, I can’t wait to get it working!

O, hmm, now that I think of it I don’t know what firmware my HBA is on. Not sure if this matters, but it clearly works fine in normal operation. But could this cause an issue? Or something about my hardware config?

Sent from my iPhone using Tapatalk

 

[kdragon75]

It could be the firmware. Check that and the logs on esxi.

 

[LIGISTX]

It could be the firmware. Check that and the logs on esxi.

Firmware is always a confusing cluster to me, what firmware do I want, and how do I verify what’s currently on the card? And since it’s a dell card does it have to be dell specific? I have never flashed the HBA before.


Sent from my iPhone using Tapatalk

 

[LIGISTX]

I am looking in ESXi's logs for the VM, and ill be honest, I have no clue what I am looking at :/

Also, I checked BIOS, Virtualization is on, and VT-d is as well.

 

[LIGISTX]

My cell internet at work is pretty horrible, so googling around is a paint. But it looks like HPE has a custom ESXi iso, any chance this is required to make this work..? I don’t think it would be seeing as my VM is working. I would think passing the HBA to the VM is not at all HP specific, that has to be an intel spec on the motherboard.

Should I maybe try 6.7? Or 6.0? Or even 6.5U1 instead of U2? I still need to figure out how and which firmware to flash my HBA with. From the eBay auction email when I purchased it, looks like it’s on P19, I think FreeNAS 11.x wants P20? Still not sure what P20 to get, or where to get the dell specific one.

Any thoughts?


Sent from my iPhone using Tapatalk

 

[LIGISTX]

It could be the firmware. Check that and the logs on esxi.

Ok, it only took me about 4 hours, but I figured out how and have successfully flashed the newest firmware. 20.00.07 or whatever it was. I only took a picture of the sas2flash -list screen before I edited anything... lol. But it is the newest as far as I know from all of the research I did, and booted up into freenas bare metal, no warning and it imported just fine.

Will hopefully have time to try ESXi tomorrow, see if it works without doing anything which I have to assume won't be the case, then I guess I will try HPE's custom ESXi version, although I have a feeling all that does is help it recognize the built in SAS controller and Ethernet ports. But I suppose we shall see, lots of reading online sorta leads to "always use the HP custom ISO for HP servers/workstations". I just hope I can get it working!!!

 

[LIGISTX]

So I am getting this in freenas. Any ideas? @Stux you are the guru, and advice..?

I will be trying the HP specific ISO right now, this was under normal ESXi 6.5 U2. But I have a feeling the HP ISO won't help as this should be completely chipset based.......

Code:

dmesg | grep mpt																									
mpt0: <LSILogic 1030 Ultra4 Adapter> port 0x1400-0x14ff mem 0xfeba0000-0xfebbffff,0xfebc0000-0xfebdffff irq 17 at device 16.0 on pci
0																																
mpt0: MPI Version=1.2.0.0																										
cd0: Attempt to query device size failed: NOT READY, Medium not present															
da0 at mpt0 bus 0 scbus2 target 0 lun 0

Scratch that, I got this

Code:

dmesg | grep mps																									
mps0: <Avago Technologies (LSI) SAS2008> port 0x4000-0x40ff mem 0xfd4f0000-0xfd4fffff,0xfd480000-0xfd4bffff irq 18 at device 0.0 on
pci3																																
mps0: Firmware: 20.00.07.00, Driver: 21.01.00.00-fbsd																			  
mps0: IOCCapabilities: 1285c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,HostDisc>								  
mps0: SAS Address for SATA device = 4f675c4ddf96b98a																				
mps0: SAS Address for SATA device = 4f696952b9b4db86																				
mps0: SAS Address for SATA device = 4f696952b9a1b785																				
mps0: SAS Address for SATA device = 4f696154dea1ca8b																				
mps0: SAS Address for SATA device = 4f626a5ac796d198																				
mps0: SAS Address for SATA device = 4f675952b9a5be72																				
mps0: SAS Address for SATA device = 4f68675ad498d274																				
mps0: SAS Address for SATA device = 4f696154deb9b888																				
mps0: SAS Address from SATA device = 4f675c4ddf96b98a																			  
mps0: SAS Address from SATA device = 4f696952b9b4db86																			  
mps0: SAS Address from SATA device = 4f696952b9a1b785																			  
mps0: SAS Address from SATA device = 4f696154dea1ca8b																			  
mps0: SAS Address from SATA device = 4f626a5ac796d198																			  
mps0: SAS Address from SATA device = 4f675952b9a5be72																			  
mps0: SAS Address from SATA device = 4f68675ad498d274																			  
mps0: SAS Address from SATA device = 4f696154deb9b888																			  
mps0: SAS Address for SATA device = 4f646a52bab5db8c																				
mps0: SAS Address for SATA device = 4f655f4fd6b2db78																				
mps0: SAS Address from SATA device = 4f646a52bab5db8c																			  
mps0: SAS Address from SATA device = 4f655f4fd6b2db78																			  
ses0 at mps0 bus 0 scbus33 target 18 lun 0																						
da1 at mps0 bus 0 scbus33 target 8 lun 0																							
da2 at mps0 bus 0 scbus33 target 9 lun 0																							
da3 at mps0 bus 0 scbus33 target 10 lun 0																						  
da4 at mps0 bus 0 scbus33 target 11 lun 0																						  
da5 at mps0 bus 0 scbus33 target 12 lun 0																						  
da7 at mps0 bus 0 scbus33 target 14 lun 0																						  
da8 at mps0 bus 0 scbus33 target 15 lun 0																						  
da9 at mps0 bus 0 scbus33 target 16 lun 0																						  
da10 at mps0 bus 0 scbus33 target 17 lun 0																						
da6 at mps0 bus 0 scbus33 target 13 lun 0

So it is seeing the controller... Wonder why it doesn't want to mount the damn ZFS filesystem.