mpt0: request <XXXX> timed out for ccb...

[Sander Jansen]

Hey, i have this timeout problem after like 2 to 3 days and when it happens i need to forcefully restart my whole esxi box because it wont shutdown the VM or itself anymore.

I have looked all over the forum and googled a lot but didnt find a solution. i followed this guide: http://www.freenas.org/blog/yes-you-can-virtualize-freenas/

Everything runs fine and fast but when this eror hits everything stops working. (all add a screenshot) (i fixed the update message, gateway settings have been updated and fixed)
I am running the free version of esxi so i cant pass through the pci cards but i dit RDM (raw disk mount) all the disks so it should be rather close to be running on real disks. From what i can find it has to do something with the hdd controller? (esxi is using LSI Logic parallel) and should be supported by freenas from what i can find.

I disabled scrub and made sure there is no write cashing. im guessing its some function that runs every few days as this is happening every other 2 to 3 days. the time in between everything functions as normal.

If there is someone with experience with this problem, please let em know how i can troubleshoot it or fix it.

 

[SweetAndLow]

Hardware specs and freenas version. Please read the forum rules before posting again.

Sent from my Nexus 5X using Tapatalk

 

[Sander Jansen]

Hardware:

asrock c2750d4i
32GB ram
15 x2 TB (3 disks in hardware CLONE for ESXI, VM storage and freenas OS VM and the rest single disks, in freenas a RAID 6)

Software:
ESXI 6.0 update 2
FreeNAS-9.10.2 (a476f16)

 

[SweetAndLow]

I'm not sure but if you can't pass the disks through it probably won't work. Zfs needs raw access to the disks.

Sent from my Nexus 5X using Tapatalk

 

[Sander Jansen]

I have the disks patched as RDM (raw disk mapping) so that should be fine but for some reason the tmp0 is being stupid

 

[joeschmuck]

The numbers don't add up, 15 drives but only 12 SATA ports?

 

[Sander Jansen]

Sorry, my mistake... the 3 disks that are in clone are external. I have a hw raid box connected that runs those 3 in clone mode

 

[joeschmuck]

I just responded in the other thread but my advice was to ensure you have the Marvel controlers updated with the latest firmware (might as well ensure the BIOS is as well). I also don't think RDM is reliable, I have yet to see a single posting where someone got it to work reliably. But I wouldn't give up if you are willing to tinker around with it and try different setups.

Also, ESXi 6.5 is out, I'd update to that as well. This will require updating via a bootable device and selecting the upgrade path for us free home users.

 

[Sander Jansen]

Ive seen that the esxi 6.5 is out, ive tested it and it works good. but the thing is that i had freenas setup before from usb and all 12 internal drives are in a raid cluster so i dont have internal devices left over and esxi refuses to work with usb for datasets... so im stuck at 6.0 if i need usb for boot and storage for vms...

what could be a solution is a esata pcie 2.0 card but those are hard to find that are supported by esxi so fot now im somewhat stuck... i also dont have the space to store 21 tb to be able to rearrange my raid setup to free up one disk...

one thing is that the esxi 6.5 does nativly support my hdd controllers, for version 6.0 what i have running now i had to add some drivers

 

[joeschmuck]

all 12 internal drives are in a raid cluster

I think you need to clearly identify your hardware configuration and how ESXi is configured for this hardware.

Based on your wording, you are going about setting up FreeNAS hard drives all wrong. For quite a while this forum would not talk about placing FreeNAS on ESXi, primarily becasue people were not disiplined enough to not cut corners and then they would lose data and no one likes to see that happen. What I'm trying to say is, if you can't properly pass the controllers then I think you will be cutting a corner. At that point you might as well just make full drive vmdk's and use those for your data drives, it would likely be safer than what you are trying right now but it's still cutting corners.

 

[Sander Jansen]

The thing is that the raid from freenas comes from a existing setup and is full of data... i dont have the space to backup 18TB of data to clear the disks and start with clean vmdk files... the cpu/motherboard does not support vt-d and i cant change hardware because there is no budget for a new system.

I needed to switch to esxi becouse the latest freenas does not support virtualbox in a jail anymore. I need to have one debian and one windows system running on the same hardware.

 

[joeschmuck]

You still have not explained your hardware setup so it will be impossible to offer further advice without that knowledge. That is true about virtualbox but it was replaced by Bhyve (Iohyve) so you would have to learn how to use that new platform in order to run those VMs.

 

[Sander Jansen]

My current setup:
asrock C2750D4I as the motherboard with 32GB ram and 15 x2 TB, 3 of witch are external (cloned) via usb (going to be esata if i find a pcie card...) and the other 12 internal.
On the software side i have ESXI installed on the cloned 3 external disks. The esxi datastore is also on those disks. I have made RDM links from all 12 disks to the dataset so i can use them. Then i have a freenas VM (and some other VMs) witch has all the disks linked (see attachment for settings) and all the other vm settings. and then i have a fresh installed freenas with the settings i need for shares and stuff...

so all 12 internal disks are directly connected to the motherboard with no raid setup or anything, just 12 disks.

trying and changing to the new VM tools that freenas offers, i would like to try and do everything to get this working with esxi... its way faster then freenas was and it would be a hell of a task to convert my other VMs to a new system... I dont care so much about lerning a new VM system... usually i can adapt pretty fast

it could be a option to create vmdk files normally for disks but then i do need to know stuff works solid... as it includes copying and spreading out at least 16-18 TB of data to a lot of places (including friends and stuff over internet) it could be done but would be a task that takes at least 4 weeks to get it all done in safe ways...

 

[joeschmuck]

Another question...

You have 12 hard drives that were previously used in a FreeNAS system and now you have them moved into an ESXi system over RDM? That makes sense to me and that you gained all of that data while using the other FreeNAS system. Now you desire to convert your system over to ESXi as I understand it.

So while I know you have desires to run everything on your machine, I do not believe it's capable to run properly using RDM. I believe the problem is that when using RDM then there is a Virtual Controller in between the VM and RAW device and this is causing the drives to drop out.

Also, in your screen capture it states that Hard Drive 2 is connected via USB, is that true? I was under the impression that your boot device was the USB drive and that all the RDM drives were via SATA.

I think you need to think about your next move as I'm unable to give you a solution with your given hardware. You really need PCI Passthrough to do this properly.

 

[Sander Jansen]

All the RDM disks are on SATA, my esxi+storage pool are on the usb disk and thus in the settings its saying its on the usb disk, that is only the RDM vmdk file. It does link to all the internal sata connectors.

im planning on looking for another motherboard that supports a lot more... but i dont like to say goodbye to the IP-KVM and a few other things that are bloody handy for headless machines.

im going to try a few more things and then im giving up... i do still have the original usb stick with freenas on it... worst case scenario i can plug that back in and continue as was... and run the other vms for a little while on my desktop system until i have found a solution...

 

[joeschmuck]

Well I'm sure that you never thought that you would want to use ESXi when you purchased the Asrock motherboard but these things happen, to me as well as I upgraded 2 years after my first FreeNAS motherboard purchase. I'm using that older system now to run ESXI with my firewall VM and a FreeNAS VM all on a few datastores in vmdk files. This FreeNAS VM is for a backup copy of my important data so if my main FreeNAS system has to go down, I still have the backup copy even thought it's not ideal but it works, and I'm only talking about maybe 300GB of data (most photos).

 

[Sander Jansen]

I was just trying to setup a VM for freenas in esxi the original way with normal vmdk files... but the webinterface wont even let me ass 13 disks in total... so i may have found part of the issue... im slowly working my way up from 2 disks and onwards... to see where the disk amount limit is... or at least when it starts complaining... and put like a few gigs in there to see how long it runs for... as i dont have the money to but now stuff right now i do need a solution... and then copying stuff all over the place seems best option untill i can get something better...

 

[Sander Jansen]

Well im dropping this all... its gotten way to complicated with more and more problems... ive installe dfreenas (latest) onto a usb stick again and im running freenas off of the hardware again... for now i have a second system with esxi for the other vms...

i would like some more information of what is possible with Bhyve but im going to start a new tret for that as this one is off topic now