FreeNAS VM corrupted(?), can I save my data? (ESXI 6.5)

[drobo]

Coming here as I am at a loss as to what to do. I currently have an ESXI 6.5 host that I setup a year ago, one of my VM's was a FreeNAS 11.1-U4 VM that I hardware pass-through four 2TB HDD's to, creating a NAS for backup/media storage. I had some friends tell me this was a bad idea and I'm starting to believe them.

I am a networking guy, so storage is outside my typical domain. Long story short, I shutdown my FreeNAS (inside FreeNAS not a forced shutdown) today and when I powered it back on again I saw "Welcome to GRUB" and then a bunch of junk symbols, at which point the symbols start to scroll and never stop. So I'm guessing my FreeNAS VM is toast.

I have a bunch of data on the four HDD's that I hardware pass-through to FreeNAS and I don't know the best way to save it. I had a standard ZFS tying them together to make a single virtual drive. I was hoping it might be possible to spin up a new VM instance of FreeNAS and hardware pass the HDD's through to the new FreeNAS and go from there?

I'm honestly pulling at straws here, I really don't want to lose my data..

Attached a pic of the symbols that scroll by very quickly and endlessly if that helps at all.

 

[HoneyBadger]

Did you pass drives through, or a controller?

Your chances at recovery improve significantly with the latter - but in either case, it shouldn't be that bad.

Either way, the sequence here should be:

1. Detach the drives containing the pool itself (un-passthrough the controller or drives as needed)
2. Reinstall FreeNAS, reboot, shut it down safely.
3. Attach the drives/controller back to the VM
4. Power up and import the pool

Assuming there is no catastrophic failure of the hardware, and you're running on a suitable platform for ESXi (not too far off the HCL) this should result in your zpool coming back right as rain. You'll have to re-configure any shares/exports/targets though, as your config can't be salvaged from the current system.

 

[Chris Moore]

one of my VM's was a FreeNAS 11.1-U4 VM that I hardware pass-through four 2TB HDD's to, creating a NAS for backup/media storage. I had some friends tell me this was a bad idea and I'm starting to believe them.

Exactly how did you pass the disks through to the VM? It makes a big difference.

So I'm guessing my FreeNAS VM is toast.

Hopefully it is just the boot device and the data might still be recoverable, depending on how you passed the drives to the VM.

I have a bunch of data on the four HDD's that I hardware pass-through to FreeNAS and I don't know the best way to save it. I had a standard ZFS tying them together to make a single virtual drive. I was hoping it might be possible to spin up a new VM instance of FreeNAS and hardware pass the HDD's through to the new FreeNAS and go from there?

This might work, but it is (as I said above) depending on how (exactly) you passed the drives into the VM. The way we suggest doing it is to pass the entire controller card into the VM so the FreeNAS instance has full, direct access to the drives.

 

[drobo]

Did you pass drives through, or a controller?

Exactly how did you pass the disks through to the VM? It makes a big difference.

Originally I wanted to pass-through a RAID controller that I had, but it was unsupported by vmware so I settled on passing through 4 different drives so that FreeNAS could ZFS them together.

I used a tutorial about vmkfstools to map each drive as a physical disk, and adding them invidually to the FreeNAS VM as 4 separate disks. So I guess it was a tad misleading, as I didn't hardware pass-through the drives.

Anyway, long story short, I was able to fix this much more simply than I thought, I created a new FreeNAS VM and added the disks to that VM, FreeNAS imported the Volume without any issues and after recreating my SMB share and users I am back up and running.

Thank you FreeNAS, made this a lot simpler to fix than I thought it would be!

Still considering making the FreeNAS box its own physical box, thoughts on this?

 

[Chris Moore]

Still considering making the FreeNAS box its own physical box, thoughts on this?

That is generally considered the most reliable way to go, but it is possible to virtualize. There are some best-practices guides to help safeguard your data though. The way you are doing it now, it generally not recommended because there is some risk of data loss or corruption.
Here are some links for you to review and consider.

Just to make sure we have all our terminology straight, you might want to look these resources over:

Slideshow explaining VDev, zpool, ZIL and L2ARC
https://forums.freenas.org/index.ph...ning-vdev-zpool-zil-and-l2arc-for-noobs.7775/

Terminology and Abbreviations Primer
https://forums.freenas.org/index.php?threads/terminology-and-abbreviations-primer.28174/

If you want to go stand-alone hardware, these guides would be useful:

Hardware Recommendations Guide Rev. 1e) 2017-05-06
https://forums.freenas.org/index.php?resources/hardware-recommendations-guide.12/

Hardware Recommendations by @cyberjock - from 26 Aug 2014 - and still valid
https://forums.freenas.org/threads/hardware-recommendations-read-this-first.23069/

Proper Power Supply Sizing Guidance
https://forums.freenas.org/index.php?threads/proper-power-supply-sizing-guidance.38811/

Don't be afraid to be SAS-sy
https://forums.freenas.org/index.php?resources/don't-be-afraid-to-be-sas-sy.48/

Building, Burn-In, and Testing your FreeNAS system
https://forums.freenas.org/index.php?resources/building-burn-in-and-testing-your-freenas-system.38/

Github repository for FreeNAS scripts, including disk burnin
https://forums.freenas.org/index.ph...for-freenas-scripts-including-disk-burnin.28/

If you want to stay virtual but do it in a safer way, this is some good reading:

"Absolutely must virtualize FreeNAS!" ... a guide to not completely losing your data.
https://forums.freenas.org/index.ph...ide-to-not-completely-losing-your-data.12714/

Build Report: Node 304 + X10SDV-TLN4F [ESXi/FreeNAS AIO]
https://forums.freenas.org/index.ph...node-304-x10sdv-tln4f-esxi-freenas-aio.57116/

There are a lot more links in the useful links in my signature if you want to dive in.

If you are able to make FreeNAS a stand-alone system, that may be the easiest route, because there are some specific hardware requirements for safe virtualization. If you have any questions, please ask. There are many people on the forum that can share their experience with you.

 

[drobo]

Awesome links, thank you I have quite a lot of reading to do.

Do NOT use VMware Raw Device Mapping.

Crap :p At least now I can start migrating my data to a backup and start thinking about some hardware. Thanks again all.