Lost Data from Rebuild?

[chris11d7]

Hey guys! I'm new here, obviously I have a crisis, any help is so much appreciated.
I have a FreeNAS box that has been running great for the past 2 years or so, consisting of 10x8tb WD Red drives in RAID-z2.
Today I decided to add some SSDs to the system to create a new pool with.
I installed a new RAID controller in HBA mode with my 16 SSDs on it, and after booting up, my main pool was degraded.
I noticed all 10 WD drives still showed up under disks, so I opened the pool status, and "replaced" the "missing" drive with the newly realized path to it.
FreeNAS proceeded to crash, which I thought was a peaceful reboot that needed to be performed for the resilver, as this was my first "failure" I've experienced.
once FreeNAS booted back up, I saw the missing drive still showed "REPLACING" and status seemed to be progressing; To my greatest discomfort, I noticed the same thing happened to ANOTHER drive, so I impulsively "replaced" that disk too, which again resulted in an unsolicited reboot.
Again, everything came back up and seemed to be okay, but after checking the status, a THIRD drive is missing...
I can still reach the SMB file share, but 99% of my files are corrupt (I assume the 1% of OK files are the ones that were rebuilt before each crash), and every sub-sub-folder is completely absent of files.
Coming from an IT background, I know backups are crucial for data longevity, and I have backups of all my non-replaceable files, luckily, so the only annoyance is all of my VM backups are gone, plus the time it took to organize all of my data on to this drive (my file backups are spread across like 20 2TB drives in my basement, not ideal, but perfect in this situation).
I really just want to know why this happened so it doesn't happen again; I realize I was too impulsive trying to rebuild right away when there was clearly a different issue, but FreeNAS should have protected from this.

Thanks!

Chris

If

 

[c77dk]

I have a FreeNAS box that has been running great for the past 2 years or so, consisting of 10x8tb WD Red drives in RAID-z2.
Today I decided to add some SSDs to the system to create a new pool with.
I installed a new RAID controller in HBA mode with my 16 SSDs on it, and after booting up, my main pool was degraded.
I noticed all 10 WD drives still showed up under disks, so I opened the pool status, and "replaced" the "missing" drive with the newly realized path to it.

Did you add or replace the controller? and is it a RAID controller or HBA? And which one?

As you stated, starting to replace wasn't a smart move - I would have ripped the new controller out ASAP. The replacement might actually have cost you the pool :-(

 

[chris11d7]

Did you add or replace the controller? and is it a RAID controller or HBA? And which one?

As you stated, starting to replace wasn't a smart move - I would have ripped the new controller out ASAP. The replacement might actually have cost you the pool :-(

The RAID controller was only for the new SSD array, the 10 original drives are plugged directly into the motherboard (Supermicro X10SRL-F) via the 10 SATA headers, all in EHCI, hot-swap support mode. So after taking out the RAID controller, the entire configuration is the same as it has been the past 2 years. The RAID controller is definitely in HBA mode, it came directly out of my decommissioned FreeNAS server.

 

[chris11d7]

I reverted the boot image to the one right before installing the latest update and physically removed the two drives being used for rebuild. Is there any way to force ada7 back into the pool without wiping it? If I could get this drive back in, there should be enough parity information to reconstruct my data.

 

[chris11d7]

I'm willing to pay for support if somebody knows how to fix this.