Boot USB Failing - 9.3

[HgTxRx]

So, my newly setup FreeNAS 9.3 server started becoming unresponsive to the web GUI and ssh login attempts. This was after I'd invested the time setting it up, adding all the users, moving the data across, configuring Crashplan, etc.

When I looked a the local output the error messages were scrolling were "CAM status: CCB request completed with an error" and the email I received was titled 'Critical Alerts' with the following text 'The boot volume state is ONLINE: One or more devices are faulted in response to IO failures.'

Searching the forum showed that the boot USB was failing. There was only one USB boot drive and since this was a new server I hadn't backed up the setup data......

I was forced to cycle the power (after closing all open documents on the clients) as I couldn't get a response from the management interfaces even thought the AFP and SMB shares were still working normally. I was concerned about cycling the power but in my case I didn't loose any data and the volume reported 'No know data errors' on the restart.

What it did to get back up and running was -

1. Leave the failing drive in the server
2. Insert a second blank USB drive into the server and the installation USB.
3. Start the installation process and pick both the new and failing USB drives for the installation.
4. Choose 'Upgrade Install' .
5. Set the root password and continue the normal installation.

When FreeNAS had finished installing and rebooting the server started as normal, I then got the 'CAM Status errors' and it was slow to respond for about 5 min, it then sent the following email 'The boot volume state is DEGRADED: One or more devices are faulted in response to persistent errors. Sufficient replicas exist for the pool to continue functioning in a degraded state.' and started to respond normally.

Now when I look at System --> Boot --> Status the failing USB drive status is 'Faulted' and I have the option to replace it.

I've order the replacement USB Stick for under $10 and backed up the config database (System --> Advanced --> Backup (located at the bottom of the page))

If you're running 9.3 using only one USB boot drive I would HIGHLY recommend spending $10 on a second and use the instructions at http://doc.freenas.org/9.3/freenas_system.html#mirroring-the-boot-device to install it and then setup the system email account if you haven't already done so.

There may be a better approach than mine but I couldn't find it so if anyone knows of something let me know or if the more experienced of you identify something that's risky in my approach let me know.

Cheers

 

[ppmax]

Hello HgTxRx--

I'm wondering if you can help me out.

I had this exact same thing happen (no ssh, web, etc...but all mounted shares working as expected), but spent the whole last day chasing my tail thinking my problems booting were related to this:
Workaround/Semi-Fix for Mountroot Issues with 9.3!

Instead, it looks like I had a failing USB stick that manifested in a similar mountroot issue.

What I'm wondering is if there is a way to get my old FreeNAS settings from that failing drive using the method you outline above? I've got my "old" failing USB that won't boot...and I just went through a fresh install on a new USB...but am dreading having to recreate my environment. The new system boots fine from the new USB...so I don't think my issues are related to setting a boot_delay or anything like that.

Any help you can lend would be highly appreciated!

thx
PP

 

[HgTxRx]

Hey PP

It sounds like you're in the same situation I was in. Checking your system details it notes 9.2 but i'm guessing you're on 9.3, let me now if your not.

These are the steps I took and it worked perfectly for me (but I guess it depends how 'broke' your USB stick is)

1. Shut FreeNAS down ( I had to pull the power out)
2. Get the second USB stick and format it as MSDOS or something so the FreeNAS installer won't recognize it
3. Plug in the failing USB stick
4. Plug in the new MSDOS USB stick
5. Connect the FreeNAS installation media (either CD or USB, however you install it)
6. Start the server and go through the installation process.
7. At the 'Choose destination media' select both the failing USB and the MSDOS formatted USB to make a mirror
8. At the 'Upgrade this FreeNAS installation' screen select 'upgrade install'
9. Follow the rest of the installation process and reboot.

On the reboot my database was installed from the failed drive to both drives in the mirror and all my setting recovered, the system reboot a couple of times as it moved the database into position and then on the final restart it was slow booting as the original failing USB caused more trouble and after about 5 min the system switch to just using the new USB and identified the old one as failing in System --> Boot --> Status.

Hope this helps, if you need more let me know.
Cheers
HgTxRx

 

[ppmax]

Hi HgTxRx--

Thanks much for the help--I appreciate it. I'm going to give it a go in just a sec so I'll provide real-time updates ;)

Out of curiosity, do you know where the settings database is located? I can't recall if that is stored on the USB boot device or in my pool...

I'm indeed on 9.3...I'll have to update my sig if this is successful.

Thanks again
PP

 

[HgTxRx]

Good luck - I believe the database is stored on the USB under /data/freenas-v1.db but I'm not 100% sure.

 

[Bidule0hm]

I confirm that the path is good ;)