FreeNAS 11.2-U6 - "panic free guard1 fail" on boot with USB hard drive attached

[james88]

Hi all,

I have a FreeNAS system that has been rock solid for some years (based on HP Microserver N54L). The boot device is a Kingston USB stick, and it was worked and rebooted flawlessly for years.

Recently I ran out of internal space for hard drives, and as a stop-gap measure before I upgrade the platform, I attached a WD USB 8TB hard drive. This works fine in FreeNAS - however it is causing issues with the boot process.

When I try to boot from the Kingston USB stick with the WD drive attached, the FreeNAS BTX boot loader begins to load, and then stops with an error that says, "panic free guard1 fail". The only option I am given is to press any key to reboot.

I've checked the BIOS settings, changed the boot order and ensured that I am booting only from the USB stick - however this makes no difference. The only way I can get the server to boot is to physically disconnect the WD USB drive, wait for the boot loader to complete and the kernel to start loading and then I can plug the WD drive back in. The boot completes flawlessly and I can use the system as normal.

The WD drive was never used for any purpose other than FreeNAS, and I put ZFS on it using FreeNAS. I didn't zero the drive first however - should I have?

The question is - why is the FreeNAS boot loader upset by the presence of this drive, and can I do something to fix it? It's really inconvenient to have to physically be at the system to unplug and plug in a drive to get it to boot. I know that my USB disk setup is far from ideal and in time I will get that fixed - however for now budget constraints mean it's the best I can do - and it strikes me this is an issue that it should be possible to workaround or resolve.

Does anyone know how to fix this? Can I zero out the boot sector or something on the WD drive to prevent the FreeNAS boot loader from getting upset?

Any and all help appreciated!

James

 

[PhiloEpisteme]

Recently I ran out of internal space for hard drives, and as a stop-gap measure before I upgrade the platform, I attached a WD USB 8TB hard drive. This works fine in FreeNAS - however it is causing issues with the boot process.

I would advice against this. Those USB enclosures add an additional layer of hardware and complexity between zfs and your drives. They have also been known to be flaky and have reliability issues. You could either buy a bigger case, pick up a jbod case, etc.

The WD drive was never used for any purpose other than FreeNAS, and I put ZFS on it using FreeNAS. I didn't zero the drive first however - should I have?

That probably wouldn't have made any difference.

The question is - why is the FreeNAS boot loader upset by the presence of this drive, and can I do something to fix it? It's really inconvenient to have to physically be at the system to unplug and plug in a drive to get it to boot. I know that my USB disk setup is far from ideal and in time I will get that fixed - however for now budget constraints mean it's the best I can do - and it strikes me this is an issue that it should be possible to workaround or resolve.

Sorry, I don't know exactly what is wrong with your boot sequence. I actually think the use of an external USB device for main storage is a bigger issue than the use of a USB key to boot from. I am curious though; how often are you rebooting your machine?

How many drives do you have in your system and how many would you be able to fit in an ideal case?

 

[Brendan9]

I have the same problem on FreeNAS-11.2-U7 on a HP Microserver N40. My system has also been working flawlessly for around 5 years.
I have 3 external 3TB USB hard drives connected to FreeNAS for backup purposes. Up until today I most recently had 2 attached USB drives (for about the past 6 months), but when I added a 3rd I got the exact same problem as you. My system will no longer boot, it stops at the BTX loader (version 1.02) after successfully detecting all internal and external drives, then goes into panic.

Note that my FreeNAS boots from an internal SSD. It has an internal RAIDZ array of 3x 3TB hard drives.

If I temporarily unplug the new USB drive during boot, then plug it in afterwards, all is ok, same as you.

Note that in the past (prior to approx 6 months ago and for approx 4 years) I have had 3 or 4 external USB hard drives for backup purposes connected to the same FreeNAS box (but it was FreeNAS 10.x then) and never had a single issue.

Were you able to resolve the issue so you can cleanly boot?

Thanks,
Brendan

 

[subhuman]

My very first step would be to completely wipe the drive, and that applies to any new drive. Most of them, especially externals, come "Formatted for Windows" who who-knows-what partitions that may or may not be flagged as boot partitions to confuse the BIOS or a bootloader. Just destroying any existing partitions and starting fresh should be enough.
Second, try different USB ports. Weird stuff sometimes happens.
Third, can you force the USB thru BIOS to run at a lower speed (2.0 or 1.0)? Higher speeds require more precise timings, and the USB-to-SATA adapters on most externals aren't the highest quality. USB is pretty resilient, but cheap crap is still cheap crap. Might be worth trying.

 

[Brendan9]

My very first step would be to completely wipe the drive, and that applies to any new drive. Most of them, especially externals, come "Formatted for Windows" who who-knows-what partitions that may or may not be flagged as boot partitions to confuse the BIOS or a bootloader. Just destroying any existing partitions and starting fresh should be enough.

Thanks for the suggestions. I'm zeroing the drive since yesterday as I was wondering the same (will take approx 2 days to complete due to the N40L only supporting USB 2.0 and I dont have an add-on USB 3 card). Although never had to do this on any other USB drive i have attached (all of them have come pre-formatted for Windows and I just used FreeNAS to make them ZFS). The one difference between the 4 external USB drives that have worked perfectly before is that they were all Western Digital but this new drive is a Seagate. Maybe Seagate do something slightly different when preparing their drives.

I'll report back once the zeroing is finished and I can try it again.

 

[Brendan9]

Update: After the zeroing completed on the drive causing the boot loader panic, it made no difference whatsoever and still halted the boot process.

What I tried next was to disable legacy USB in the BIOS to see if that made any difference. Low and behold it worked. Seems the boot loader didn't like this particular Seagate 3TB desktop USB drive being detected by the BIOS first.

Not really an issue for me having no USB devices available before the OS drivers kick in so I'm happy to leave legacy USB support off.

 

[subhuman]

Congrats! Hopefully @james88 sees your post and it helps him, too.

 

[james88]

Glad I'm watching this thread - that's really helpful - thank you @Brendan9. I'm away at the moment but will test this as soon as I can when I return.

 

[james88]

@Brendan9 can you elaborate on what you did to get this working? I'm in the BIOS of my N54L Microserver (original, unmodded BIOS) and I can't find anywhere to disable Legacy USB. Would you mind sharing with me what you did (e.g. menu items in the BIOS), and I'll try and replicate it here. I note that I'm on an N54L and you were on an N40L so perhaps there's a difference in the BIOS? Possibly also worth loading the modded BIOS?

Thanks in advance!