Updates kick USB out of boot order

[Mike Haskell]

I just upgraded to 9.2.1.9 from 9. 2.1 and experienced the same boot problem as well

 

[sef]

I didn't expect it would.

There are only two things I can think of that could be going on from the FreeNAS side:

1. The boot sector is not recognized by the BIOS. But that doesn't explain why it would be recognized sometimes, but not others. (Currently, no shiny! update touches the boot block.)
2. Data aren't being synced out in time. But that comes back to the above -- the boot block is what the BIOS will be looking at, and it's not changing. (Even if it's GPT vs MBR, those portions of the drive are still not changing!)

Given that, it seems more likely to be something about the thumb drives or the MB/BIOS not working the way it is expected. And it probably always worked this way, but there are more frequent updates now.

It's also possible that the BIOS is keeping track of writes to the boot device, but that also doesn't make much sense to me. An edge case where the BIOS gets confused about the new GPT (even though there's a "protective MBR") layout is also possible.

 

[Pasquale61]

The thing about the boot sector not being recognized by the BIOS can't be 100% true because a power cycle seems to fix it, and power cycling should not rewrite it. For those where it seems to put the USB boot device in a different order, it seems to also work if they put it back in the right order or reboot, so that would also also suggest the boot sector is fine.

I wonder why doing the update via command line seems to work every time for me. It seems that the solution may be embedded in whatever additional (or different) commands the WebGUI is doing than the shell commands: "freenas-update check" and "freenas-update update" and "shutdown -r now" Maybe there is something that is signaling a change in the USB device state somewhere in the shiny update process? Or am I way off here?

 

[sef]

A power cycle would reset the USB. One interesting question is whether pulling and reseating the thumb drive, rather than power cycling, would also change the behaviour.

The WebUI update does almost nothing different from freenas-update -- freenas-update is a small wrapper around several functions, which are also what the WebUI does. The main difference is that the WebUI then calls shutdown, while you have to do that manually with the CLI -- but you could try doing

freenas-update -C /tmp/cache check && freenas-update -C /tmp/cache update && shutdown -r now​

for similar timing.

The most likely thing I can think of is the thumb drive being flaky, and taking too long to respond after a USB reset happens as part of the boot.

 

[indivision]

The most likely thing I can think of is the thumb drive being flaky, and taking too long to respond after a USB reset happens as part of the boot.

Is there a list of known non-flaky USB drives?

I've used many different brands, sizes, etc. I can't say that I know if they made a difference either way as I paid less attention to the issue when updates were less frequent...

 

[Pasquale61]

A power cycle would reset the USB. One interesting question is whether pulling and reseating the thumb drive, rather than power cycling, would also change the behaviour.

I tested pulling and reseating earlier and that does work for me. I only need a soft boot after that.

I will try the command you suggested to see how it behaves.

 

[indivision]

I just ran the update with "shutdown" from CLI for the second time and it worked perfectly. When using the GUI it has never worked for me.

 

[Pasquale61]

The WebUI update does almost nothing different from freenas-update -- freenas-update is a small wrapper around several functions, which are also what the WebUI does. The main difference is that the WebUI then calls shutdown, while you have to do that manually with the CLI -- but you could try doing

freenas-update -C /tmp/cache check && freenas-update -C /tmp/cache update && shutdown -r now​

for similar timing.

So, I tried the this command and I got the exact same results as I do in the WebGUI! (It doesn't work right.) I even tried putting in 10 second sleeps between the 3 commands to no avail. Yet, if I do the 3 commands individually it works every time. I also tried changing "now" in the shutdown command to a +1 for a minute delay and that did nothing other than waste a minute. Hmm...

I'll try some other USB name brand thumb drives when I get a chance. So far I have tried an 8G and 4G SanDisk Cruzer, and a generic one.

 

[sef]

This is dumb, but you could try

freenas-update -C /tmp/cache check && freenas-update -C /tmp/cache update && sync && sync && sync && shutdown -r now
​

But: as I said, I really suspect this is something to do with the thumb drives, which means timing is going to be the issue.

 

[Pasquale61]

I'm trying it right now before I go to sleep. I'll also see if I can find some other brand of thumb drive tomorrow night.

 

[cyberjock]

The official answer we typically give for brands of USB sticks to use is something like "name brand that is at least 4GB in size". Of course I recommend 8GB with 9.3 now, but the name brand is still very valid. I've always used and recommended Corsair Voyager and never had problems. ;)

Of the "name brands" I'd consider Sandisk, Corsair, Mushkin to be the only 3 that fall into that category. Kingston has had some issues lately with their quality control, so I recommend they be ignored just because it's not particularly difficult to ignore them for the potential benefits.

 

[indivision]

freenas-update -C /tmp/cache check && freenas-update -C /tmp/cache update && shutdown -r now

What do the additional cache flags do? Could they significantly change how the shutdown process goes in terms of timing and/or drive status?

The official answer we typically give for brands of USB sticks to use is something like "name brand that is at least 4GB in size". Of course I recommend 8GB with 9.3 now, but the name brand is still very valid. I've always used and recommended Corsair Voyager and never had problems. ;)

Of the "name brands" I'd consider Sandisk, Corsair, Mushkin to be the only 3 that fall into that category. Kingston has had some issues lately with their quality control, so I recommend they be ignored just because it's not particularly difficult to ignore them for the potential benefits.

Thank you. I'll look at those options.

What about Patriot? I was planning on using a Patriot "TAB" in the new Supermicro build I'm putting together...

 

[sef]

What do the additional cache flags do? Could they significantly change how the shutdown process goes in terms of timing and/or drive status?

No, they are where it puts the update, that is all. Default is /var/db/system/update

 

[Pasquale61]

The most likely thing I can think of is the thumb drive being flaky, and taking too long to respond after a USB reset happens as part of the boot.

OK, I made some progress for the first time using a different brand. All I could find laying around was a Kingston DT 101 G2 16GB, so I did a fresh install on it, and rebooted it a couple of times just to make sure it was fine. Then I ran the update through the WebGUI and it worked!

So to sum it up:
SanDisk Cruzer 4GB and SanDisk Cruzer Glide 8GB both had problems after an update.
Kingston DT 101 G2 16GB worked the first time after an update.

 

[craigyb]

Mine was an 8GB SanDisk Cruzer, so presumably a known good make.