DrKK
FreeNAS Generalissimo
- Joined
- Oct 15, 2013
- Messages
- 3,630
So, there were a few pitfalls with this; I assume most of these pitfalls loosely qualify as "bugs", so I don't know how long what I say below is relevant before they're fixed:
Situation: I have been booting my FreeNAS for a long time with a single 8GB Kingston Micro DT thumb drive. With the ability, now, to save boot images, and to run ZFS-on-boot, I thought it made sense to go up to 16GB, and in fact, to go to two devices in a mirrored boot pool. However, I only have one USB port open (my supermicro X10SLM has only 4 ports on the back, 2 are in use for peripherals, and 1 is in use for the USB boot stick). Therefore, I can't plug in both boot devices until the original one is removed. Can this be done without even turning off the NAS and/or recreating a new bootable OS?
Yes, if you know the pitfalls.
So here is what you do (even if you have two free USB ports, I still recommend doing it this way, one at a time):
1) Insert one new USB boot device. The kernel/devd should pick it up right away, and 'camcontrol devlist' will probably understand the drive as /dev/da1
2) Go to the boot pool in the system menu, highlight the pool name, and click "attach".
3) It will prep the drive, then it will ZFS resilver it. This will take 5-30 minutes, depending.
3a) Wait for the resilver to COMPLETELY FINISH before going on to the next step. You can monitor your resilver progress by doing "zpool status" and looking at what it says under freenas-boot.
4) IMPORTANT STEP: Slight booboo. Turns out (at least until they fix it), the GRUB install isn't set right, and never makes it to your thumbdrive. Thus, even though your device contains the boot pool's root data, it wouldn't boot, if you needed it to. You must manually make your new device bootable. Fortunately, this is easy, one command, in the CLI:
where of course you replace "da1" as necessary with whatever the device name for the new device.
Repeat steps 1-4 for each new boot device, EXCEPT THE LAST DEVICE. (so, if you are replacing a single boot device with two boot devices, you will only go through steps 1-4 the one time.)
5) Now, physically remove the *OLD* boot device right from the appliance. Just take it right out.
6) Now, insert your final device where da0 used to be. Repeat Steps 2-4.
7) Now, under the boot->status screen in the GUI, you'll see the now-pulled USB drive is shown as removed. Highlight it (you'll know which one it is, because it will show as 'removed'). "Detach" it, permanently.
8) IF YOU REPLACED WITH LARGER DEVICES, turns out, ZFS autoexpand may not have been on, so the device pool won't increase in size (verify this with "zfs list" at the CLI). If your size did not increase with the replacement, follow the directions in my bug ticket on the subject.
OK, that should do it!
Situation: I have been booting my FreeNAS for a long time with a single 8GB Kingston Micro DT thumb drive. With the ability, now, to save boot images, and to run ZFS-on-boot, I thought it made sense to go up to 16GB, and in fact, to go to two devices in a mirrored boot pool. However, I only have one USB port open (my supermicro X10SLM has only 4 ports on the back, 2 are in use for peripherals, and 1 is in use for the USB boot stick). Therefore, I can't plug in both boot devices until the original one is removed. Can this be done without even turning off the NAS and/or recreating a new bootable OS?
Yes, if you know the pitfalls.
So here is what you do (even if you have two free USB ports, I still recommend doing it this way, one at a time):
1) Insert one new USB boot device. The kernel/devd should pick it up right away, and 'camcontrol devlist' will probably understand the drive as /dev/da1
2) Go to the boot pool in the system menu, highlight the pool name, and click "attach".
3) It will prep the drive, then it will ZFS resilver it. This will take 5-30 minutes, depending.
3a) Wait for the resilver to COMPLETELY FINISH before going on to the next step. You can monitor your resilver progress by doing "zpool status" and looking at what it says under freenas-boot.
4) IMPORTANT STEP: Slight booboo. Turns out (at least until they fix it), the GRUB install isn't set right, and never makes it to your thumbdrive. Thus, even though your device contains the boot pool's root data, it wouldn't boot, if you needed it to. You must manually make your new device bootable. Fortunately, this is easy, one command, in the CLI:
Code:
grub-install --modules='zfs part_gpt' /dev/da1
where of course you replace "da1" as necessary with whatever the device name for the new device.
Repeat steps 1-4 for each new boot device, EXCEPT THE LAST DEVICE. (so, if you are replacing a single boot device with two boot devices, you will only go through steps 1-4 the one time.)
5) Now, physically remove the *OLD* boot device right from the appliance. Just take it right out.
6) Now, insert your final device where da0 used to be. Repeat Steps 2-4.
7) Now, under the boot->status screen in the GUI, you'll see the now-pulled USB drive is shown as removed. Highlight it (you'll know which one it is, because it will show as 'removed'). "Detach" it, permanently.
8) IF YOU REPLACED WITH LARGER DEVICES, turns out, ZFS autoexpand may not have been on, so the device pool won't increase in size (verify this with "zfs list" at the CLI). If your size did not increase with the replacement, follow the directions in my bug ticket on the subject.
OK, that should do it!
