SOLVED Server inaccessible over network and console root login not possible due to long password

[trionic]

I just moved home and set-up my TrueNAS SCALE server on the home network. Just connected it into the switch, nothing more. All was fine until I discovered that a Windows VM could not access a SMB share. I have a bridge interface set-up for this.

I changed the bridge interface IP address but reverted the change and then the server disappeared off the network. Admin app for the router showed server as online for a while but not showing as offline. Tried differen ethernet cables but no change; the LED on the switch for the port is green and blinking every second.

Hooked up a monitor and keyboard which shows console output, but I can't log-in through that as the password is 100 random characters and TBH I would not know what to change if I did manage to log-in.

I tried using different ethernet ports on the server to see if that would re-initialise a connection, but to no avail. No idea what to try next!

I have a lot of computing experience but my networking knowledge is shamefully pathetic, hence I am asking for help.

 

[danb35]

So you're seeing here the trade-off between (perceived) security and convenience. You chose to change the system default to require a login to show the console menu, and you also chose a ridiculously-long password for the root user, both presumably in the interest of security. The result of that is the situation you now face: you must manually type a 100-character random password to log in in order to change settings at the console. If you don't get the console menu when you log on, type cli --menu at the shell prompt.

 

[mistermanko]

the password is 100 random characters

My deepest consolations. May I ask why?

 

[trionic]

The reason I chose such a long password is as Dan says: "perceived" security. I should not have enabled console password authentication and should not have chosen such a long password.

I am screwed here because the console login refreshes when I have typed only about half of the password and then I have to start again. I cannot see way to type the full password before the console refreshes.

The console reports the GUI console IP address of 192.168.0.205 but is unreachable and the LinkSys router reports that the device is offline, plus the router's IP range begins at 192.168.1.x (and cannot be changed to 192.168.0.x). I think the 192.168.0.205 is what I configured on the router at my previous house.

So I cannot seem to get the server on the network and I can't type in the long password in time, so what are my options here?

 

[Samuel Tai]

So I cannot seem to get the server on the network and I can't type in the long password in time, so what are my options here?

You can try mounting your boot pool on another system, and then directly editing the configuration DB /data/freenas-v1.db via sqlite3 to remove the console password by setting the adv_consolemenu key in the system_advanced table to true, but this requires you to know a bit of SQL-fu:

Code:

sqlite> SELECT adv_consolemenu FROM system_advanced;
1 <- Default of consolemenu w/o password
0 <- Your likely value

sqlite> UPDATE system_advanced SET adv_consolemenu=TRUE;

Then you can move your boot pool back into your current system, and boot up as normal. The other system can be an Ubuntu 20.04 system. For Ubuntu, the mount command is: zpool import -f -R /mnt boot-pool, and the unmount is: zpool export boot-pool. You may need to navigate to /mnt/ to locate data/freenas-v1.db.

Faster might be just reinstalling, and reconfiguring from scratch. Your data pool is safe, and only needs to be reimported after the reinstallation.

 

[GBillR]

I use a yubikey to store a static pw to unlock a bitlocker drive on one of my PCs... this option might work for you if you want to try.

You would need to purchase a yubikey and load one of the slots with you super password. The key works via usb and I would think should be able to regurgitate the password and the return fast enough to log in at the console.

 

[Samuel Tai]

I use a yubikey to store a static pw to unlock a bitlocker drive on one of my PCs... this option might work for you if you want to try.

You would need to purchase a yubikey and load one of the slots with you super password. The key works via usb and I would think should be able to regurgitate the password and the return fast enough to log in at the console.

Unfortunately, there is NO YubiKey integration with TrueNAS Scale nor CORE. Plus, OP already stated he doesn't remember his 100-character long super password.

 

[Whattteva]

the password is 100 random characters

The correct answer is key-based auth, not 100-character long password.

Honestly, the road to least resistance here is probably just to reinstall TrueNAS, import your pool and rebuild your config from scratch.

 

[trionic]

Thank you everyone for your replies. In particular, I am greatly reassured that the data pool is safe. This server has been around in various guises since FreeNAS 9.2 and contains a lot of important data.

I modified the subnet mask in the router settings to 255.255.0.0 so that allocated IP addresses begin at 192.168.0.x instead of 192.168.1.x, in the hope that would be a step to reinstating it on the network under its configured 192.168.0.205 address but to no avail. I suspect I have mis-configured something else. LEDs on the server's network interface are flashing yellow and constant orange; on the switch they're flashing green. I tried bypassing the switch with a direct connection to the router but with no difference. In addition, the server has about seven Ethernet ports and I tried all of them. In all cases, the LinkSys admin GUI does not show the server as connected to the network (or list it at all).

If the server had still been on its original chassis, with Supermicro motherboard with IPMI, I might have been able to paste in the password (which I do know) into the console but a while ago I bought a QNAP chassis and transferred over all of the data and that has no IPMI.

SQL-fu and reinstallation would both require either a reboot or power-off. How can the server be safely rebooted/powered-off without access to the GUI or console?

 

[Whattteva]

SQL-fu and reinstallation would both require either a reboot or power-off. How can the server be safely rebooted/powered-off without access to the GUI or console?

Someone else with more knowledge can probably shed more insight, but as long as there are no pending transfers going on and you're not running any VM's (probably can shutdown VM through its own remote console) on the data store, you can probably just power it off. ZFS is a CoW file system and pointer reassignments are atomic, so an improper shutdown is really a non-event for the most part. My TrueNAS server has gotten' its power unplugged probably a dozen of times (little devil kids at home) over the years and I have yet to notice any adverse effect from it.

 

[Samuel Tai]

SQL-fu and reinstallation would both require either a reboot or power-off. How can the server be safely rebooted/powered-off without access to the GUI or console?

Try just pushing the power button. ACPI should initiate a clean shutdown.

 

[trionic]

A momentary press on the power button does not initiate a clean shutdown, which leaves a brute-force shutdown.

There is currently no disk activity and I know that no VMs are running.

 

[GBillR]

Unfortunately, there is NO YubiKey integration with TrueNAS Scale nor CORE. Plus, OP already stated he doesn't remember his 100-character long super password.

FYI, there is no integration required. FreeBSD recognizes the yubikey when connected to a usb port, same as a usb keyboard. If he does remember the PW, this would work. I just tested it to confirm. Plug in the yubikey, press the button, and the string loaded into the yubikey slot is entered into the console.

EDIT: I am assuming this would also work with the linux os underlying SCALE, but I have not tested that.

EDIT 2: It also seems as if he does have the PW written down somewhere:

I am screwed here because the console login refreshes when I have typed only about half of the password and then I have to start again. I cannot see way to type the full password before the console refreshes.

 

[trionic]

Yes, the TrueNAS password is in my LastPass vault.

I do have YubiKeys here that I use for website logins and it might be worth seeing if I can figure out how to get the password onto the key such that I can use that to login to the console. That would be easier than SQL-fu on the boot pool or reinstalling.

Am I okay to turn off the power to this server given there's no disk activity or VMs running?

 

[GBillR]

Yes, the TrueNAS password is in my LastPass vault.

I do have YubiKeys here that I use for website logins and it might be worth seeing if I can figure out how to get the password onto the key such that I can use that to login to the console. That would be easier than SQL-fu on the boot pool or reinstalling.

I use https://www.yubico.com/support/download/yubikey-manager/

You can program either slot with your 100 character pw. I use slot 1 for mine and it also enters the return at the end of the string for me.

 

[trionic]

Me and my stupid long passwords.

The YubiKey idea was great and would have worked beautifully, but alas the documentation states that static passwords are limited to 64-characters. However, on my YubiKey 5 NFC and Nano keys, the 100-character password was truncated at 38 characters. Perhaps newer YubiKeys support 64-characters, in which case I could have set one half of the password to one slot (with no carriage return) and the other half to the other slot, thus using two key presses to input the whole password in two halves. So close!

The TrueNAS boot pool is on an NVMe drive and I have ordered an enclosure so that I can connect that to USB and do the SQL trick to remove console password authentication.

I do have Ubuntu and SystemRescue boot USB sticksa so if I can force power-off the server I could boot from one of those and do the SQL trick straightaway.

 

[GBillR]

Me and my stupid long passwords.

The YubiKey idea was great and would have worked beautifully, but alas static passwords are limited to 64-characters so my 100-character password was truncated.

So... If you are interested, you could program the first 64 into slot 1, program the remaining into slot 2. While I am unable to find a way to avoid the return at the end of the slot using the yubikey gui manager, there is a way to disable the return at the end of the string with some CL work.

A bit more work, but I do think this would potentially allow you to enter the pw prior to the console refreshing. Older versions of the yubikey GUI allowed for disabling the return at the end of the string, and I have setup a slot without that return before.

See: https://docs.yubico.com/software/yubikey/tools/ykman/OTP_Commands.html

 

[trionic]

I edited my post just as you were writing your reply!

Alas, the keys I have here truncate the password at 38-characters.

 

[GBillR]

I edited my post just as you were writing your reply!

Alas, the keys I have here truncate the password at 38-characters.

Well that does suck. That would require 2 yubikeys and 3 slots.... and now the chances of successfully entering 3 slots worth of string prior to the console refresh are definitely lower.

 

[Patrick M. Hausen]

There's a lesson here

@trionic not trying to make fun of you. Just want to share a story.

Incidentally I begged (and finally convinced) my colleagues to keep our root passwords at length 20. And possibly invoke pwgen a couple of times or by some other means exclude y and z and some of the special characters, because english and german keyboard layouts ...

It sucks to be forced to enter 40 characters in the middle of the night for an emergency recovery over VGA based IPMI console (so no copy&paste as in your case).

20 random characters is more than enough entropy in my opinion.