mobrien118
Dabbler
- Joined
- Jun 22, 2020
- Messages
- 25
Hello,
I have recently started using FreeNAS/TrueNAS and I'm pretty impressed, overall, with what the community and iXSystems has built!
I'm having some issues with performance, that have been multiplied since upgrading from 11.3 (my starting point) to 12.
Now, I have to say, my hardware is not exactly top notch. I'm using a ~8 year old desktop PC with 3 2TB hard drives as the storage array and 1 500GB SSD as the system disk. To top that off, I'm running it inside of a KVM/QEMU VM (with direct access to the storage array disks, but the host is running in a VM disk).
It's complicated how I got here, but the BIOS of this PC has a "bug" that FreeBSDdoes not handle that will not allow it to boot FreeBSD natively. I found a bugfix request in the FreeBSD bootloader's bug list, but that code has not been updated in 8 years and it dosn't look like anyone is going to implement the fix (although it is known) because this hardware is so old anyway.
That was all good and fine, with 11.3. it wasn't a speed demon, but it got the job done. Now that I'm on 12, it seemed OK at first, even, but now I'm seeing something different entirely.
First of all, it sits in a boot loop for about 1-2 days. It posts an error about the pool and reboots. I have a screenshot of this stack trace, but it's very difficult to capture on the KVM console (over SSH/X11). It references a zfs_panic_recover(). However, at some point, it will randomly boot. I've had this experience twice in a row, and I won't reboot it again without expecting sitting for a couple days waiting for it to come back.
The first time it came up like this, I noticed that the "Dashboard" simply will not load. It's just blank. I didn't do much other than check for updates and then reboot. Then the waiting began again... Now it's up again, and I'm seeing something strange:
The system is sitting damn close to 0% CPU, but the "system load" for 5 and 15 minutes are both showing about 3.0. If I click something in the GUI (for instance, "plugins") CPU usage will spike to 100%, the page will start "spinning" in the GUI, and ultimately fail to load, and the "system load" for the 1 minute interval will rise up to about 6.0+ then drop back down to the 3.0 level.
Also, my incoming RSYNC jobs were working after the first boot loop/reboot, now they are failing. So, it seems like performance is getting worse as time goes on.
Short of migrating this to better hardware, what should I be looking at here? I'm certainly no ZFS expert, but based on the kernel panic at boot, it seems like something is wrong with the ZFS pool. I just looked in the "Pools" tab in the GUI and it says there are no pools (this was not the case during the last boot, though).
I don't want to have to rebuild this, but if I have to, I guess I have to. I would assume that there is some way to recover the pool, at least, though...
Thanks for reading all this! Hopefully there are some clues in here and someone has seen something like this before!
--mobrien118
I have recently started using FreeNAS/TrueNAS and I'm pretty impressed, overall, with what the community and iXSystems has built!
I'm having some issues with performance, that have been multiplied since upgrading from 11.3 (my starting point) to 12.
Now, I have to say, my hardware is not exactly top notch. I'm using a ~8 year old desktop PC with 3 2TB hard drives as the storage array and 1 500GB SSD as the system disk. To top that off, I'm running it inside of a KVM/QEMU VM (with direct access to the storage array disks, but the host is running in a VM disk).
It's complicated how I got here, but the BIOS of this PC has a "bug" that FreeBSDdoes not handle that will not allow it to boot FreeBSD natively. I found a bugfix request in the FreeBSD bootloader's bug list, but that code has not been updated in 8 years and it dosn't look like anyone is going to implement the fix (although it is known) because this hardware is so old anyway.
That was all good and fine, with 11.3. it wasn't a speed demon, but it got the job done. Now that I'm on 12, it seemed OK at first, even, but now I'm seeing something different entirely.
First of all, it sits in a boot loop for about 1-2 days. It posts an error about the pool and reboots. I have a screenshot of this stack trace, but it's very difficult to capture on the KVM console (over SSH/X11). It references a zfs_panic_recover(). However, at some point, it will randomly boot. I've had this experience twice in a row, and I won't reboot it again without expecting sitting for a couple days waiting for it to come back.
The first time it came up like this, I noticed that the "Dashboard" simply will not load. It's just blank. I didn't do much other than check for updates and then reboot. Then the waiting began again... Now it's up again, and I'm seeing something strange:
The system is sitting damn close to 0% CPU, but the "system load" for 5 and 15 minutes are both showing about 3.0. If I click something in the GUI (for instance, "plugins") CPU usage will spike to 100%, the page will start "spinning" in the GUI, and ultimately fail to load, and the "system load" for the 1 minute interval will rise up to about 6.0+ then drop back down to the 3.0 level.
Also, my incoming RSYNC jobs were working after the first boot loop/reboot, now they are failing. So, it seems like performance is getting worse as time goes on.
Short of migrating this to better hardware, what should I be looking at here? I'm certainly no ZFS expert, but based on the kernel panic at boot, it seems like something is wrong with the ZFS pool. I just looked in the "Pools" tab in the GUI and it says there are no pools (this was not the case during the last boot, though).
I don't want to have to rebuild this, but if I have to, I guess I have to. I would assume that there is some way to recover the pool, at least, though...
Thanks for reading all this! Hopefully there are some clues in here and someone has seen something like this before!
--mobrien118
