mgd
Dabbler
- Joined
- Jan 8, 2017
- Messages
- 46
EDIT:
See the conclusion at the bottom of this post with instructions.
Quick summary: 11.2 requires more from its boot devices compared to 11.1. If you experience very long boot times after upgrading to 11.2, then replace your boot devices with SSD drives.
---
ORIGINAL QUESTION
I have a FreeNAS system based on
Boot
Now, the system boots and works but is really slow booting and shutting down. After power on and after the motherboard is done with POST (which takes some time) the boot begins and seems to progress fine with messages scrolling down at the console.
However, when I get to the message…
…I am stuck here for minutes (we are talking about more than 5-10 minutes).
After that the boot continues by loading plug-ins which progresses but much slower than the initial part of the boot. Eventually, the system comes online.
Shutdown / Reboot
When shutting down the system I see loads of messages like this:
Finally, the system is syncing disks and reports
Ideas?
I have tried to check if there is something wrong with the boot pool but it seems to look fine and was scrubbed yesterday:
However, it looks like a scrub of a 16GB ZFS pool taking almost 4 hours is pretty slow.
Any ideas to debug and resolve this problem is highly appreciated.
---
EDIT:
I checked out this thread
* https://www.ixsystems.com/community/threads/rrdcached-plugin-failed-to-connect.76516/
as it seemed related.
After checking the checkbox “System > System Dataset > Reporting Database” I might have gotten rid of the “RRDCacheD” messages. However, the system is still taking around 20 minutes to startup. And now I have these messages:
This led me to this thread which suggests 11.2 has a serious problem booting of USB devices:
* https://www.ixsystems.com/community...r-attempt-to-write-a-readonly-database.70827/
Any comments?
---
CONCLUSION
FreeNAS 11.2 needs a fast boot device – Use SSDs
Apparently, FreeNAS 11.2 requires more from its boot devices that the previous version 11.1 did. If you have a slow USB boot device (maybe it has become slow over the years due to wear) this is the time to replace it to an SSD boot device. These are so inexpensive now that here in Denmark I could buy two 120 GB Kingston SSDNow A400 drives and a SATA power splitter for 400 DKK ≈ 59,50 USD ≈ 53,50 EUR. Replacing my 16 GB mirrored USB boot pool with these two drives gives me a 120GB boot pool instead – much more than I need but it allows for wear leveling over the SSD to make it last much longer
Preparation
Before replacing your boot pool, make sure to
Upgrading the boot pool to SSD
We are now ready to upgrade the boot pool. Follow these steps:
See the conclusion at the bottom of this post with instructions.
Quick summary: 11.2 requires more from its boot devices compared to 11.1. If you experience very long boot times after upgrading to 11.2, then replace your boot devices with SSD drives.
---
ORIGINAL QUESTION
I have a FreeNAS system based on
- ASRock Rack C2550D4I
- 16GB DDR3 ECC RAM
- 2 x 4TB WD RED drives (mirrored data pool)
- 2 x 16 GB Kingston DataTraveler Micro (mirrored boot devices)
Boot
Now, the system boots and works but is really slow booting and shutting down. After power on and after the motherboard is done with POST (which takes some time) the boot begins and seems to progress fine with messages scrolling down at the console.
However, when I get to the message…
Trying to mount root from zfs:freenas-boot/ROOT/11.2-U7 []...
…I am stuck here for minutes (we are talking about more than 5-10 minutes).
After that the boot continues by loading plug-ins which progresses but much slower than the initial part of the boot. Eventually, the system comes online.
Shutdown / Reboot
When shutting down the system I see loads of messages like this:
freenas collectd[...]: rrdcached plugin: Failed to connect to RRDCacheD at unix:/var/run/rrdcached.sock: Unable to connect to rrdcached: No such file or directory (status=2)
Finally, the system is syncing disks and reports
All buffers synced.
and then I have to wait a long time for the system to actually turn off or start to boot.Ideas?
I have tried to check if there is something wrong with the boot pool but it seems to look fine and was scrubbed yesterday:
Code:
$ zpool status freenas-boot pool: freenas-boot state: ONLINE scan: scrub repaired 0 in 0 days 03:45:00 with 0 errors on Sun Dec 22 07:30:01 2019 config: NAME STATE READ WRITE CKSUM freenas-boot ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 da1p2 ONLINE 0 0 0 da0p2 ONLINE 0 0 0 errors: No known data errors $
However, it looks like a scrub of a 16GB ZFS pool taking almost 4 hours is pretty slow.
Any ideas to debug and resolve this problem is highly appreciated.
---
EDIT:
I checked out this thread
* https://www.ixsystems.com/community/threads/rrdcached-plugin-failed-to-connect.76516/
as it seemed related.
After checking the checkbox “System > System Dataset > Reporting Database” I might have gotten rid of the “RRDCacheD” messages. However, the system is still taking around 20 minutes to startup. And now I have these messages:
Error: attempt to write to a readonly database
This led me to this thread which suggests 11.2 has a serious problem booting of USB devices:
* https://www.ixsystems.com/community...r-attempt-to-write-a-readonly-database.70827/
Any comments?
---
CONCLUSION
FreeNAS 11.2 needs a fast boot device – Use SSDs
Apparently, FreeNAS 11.2 requires more from its boot devices that the previous version 11.1 did. If you have a slow USB boot device (maybe it has become slow over the years due to wear) this is the time to replace it to an SSD boot device. These are so inexpensive now that here in Denmark I could buy two 120 GB Kingston SSDNow A400 drives and a SATA power splitter for 400 DKK ≈ 59,50 USD ≈ 53,50 EUR. Replacing my 16 GB mirrored USB boot pool with these two drives gives me a 120GB boot pool instead – much more than I need but it allows for wear leveling over the SSD to make it last much longer
Preparation
Before replacing your boot pool, make sure to
- Export your FreeNAS configuration from System > General and pressing the Save Config button. Consider ticking Export Password Secret Seed if you have stored passwords for 3rd parties (e.g. SMTP configuration)
- Important: Download your encryption keys if you have encrypted pools under Storage > Pools. (Read exactly how in the manual)
- [Optional]: Take a backup of the SSH host keys in
/usr/local/etc/ssh/ssh_host_*
Upgrading the boot pool to SSD
We are now ready to upgrade the boot pool. Follow these steps:
- Physically install the two new SSD devices in your system
- Boot the system
- Go to System > Boot Environments and click Boot Pool Status
- If your boot device is already mirror, you need to detach one of the devices from the boot pool because FreeNAS unfortunately does not support 3-way mirrors: Click the three vertical dots next to one of the devices and select Detach
- Attach the first SSD:
- Click the three vertical dots next to the Pool and click Attach
- Select one of the new SSDs in the dropdown
- Remember to tick “Use all disk space” because otherwise you boot device will not grow.
- Click Save
- Wait for the mirror to completely resilver. There is a spinning wheel in the top bar of the GUI as long as the resilver takes place. Please note, this can take very long time if your USB is very worn. For me it took 4 hours to resilver 7.5 GB of data.
- Detach the last USB drive: Click the three vertical dots next to the USB device and select Detach
- Attach the second SSD just like you attached the first. Remember to tick “Use all disk space”.
- Wait for the mirror to completely resilver. This is fast. For me it took around 2 minutes.
- Shutdown your NAS, remove the USB pen drives and restart.
Last edited: