11.2 - Who has installed?

[dlavigne]

One other thing - Are there any 'gotchas' on these prior releases regarding pool upgrades, or SMART issues with FreeNAS on USB?
Just want to be prepared as these were some reasons I balked at upgrading over the years.

Those are detailed in the Upgrading the Storage Pools section of https://www.ixsystems.com/blog/library/freenas-11-2-release/.

 

[hugovsky]

Hi all.
As per 11.2 user guide, hardware section, "Upgrading the firmware of Broadcom SAS HBAs to the latest version is recommended."

While everything is working very well, do I have/should upgrade my sas controller?

Code:

root@nas:~ # sas3flash -list
Avago Technologies SAS3 Flash Utility
Version 16.00.00.00 (2017.05.02)
Copyright 2008-2017 Avago Technologies. All rights reserved.

        Adapter Selected is a Avago SAS: SAS3008(C0)

        Controller Number              : 0
        Controller                     : SAS3008(C0)
        PCI Address                    : 00:01:00:00
        SAS Address                    : xx
        NVDATA Version (Default)       : 0e.00.00.00
        NVDATA Version (Persistent)    : 0e.00.00.07
        Firmware Product ID            : 0x2221 (IT)
        Firmware Version               : 14.00.00.00
        NVDATA Vendor                  : LSI
        NVDATA Product ID              : SAS9300-8i
        BIOS Version                   : N/A
        UEFI BSD Version               : N/A
        FCODE Version                  : N/A
        Board Name                     : SAS9300-8i
        Board Assembly                 : N/A
        Board Tracer Number            : N/A

        Finished Processing Commands Successfully.
        Exiting SAS3Flash.

Intel(R) Xeon(R) CPU E3-1230 v5
X11SSH-CTF with 32GB
6x 3GB WD Red
Freenas 11.2

 

[cods69]

Those are detailed in the Upgrading the Storage Pools section of https://www.ixsystems.com/blog/library/freenas-11-2-release/.

Thanks for that. It was actually the prior releases/upgrade versions you suggested which I was concerned about, but that's ok.
I'm going to hold off for a while, just to let the dust settle. I can't afford time around XMAS stuck without storage, so later in the new year I might stick a toe in the pool.
I appreciate the advice!

 

[SweetAndLow]

Ran my first scrub with 11.2 and it takes half the time of my old scrubs. Old scrubs were 24h and new scrubs are 12h. Newer zfs improvements are awesome!

 

[Zugschlus]

I had the same issue. USB Flash Drives are just too slow for FreeNAS now. Even the USB 3 ones. You should upgrade to SSDs for boot drives. They are like $24 now. Save your config, install fresh on SSD, load config, and you should be good to go. My server also took AGES to reboot and hours to do upgrades until I installed on some cheap 120GB SSDs. iXsystems does not even recommend Flash Drives any more.

That makes FreeNAS even harder to use on the rather common HPE MicroServer which only has four drive bays, two of which would be occupied by the system drives. I would love to have that issue remedied and USB drives possible to use again.

Greetings
Marc

 

[Jailer]

Ran my first scrub with 11.2 and it takes half the time of my old scrubs. Old scrubs were 24h and new scrubs are 12h. Newer zfs improvements are awesome!

Just had my first scrub last night after upgrading and it's the same for me. Used to take 8+ hours previously, last nights scrub finished in 5.

So far I'm very happy with 11.2.

 

[IanWor]

I am hearing a lot about putting the boot drives on SSD. Mine are currently on mirrored USBs and I want to keep them that way for now. Will this be an issue going forward? I mean, will they not work at all?

Adding mirrored SSD's for me will be a PITA as I lack the 2 SATA ports that are needed. If this is the case then a new SATA board will be needed.

 

[gary_1]

I am not a particular 'fan' of the feature where dataset encryption pass-phrases are removed as part of the upgrade process.

I'm sure there is a reason but FreeNAS might want to inform users of this explicitly during the process.

I was a little wary of this, so I rebooted and didn't unlock my data volume before installing the 11.2 stable upgrade. After upgrade completed (took quite a long time) my data volume retained its encryption settings and required a password to unlock as normal.

I have no idea if I'd have encountered the same bug as the above poster had I done the upgrade with my data volume unlocked, but if that issue crops up with others with encrypted pools, this is a simple enough workaround.

 

[jgreco]

That makes FreeNAS even harder to use on the rather common HPE MicroServer which only has four drive bays, two of which would be occupied by the system drives. I would love to have that issue remedied and USB drives possible to use again.

You can remedy this by buying high quality USB flash drives and putting them on USB3 ports. This is not a "new" problem and has basically been around since Day 1 with FreeNAS. The base image has grown from the days when it was installable on a 1GB USB thumb drive. Even back then, it could take a substantial amount of time (half an hour sometimes?) to install. So if you are still using USB2 and crappy low throughput USB thumb drives, the time to perform an upgrade will be substantially greater than it used to be, because the image is much larger. It isn't clear that there's a workable fix to that issue other than to get less crummy USB devices. Or to strip the system down to bare functionality. Which some of us wouldn't mind, but isn't a reasonable business decision for iXsystems, as all the competitive products offer abilities to run VM's and add-ons and all that extra stuff.

It is helpful to remember that FreeNAS exists as a way for iXsystems to get testing and feedback on their TrueNAS product. They have to pay developers to develop this stuff, which comes from the paid product. As nice as it would be to have a stripped down 1GB-image version of FreeNAS without all the bells and whistles, that's not likely to happen. Part of the price we pay as users of the free product is that we get our stuff for "free" but we are basically hitchhiking on the TrueNAS product development. Sometimes it's necessary to move on. We used to have a bunch of FreeNAS users who had built dirt-cheap AMD E350 APU systems with 4GB or less of memory. That's no longer supported. Could it be? Sure, but iXsystems isn't going to sell a non-ZFS TrueNAS box, so it isn't supported anymore. Likewise, it is probably unfortunate that your MicroServer only has four bays. I hacked the one here years ago to put a six-2.5-in-3.5 dealie in the optical bay, and was actually running RAID1 SSD boot years before the ZFS transition happened. You can still use your USB boot drives, it just takes for-freakin'-ever. :-/

I am hearing a lot about putting the boot drives on SSD. Mine are currently on mirrored USBs and I want to keep them that way for now. Will this be an issue going forward? I mean, will they not work at all?

Adding mirrored SSD's for me will be a PITA as I lack the 2 SATA ports that are needed. If this is the case then a new SATA board will be needed.

You want to use SSD because the USB thumb drives have proven inconsistent at best over the last seven years. You can make USB thumb drives more reliable by putting them in a mirror pair (creates redundancy) and moving the system dataset into your hard disk pool (reduces writes). USB thumb drives tend to be slow, don't usually have meaningful wear leveling, and have other negatives associated with them. There isn't really a way to fix that, except to cope with reality, and ideally to buy high quality ones, to reduce the likelihood of problems.

To the best of my knowledge, no one is talking about deprecating support for USB. It's just that they suck, they've always sucked, and they suck worse now that the system is using ZFS for the root filesystem, and there's more stuff that's built into FreeNAS, so the FreeNAS image is bigger, and this means that slow USB devices that merely sucked badly in years past... suck terribly now.

Lots of suck.

 

[Constantin]

I was a little wary of this, so I rebooted and didn't unlock my data volume before installing the 11.2 stable upgrade. After upgrade completed (took quite a long time) my data volume retained its encryption settings and required a password to unlock as normal.

The issue was apparently related to my simultaneous addition of a L2ARC. Apparently, adding a SLOG or a L2ARC drive will automatically require new encryption parameters, such as re-applying a passphrase. This is a current limitation of ZFS that may change in the future.

 

[danb35]

That makes FreeNAS even harder to use on the rather common HPE MicroServer which only has four drive bays, two of which would be occupied by the system drives.

Mirrored SSDs are really unnecessary, and there's no reason that a single SSD needs to take a drive bay--just Velcro it to a convenient spot on the inside of the chassis.

 

[Ericloewe]

The issue was apparently related to my simultaneous addition of a L2ARC. Apparently, adding a SLOG or a L2ARC drive will automatically require new encryption parameters, such as re-applying a passphrase. This is a current limitation of ZFS that may change in the future.

It has nothing to do with ZFS and everything to do with FreeNAS' management of GELI. The L2ARC part is a bit silly, since it's treated as volatile, but I guess it's nice if persistent L2ARC ever becomes a thing.

 

[jgreco]

Mirrored SSDs are really unnecessary, and there's no reason that a single SSD needs to take a drive bay--just Velcro it to a convenient spot on the inside of the chassis.

I had one of the mirrored RAID1 SSD's in my N40L die. "Unnecessary" is a function of how inconvenient downtime is.

 

[kjsdad]

Updated from 11.2-RC1

S.M.A.R.T. service failed to start.

I have disabled smart on all disks and restarted the system. Service still refuses to start.

I found that you can enable them on all disks but not the usb drives.

 

[samulen]

I got my server up last night!

I decided to forget about booting from the 11.2 install media. I decided to run the 11.1 install media (which works flawlessly). I installed it on my SSD ... took about 5 minutes. Rebooted to SSD and I was in! I then ran the upgrade to 11.2 (by changing trains) on the SSD.

After it finished, I was booted into 11.2. I then hooked up one of the USB drives I had installed 11.2 to but couldn't boot from. Somehow, it ended up booting off this USB because all my certificates, etc. were there ... jails, VMs, etc. I then recreated my boot mirror and I'm back in business ... now to upgrade jails...

My end state: 11.2 installed on 1 SSD and 1 USB drive. SSD is my boot device.

This is what I resorted to do too! Installing only worked with update to 11.2 from 11.1, not with the 11.2 installer.

 

[Constantin]

Upgraded pool, changed cases, improved CPU cooling. Added a second SSD to mirror the SLOG. (both are same model S3710 200GB from Intel). New pool has one less disk (8 total) and 2.5x the capacity of the old one.

Started repopulating the NAS with content and am now consistently hitting 80% processor utilization due to multiple concurrent AFP streams hammering the NAS. However, I doubt the cooling was so effective as to achieve a die temperature 20*C below ambient.


...Seems ambitious.


The legacy UI seems to graph the same data differently and more believably.

 

[Dan Tudora]

That makes FreeNAS even harder to use on the rather common HPE MicroServer which only has four drive bays, two of which would be occupied by the system drives. I would love to have that issue remedied and USB drives possible to use again.

Greetings
Marc

hello
Microserver Gen 8 have 5 sata ports, but can't boot from ODD port
put a small SSD on the 5 port and "mod" a power connector from the ODD power wire
install FreeNAS on the SSD
install GRUB on a microSD card or on a usb stick
edit grub.cfg to chainload and boot from SSD
read this post https://forums.freenas.org/index.php?threads/bootloader-on-usb-freenas-on-ssd.56430/#post-395761
enjoy

 

[IanWor]

Hi guys. Looking for some advice here.

My current system - 11.1-U6 is running on mirrored USB boot drives. I have arriving today 2 SSD drives and a new SATA card to add the additional required ports to run them.

What would be the best way for me to move the boot off the USB drives and onto the SSD drives?
Also, do you think there will be any got-cha's in replacing a 4 port SATA card with an 8 port one? Something that I have never done before...

Thanks guys!


Build FreeNAS-11.1-U6 Platform Intel(R) Core(TM) i3-6100 CPU @ 3.70GHz Memory 32616MB
Drives 6 x 2TB / 3 x 3TB
"It's a cube!"

 

[MatthewSteinhoff]

What would be the best way for me to move the boot off the USB drives and onto the SSD drives?

Add the SSDs, expand the boot VDEV from a mirror to a quad. Remove the USB sticks from the group. If everything is hot-swap, you can do it without turning off the server.

Also, do you think there will be any got-cha's in replacing a 4 port SATA card with an 8 port one?

That shouldn't be a problem at all. Be sure the new card's firmware is up to date. Otherwise, one of the best features of FreeNAS/ZFS is that it is controller agnostic. Any port in a storm.

Cheers,
Matt

 

[IanWor]

Add the SSDs, expand the boot VDEV from a mirror to a quad. Remove the USB sticks from the group. If everything is hot-swap, you can do it without turning off the server.



That shouldn't be a problem at all. Be sure the new card's firmware is up to date. Otherwise, one of the best features of FreeNAS/ZFS is that it is controller agnostic. Any port in a storm.

Cheers,
Matt

Thanks! As always this group is awesome!