To boot with USB or SSD or NVMe ?

[GChuck]

I have finally received the hardware for my new FreeNAS server. Probably not up to the spec most gurus here request, but it was what was both available and in my price range. Motherboard is ASUS P11C-M/4L, with Xeon E-2236 cpu, 32GB ECC memory and 4 x 4TB Seagate IronWolf drives. This motherboard has support for 2 NVMe devices. So at the same time I also purchased a 256GB NVMe stick, a 250GB WD Blue SSD and four Transcend USB sticks.

So the question I'm posing is it better to use a single NVMe or SSD as a FreeNAS boot drive, or to continue using 2 USB sticks in a mirror?

The cost for the USB sticks is minimal, but I have had two USB failures in the two months I've been testing FreeNAS. Under 11.2-U8, replacing the USB stick on the fly was pretty simple. But that menu item is now missing in the 11.3-U1 GUI.

Both the NVMe and SSD cost about 5 times more than the USBs. I've used a WD Blue SSD as my boot drive on my Windows box for about 3 years with absolutely no problems and I suspect the NVMe would have the same error free longevity. I could mirror either the NVMe or the SSD, but as rebuilding a FreeNas server seems quite trivial, I don't see the need.

I just need some suggestions before I make my final build. I just want to get it right.

Thank you.

Greg ...

 

[Yorick]

SSD for sure, in whatever form. USB is just a pain. The older 2.0 sticks don't die as quickly and, still a pain.
Side note: SATA and NVMe are protocols, SSD is a storage technology. An "NVMe stick" is an SSD.

No need to mirror the SSD, as long as you back up your config daily.

You can get 40GB Intel SATA SSDs for roughly 20 bucks on eBay, those will do great. Or use something new, of course. M.2 NVMe x4 goes for about 40-45 bucks new (120 to 128GB). That board has M.2, so that'd be an easy way to boot and not need a SATA port for it.

 

[GChuck]

Thank you Yorick.

Since I already have the NVMe stick and have tested that it will boot FreeNAS, I think that's what I'll use. It maybe overkill for a boot drive but hopefully, it won't fail anytime soon!

Greg ...

 

[JaimieV]

Remember that you can't use the boot device for anything except the boot pool - so a few gig of FreeNAS plus logs. If you have a diddly little crap NVMe or SATA SSD I'd use that instead of tying up a 256gig one you could use elsewhere.

 

[Jailer]

Remember that you can't use the boot device for anything except the boot pool - so a few gig of FreeNAS plus logs. If you have a diddly little crap NVMe or SATA SSD I'd use that instead of tying up a 256gig one you could use elsewhere.

Or a cheap server pull SSD from ebay for $10-$15 would be a good option. Lots of good Intel ones out there with plenty of life left in them for a boot device.

 

[alprio]

Hello,
My mobo has 4 sata ports.
I want to free the 1(the boot drive), cause I want to add an ssd for my vm's.
Can I transfer my boot disk to an external case for booting from usb?
Or is better to get 2 16gb flash drives for mirroring booting ?

 

[Yorick]

USB to SATA works okay, as long as the adapter is decent.

An idea using an adapter for M.2 drives is here:https://www.truenas.com/community/t...my-usb-mirrored-boot-drive.86174/#post-596973

 

[alprio]

USB to SATA works okay, as long as the adapter is decent.

An idea using an adapter for M.2 drives is here:https://www.truenas.com/community/t...my-usb-mirrored-boot-drive.86174/#post-596973

USB to SATA works okay, as long as the adapter is decent.

An idea using an adapter for M.2 drives is here:https://www.truenas.com/community/t...my-usb-mirrored-boot-drive.86174/#post-596973

Very good idea. Thank you.
I will order this system.
can I use mirror ?

 

[joeschmuck]

Or is better to get 2 16gb flash drives for mirroring booting ?

Absolutely not a good idea. You do not need to mirror the boot drives. A decent USB 2.0 drive will work fine but I too like the USB to M.2 adapter, hopefully it works well for you and you only need one.

 

[alprio]

Absolutely not a good idea. You do not need to mirror the boot drives. A decent USB 2.0 drive will work fine but I too like the USB to M.2 adapter, hopefully it works well for you and you only need one.

Thank you very much.I have a usb2 HDD case. I will use it until I purchase the m2 adapter.

 

[Dan Tudora]

Absolutely not a good idea. You do not need to mirror the boot drives

hello
why not ??
I have a old system (HP Microserver Gen8) with that kind of setup (2 USB2 stick in mirrror kingston/metal gold, gold is color ) and is working for many years now
that system have 9.3/9.10/11.1 FreeeNAS version
and YES is in this moment is "backup of the backup server" and working
BUT for the OP is much good solution to go with the USB=>SATA soution
cheers

 

[Herr_Merlin]

Absolutely not a good idea. You do not need to mirror the boot drives. A decent USB 2.0 drive will work fine but I too like the USB to M.2 adapter, hopefully it works well for you and you only need one.

Why the heck not?
It's one of the best ideas actually

 

[Jailer]

Why the heck not?

why not ??

Because USB drives as a boot device are notoriously unreliable. Even iX doesn't recommend them anymore.

 

[Yorick]

Maybe distinguishing between USB “flash sticks” and USB to SATA m.2 adapters makes sense. Granted it’s only been a year or so that people have been using these, and, no reports of failures so far.

 

[joeschmuck]

Why the heck not?
It's one of the best ideas actually

My comment was against USB Flash Drives as indicated in the posting, it is not a good idea and my opinion stands, it is not a good idea at all. USB Flash drives don't last. Sure you might be able to get some that last a few years as a boot device but they do fail often. USB 3.0 devices fail more frequently and that has been well documented already, no reason to go over it again. The M.2 to USB adapter sounds like a viable option but I would not mirror it, there is no need if you maintain a backup of your config file. Recovery is super fast.

 

[G8One2]

I think I read somewhere here on the forums, that earlier versions of FreeNas were compatible with USB boot devices because once booted, all the OS activity was handled on the USB. Newer versions of FreeNas/TrueNas have much more OS activity than that of earlier versions. So much so that USB flash drives arent reliable enough to keep up with constant read/write activity. This is why it is no longer recommended to use USB flash drives as the OS boot device. Now i could be completly wrong about this, but I'm fairly certain I read something along those lines about that, a while back.

 

[alprio]

I think I read somewhere here on the forums, that earlier versions of FreeNas were compatible with USB boot devices because once booted, all the OS activity was handled on the USB. Newer versions of FreeNas/TrueNas have much more OS activity than that of earlier versions. So much so that USB flash drives arent reliable enough to keep up with constant read/write activity. This is why it is no longer recommended to use USB flash drives as the OS boot device. Now i could be completly wrong about this, but I'm fairly certain I read something along those lines about that, a while back.

I read that too somewhere. Thats why I asked. I moved my HDD to an external usb 2 case. Works like a charm until I buy the m2 USB adapter.
SSD for VM's works too. So rn I am good.
I placed the hdd case inside the case. You never know what is going to happen if you let It out..

 

[Dan Tudora]

Because USB drives as a boot device are notoriously unreliable. Even iX doesn't recommend them anymore.

OK. that is correct and DO not FOLLOW THAT WAY

My comment was against USB Flash Drives as indicated in the posting, it is not a good idea and my opinion stands

for OP fallow "an Old Man advice" go with other solution for boot device, not USB stick, USB3 or USB2 whatever

Thats why I asked. I moved my HDD to an external usb 2 case

that is for DATA HDD ??
if YES is a weird solution

 

[Constantin]

I consider a mirrored pool to be good insurance but it comes down to use case.

I have a mirrored boot pool (2x64GB SATADOM and 2x128GB SATA SSD) in both of my servers. BUT, both of my servers have plenty of SATA ports (20, 12, respectively) and losing your boot with a GELI encrypted pool has been a problem for others in the past. With TrueNAS 12, this may no longer be as big of an issue but for me, 2 system drives per server is easily accommodated and I sleep better.

Another perfect use case are remote systems (for backup) where it might take you a while to get to them.

SSDs that are well-suited for the boot pool (<=128GB) now cost less than $20 new or you can go for data-center grade gear at the same price used. So if you have the space / ports, there is nothing wrong with having a mirrored boot pool other than minimal added power consumption. Compared to the HDDs, that's peanuts... and... how much is that data worth to you? (esp. if it's encrypted).

 

[joeschmuck]

I consider a mirrored pool to be good insurance but it comes down to use case.

I couldn't agree more, the use case matters.

One of the big things with using a mirrored boot pool is that in most configurations I have seen here, if the primary boot drive fails then the user has a dead system until they reconfigure the motherboard to boot from the alternate/good device and sometimes that also means physically removing the failed device. The only true way to implement a mirrored boot pool that would bootstrap even with a fault is to use a RAID card that would control this action, software RAID can't do this. Many people think that because FreeNAS/TrueNAS in configured as a mirror boot pool that it is fully automatic, It isn't. Use case matters here and I feel that if you do have a mirrored boot pool AND are remotely managing the device, you could save the day remotely via IPMI and reconfigure the boot device and get the system back online. If you are a local user then rebuilding the boot drive is super easy and fairly quick if you have a copy of your config file, or you could use a mirror and then reconfigure it once you figure out which drive failed.

So the reason you will see me recommend no mirrored boot pool is for the above reasons: 1) It doesn't mean the system will bootstrap if the primary boot device fails, 2) It easy to maintain a copy of the config file and just rebuild the boot device.

I just think that anyone who uses FreeNAS/TrueNAS and is using a mirrored boot pool should understand what they are getting and not getting.

I'm crawling down off my soapbox now.

P.S. This is just my opinion and advice. No one needs to accept or take it. And I hope everyone is enjoying the holidays!