Mirroring the FreeNAS installation itself

[Eric Bautsch]

Hi.

Please point me in the right direction if I have overlooked this topic somewhere...
I've got a FreeNAS system with a USB stick and two disks where the disks are for data only and the USB stick carries the FreeNAS installation.
When I set it up, I tried to mirror the USB stick to another USB stick and run the entire thing mirrored, but following the FreeBSD guides to mirroring failed miserably (I'm a Solaris man myself). I eventually gave up and wanted to come back to it when I had a bit more time on my hands, but alas, over night, my FreeNAS died because after less than a year, my USB stick seems to have had it.
I'm currently recovering my FreeNAS using a new USB stick, but is there a way to mirror the FreeNAS installation itself? Does anyone have instructions?
Thanks for your help.
Ciao,
Eric

 

[gpsguy]

No, it won't work - there was a discussion about it yesterday, too.

If you're worried about it, make another (second) USB stick now, and restore the current configuration file to it. Tape it to the box.

Consider creating a cronjob to backup /data/freenas-v1.db on a regular basis. This is the configuration file.

 

[Eric Bautsch]

Thanks for the quick response.
Also thanks for the location of the config file, that would have been my next question... ;-)

Shouldn't it work though? RFE perhaps?

Eric

 

[cyberjock]

No. The expectation is that:

1. You'd use a USB stick(not a hard drive or RAID controller)
2. Your USB stick would be a name brand(so as to not die from poor quality components).
3. The USB stick will be mounted read-only to extend the USB stick lifespan.
4. The USB stick will last so long you'll probably never have to replace it.
5. You'll be smart enough to backup your config file.
6. Recovery will take 15 mins(make a USB stick and restore your config) so why care about RAIDs?

 

[Eric Bautsch]

Assuming you're not trolling (and it's a reach)

1. I am using a USB stick
2. It was a brand name
3. I haven't asked it to be mounted any particular way, surely (since you're suggesting this all should be outside of user config reach) FreeNAS would take care of mounting it read only.
4. Well, it didn't.
5. I did, but that doesn't help me keep the system up when the only USB stick in the system fails.
6. Have you actually heard about data availability?

Eric

 

[Knowltey]

Yeah #3 should be the default behaviour. It's intended to be a running image loaded into RAM at boot.

#4 - Shit happens, even the best components can still fail. But recovery takes no longer than 15 minutes. If it's a big concern you can always keep a spare USB with the config updated to it whenever you make a change and at that point it'll literally be plug in and go. Just save the config out of FreeNAS and then have the other USB boot up and Upload Config to it. You may be able to even do it from an alternate machine, not sure if the config requires the drives actually be present or not? (Someone more knowledgeable can probably tell you if this is indeed true)

#5 - USB stick failure is so rare it's not a major concern. The USB is literally only ever being touched (read or write) in two situations: A- at boot. B- When making a configuration change to certain options. Any other time the USB is sitting there doing absolutely nothing. Given a non-fault USB and a non-fault HDD, your HDD will statistically fail at least 10 times before the USB stick ever does simply from mechanical wear.

#6 - See response to #5. USB Failure is extremely rare in this use scenario. If it really concerns you keep a preconfigured USB handy and dowtime will be literally the time it takes for the machine to boot, probably 1 or 2 minutes at most. Judging by your concern about data availability I'll assume you intend to leave your FreeNAS machine on 24/7. This means that you'll load the 2GB off the USB stick once and never again, and then you'll only be writing a very miniscule amount of data to the USB stick if you ever need to change a configuration setting.

 

[cyberjock]

I'm really not trolling. I was trying to explain the constraints of what the FreeNAS designers expected.

FreeNAS does take care of mounting it read-only. You don't have to do anything special as this is designed in. :)

I currently manage 6 FreeNAS servers. The oldest ones are 2 years old. So far I've had zero failures.

One problem is what people define as "name brand". While I think "Corsair, Kingston, Samsung" we've had ALOT of people mention brands I've never heard of, some of them being local companies in that country that are everywhere and so taken to mean "name brand". Not surprisingly, they don't always last.

Additionally, a large number of people choose to use SD card with a USB converter which isn't the same as a USB stick. Those people will claim up and down that "I'm using a USB stick" until you start asking detailed questions and then they admit "it's not a USB stick but an SD card with converter is the same". It's not. So you also have to consider people that won't really use a real tried and true USB stick but will use something they consider to be identical.

I have heard of data availability. But, based on how many people use USB and how low the failure rate is, it's not a big problem. When you choose to use systems that aren't designed by you, you have to accept some engineering decisions. You should also be doing regular upgrades of FreeNAS for security reasons alone, and just the reboot time for upgrades throughout the year could ruin your contractually obligated availability.

If you are looking for reliability I'd recommend you get the smallest/cheapest Intel SSD you can find. Those things seem to never fail(0.4% per year statistically) beating virtually every other brand and media type that exists, sometimes by an order of magnitude. There's no reason why an Intel SSD won't work. You'll just have a bunch of wasted space on the drive. We have a mod that uses an SSD for just that reason.

 

[Dusan]

Yeah #3 should be the default behaviour. It's intended to be a running image loaded into RAM at boot.

Well, almost. /data is mounted read-write and the reporting history (rrd databases) are saved there every hour.

 

[jgreco]

I think part of the problem is that there's a bit of a quality difference between SATA DOM units (which is what I'm guessing that TrueNAS uses for system boot) and the reality of make-it-cheap USB sticks ... even the "name brands" may suffer. Local experiences with USB sticks for servers are not fantastic, even "good" ones. It should be noted that even with a sophisticated controller managing flash storage, even SSD's do die now and then. I do not think your average USB flash enjoys anything so complex. Especially for complex behaviours such as wear-leveling, I do not trust them to be present in a USB stick, and the constant flushing of RRD updates to the stick cannot be all that great for it.

From my perspective, this is one of the things that makes virtualizing FreeNAS somewhat attractive. It is difficult to justify the cost and complexity of boot redundancy for a handful of gigs, but for a bunch of VM's, there it is not so difficult. Nothing ticks me off more than to reboot a FreeNAS box and have it NOT come back up because flash is wonky. Seen that happen several times, almost always at an inconvenient time. So now even the MicroServer boots off a pair of cheap old SSD's in RAID1. Plus it is plenty fast that way. And it is very nice booting off protected storage under ESXi too.