Is this safe to do? / I might have found a bug in FreeNAS 11.1

[NASbox]

I am writing a script to facilitate the partial automation of backups, and I have a single drive pool that goes in a hot swap on ada3.

Assuming a pool name BACKUP01 is there any reason for me not to manually import the pool with zpool import BACKUP01?

This command defaults to mounting the pool as /BACKUP01 instead of under /mnt and it doesn't show up in the GUI, which is fine with me. Under normal circumstances only my scripts should interact with this pool, and I prefer the command line for this type of work since I can achieve a high degree of automation.

(I am hoping to automate to the point where offline backups are "load drive, run script, answer a couple of confirming questions, wait for backup to finish, check the output for errors and then remove the drive.)

Possible bug alert:
FreeNAS 11.1: I used the GUI to import, and then after use export three pools BACKUP01, BACKUP02, and BACKUP03 all on /dev/ada3. I noticed sometime later that I had a warning flashing in the GUI, and FreeNAS was complaining about not being able to mount ada3! (I cleanly exported each pool with the GUI, and got no error messages.) The message disappeared when I rebooted FreeNAS.

 

[jgreco]

Well, and don't take this the wrong way, it's probably not the "smartest" thing to do, because FreeNAS is really an appliance and is expecting that it is supposed to be managing the entire system for you, but it *is* basically a FreeBSD system under the hood, and my *opinion* is that it is probably safe to do this, but mainly because FreeNAS is pretty paranoid about doing damaging, stupid things to disks unless someone has actually confirmed it through the GUI. So as long as you're the only one working with the system, and you aren't going to tell the GUI to import the disk, it works, but it's paddling upstream in a vaguely risky and potentially dangerous way because you're doing something unanticipated and also untested.

That having been said, I abuse FreeNAS boxes for manipulating disks from the CLI all the time.

 

[joeschmuck]

Possible bug alert:
FreeNAS 11.1: I used the GUI to import, and then after use export three pools BACKUP01, BACKUP02, and BACKUP03 all on /dev/ada3. I noticed sometime later that I had a warning flashing in the GUI, and FreeNAS was complaining about not being able to mount ada3! (I cleanly exported each pool with the GUI, and got no error messages.)

I assume that the alarm message is now clear? You could submit a bug report however if it were me, I'd do a bit of initial troubleshooting. Try to replicate the problem because if the developers cannot recreate it then odds are nothing will happen with it. Include the step by step process on how to replicate this problem in the bug report.

 

[NASbox]

Well, and don't take this the wrong way, it's probably not the "smartest" thing to do, because FreeNAS is really an appliance and is expecting that it is supposed to be managing the entire system for you, but it *is* basically a FreeBSD system under the hood, and my *opinion* is that it is probably safe to do this, but mainly because FreeNAS is pretty paranoid about doing damaging, stupid things to disks unless someone has actually confirmed it through the GUI. So as long as you're the only one working with the system, and you aren't going to tell the GUI to import the disk, it works, but it's paddling upstream in a vaguely risky and potentially dangerous way because you're doing something unanticipated and also untested.

That having been said, I abuse FreeNAS boxes for manipulating disks from the CLI all the time.

Thanks for the comment - it's a catch 22 - FreeNAS is a great appliance that does 98% of what I want the way I want it to... it's that last 2% that needs the work around, and that's exactly why I chose FreeNAS over a sealed commercial appliance because I am free to invent a solution. I've got a huge pool to back up, and my current reality is the only practical way to do it is on external disks. I've found a great hot swap system that seems to work very very well, now I just need the scripts to go around it. Since there is no FreeNAS API for the mounting of disks, I don't have much choice, and in many ways I might be better off becuase FreeNAS isn't likely going to touch a disk that isn't part of the pools under it's control.

I assume that the alarm message is now clear? You could submit a bug report however if it were me, I'd do a bit of initial troubleshooting. Try to replicate the problem because if the developers cannot recreate it then odds are nothing will happen with it. Include the step by step process on how to replicate this problem in the bug report.

I'll see what I can do when a get a few moments. I just put it out there so that if anyone is building or doing other testing they could try import, export and detach and see if it causes problems. If anyone else has a problem, then it's definitely bug report time. Again the issue here is I don't think anyone envisioned 3 different pools on one device, and then having the pool disappear all together even if it was otherwise done completely in accordance with normal established procedures.

Good question "Is the message clear?" I updated the OP to indicate that it cleared on reboot.

 

[jgreco]

Thanks for the comment - it's a catch 22 - FreeNAS is a great appliance that does 98% of what I want the way I want it to... it's that last 2% that needs the work around, and that's exactly why I chose FreeNAS over a sealed commercial appliance because I am free to invent a solution. I've got a huge pool to back up, and my current reality is the only practical way to do it is on external disks. I've found a great hot swap system that seems to work very very well, now I just need the scripts to go around it. Since there is no FreeNAS API for the mounting of disks, I don't have much choice, and in many ways I might be better off becuase FreeNAS isn't likely going to touch a disk that isn't part of the pools under it's control.

I agree, and I suspect it is safe, but I'm always a little skeptical of things being done that are significantly unusual. In my experience that's where the risk is.

 

[NASbox]

I agree, and I suspect it is safe, but I'm always a little skeptical of things being done that are significantly unusual. In my experience that's where the risk is.

Thanks for the confirmation and challenging my thinking.

<rant>
Back in the day I used to backup with a tape drive on mag tape and it was like playing Russian roulette wondering if the tape could be successfully read if/when required. After awhile I gave up and used a removable drive caddy. They weren't hot swap, and because of poor manufacturing they were very hard to install and remove, but they were much better than tape!

I guess today large scale enterprise replicates data with multiple servers.

Home users/small business either uses a cloud service or some sort, a USB drive, or nothing at all.

I guess USB has made it too easy (although less reliable), so hot swap bays aren't mainstream. I got a removable cartridge system from Addonics (who have ZERO clue how to market) that fits well (seems to have really good connectors) and adds < $50 to the cost of a backup drive. Good luck pushing 6-8TB to the cloud over a 20Mbps cable connection!
</rant>

 

[jgreco]

Addonics makes some interesting stuff that is hard to find elsewhere.

Also, I'll follow up to your rant with a comment that pushing 6-8TB to the cloud is itself an issue, in that people expect "the cloud" to be some magic thing that magically protects their data. "The cloud" is just someone else's hardware, and it costs them money, so it is worth figuring out how them storing your data is paying for itself.

 

[NASbox]

Addonics makes some interesting stuff that is hard to find elsewhere.

Also, I'll follow up to your rant with a comment that pushing 6-8TB to the cloud is itself an issue, in that people expect "the cloud" to be some magic thing that magically protects their data. "The cloud" is just someone else's hardware, and it costs them money, so it is worth figuring out how them storing your data is paying for itself.

So VERY true!

As I'm thinking about this, I'm wondering how it would scale to a small enterprise.
(Say 8 x 8 TB RaidZ2)

Set up a second server pulling snapshots but with a fully removable media.

Once per day/two days/week (as appropriate) export the pool, pull all 8 drives and ship them to some secure location. Then load a pool from an earlier rotation for update.

I think you would have a very hard job providing that level of data security (and privacy) without massive Internet infrastructure. I'll bet that the cost of cloud storage would pay for the hardware in less than a year. (And how long will it take you to get your data back if you need to recover it from the cloud?) Depending on the value of the value of the assets involved, you may want to do both if you are worried about natural disaster. Or just keep the data from between media rotations on the cloud. If transfer in is free, and you rotate to keep the volume low, it cost would be a lot lower. You only get hit with bandwidth costs if you need to recover, and you would be very happy to pay it under those circumstances.

 

[jgreco]

You only get hit with bandwidth costs if you need to recover, and you would be very happy to pay it under those circumstances.

Yeah? But you don't discover the other logistical nightmares until you actually try to do the recovery. It's funny how many people have never tried their recovery plan. Consider, for example, restoring file level backups to HDD. Then you find out that you need to recover a terabyte of small files, slowly archived over many years, but restoring them will take a month of seeks. ...sigh