Learning freenas offline disk how to online?

[m3ki]

Hi,

I have been digging through the manual and can't find the answer to this question.

I have a 24 bay hot swap freenas machine, I am setting up. 4 stripes of 6x RaidZ2 (Encrypted), 32GB ECC ram ...
Hypothetical question:
If I take one drive in the pool and do OFFLINE.
How do I get it back into the pool?
Once I pull the drive then put it back I only have "Replace button" and no way to put it back into the pool except to press replace then do force add. Then resilver.

Am I missing something?

Sorry if this is a dumb question.

edit: added encrypted

 

[Bidule0hm]

The most simple thing to do is to reboot. It should take the drive back in the pool during the import.

 

[m3ki]

Hmm... I was hoping with hotplug I wouldn't need to reboot.
So freenas doesn't recognize the drive was put back in?

 

[Bidule0hm]

Ok, then there's the CLI but you need to know exactly what you're doing or you can ruin everything pretty easily. I'll let a more knowledgeable member to answer this one (ping @jgreco ).

 

[m3ki]

Thank you that would be helpful, I was wondering what people do in production environments.

I also saw this thread, I am not sure it's same scenario (answer by cyberjock):

First.. I hope you didn't just unplug the device with the server running if you have important data on the zpool. The manual specifically states that's a very bad idea if you value your data.

Second.. All you have to do(if your system supports hotswap) is to plug the drive in and after the system reacknowledges the drive it will automatically add it to the array again. Then you just need to do a scrub to fix any issues with data inconsistencies between the zpool and the drive you removed.

edit: formatting

 

[jgreco]

Thank you that would be helpful, I was wondering what people do in production environments.

This is going to sound not-nice, but in production environments we don't offline disks for trite reasons.

It's a particularly dangerous game to play with the "add-on" encryption layer FreeNAS uses to get the encrypted pool functionality.

I don't know for sure how the encryption layer will play with the offlining of the disk - it may have no impact at all.

So for a production environment where I had actually done such a thing, I would then go over to my testbed environment, repeat the experiment in small (probably virtualized) scale and then figure out whether or not a simple reboot would resolve the issue, which I'm *guessing* it would.

Then I would probably cause myself to write "The time to play is during burn-in" a hundred times on the whiteboard. Heh.

But in all seriousness, reducing the redundancy of your pool is a dangerous game.

 

[m3ki]

This is going to sound not-nice, but in production environments we don't offline disks for trite reasons.

I absolutely agree with you :)
My train of thought was more along the lines, cable goes bad or something like that and drive falls out of the array. (Drive is not bad but cable is for example)

Can I s.... Nevermind.... I would have to shut down the system anyway to replace that...

So what's the point of "Offline Disk" functionality if you cannot online it again?

Let's say I made a mistake (admittedly stupid) and offlined the wrong disk, when trying to replace another one. There is no way to bring it back online without rebooting?

This is going to sound not-nice, but in production environments we don't offline disks for trite reasons.
It's a particularly dangerous game to play with the "add-on" encryption layer FreeNAS uses to get the encrypted pool functionality.

Are you saying I shouldn't use encryption layer on freenas?

Am I going to be in trouble when disk fails if I use encryption? (I followed few procedures outlined in the docks how to replace encrypted disks and it seemed ok)

This is going to sound not-nice, but in production environments we don't offline disks for trite reasons.
I don't know for sure how the encryption layer will play with the offlining of the disk - it may have no impact at all.

So for a production environment where I had actually done such a thing, I would then go over to my testbed environment, repeat the experiment in small (probably virtualized) scale and then figure out whether or not a simple reboot would resolve the issue, which I'm *guessing* it would.

makes sense

This is going to sound not-nice, but in production environments we don't offline disks for trite reasons.
Then I would probably cause myself to write "The time to play is during burn-in" a hundred times on the whiteboard. Heh.

But in all seriousness, reducing the redundancy of your pool is a dangerous game.

Well yes, now is my playtime :) Before I move all my data to freenas.

 

[jgreco]

I absolutely agree with you :)
My train of thought was more along the lines, cable goes bad or something like that and drive falls out of the array. (Drive is not bad but cable is for example)

Of course.

So what's the point of "Offline Disk" functionality if you cannot online it again?

Offlining the disk is the first step in replacing the disk. You tell ZFS that you're taking part of one of its vdevs offline so that ZFS stops accessing the device.

Let's say I made a mistake (admittedly stupid) and offlined the wrong disk, when trying to replace another one. There is no way to bring it back online without rebooting?

Of course there is, but technically now that disk is out of sync with the rest of the pool and it needs to be resilvered back into the pool. This is actually a fast operation as long as things haven't diverged much.

I doubt any developer time has been spent creating a "Just kidding-bring disk back online" button because in the normal operational scheme, you'd never do this. There are definitely ZFS commands to do this.

http://docs.oracle.com/cd/E19253-01/819-5461/gazgm/index.html

Are you saying I shouldn't use encryption layer on freenas?

The encryption layer is an idea that sounds good in theory but the implementation has always been a bit rocky.

Am I going to be in trouble when disk fails if I use encryption? (I followed few procedures outlined in the docks how to replace encrypted disks and it seemed ok)

I don't know. That mostly depends on whether you've dotted all your i's and crossed all your t's. Enough people have screwed it up that I don't feel comfortable suggesting encryption to users.

makes sense
Well yes, now is my playtime :) Before I move all my data to freenas.

Well, good You're already ahead of many of our overly anxious-to-deploy users, and I congratulate you for that. Now is the perfect time to figure all this stuff out.

 

[m3ki]

Of course.


The encryption layer is an idea that sounds good in theory but the implementation has always been a bit rocky.



I don't know. That mostly depends on whether you've dotted all your i's and crossed all your t's. Enough people have screwed it up that I don't feel comfortable suggesting encryption to users.

Thank you very much for your reply.

Now you got me worried... re encryption.

I love the idea of if the drive fails I just send it back to the manufacturer. without using the sledgehammer.

So if I follow these steps will I be ok?
Or is the implementation of encryption layer is crappy?( meaning stuff just breaks randomly libraries/gremlins...)

1. Disk is dead/Bad sectors errrors etc...
2. Offline disk
3. Put new disk in
4. Rekey
5. Create passphrase
6. Download key
7. Create recovery key
8. Download recovery key

 

[jgreco]

Whatever procedure that's in the manual is expected to work and we've beaten both the developers and the doc writers when that hasn't been true. Still, it is a finicky process with little room for error.

For those of us who entrust our data to ZFS because we want data security, the idea of introducing something that could break the pool is a scary idea.

My recollection is that encryption is either not going to be supported or will be supported in a different fashion moving forward, but I leave that to you to research. Sorry.

 

[m3ki]

Oh boy you got me worried, I already started moving data into freenas, though I can restart and start anew with unencrypted volume I really wanted to use encryption. It was one of the reasons I went with freenas.

I also just ran into a weird issue with importing an encrypted volume, and I don't know what to do(regarding to encrypt or not to encrypt).

Steps to reproduce:

1. Detach an encrypted volume (keep second (main) volume attached)
2. Try to reimport on step 2 of 3 no disks show up.
3. Reboot
4. Unlock main volume (all good)
5. Try to import secondary encrypted volume same issue no disks show up
6. Reboot
7. Do not unlock main volume
8. Try to Import secondary volume (disks show up)
9. import (all good)
10. Unlock Main volume

What gives!?

I am worried now, Should I bother with encryption (though I really want to use it)

Or should I wait for freenas 10?

 

[maglin]

Use another form of encryption or just roll with it.

As far as a HDD with data just degaussing is enough to take care of that issue.


Sent from my iPhone using Tapatalk

 

[Mirfster]

Personally I would avoid encryption. I think others have suggested using other tools/software to just encrypt a folder/container (Think VeraCrypt was mentioned). Even if I had a disk that was encrypted and had to send it back for RMA, I would still DoD Wipe it anyways, degauss or sledgehammer.

/Eh maglin beat me to it

 

[m3ki]

Yeah, hmm... what if the drive is clicking and you cannot access it to use dban?

Roll with it (encryption) Lol I just made a mistake while testing didn't backup a key again. Thus locked myself out of the volume. ... current implementation kinda sucks. And I am obsessive.

Is it true that Freenas 10 wont support encryption in its current form?

I found one thread but it's vague.

How much data can be recovered if one uses 4x6 RaidZ2 ? if I send one drive back? (personal financial docs)

 

[jgreco]

Yeah, hmm... what if the drive is clicking and you cannot access it to use dban?

You make minor adjustments with the five pound sledge until it is in an appropriate operational profile.

Some people like thermite, but there's too much room for confusion in these days of terrorist-paranoia.

Roll with it (encryption) Lol I just made a mistake while testing didn't backup a key again. Thus locked myself out of the volume. ... current implementation kinda sucks. And I am obsessive.

Which is why we don't suggest it for "would be nice to have" scenarios.

Is it true that Freenas 10 wont support encryption in its current form?

It's been said. But lots of stuff gets said that is 30% B.S. at least.

I found one thread but it's vague.

How much data can be recovered if one uses 4x6 RaidZ2 ? if I send one drive back? (personal financial docs)

About 1/4th of it, if you look at it directly. But the other half of that question would have to be, why would anyone try? For a compressed pool, decompression of partial ZFS records would be fairly daunting to recover anything meaningful. So the practical answer is more likely to be "not very much." And then on top of it, if you're sending your drive for depot repair, you use a secure trackable shipper like FedEx or UPS, and it arrives at the facility, and part of the process of recertifying the drive once repaired is to blank the drive. It isn't clear that there's a major risk here. If you're worried about your personal financial data, it's easier for someone to trojan e-mail you.