Best practices for drive replacement

[wags22]

Running Freenas 9.2 configured with four 1 TB drives in a RAID 5 configuration with a spare. One of the drives experienced mechanical failure and had to be removed. System is back up, but in a degraded state.

First question: How do I "stabilize" the system with the remaining 3 drives.

Second question: I'm planning on replacing the 1 TB drives with 2 TB drives in same configuration. I have one drive that I want to add and move the data to, then add the other 3, setup the raid 5, transfer data to it, then add the first drive as the spare. Right now I want to start with adding 1st drive and transferring date. What is the best way to do it?

Thanks for any assistance in advance.
Wags

 

[DifferentStrokes]

The FreeNAS manual outlines this well. There is a link at the top of the forum page.

 

[Robert Trevellyan]

How do I "stabilize" the system with the remaining 3 drives.

Not sure what you mean. The pool will remain degraded until you restore the full complement of working drives.

I'm planning on replacing the 1 TB drives with 2 TB drives in same configuration. I have one drive that I want to add and move the data to, then add the other 3, setup the raid 5, transfer data to it, then add the first drive as the spare. Right now I want to start with adding 1st drive and transferring date. What is the best way to do it?

After replacing the currently failed drive, follow the directions in the manual for replacing drives to grow a ZFS pool. This will be much safer than copying everything to one drive and destroying and recreating the pool.

 

[wags22]

I have a full complement of drives - 3 1TB drives and a single 2TB drive. The manual isn't very clear on how to repair it via the gui, especially since I wasn't able to nicely remove the failed drive - hardware failure that prevented system boot. What I'm looking for is a nice step by step guide for doing it through the shell. I've done it before, but don't remember where I found the guide.

What you're suggesting, Robert, is to grow the pool to the new drive, then "remove" the smaller drives via the gui? If so, I hadn't thought of that, but I think I will try it.

 

[SweetAndLow]

I wouldn't suggest using the cli. All you have to do is offline the drive in the GUI if it isn't already offline. Then put in the new drive. Then click replace and select the new drive.

 

[wags22]

I wouldn't suggest using the cli. All you have to do is offline the drive in the GUI if it isn't already offline. Then put in the new drive. Then click replace and select the new drive.

You obviously didn't read my post, or follow-up post. The hard drive failed, it failed so badly that the system crashed and wouldn't boot up because of that drive...I had to shut it off and figure out which drive was failing and remove it to get the system to boot again.

I don't trust GUI's as much as CLI's, but I'm not familiar enough with freenas, just want someone to point me to a good cheatsheet or post that has what I need to know to restore the pool and expand it via CLI.

 

[danb35]

There is no "good cheatsheet or post" on how to replace a drive using the CLI, because it shouldn't be done from the CLI. The manual has click-by-click instructions for replacing a disk using the GUI; follow those.

 

[SweetAndLow]

You obviously didn't read my post, or follow-up post. The hard drive failed, it failed so badly that the system crashed and wouldn't boot up because of that drive...I had to shut it off and figure out which drive was failing and remove it to get the system to boot again.

I don't trust GUI's as much as CLI's, but I'm not familiar enough with freenas, just want someone to point me to a good cheatsheet or post that has what I need to know to restore the pool and expand it via CLI.

If you don't trust gui's then don't use freenas because that is what freenas is. If you just want cli use plain freebsd. Not trusting the GUI doesn't make any sense, it just uses the cli under the covers but it does it's own special way of doing things and stores information in a database so it knows the current state. It sounds like you don't quite know enough yet to be using the cli in this manor. Anyone who wants to do something like this with the cli shouldn't have to ask how to do it.

 

[Robert Trevellyan]

What you're suggesting, Robert, is to grow the pool to the new drive, then "remove" the smaller drives via the gui? If so, I hadn't thought of that, but I think I will try it.

Not exactly. The procedure is actually to replace the 1TB drives one at a time with 2TB drives, using the procedure described in the documentation for growing a pool by replacing drives with larger drives.

 

[cyberjock]

You obviously didn't read my post, or follow-up post. The hard drive failed, it failed so badly that the system crashed and wouldn't boot up because of that drive...I had to shut it off and figure out which drive was failing and remove it to get the system to boot again.

I don't trust GUI's as much as CLI's, but I'm not familiar enough with freenas, just want someone to point me to a good cheatsheet or post that has what I need to know to restore the pool and expand it via CLI.

What @SweetAndLow said. If you aren't going to trust the WebGUI, you are the weakest link in your server. Mixing your own "secret sauce of commands" from the CLI when FreeNAS expects to do things "its way" is a recipe for disaster. Use the WebGUI. Choosing to do the CLI means things will appear to be fine. You'll have no warnings, no errors, nothing. Then one day you'll be here asking where all your data went and how to recover, and the answers will be "you don't recover... that's it".

The reason you can't find a cheatsheet is because it's more of a "how to thorough f*ck your server from the CLI" sheet. The "cheatsheet" is to use the WebGUI. That is the only proper way to admin your FreeNAS server. If you don't like this you should be using FreeBSD so that you can have total control of everything.

You do realize that companies pay iXsystems money to manage their servers, and they do so all the time from the WebGUI.

 

[wags22]

If the manual's way is the only way, then the manual is thoroughly lacking as it doesn't address my situation. At this point I've used the gui to add two new drives in as spares, but I'm still showing a degraded status. The manual gives no guidance on how to add/replace a drive that was not removed via the gui.

I've been a linux admin for 15 years. I don't trust gui's because they don't always work the way they're advertised. With cli I know what command is being input and can watch the output. I guess what I was really asking for is cli manual for Freenas.

 

[SweetAndLow]

If the manual's way is the only way, then the manual is thoroughly lacking as it doesn't address my situation. At this point I've used the gui to add two new drives in as spares, but I'm still showing a degraded status. The manual gives no guidance on how to add/replace a drive that was not removed via the gui.

I've been a linux admin for 15 years. I don't trust gui's because they don't always work the way they're advertised. With cli I know what command is being input and can watch the output. I guess what I was really asking for is cli manual for Freenas.

It doesn't mean anything when you say you have been an linux admin for 15 years. There are kids coming straight out of college with the same skill as you. What makes the difference is the ability to follow directions. I'm not sure where you got the idea that adding spares would fix your degraded pool. Spares are just a way to keep a drive warm in your server for when you need to replace a failed drive. To actually fix a degraded pool you need to offline the bad drive and replace it with a new drive. However you go about doing that is up to you. I highly suggest using the gui since it is a matter of clicking 2 buttons, the CLI will be roughly 5 commands that you can get wrong.

 

[wags22]

I must be missing something, nowhere in the gui do i see the option to offline or replace drives. The only options I have under View Volumes are the Volume managers, import volumes, or view disks. Under View disks, the only options I have when I highlight a disk is to edit or wipe.

Also, the only drives that show up in the gui are the 4 currently attached and running ones. The two drives that failed had to be physically removed as they prevented the system from booting.

I was able to replace one of the missing failed drives via the cli with zpool replace. Now I just have to figure out how to remove the remaining drive from being a hot spare to being part of the pool.

 

[pirateghost]

..........

http://doc.freenas.org/9.3/freenas_storage.html#replacing-a-failed-drive


covered pretty well with pics

 

[danb35]

I must be missing something, nowhere in the gui do i see the option to offline or replace drives.

Yes, you're missing something. @pirateghost gave the link to the manual, but you go to Storage, click on the pool , then click on the volume status button (at the bottom, it looks like a sheet of notebook paper). From the volume status page, click on one of the disks, and you'll see the Edit Disk, Offline, and Replace buttons appear at the bottom.

Edit: ...but these instructions (and @pirateghost's link) are for 9.3, and you said you're running 9.2. The 9.2.1 manual is at http://web.freenas.org/images/resources/freenas9.2.1/freenas9.2.1_guide.pdf, and the instructions on replacing a failed disk are in section 6.3.12.

 

[wags22]

Yes, you're missing something. @pirateghost gave the link to the manual, but you go to Storage, click on the pool , then click on the volume status button (at the bottom, it looks like a sheet of notebook paper). From the volume status page, click on one of the disks, and you'll see the Edit Disk, Offline, and Replace buttons appear at the bottom.

Edit: ...but these instructions (and @pirateghost's link) are for 9.3, and you said you're running 9.2. The 9.2.1 manual is at http://web.freenas.org/images/resources/freenas9.2.1/freenas9.2.1_guide.pdf, and the instructions on replacing a failed disk are in section 6.3.12.

Thank you danb35. I finally saw what everyone was talking about, however, my cli solution worked, and after resilvering, I now have a healthy pool again.
Apparently I can't add the 2 TB drive to the raid pool, so I will either have to set it up as a separate pool, or just leave it offline until I'm ready to start replacing the current drives.

 

[Robert Trevellyan]

Apparently I can't add the 2 TB drive to the raid pool

No, the only type of vdev you can currently modify is a mirror (or a single-disk stripe, which you can turn into a mirror). For a while there's been talk of that situation improving, but nothing has landed yet. You might want to read @cyberjock 's guide for newbies, linked from his signature, to make sure you have a good understanding of the basics.