Replace disk with larger disk (disk almost full)

[bassengunder]

Hi,

I have a FreeNAS 11 (stable) server with only one disk, but with several ZFS volumes/shares. The disk is now nearly full (93% capacity used), so I simply want to replace my 4 TB disk with a larger 10 TB disk. Since there is only one disk, there is no raid. What is the easiest way to copy the disk content and volume config to the new disk, i.e. migrate to the newer and larger disk? The new disk is already mounted in server computer, and appears in the FreeNAS web-GUI -> View Disks (ada1 10 TB).

I FreeNAS web-GUI, I tried the "replace" option from FreeNAS web-GUI -> View Volumes -> Volume Status -> Replace. The resilver process completed, but with an error. Maybe this is not an option in raid0?

Any help is much appreciated!

 

[danb35]

The resilver process completed, but with an error.

What did the error say?

 

[wblock]

See Replication Tasks. The sending and receiving server are the same, in this case.

Of course, with no redundancy, a single drive failure could cause the loss of all of your data. Or a user error, like trying to replace the only drive in a stripe. Before the new drive has a lot of data on it, why not get another one and make a 10TB mirror?

 

[danb35]

See Replication Tasks. The sending and receiving server are the same, in this case.

Sure, replication will do the job, but you should be able to replace a single disk with another single disk.

 

[Arwen]

Single disk replacements for growing a single disk pool works just like normal. I did it with ZFS on Linux, but it should work the same on FreeNAS. That said, I did it from the command line.

 

[Stux]

Single disk replacements for growing a single disk pool works just like normal. I did it with ZFS on Linux, but it should work the same on FreeNAS. That said, I did it from the command line.

And if it doesn't, that's a bug, and should be reported as such

Obviously, you can't offline the disk first.

 

[bassengunder]

What did the error say?

I have not found out where I can read the error message. In the "Volume Status" GUI, where it says "Completed" some hours after each reboot, it is only counting the number of errors, 1 or 2 errors, but I don't know where to read the error message. I am quite dissapointed in how difficult it seems to be to try such as basic task, as replacing to almost full disk containing ZFS volumes, with a larger disk.
My situation now:
- each time I restart my FreeNAS Server, it starts resilvering it. Sometimes both disks shows up as healthy/online in the start of the resilvering. However, when resilvering has executed a while, or is finished, the old one (ada0 4 TB) says "Degraded", and the new one (ada1 10 TB) says "Online", see the attached pictures, and always when finished, resilvering failed, thus resilvering restarts in next boot.
- Even though when the disk/stripe status says "degraded", I can access the volumes from my windows PC. I also tried to physically disconnect the old (4TB) disk, so only the new 10 TB disk was connected, still I was able to access the files/volumes on the FreeNAS server. So it looks like the content of the old disk is copied to new disk in the replace operation, but since the resilvering was not successfully, I still have to deal with the 4TB disk somehow, including it limitation of storage capacity, which is not desired. In the "Volume Status" GUI view (in attached files), I only want to see the 10 TB drive, not the 4 TB, so that the disk can continue to grow.
- At a time today, early in the resilvering process, all "Volum Status" was online on both old and new disk (see attached picture). When I tried to offline the old disk, I got error message top-side in the GUI, with no further information. Where can I see details of the error message?
- I have an NTFS backup disk that I think is current/updated (yesterday), if not some of the content is corrupt according the the degraded message. But when I look at the NTFS backup disk using my SATA-to-USB adapter, it all looks ok.
- All I want to do, is to migrate to a larger disk, and eliminate the problem of the old disk, having to small capacity now (94%). See Alert in attachment. And actually, I think my new disk (ada1 10 TB) already have a working copy of the old disk, but I need to remove the old disk from the system/config.

Any help or idea how I could proceed, is once again, much appreciated.

 

[danb35]

It took you two and a half months to follow up, and you're the one who's disappointed? Especially when you don't even know how your pool's configured (hint: it isn't a single disk)?

You've got checksum errors on all three disks, but especially on the 4 TB disk. Since your pool is striped, this means you have data corruption. I don't know what that's going to do the resilvering process.

 

[bassengunder]

It took you two and a half months to follow up, and you're the one who's disappointed? Especially when you don't even know how your pool's configured (hint: it isn't a single disk)?

You've got checksum errors on all three disks, but especially on the 4 TB disk. Since your pool is striped, this means you have data corruption. I don't know what that's going to do the resilvering process.

I agree that my follow up was late, and I am sorry for that. My "disappointed", which you are referring to, did not point on either my or any of yours follow ups in this thread, but the user friendliness of FreeNAS on how to migrate to a larger drive. I did not had problems with FreeNAS, before I tried to replace the disk, except the growing disk alert, more than 80% of the disk capacity. The checksum error/degraded status, is something that has appeared after I tried the replace function for the first time.

You wrote: "Especially when you don't even know how your pool's configured". Well, not sure what you mean here. The data has always been on only one disk previously, until I tried to replace that disk for a couple of months ago. The other disk (ada2) is for the FreeNAS OS. Now, it is two disks with (probably with the same or almost same data), the old and the new disk, and I want to remove the old disk from the system, both by means of software/config and hardware.

Again, any good suggestions would be appreciated, but I understand if you will not help more, due to my late response.

 

[bassengunder]

Single disk replacements for growing a single disk pool works just like normal. I did it with ZFS on Linux, but it should work the same on FreeNAS. That said, I did it from the command line.

Hi, any hint on what command could be used to do this from command line, is much appreciated. I have not previously been working with command line interface for this functionality on Linux, neither FreeNAS.

 

[Arwen]

Hi, any hint on what command could be used to do this from command line, is much appreciated. I have not previously been working with command line interface for this functionality on Linux, neither FreeNAS.

It's not that hard;

zpool set autoexpand=on POOL
zpool replace POOL OLD_DEVICE NEW_DEVICEC

But, there is a reason why FreeNAS has a GUI. There are things FreeNAS does with the disks, like add a swap space that complicates the procedure. So my simple example was with ZFS on Linux, with a OS completely configured by me.

 

[garm]

Ya but at this point it won’t matter. The disks are striped and as far as I know can’t be broken up. All you can do now is destroy the pool and reburn the drives you intend to use (becasue of the checksum errors). Sins you are running a single disk pool I assume there are multiple backups, right?

 

[Jailer]

With checksum errors on all of the drives there is some sort of hardware issue going on. Please list all of your hardware.

 

[danb35]

You wrote: "Especially when you don't even know how your pool's configured".

Yep, that was my mistake--the GUI is showing three entries for the pool status, and I incorrectly read that as there being a third disk there.

 

[bassengunder]

Dear all, thank you for your support. I managed to find a solution to the issue.

The solution for me was to type 'zpool status -v' in the FreeNAS shell, which then listed the corrupt file with the checksum error on my new replacement disk ada1p2 (10TB). The old disk to be replaced listed "To many errors" or something similar, while the new disk only listed one file with checksum error. I deleted that file and replaced it with the same file from my older backups. After a new FreeNAS reboot, the replace disk resilvering process finally ended successfully, and the old disk
disappeared from the Volum Status view, and now status is ONLINE.

Thanks for your suggestions and support!

 

[bassengunder]

Ya but at this point it won’t matter. The disks are striped and as far as I know can’t be broken up. All you can do now is destroy the pool and reburn the drives you intend to use (becasue of the checksum errors). Sins you are running a single disk pool I assume there are multiple backups, right?

Hi, I was in the middle of a single disk replacement, and it looks like the old disk had been copied to new disk already in the replace operation, but since one file was corrupted on the new disk, the re-silvering process did not complete, until I replaced that corrupt file with a good backup file. Meanwhile, and before I found the solution of replacing the corrupt file, it seems like the old disk got more checksum errors. However, as long as the replace operation succeeded, that does not matter, since the old disk was then offlined. Now the new disk is operating good, without the need of destroy the pool and reburn the drive.