Increasing Storage Via Drive Swap

[Jason Hamilton]

Hey Everyone,

So I've been doing a lot of reading about zfs, which I already knew to be totally awesome but just doing some refresher never hurts. Anywho I'm currently in the process now of moving my drives from my old FreeNAS box into my new one. Since I moved all of the data its time to move my 2TB drives in here to replace the 1TB ones. Yes I do have a backup of my data with crashplan so I don't have it onsite and this is at home so not mission critical. Anyways I understand that ZFS will increase the pool once all of the drives have been replaced.

Here is my question per a recommendation from the forum and documentation my new box has 12 drives in it. I split the 12 drives up into 2 Vdevs of 6 drives a piece in RaidZ1.

So lets call it Vdev1 and Vdev2

I am on my 3rd drive swap of vdev1 with the 2TB drives. Once I have all 6 drives swapped out, will my storage increase in that vdev? Or will it only increase once I do both vdevs? And now after typing this out I think that I just answered my own question. It will increase in capacity once both vdevs have the same equal amount of disk available. Because the 2 Vdev's have to mirror eachother in amount of disk capacity and number of drives. So there you have it, if I'm wrong please let me know but yeah you know how it goes lol.

 

[danb35]

And now after typing this out I think that I just answered my own question. It will increase in capacity once both vdevs have the same equal amount of disk available.

This is not correct...

Because the 2 Vdev's have to mirror eachother in amount of disk capacity and number of drives.

...because this is not correct. Really, none of this is correct--vdevs in a pool don't have to match in disk capacity, number of disks, RAIDZ level, or any other way. You can have, for example (and many people seem to inadvertently generate this configuration) a RAIDZ vdev striped with a single disk. That single disk's capacity will be added to the pool, and that disk will now be critical--if it fails, the entire pool is lost.

My own pool is an example of this. It consists of one RAIDZ2 vdev of 3 x 2 TB + 3 x 3 TB disks, and another RAIDZ2 vdev of 6 x 4 TB disks. The smaller vdev's capacity is limited by the 2 TB disks, so it has a net capacity of ((6 - 2) x 2) = 8 TB. It's not optimal, but it certainly works well enough to saturate GbE connections.

Pools are striped across any member vdevs. Once you increase the capacity of one vdev, you increase the capacity of the pool. I'll note, though, that RAIDZ1 is disfavored.

 

[joeschmuck]

If you are planning to mirror two RAIDZ1 vdevs, I don't get it.

 

[gpsguy]

I don't think the OP was trying to "mirror" the drives in the same context you're thinking. My guess, is that he (incorrectly) thought that when adding vdev's to a pool, that one needed to use the same quantity of disks and same size as the original ones.

 

[Jason Hamilton]

No I'm not planning to mirror them. It was just 1 vdev created with 6 1TB drives and then a second one used to extend the first one as was suggested in my original forum post here and looking back now I can see that I misread it and so I created them as z1 when they should've been z2. Oh well too late now. Anywho then I shall continue hammering away with my drive replacements and someday when I have the time and space I'll move my storage and redo my pools.

 

[Jason Hamilton]

I apologize for resurrecting my old thread here. So I have all 12 of my drives now replaced with 2TB drives. With zpool list I see this

Code:

tbone1        21.8T  11.1T  10.7T         -    25%    50%  1.00x  ONLINE  /mnt

Yeah so I have an interesting name for my pool :)

So I see it now shows the pool with the increased size however my zvols do not report that available space

For example

Code:

tbone1/share                                                11T    4.7T    6.7T    41%    /mnt/tbone1/share

followed the guide here https://forums.freenas.org/index.ph...ool-properly-after-adding-larger-drives.9920/ and I had to reboot since my pool was busy (no surprise there its got a lot going on with jails and iscsi mounts) so no biggie there.

But still after the reboot I can't get it to allocate the freespace to zvol. Is there something else that I need to do here? I don't have any quotas setup on most of my mounts a few do but my share does not.

 

[gpsguy]

You've got the terms wrong. Your pool is comprised of one or more vdevs. As you correctly stated, your pool shows the increased size.

I'm a little confused by your statement about quoata's. Is "share" a dataset? And, does it have a quota?

 

[Jason Hamilton]

I apologize for the wrong terms. Yes share is a dataset and no it doesn't have a quota on it.

 

[Jason Hamilton]

Do you think that this "bug" is the issue? https://forums.freenas.org/index.ph...ets-showing-misleading-incorrect-sizes.15828/ So I really do have 20TB of storage just not showing up properly? Even my nfs mounts for share report 12T total with 4.8T used and 6.7T Free

 

[Jason Hamilton]

And I just took a screenshot of what my pool list looks like in the GUI. I understand that I loose some to parity since my disks are setup like this

12 2TB disks
I then created 1 raidz2 with the first 6 disks then I extended the zvol into 2nd set of disks in a raidz2 so now I can handle 4 drive failures.

According to http://wintelguy.com/raidcalc.pl I should have 16T of total usable space. I know that I have allocated some of it off to various places with quotas like for example my mythtv backend storage (2TB) is kept here on an iscsi mount (which the performance of that over my previous setup with nfs is amazing). So I get with that why I wouldn't see the full 16TB since 2TB is being used up by myth.

For some reason now on my local box my share is now showing up as 12TB of total storage however mp3 and users still only report in at 6.7TB so I don't know what I did to make the share check in at a better amount than it did before. Continuing to research and what not but if anyone has any tips please feel free to point me in that direction.

 

[gpsguy]

It's been so long, I forgot you had 2 vdev's of 6 drives each. One of the "problems" you have is the age old issue, wherein the drives manufacturers don't use the same numbering system that most OS's use. So FreeNAS, Windows, etc. only sees ~1.8TiB of space. At best, you'd start out with ~14TiB, before substracting any overhead for FreeNAS.

See @Bidule0hm calculator - https://forums.freenas.org/index.php?threads/zfs-raid-size-and-reliability-calculator.28191/

Instead of putting in 12 disks, 2TB disks with RAIDz2 - use 6 and multiple the results by 2.

 

[Jason Hamilton]

Thank you that would explain the 12T that my share is now reporting. Just weird that the screenshot shows so much but the datasets don't show the same.

Sent from my LG-V700 using Tapatalk

 

[Bidule0hm]

What you see on the first line of the screenshot is the total space, including parity and ZFS overhead.

 

[Jason Hamilton]

thank you @Bidule0hm for your calculator and the explanation. I appreciate it.

 

[Jason Hamilton]

So on that I think that its time to start purchasing 3TB drives as my 2TB fail just stick a 3 or heck maybe even a 4 in there :).

 

[Bidule0hm]

You're welcome ;)

 

[gpsguy]

I would skip the move to 3TB drives and use 4's instead. Since you have 2 vdevs you should see your pool grow after replacing the first 6 drives.


Sent from my phone

 

[Jason Hamilton]

that shall be the route I think I'll take and well it will give me a bigger gain as well.