replacing the right drives for expansion

[oppland]

My FreeNAS Mini has been so reliable that I feel like I have forgotten much of what I knew about ZFS. I've had four 3TB drives for years and just bought two 8TB drives on Prime Day.

To get the full size expansion, I think I need to replace ada1 and ada3 (or ada2 and ada4). Is that right?


NAME STATE READ WRITE CKSUM
data ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
gptid/0b19ab96-774b-11e5-bb9e-d050996fe268 ONLINE 0 0 0
gptid/0b90c5a0-774b-11e5-bb9e-d050996fe268 ONLINE 0 0 0
mirror-1 ONLINE 0 0 0
gptid/0c083dac-774b-11e5-bb9e-d050996fe268 ONLINE 0 0 0
gptid/147a98b2-fd94-11e5-b742-d050996fe268 ONLINE 0 0 0


ada1
data
WD-WCC4N1HND4RU
2.73 TiB

ada2
data
WD-WCC4N2NNAUNC
2.73 TiB

ada3
data
WD-WCC4N3NZ4EZN
2.73 TiB

ada4
data
WD-WCC4N1XYTJ9Z
2.73 TiB

 

[danb35]

Is that right?

How would we know? You haven't shown anything that would connect a gptid to a disk name. glabel status would let you correlate the gptids to the disk designations, but it'd be much easier to go into pool status in the GUI, as it shows the disks by adan designations. And that's where you'd need to go anyway to do the replacement.

 

[oppland]

Sorry about that. I haven't really touched this stuff in years.

Name Status Components
gptid/a172ef57-7748-11e5-8668-d050996fe268 N/A ada0p1
gptid/a176ce30-7748-11e5-8668-d050996fe268 N/A ada0p2
gptid/0b19ab96-774b-11e5-bb9e-d050996fe268 N/A ada1p2
gptid/0b90c5a0-774b-11e5-bb9e-d050996fe268 N/A ada2p2
gptid/0c083dac-774b-11e5-bb9e-d050996fe268 N/A ada3p2
gptid/147a98b2-fd94-11e5-b742-d050996fe268 N/A ada4p2


So it looks like I've got:

mirror0
gptid/0b19ab96-774b-11e5-bb9e-d050996fe268 N/A ada1p2
gptid/0b90c5a0-774b-11e5-bb9e-d050996fe268 N/A ada2p2

mirror1
gptid/0c083dac-774b-11e5-bb9e-d050996fe268 N/A ada3p2
gptid/147a98b2-fd94-11e5-b742-d050996fe268 N/A ada4p2

 

[danb35]

So it looks like I've got:

Looks like it to me too. So you could replace ada1 and ada2, or ada3 and ada4. But again, if you do it through the GUI (which you should anyway), you don't need to worry about the gptid designations, as the GUI hides them--you'd just use adan.

 

[oppland]

Thank you. The first one is resilvering now.

 

[tfran1990]

At first glance i would have guessed 1 and 3. explain.

ada1 would copy to ada3 (M0) and ada2 copy to ada4, (M1)

to me i see it as ada1(8TB disc) with ada2 (4TB disc) as M0 = 12
then ada3(8TB disc) with ada4(4TB disc) M1 = 12

 

[danb35]

to me i see it as ada1(8TB disc) with ada2 (4TB disc) as M0 = 12

No, mirrors take the capacity of the smallest member device (just like RAIDZn does). So if OP does it your way, he has two mirrors, each with 3 TB of capacity, and each with 5 TB of wasted space. Replace both disks in one mirror, and that mirror grows--then he has one 8 TB mirror, and one 3 TB mirror.

Mirror-0 has ada1 and ada2; mirror-1 has ada3 and ada4.

 

[carleycr]

I'm in a similar situation as the OP. I've had my FreeNAS box running pretty reliable for about 5 years now. I recently got a warning that my storage is at 81% used. I want to expand the storage. Any recommendations on what drives to replace? ZFS was setup originally, so out of the 18TB total storage, only about half of that is usable. Screenshot of current drives below.

 

[danb35]

Screenshot of current drives below.

Not at all helpful, unfortunately--what does the volume status page look like?

 

[carleycr]

Not at all helpful, unfortunately--what does the volume status page look like?

Sorry, also not super technical. is this what you're looking for?

 

[danb35]

is this what you're looking for?

No, the volume status page. Toward the bottom of that screen is a button that looks like a blank sheet of notebook paper--if you mouse over that you'll see a tool tip saying "volume status". Click that and post what you see.

 

[carleycr]

No, the volume status page. Toward the bottom of that screen is a button that looks like a blank sheet of notebook paper--if you mouse over that you'll see a tool tip saying "volume status". Click that and post what you see.

Here you go.

 

[danb35]

That's it. With that arrangement, you'd lose about 1/3 (not 1/2) of your total capacity to redundancy. The drawback to this volume arrangement is that you'll need to replace all of your disks to increase its capacity.

 

[carleycr]

That's it. With that arrangement, you'd lose about 1/3 (not 1/2) of your total capacity to redundancy. The drawback to this volume arrangement is that you'll need to replace all of your disks to increase its capacity.

Damn. I was hoping that wasn't the case. I'm assuming the other option would be to just add more 3TB Red drives?

 

[myoung]

Damn. I was hoping that wasn't the case. I'm assuming the other option would be to just add more 3TB Red drives?

RaidZ expansion doesn't exist yet. Your only other option is to add another 6-disk RaidZ vdev. Either way you need to buy 6 more disks.

 

[danb35]

Your only other option is to add another 6-disk RaidZ vdev.

....or another RAIDZ vdev of any size, though matching numbers of disks would be optimal.

I was hoping that wasn't the case.

That's a large part of the reasoning behind this blog post: https://jrs-s.net/2015/02/06/zfs-you-should-use-mirror-vdevs-not-raidz/

I don't necessarily agree with his conclusions, but he raises some good points.

 

[oppland]

My second drive just finished resilvering, and I now have about 10TB of storage.
Thanks again.

 

[carleycr]

That's a large part of the reasoning behind this blog post: https://jrs-s.net/2015/02/06/zfs-you-should-use-mirror-vdevs-not-raidz/

I don't necessarily agree with his conclusions, but he raises some good points.

So I setup my server back in 2014. At the time I don't recall mirror vdev being an option, but maybe I missed that "hot new" technology. Is there any documentation out there to move this existing volume to vdevs, or is that not a thing/good idea?

Based on my signature specs, considering that my case is limited to 6x3.5" bays and all 6 bays are full, what would your recommendation be to "expand storage"? Based on the article linked, I think the cheapest long term solution would be to fail a drive, replace with larger, resilver, repeat for every drive. Is that correct, or am I missing something?

Another solution would be to build a brand new server, but I'm still stuck buying all the HDD's, plus cost of components. This doesn't seem very cost effective to me.

 

[danb35]

At the time I don't recall mirror vdev being an option

It was.

Is there any documentation out there to move this existing volume to vdevs

Your pool currently consists of one vdev containing all six disks. What you'd presumably want to do is change it to three mirrored vdevs, and that isn't possible. Even if it were, you don't have enough space for that--your pool currently has ~ the capacity of four of your disks, and it's 86% full. If you were to change it to three mirrored pairs (even if that were possible), you'd lose 25% of your current available capacity--you just have too much data for that to work, even if it were otherwise possible.

Based on the article linked, I think the cheapest long term solution would be to fail a drive, replace with larger, resilver, repeat for every drive. Is that correct, or am I missing something?

You aren't missing a thing.

Another solution would be to build a brand new server

A related solution would be to buy a used server with more drive bays--used enterprise gear can be pretty cheap on eBay, and still gives very good service. Then you'd still be buying new disks, but you wouldn't be throwing out the old (and presumably perfectly good) ones.

 

[carleycr]

A related solution would be to buy a used server with more drive bays--used enterprise gear can be pretty cheap on eBay, and still gives very good service. Then you'd still be buying new disks, but you wouldn't be throwing out the old (and presumably perfectly good) ones.

Didn't think of that. Any suggestions on chasis that I swap over my existing hardware and add 6 more hard drives?