Adding multiple cache drives

[mackwage]

I have two ZFS arrays in my system, we shall call them Array1 and Array2. I added two Intel 120gb SSDs to my box and the intent was to add each SSD as a cache drive to each array.. so each array would have a single cache drive.

When I went into storage manager section and the add volume, it would only let me add both SSDs as a single cache drive. Seeing this, I change it to the manual setup menu and added a single SSD as a cache drive to Array1.

Here is where the fun starts... it added the cache drive successfully but then it added the second SSD drive a single stripe drive. See the image; it is the ada3p2. Since it is not designated as a log or cache drive, it will NOT allow me to remove it.

So 1. why would the initial wizard only allow me to add BOTH drives a striped cache drive to a single array as oppose to 1 to 1?

2. How can I remove this from the array safely?

 

[mackwage]

Also note.. the cache drive that was present is not shown as I removed it after see what had happened.

 

[cyberjock]

You've got a bigger problem.. see that "stripe" drive.. ada3. If you lose that drive you lose your pool. I think you know this though.

Also, those checksum errors, you ahve some kind of hardware issue, those should be zero pretty much all the time. I've had my system for 9 months now, never had a single checksum error.

For your questions:

1. Not sure. I don't exactly understand what you are saying. You can most certainly add a single SSD as an L2ARC. I've done it on several systems in recent weeks.

2. You can't remove that stripe. The only way to remove it is to destroy your pool and restore from backup. :(

I'm just gonna say this and it's nothing personal... I'm thinking this is more of user error than anything else. We've seen stripes added around here before.. it's always been user error. But, if this is a bug, this is a serious bug. I'd try to reproduce what you did in a VM and if you can reproduce it DEFINITELY put a ticket in.

 

[mackwage]

Thanks! I am pretty sure I can reproduce it as I have run into this several times on my system. I'll try to reproduce after restoring from my backup and include step by step screen shots.

 

[mackwage]

Got it. It is a mixture of user-error and a mixture of the UI on one part could have been more clear.

First, added two physical 120 gb ssds to the system. When you go to the vol mgr you see..

When you click the PLUS sign, it automatically adds both drives to your layout.

I did not realize at first that you can use the slide to clear the extra volume.

If you go to Manual Setup, you see this.

My intent was to add ada3 as a cache disk. What I did not see is that under ZFS Extra, the device you are setting that attribute on is the drive opposite of the one you click. So if you click ADA3 and set Cache on ADA4, it will add ADA4 as a cache and add ADA3 as a stripe.

 

[cyberjock]

oh my.. that's pretty messed up! That should probably be fixed..definitely put a ticket in for that.

 

[thirdgen89gta]

Yes, I also ran afoul of this very issue shortly after first setting up my NAS. Thankfully I didn't have much data on it and just destroyed it to start over.

I like the OP accidentally added a single drive as part of the array, instead of as a cache or log. I was forced to destroy it after backing up the data and starting over. Thankfully, I didn't have alot of data on there, and what I did have was backed up elsewhere.

 

[cyberjock]

Now you see why "manual setup" is for advanced users only. ;)

Literally the manual setup is with all of the "seatbelts" off. You can do anything and everything you want, and you can do all the stupid things that most people wouldn't think of. You want to add single disks? Go ahead. You want to add oddball sized disks to a vdev? Go ahead. The standard volume manager won't let you do one and the other comes with a hefty warning.