[solved] "You are trying to add a virtual device consisting of 2 device(s) in a pool that..." error?

[Greg10]

I have a FreeNAS box with twelve disks installed (a mixture of 250gb and 500gb SSD drives) in a RAID-10 mirror configuration and I am trying to add two more 500gb drives.

When I attempt to expand the volume I receive:

"You are trying to add a virtual device consisting of 2 device(s) in a pool that has a virtual device consisting of 4 device(s)" and I cannot proceed.

Can someone please explain this behavior to me?

Here is the output of zpool status:

Code:

		NAME											STATE	 READ WRITE CKSUM
		vol0											ONLINE	   0	 0	 0
		  mirror-0									  ONLINE	   0	 0	 0
			gptid/8d6286f3-c34e-11e8-85ac-00259064e5e2  ONLINE	   0	 0	 0
			gptid/8db913a9-c34e-11e8-85ac-00259064e5e2  ONLINE	   0	 0	 0
		  mirror-1									  ONLINE	   0	 0	 0
			gptid/8e171bfd-c34e-11e8-85ac-00259064e5e2  ONLINE	   0	 0	 0
			gptid/8e6f48eb-c34e-11e8-85ac-00259064e5e2  ONLINE	   0	 0	 0
		  mirror-2									  ONLINE	   0	 0	 0
			gptid/8ed1dc63-c34e-11e8-85ac-00259064e5e2  ONLINE	   0	 0	 0
			gptid/8f2761e6-c34e-11e8-85ac-00259064e5e2  ONLINE	   0	 0	 0
			gptid/8f7d3d35-c34e-11e8-85ac-00259064e5e2  ONLINE	   0	 0	 0
			gptid/8fd620d2-c34e-11e8-85ac-00259064e5e2  ONLINE	   0	 0	 0
		  mirror-3									  ONLINE	   0	 0	 0
			gptid/903e47e4-c34e-11e8-85ac-00259064e5e2  ONLINE	   0	 0	 0
			gptid/90b0b695-c34e-11e8-85ac-00259064e5e2  ONLINE	   0	 0	 0
			gptid/910c8e94-c34e-11e8-85ac-00259064e5e2  ONLINE	   0	 0	 0
			gptid/916fb64c-c34e-11e8-85ac-00259064e5e2  ONLINE	   0	 0	 0

The two two-disk mirrors are the 500gb SSDs and the two four-disk mirrors are the 250gb SSDs. Surely I should be able to add additional disk to vol0?

 

[kdragon75]

While mixed vdevs are not encouraged, they are definitely supported... How did your pool start? The 4 drives at the top? Perhaps this is some oddity with stripe width?

 

[Greg10]

It was an oddity with the way I set up the striped mirrors. Mirror-2 and Mirror-3 were basically RAID-1 with two hot spares by accident and that, plus a combination of 500 and 250gb drives caused this error.

I was able to migrate all the data off of this SAN, destroy vol0 and rebuild the volume using pairs of drives in RAID-1:

Code:

 NAME											STATE	 READ WRITE CKSUM
		vol0											ONLINE	   0	 0	 0
		  mirror-0									  ONLINE	   0	 0	 0
			gptid/5ae20ae6-d0cf-11e8-94f8-00259064e5e2  ONLINE	   0	 0	 0
			gptid/5b35c7df-d0cf-11e8-94f8-00259064e5e2  ONLINE	   0	 0	 0
		  mirror-1									  ONLINE	   0	 0	 0
			gptid/5b91becf-d0cf-11e8-94f8-00259064e5e2  ONLINE	   0	 0	 0
			gptid/5be4f4aa-d0cf-11e8-94f8-00259064e5e2  ONLINE	   0	 0	 0
		  mirror-2									  ONLINE	   0	 0	 0
			gptid/8cc33aca-d0cf-11e8-94f8-00259064e5e2  ONLINE	   0	 0	 0
			gptid/8d1971f4-d0cf-11e8-94f8-00259064e5e2  ONLINE	   0	 0	 0
		  mirror-3									  ONLINE	   0	 0	 0
			gptid/aeda27ea-d0cf-11e8-94f8-00259064e5e2  ONLINE	   0	 0	 0
			gptid/af31d0ba-d0cf-11e8-94f8-00259064e5e2  ONLINE	   0	 0	 0
		  mirror-4									  ONLINE	   0	 0	 0
			gptid/cb70d267-d0cf-11e8-94f8-00259064e5e2  ONLINE	   0	 0	 0
			gptid/cbca5588-d0cf-11e8-94f8-00259064e5e2  ONLINE	   0	 0	 0
		  mirror-5									  ONLINE	   0	 0	 0
			gptid/ded88b53-d0cf-11e8-94f8-00259064e5e2  ONLINE	   0	 0	 0
			gptid/df37ef7e-d0cf-11e8-94f8-00259064e5e2  ONLINE	   0	 0	 0
		  mirror-6									  ONLINE	   0	 0	 0
			gptid/f1496137-d0cf-11e8-94f8-00259064e5e2  ONLINE	   0	 0	 0
			gptid/f1aaa8a1-d0cf-11e8-94f8-00259064e5e2  ONLINE	   0	 0	 0

This is a lot cleaner of an installation and frees up the 1TB of raw disk space that had gone missing. :D

 

[kdragon75]

Good call.