Cant' remove device from my volume

Status
Not open for further replies.

lohmag

Dabbler
Joined
Dec 2, 2013
Messages
38
Hello. Please help. I tried to add device to my volume now it's stuck somewere and I cant remove it.
34s0aj8.jpg


zpool status doesn't show it


[root@freenas] /usr/libexec# zpool status
pool: VOL1
state: ONLINE
scan: scrub repaired 0 in 22h38m with 0 errors on Sun Dec 1 16:38:53 2013
config:

NAME STATE READ WRITE CKSUM
VOL1 ONLINE 0 0 0
raidz2-0 ONLINE 0 0 0
gptid/fefe552b-b622-c361-add3-83a7b7237dfa ONLINE 0 0 0
gptid/ca954a9e-95a4-4fcb-8eae-b83a1f0b2c7c ONLINE 0 0 0
gptid/4f83063a-b094-e548-9527-a1bcc79492db ONLINE 0 0 0
gptid/d320c540-8e47-1060-85ee-e6ab60b0cff6 ONLINE 0 0 0
gptid/3839d81c-9309-2c66-b725-81857fb8f5c9 ONLINE 0 0 0
gptid/45f6090e-1834-a7e9-c91f-aeb23d24948f ONLINE 0 0 0
gptid/3b12818f-601e-97eb-ffe4-cf771d589c70 ONLINE 0 0 0
gptid/fe558ed3-131d-544c-ef2d-bf3b1784a1e9 ONLINE 0 0 0
gptid/0aa39653-b53b-ea67-fedc-c3e7070c5b01 ONLINE 0 0 0
gptid/a4c73134-5b29-11e3-bb71-00259092d588 ONLINE 0 0 0
cache
gptid/2606ffb0-5b29-11e3-bb71-00259092d588 ONLINE 0 0 0

errors: No known data errors

pool: VOL2
state: ONLINE
scan: none requested
config:

NAME STATE READ WRITE CKSUM
VOL2 ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
gptid/9d28d97d-5b3d-11e3-bb71-00259092d588 ONLINE 0 0 0
gptid/9eb86534-5b3d-11e3-bb71-00259092d588 ONLINE 0 0 0
gptid/a045d22f-5b3d-11e3-bb71-00259092d588 ONLINE 0 0 0
gptid/a1c590cf-5b3d-11e3-bb71-00259092d588 ONLINE 0 0 0
gptid/a34a0508-5b3d-11e3-bb71-00259092d588 ONLINE 0 0 0

errors: No known data errors

How to remove it from VOL1 and then add to VOL1 raidz2 pool?
Second question can I join VOL1 and VOL2 to share it as one massive iSCSI disk to Win?
Why me upload/download speed very slow(20mb/sec)? I have 4gb ram and 60gb ssd cache device
Thank you.
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
First, you need to post the zpool status output in CODE tags. The formatting of the text is CRUCIAL to understanding what your pool is doing. Consider these 2 outputs:

Code:
        NAME                                            STATE    READ WRITE CKSUM
        tank                                            ONLINE      0    0    0
          raidz1-0                                      ONLINE      0    0    0
            gptid/6fbb91d5-4a95-11e2-bca4-0015171496ae  ONLINE      0    0    0
            gptid/70448fd2-4a95-11e2-bca4-0015171496ae  ONLINE      0    0    0
            gptid/70c0c7b3-4a95-11e2-bca4-0015171496ae  ONLINE      0    0    0
            gptid/713de0d5-4a95-11e2-bca4-0015171496ae  ONLINE      0    0    0
 
 
        NAME                                            STATE    READ WRITE CKSUM
        tank                                            ONLINE      0    0    0
          raidz1-0                                      ONLINE      0    0    0
            gptid/6fbb91d5-4a95-11e2-bca4-0015171496ae  ONLINE      0    0    0
            gptid/70448fd2-4a95-11e2-bca4-0015171496ae  ONLINE      0    0    0
            gptid/70c0c7b3-4a95-11e2-bca4-0015171496ae  ONLINE      0    0    0
          gptid/713de0d5-4a95-11e2-bca4-0015171496ae    ONLINE      0    0    0


The first output is 4 disks in a RAIDZ1.

The second output is 3 disks in a RAIDZ1 + 1 disk striped. Money says you have a striped disk at the end and the GUI has noticed that extra space. See how important a single "space" is? Imagine all the noobs that are so lost because they aren't used to that level of attention required and that kind of importance for a space in Windows. :p

First: You cannot remove that stripe disk from the pool now no matter what. If you can destroy and recreate the pool that's your option. But your bigger problem is that if that disk fails right now you will lose your entire pool. Not just some data, the entire pool. So what you need to do now is get another disk and mirror that disk so you have some redundancy ASAP. This will require you to do it from the CLI since you can't add a mirror disk to a vdev from the GUI. This should be very high on your priority. Searching around the forums will allow you to find the appropriate command to add the mirror. We've had some people lose their pool in this way before they could get their replacement disk, so don't waste time!

Second: There is no way to take 2 pools and combine them into 1 big iscsi disk.

Third: You cannot add single disks to a vdev. This is explained in my presentation(link in my sig). You can never add disks to a vdev once it has been created with one exception; adding disks to a vdev to make mirrors. You were probably hoping to add another disk to "expand" the zpool. You can't do that without giving up redundancy. That's why we tell people to oversize their zpool because adding more space later is not trivial. If you don't oversize your pool you'll be forced to buy multiple disks to ensure redundancy. This is a limitation of ZFS and there is no fix ever coming for this. It's not considered a problem since ZFS is considered an enterprise class product. Typically, buying a dozen or more disks isn't a problem for corporations.
 

lohmag

Dabbler
Joined
Dec 2, 2013
Messages
38
Could I make a snapshot of vol1 then move it to vol2 then recreate vol1?
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
If you have enough free disk space on vol2, then yes.
 

Dusan

Guru
Joined
Jan 29, 2013
Messages
1,165
Why me upload/download speed very slow(20mb/sec)? I have 4gb ram and 60gb ssd cache device
Minimum recommended memory for ZFS is 8GB (and much more if you expect decent performance with larger pools). Also, that 60GB L2ARC is probably actually hurting your performance with that little memory (or it is completely disabled as FreeNAS disables ARC when you have only 4GB of RAM).
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
Good catch Dusan. I forgot to mention that while trying to explain the zpool stuff.

Per the manual you should have 1GB of RAM per TB of disk storage, but never less than 8GB of RAM.

Dusan is also right that your L2ARC is probably hurting your performance. An L2ARC should never be considered until you've maxxed out your system. Even then, your L2ARC should never exceed 5x your ARC(ARC size is typically 70% of your system RAM over 8GB). So you have a few things to fix.
 

lohmag

Dabbler
Joined
Dec 2, 2013
Messages
38
I have Supermicro X7SPA controller it can't handle more than 4gb RAM.
Again about snapshot - how can I take it if it says "Snaphot could not be taken: out of space" ? What can be deleted/destroyed? How can I move space from VOL1/VOL1 to parent filesystem?
I have unlocated 10tb of disks only data on VOL1 is 3TB
Code:
 zfs list -o space
NAME        AVAIL  USED  USEDSNAP  USEDDS  USEDREFRESERV  USEDCHILD
VOL1        2.83T  18.5T        0    103K              0      18.5T
VOL1/VOL1  18.4T  18.5T        0  2.96T          15.6T          0
VOL1/jails  2.83T  115K        0    115K              0          0
VOL2        5.33T  719K        0    192K              0      527K
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
I have Supermicro X7SPA controller it can't handle more than 4gb RAM.

In all seriousness, that's why this is in the manual too:

FreeNAS® with ZFS typically requires a minimum of 8 GB of RAM in order to provide good performance and stability.

If you don't have at least 8GB of RAM with ZFS or 2GB of RAM with UFS, you should consider getting more powerful hardware before using FreeNAS® to store your data otherwise data loss may result.

Don't be fooled and think you can get by with less. That warning was added because we've seen so many people lose their pools without warning.
 

lohmag

Dabbler
Joined
Dec 2, 2013
Messages
38
I don't bought this device all i should do make it work somehow as fast as possible
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
Ok. Well, good luck to you. Can't say I didn't warn you when things go very badly suddenly.
 

Knowltey

Patron
Joined
Jul 21, 2013
Messages
430
I don't bought this device all i should do make it work somehow as fast as possible

Sorry, 4GB of RAM simply isn't enough for FreeNAS and ZFS. I'm currently running that config as well on my testing machine pending a hardware update and I'm lucky that I'm able to pull 60MB/s out of it. If you want more performance for FreeNAS the first thing you'll want to consider is getting your machine up to the minimum requirements, especially in the RAM department.
 

lohmag

Dabbler
Joined
Dec 2, 2013
Messages
38
ok I understand that and i don't argue with you about RAM, but thing is it's already running and people use it. First need is to make redundacy then think about change controller and ram. This thing was sold as ready device to that firm at least year ago no chance to send it back. If you can help with snapshoting i'll appreciate that.
 

Knowltey

Patron
Joined
Jul 21, 2013
Messages
430
Wait, let me get this straight, you have a FreeNAS machine with under 4GB of RAM, and you're running it IN BUSINESS PRODUCTION!?

Honey, snapshots aren't going to do you a bit of good when* your entire server destroys itself from lack of RAM, which is especially likely using it in business production.

Upgrading your RAM to minimum requirements should absolutely be your first order of business and should be done ASAP. As it stands you're running a ticking time bomb of data destruction. Adding more drives to create redundancy is only going to make your limited RAM situation even worse.

* - Yes I said when not if.
 

gpsguy

Active Member
Joined
Jan 22, 2012
Messages
4,472
Hopefully the firm backs up the data stored on the FN box.


Sent from my phone
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
Why do you think I said what I said above? No way I want to be blamed in any way, shape, or forum when this goes bad. FreeNAS doesn't always behave right and is unstable with less than the required RAM.
 

lohmag

Dabbler
Joined
Dec 2, 2013
Messages
38
At least i should backup all data first. Is there an option to bckup my zvol which is shared via iscsi to another volume on the same nas? Can't find anything on doc about that.
206hnja.jpg
 

Knowltey

Patron
Joined
Jul 21, 2013
Messages
430
Since the inadequacies are hardware based don't backup your FreeNAS to your FreeNAS (You should NEVER EVER EVER be backing up to the same physical machine in the first place regardless, FreeNAS or otherwise, that's not a backup (Oh look the building burned down))

Pull all the data off and store it somewhere, then push it all back down once you have the machine meeting minimum specs.

Making a backup to the same physical machine is like having a piece of paper in your hands with a bunch of important information on it, saying to yourself "I should probably make a copy of this in case I ever lose it" and then flipping over said piece of paper and copying all of the information from the front to the back of the paper. Guess what happens to your "copy" when you lose the front of that paper?

9 times out of 10, events that would cause you to need a backup are things that effect the entire machine.
 

lohmag

Dabbler
Joined
Dec 2, 2013
Messages
38
I speak with the supplyer they'll replace mother with something apropriate for zfs and put more ram.
Again how can i make a snapshot can't find a way
I have only 3tb of data in zpool VOL1/VOL1

Code:
zfs snapshot VOL1/VOL1@snap1
cannot create snapshot 'VOL1/VOL1@snap1': out of space

Code:
zfs list -o space VOL1
NAME  AVAIL  USED  USEDSNAP  USEDDS  USEDREFRESERV  USEDCHILD
VOL1  2.83T  18.5T        0    100K              0      18.5T

Code:
zfs list
NAME        USED  AVAIL  REFER  MOUNTPOINT
VOL1      18.5T  2.83T  100K  /mnt/VOL1
VOL1/VOL1  18.5T  18.4T  2.95T  -
VOL2      10.3T  409G  275K  /mnt/VOL2
VOL2/VOL2  10.3T  10.3T  468G  -
 

Dusan

Guru
Joined
Jan 29, 2013
Messages
1,165
Is any reservation or refreservation set?
zfs list -o reservation,refreservation VOL1
 

lohmag

Dabbler
Joined
Dec 2, 2013
Messages
38
Thanks man you are genius
Code:
 zfs get refquota,reservation,refreservation
NAME      PROPERTY        VALUE      SOURCE
VOL1      refquota        none      local
VOL1      reservation    none      default
VOL1      refreservation  none      local
VOL1/VOL1  refquota        -          -
VOL1/VOL1  reservation    none      default
VOL1/VOL1  refreservation  18.5T      local
VOL2      refquota        none      default
VOL2      reservation    none      default
VOL2      refreservation  none      default
VOL2/VOL2  refquota        -          -
VOL2/VOL2  reservation    none      default
VOL2/VOL2  refreservation  10.3T      local

Probably refreservation sets automatically when youre creating zvols.
Than i did:
Code:
zfs set refreservation=5T VOL1/VOL1
zfs snapshot -r VOL1/VOL1@snap1
zfs send VOL1/VOL1@snap1 | zfs recv VOL2/VOL2

Now it's processing
 
Status
Not open for further replies.
Top