Upgrade 9.1 to 9.1.1 zpool Volumes changed size

[mircsicz]

Hi hi,

don't know how to discribe, so I'll just begin...

Have installed 9 on a HP N54 Microserver with 4SATA slots... ZFS RAIDZ1 setup for the 4 1,5TB drives...

Wednesday I upgraded from 9.1 to 9.1.1, afterwords I realized my network share's refused to be writeable! First one I realized it with was the TM share for my MacBook-Pro...

The traffic light is green and so it doesn't have a known error...

Then I had a look on FreeNAS:

Code:

[root@nas] ~# zpool list
NAME    SIZE  ALLOC  FREE    CAP  DEDUP  HEALTH  ALTROOT
RAIDZ  5.44T  5.08T  362G    93%  1.00x  ONLINE  /mnt

Code:

[root@nas] ~# zpool status
  pool: RAIDZ
state: ONLINE
status: The pool is formatted using a legacy on-disk format.  The pool can
        still be used, but some features are unavailable.
action: Upgrade the pool using 'zpool upgrade'.  Once this is done, the
        pool will no longer be accessible on software that does not support feature
        flags.
  scan: scrub repaired 561K in 12h54m with 0 errors on Sun Sep 22 12:54:46 2013
config:
 
        NAME                                            STATE    READ WRITE CKSUM
        RAIDZ                                          ONLINE      0    0    0
          raidz1-0                                      ONLINE      0    0    0
            gptid/7cfc6f55-b96f-11e2-9c2d-28924a2bfe62  ONLINE      0    0    0
            gptid/7daab82c-b96f-11e2-9c2d-28924a2bfe62  ONLINE      0    0    0
            gptid/7e1041f3-b96f-11e2-9c2d-28924a2bfe62  ONLINE      0    0    0
            gptid/7e73b3e6-b96f-11e2-9c2d-28924a2bfe62  ONLINE      0    0    0
 
errors: No known data errors

Here's an example of a share that is smaller than before the upgrade:

Code:

[root@nas] ~# zfs list
NAME USED AVAIL REFER MOUNTPOINT
RAIDZ/home/tm-mbp 834G 0 834G  /mnt/RAIDZ/home/tm-mbp

This Volume had 1.1TB before I did the upgrade!

I hope you can help, I don't know what to do...

 

[cyberjock]

Trust me. It didn't shrink because of the upgrade. There was a bug that was fixed that made it appear to have more free space than it actually had. It was fixed recently. This may or may not apply to you.

Some people also have had coincidences where they weren't really watching their server closely, and now they are and they are thinking that there should be more space than there is and they are realizing later that nothing changed. Check your graphs in the WebGUI and see if the diskspace made a large marginal change suddenly. If you have the charts then either you won't see a marginal change or you will. If you do my bet is its because of the bug that was fixed.

What you do need to do is free up space though. ;)

And if you look at your zpool status you can see that 561k was fixed. You should probably check out your disk attributes in SMART to make sure everything is okay. Usually, if fixes are having to be done that means that something was wrong in the past between the previous scrub and the most recent one that you performed.

 

[mircsicz]

Hi cyberjock, thx for the fast reply...

From what I can see in the "zfs list" output:

Code:

[root@nas] ~# zfs list
NAME                                                        USED  AVAIL  REFER  MOUNTPOINT
RAIDZ                                                      4.00T      0  1.16G  /mnt/RAIDZ
RAIDZ/home                                                3.76T      0  61.3K  /mnt/RAIDZ/home
RAIDZ/home/homedirs                                        6.15G      0  44.9K  /mnt/RAIDZ/home/homedirs
RAIDZ/home/homedirs/mirco                                  6.15G      0  6.15G  /mnt/RAIDZ/home/homedirs/mirco
RAIDZ/home/iPhoto                                          310G      0  310G  /mnt/RAIDZ/home/iPhoto
RAIDZ/home/iTunes                                          200G  12.6G  187G  /mnt/RAIDZ/home/iTunes
RAIDZ/home/tm-ac100                                        43.4K      0  43.4K  /mnt/RAIDZ/home/tm-ac100
RAIDZ/home/tm-embedded                                      65G  64.8G  207M  /mnt/RAIDZ/home/tm-embedded
RAIDZ/home/tm-g5                                          43.4K      0  43.4K  /mnt/RAIDZ/home/tm-g5
RAIDZ/home/tm-mbp                                          834G      0  834G  /mnt/RAIDZ/home/tm-mbp
RAIDZ/home/tm-t420s                                        100G  100G  4.83M  /mnt/RAIDZ/home/tm-t420s
RAIDZ/home/tm-zotac                                          50G  26.7G  23.3G  /mnt/RAIDZ/home/tm-zotac
RAIDZ/home/video                                          2.23T      0  2.23T  /mnt/RAIDZ/home/video
RAIDZ/jails                                                2.98G      0  146M  /mnt/RAIDZ/jails
RAIDZ/jails/.warden-template-9.1-RELEASE-amd64              347M      0  347M  /mnt/RAIDZ/jails/.warden-template-9.1-RELEASE-amd64
RAIDZ/jails/.warden-template-9.1-RELEASE-amd64-pluginjail  891M      0  891M  /mnt/RAIDZ/jails/.warden-template-9.1-RELEASE-amd64-pluginjail
RAIDZ/jails/bit_1                                          202M      0  1.06G  /mnt/RAIDZ/jails/bit_1
RAIDZ/jails/dlna_1                                          432M      0  1.29G  /mnt/RAIDZ/jails/dlna_1
RAIDZ/jails/mt-daapd_1                                      221M      0  1.08G  /mnt/RAIDZ/jails/mt-daapd_1
RAIDZ/jails/pyload_1                                        815M      0  1.13G  /mnt/RAIDZ/jails/pyload_1
RAIDZ/office                                                243G      0  46.4K  /mnt/RAIDZ/office
RAIDZ/office/backup                                        241G      0  241G  /mnt/RAIDZ/office/backup
RAIDZ/office/one                                              2G  1.42G  591M  /mnt/RAIDZ/office/one

I've 4TB but "zpool list" says it's 5.4TB, I don't understand that difference?

And what about that that message about "zpool upgrade" from "zpool status", does it help to do that?

I can freeup a few GB's but the most is important...

 

[cyberjock]

zpool list includes available space, including redundancy disks. If I have a 3 disk RAIDZ1 of 2TB disks I should expect something like 5.7TB or so despite having only 3.7TB or so of actual storage space. The disparity is for parity data.

The zpool upgrade is only to allow you to upgrade the pool from your current version to the newer version supported by FreeNAS. It doesn't affect free space at all. See http://doc.freenas.org/index.php/Known_Issues as this is a limitation of ZFS.

Pools should not be allowed to be filled beyond 90%(80% is preferred). Performance can take a nosedive otherwise. If you can't free up enough space to get to either of those you need to seriously stop and look at upgrading your pool. Pools with no free space can suffer corruption. It should be your highest priority to get at least 50GB of free space immediately for regular disk activity.

 

[mircsicz]

Yeap, seems I was hit by the bug...

Free'd some space and now I think bout exchanging the drive with the SMART error's or directly upgrading all 4 HDD's to 2 or 3TB Modell's!

Is there a upgrade path like exchanging one drive after another and letting ZFS handle the new Space? Or am I forced to transfer all the data to external drive's and copy it back later?

EDIT: just read your NOOB doku, thank's for that great write down... I'll exchange the 1,5TB HDD with a 3TB and continue with the other's later...