Few Questions

[Chadi]

Hi folks,

So I hired someone a while back to install FreeNAS on a custom built 1u server. Below is the odd setup.

[root@freenas] ~# df -h
Filesystem Size Used Avail Capacity Mounted on
/dev/ufs/FreeNASs2a 926M 748M 104M 88% /
devfs 1.0k 1.0k 0B 100% /dev
/dev/md0 4.6M 3.3M 904k 79% /etc
/dev/md1 823k 4.0k 754k 1% /mnt
/dev/md2 149M 37M 100M 27% /var
/dev/ufs/FreeNASs4 19M 3.1M 15M 17% /data
Data 3.6T 1.4T 2.2T 38% /mnt/Data
Data/.samba4 2.2T 550k 2.2T 0% /mnt/Data/.samba4
Data/jails 2.2T 31k 2.2T 0% /mnt/Data/jails
Data/jails/.warden-template-pluginjail-9.2-RELEASE-x64 2.2T 1.1G 2.2T 0% /mnt/mnt/Data/jails/.warden-template-pluginjail-9.2-RELEASE-x64

Not sure why there's these 3 drives being wasted. How can I rectify these to use for my archives/backups?

Data/jails
Data/.samba4
Data/jails/.warden-template-pluginjail-9.2-RELEASE-x64

I'm also trying to figure out how to use the SMART tool, but a bit confused on the config in the GUI. I just want to run manual tests via command line at least to see the drives' health and conditions.

 

[joeschmuck]

I hope you didn't pay them too much because this is a UFS files system not ZFS and it's not supported in FreeNAS 9.3 or later. What do you get for "ls /dev/".

Also, list your hardware specs and version of FreeNAS you are running.

 

[Chadi]

I also tried to update the firmware to 9.3

Oct 3 12:29:42 freenas manage.py: [middleware.exceptions:38] [MiddlewareError: The firmware does not meet the pre-install criteria: mktemp: mkstemp failed on /mnt/Data/jails/owncloud_1/var/tmp/install_worker.gui_error_log.peYChh0s: No such file or directory mktemp: mkstemp failed on /mnt/Data/jails/owncloud_1/var/tmp/install_worker.verboselog.yKzBsLzv: No such file or directory mktemp: mkstemp failed on /mnt/Data/jails/owncloud_1/var/tmp/install_worker.usage_log.OY2zyMwB: No such file or directory bin/install_worker.sh: cannot create : No such file or directory usage: rm [-f | -i] [-dIPRrvW] file ... unlink file ]

Code:

[root@nas] /var/log# ls /dev/
./          bpf0@       cuau0       cuau2       da0p2       da1s3       fd/         iscsi       kmem        mdctl       pci         stderr@     ttyu0.lock  ttyu2.lock  ttyv5       ttyvb       ufs/        ugen1.1@    usb/
../         cam/        cuau0.init  cuau2.init  da1         da1s4       fido        kbd0@       led/        mem         pf          stdin@      ttyu1       ttyv0       ttyv6       ttyvc       ufsid/      ugen1.2@    usbctl
acpi        console     cuau0.lock  cuau2.lock  da1s1       devctl      geom.ctl    kbd1@       log@        nfslock     ptmx        stdout@     ttyu1.init  ttyv1       ttyv7       ttyvd       ufssuspend  ugen1.3@    xpt0
atkbd0      consolectl  cuau1       da0         da1s1a      devstat     ggctl       kbd2@       md0         null        pts/        sysmouse    ttyu1.lock  ttyv2       ttyv8       ttyve       ugen0.1@    ukbd0       zero
audit       crypto      cuau1.init  da0p1       da1s2       dtrace/     gptid/      kbdmux0     md1         pass0       random      ttyu0       ttyu2       ttyv3       ttyv9       ttyvf       ugen0.2@    ums0        zfs
bpf         ctty        cuau1.lock  da0p1.eli   da1s2a      dumpdev@    io          klog        md2         pass1       snp         ttyu0.init  ttyu2.init  ttyv4       ttyva       tws0        ugen0.3@    urandom@

Build FreeNAS-9.2.1-RELEASE-x64 (bd35c86)
Platform Intel(R) Core(TM) i3-2100 CPU @ 3.10GHz
Memory 8143MB

 

[joeschmuck]

What is your specific hardware? Motherboard, do you have a RAID controller, Hard drives installed and model numbers, etc... Keep in mind we are trying to answer your question but we need certain data in order to that that properly. You cannot use a hardware RAID, ZFS prefers a software RAID, if you didn't know that already.

It looks like you have a CPU and the 8GB of RAM needed to run FreeNAS 9.3 but you will need to backup any data on your system that you want to keep and then destroy the pools you have now and recreate them as ZFS via the GUI. Follow the user manual, it actually works well but I would obtain a new USB Flash Drive that was 8GB to 16GB is capacity and load it on that device, do not use your old one.

 

[danb35]

@joeschmuck, what is telling you that he's using UFS? If anything, the fact that Data/.samba and Data/jails are listed as separate filesystems strongly suggests that Data is a ZFS pool, albeit a poorly-named one, and .samba and jails are datasets on that pool. The boot device is UFS, of course, as it always is with 9.2.x.

@Chadi, nothing you've posted indicates that anything is being "wasted". ZFS supports things called datasets, which act kind of like directories, and kind of like separate filesystems. Part of the "kind of like separate filesystems" is that they're listed separately when you do "df". There's no need to be in a hurry to upgrade, but to help see what's going on with your system, what's the output of "zpool status"? Edit: and "camcontrol devlist"?

 

[Chadi]

Thanks for the help. So basically, having a hard time upgrading to 9.3. Plus, hoping to free up those 3x 2TB drives, if possible. I also want to enable SMART to check the health and I/O of the drives.

hw.model: Intel(R) Core(TM) i3-2100 CPU @ 3.10GHz

Code:

[root@nas] ~# dmesg | grep CPU
coretemp0: <CPU On-Die Thermal Sensors> on cpu0
p4tcc0: <CPU Frequency Thermal Control> on cpu0
coretemp1: <CPU On-Die Thermal Sensors> on cpu1
p4tcc1: <CPU Frequency Thermal Control> on cpu1
coretemp2: <CPU On-Die Thermal Sensors> on cpu2
p4tcc2: <CPU Frequency Thermal Control> on cpu2
coretemp3: <CPU On-Die Thermal Sensors> on cpu3
p4tcc3: <CPU Frequency Thermal Control> on cpu3
SMP: AP CPU #1 Launched!
SMP: AP CPU #3 Launched!
SMP: AP CPU #2 Launched!

[root@nas] ~# sysctl -a | egrep -i 'hw.machine|hw.model|hw.ncpu'
hw.machine: amd64
hw.model: Intel(R) Core(TM) i3-2100 CPU @ 3.10GHz
hw.ncpu: 4
hw.machine_arch: amd64

System Information
Manufacturer: Supermicro
Product Name: X9SCL/X9SCM

[root@nas] ~# dmesg | grep -i RAID
GEOM_RAID5: Module loaded, version 1.1.20130907.44 (rev 5c6d2a159411)

Code:

[root@nas] ~# zpool status
  pool: Data
state: ONLINE
status: The pool is formatted using a legacy on-disk format.  The pool can
        still be used, but some features are unavailable.
action: Upgrade the pool using 'zpool upgrade'.  Once this is done, the
        pool will no longer be accessible on software that does not support feature
        flags.
  scan: scrub repaired 0 in 1h9m with 0 errors on Sun Sep 27 01:09:23 2015
config:

        NAME                                          STATE     READ WRITE CKSUM
        Data                                          ONLINE       0     0     0
          gptid/213297e7-4886-11e2-a5b8-002590a4e942  ONLINE       0     0     0

errors: No known data errors

 

[danb35]

I don't understand what you mean by "hoping to free up those 3x 2TB drives"--as far as FreeNAS can see, you have one 4 TB drive. I'm now back with @joeschmuck, suspecting that you have a hardware RAID5 array set up with 3 x 2 TB drives, and your pool created on that array. That's a very bad idea for many reasons, one of which is that it prevents FreeNAS from checking the SMART attributes of your individual disks.

What's the output of "camcontrol devlist"?

Edit: Your hardware otherwise looks pretty good. Your RAM is at the minimum, but likely adequate. You have a good motherboard and CPU. But hardware RAID and FreeNAS are a bad combination.

 

[Chadi]

[root@nas] ~# camcontrol devlist
<LSI 9750-4i DISK 5.12> at scbus0 target 0 lun 0 (da0,pass0)
<SanDisk Cruzer 1.26> at scbus8 target 0 lun 0 (pass1,da1)

 

[joeschmuck]

Yea, I think the controller is a RAID configuration but I'm not an expert on hardware RAIDs.

 

[danb35]

Yep, refer back to my post from about an hour ago. Your RAID controller is presenting itself as a disk device, and FreeNAS won't be able to see past that. Since you mention 3x2TB disks, my assumption is that you have them configured as RAID5 on your RAID controller, which would result in 4 TB (or 3.6 TiB) of capacity, which is what you have.

This means you won't be able to (meaningfully) enable SMART in FreeNAS, since your RAID controller is pretending to be a hard drive. Your best bet is to back up your data, scrap the RAID controller, recreate your array on the disks through FreeNAS, and restore to that.

 

[jgreco]

[root@nas] ~# camcontrol devlist
<LSI 9750-4i DISK 5.12> at scbus0 target 0 lun 0 (da0,pass0)
<SanDisk Cruzer 1.26> at scbus8 target 0 lun 0 (pass1,da1)

Also I think that is a 3Ware 9750 which is likely to be a frickin' train wreck sooner or later.

 

[joeschmuck]

So there is a silver lining here... If you have only three hard drives, you can remove the RAID card and plug the drives into the motherboard directly and that would solve your RAID issue, HOWEVER you must backup all your data first that you want to retain. Once you have your data saved, destroy your pool, turn off your machine, open that case up and disconnect remove that RAID card, plug your drives directly into the motherboard, get a new USB boot flash drive or even a small SSD as a boot device, follow the user manual and install FreeNAS 9.3.1, create a new pool using your hard drives and create a dataset to be shared via Samba/CIFS.

The one thing you need to do here is figure out your storage solution. You stated earlier that you wanted to free up the three 2TB drives, what do you mean by "free up", does that mean you want to remove them from your system? I'm still unclear as to how many and what model hard drives you have in your system. The next thing you need to do is figure out how much storage capacity you need and if the data being stored is important or not. If it's not important then a RAIDZ1 format will do fine but if some of the data being stored is important and you want to protect it from multiple drive failure then a RAIDZ2 is highly recommended and what most of us here run. So if you could be a little clearer on your configuration, that would be helpful.

 

[Chadi]

Thanks for all the responses. I want to clarify this is 4 x 2TB, likely RAID5 from what I remember.

The reason I said free up the other 3 TB, is because they seem to be wasted on whatever the jails and samba stuff is.

 

[danb35]

If you have 4x2 TB disks, they're in a RAID6 configuration. Your volume has a total capacity of 4 TB (3.6 TiB), which is the capacity of two of your disks. Assuming that all four of the disks are part of the array, that means that you're using two disks' worth of parity. I guess it could be RAID 10 instead.

There's nothing to "free up" wrt the samba and jails datasets. They're taking up about 1 GB on your pool.

 

[joeschmuck]

If you converted your system over to use the recommended RAIDZ2 system (without the RAID controller) you would still be left with approximately 3.6TB of storage.

 

[cyberjock]

I don't know where you are getting the idea that space is being wasted. But df/du are not appropriate for ZFS file systems, so the numbers you are using are meaningless at the least, and wholly inaccurate at the worst.

Frankly, you've got bigger problems to deal with than some missing free space. ZFS + hardware RAID = lost data. And nobody noticed this, but "Data" is a zpool name that shouldn't be used in FreeNAS because it can cause all sorts of problems. The manual explicitly says not to use it.

 

[danb35]

And nobody noticed this, but "Data" is a zpool name that shouldn't be used in FreeNAS because it can cause all sorts of problems.

Data is a ZFS pool, albeit a poorly-named one

 

[rogerh]

Actually the Oracle documentation has nothing against 'Data', and the manual just says 'data' would be a bit bland and hard to pick out. And, to be pedantic, it says nothing about 'Data' which actually stands out a lot better in a non-Windows context than 'data'. Just saying.

If there are more important reasons for not choosing this pool name then they aren't in the manual.

 

[cyberjock]

Actually the Oracle documentation has nothing against 'Data', and the manual just says 'data' would be a bit bland and hard to pick out. And, to be pedantic, it says nothing about 'Data' which actually stands out a lot better in a non-Windows context than 'data'. Just saying.

That doesn't matter. We aren't using Oracle's OS, we aren't using Oracle's ZFS. So the fact that Oracle has nothing against "data" is inconsequential to FreeNAS saying not to use it.

If there are more important reasons for not choosing this pool name then they aren't in the manual.

So one reason isn't enough?

 

[danb35]

"it is recommended to choose a name that will stick out in the logs (e.g. not data or freenas)" != ""Data" is a zpool name that shouldn't be used in FreeNAS because it can cause all sorts of problems. The manual explicitly says not to use it."

"Pool name doesn't stick out in the logs" != "can cause all sorts of problems."