8-bay FreeNAS 11.2-U5 with qbittorent in BhyveVM - one drive is running extra hot?

[victorhooi]

I have a small NAS running FreeNAS 11.2-U5.

It has 8 x 12TB 3.5” HDDs, in RAID-Z1:

Code:

root@freenas[~]# zpool list
NAME               SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
bhyve-vms         1.81T  68.3G  1.75T        -         -     1%     3%  1.00x  ONLINE  /mnt
freenas-boot        57G  2.23G  54.8G        -         -      -     3%  1.00x  ONLINE  -
naulty-datastore    87T  34.0T  53.0T        -         -     0%    39%  1.00x  ONLINE  /mnt

Code:

root@freenas[~]# zpool status
  pool: bhyve-vms
 state: ONLINE
  scan: scrub repaired 0 in 0 days 00:01:14 with 0 errors on Sun Jun 23 00:01:14 2019
config:

        NAME                                          STATE     READ WRITE CKSUM
        bhyve-vms                                     ONLINE       0     0     0
          gptid/23aa0022-78f1-11e9-bb8f-ac1f6b4072fc  ONLINE       0     0     0

errors: No known data errors

  pool: freenas-boot
 state: ONLINE
  scan: scrub repaired 0 in 0 days 00:00:51 with 0 errors on Tue Jul  2 03:45:51 2019
config:

        NAME        STATE     READ WRITE CKSUM
        freenas-boot  ONLINE       0     0     0
          da0p2     ONLINE       0     0     0

errors: No known data errors

  pool: naulty-datastore
 state: ONLINE
  scan: scrub repaired 0 in 1 days 00:46:22 with 0 errors on Mon Jun  3 00:46:25 2019
config:

        NAME                                            STATE     READ WRITE CKSUM
        naulty-datastore                                ONLINE       0     0     0
          raidz1-0                                      ONLINE       0     0     0
            gptid/1f335dba-665a-11e9-a056-ac1f6b4072fc  ONLINE       0     0     0
            gptid/22682cff-665a-11e9-a056-ac1f6b4072fc  ONLINE       0     0     0
            gptid/259baf96-665a-11e9-a056-ac1f6b4072fc  ONLINE       0     0     0
            gptid/28cd6540-665a-11e9-a056-ac1f6b4072fc  ONLINE       0     0     0
            gptid/2bfb1f24-665a-11e9-a056-ac1f6b4072fc  ONLINE       0     0     0
            gptid/2f332199-665a-11e9-a056-ac1f6b4072fc  ONLINE       0     0     0
            gptid/327a1be9-665a-11e9-a056-ac1f6b4072fc  ONLINE       0     0     0
            gptid/35b92bc9-665a-11e9-a056-ac1f6b4072fc  ONLINE       0     0     0

errors: No known data errors

I have a Bhyve VM running Ubuntu 18.10 - and qBittorent running in a Docker container on here. The VM disk images are on bhyve-vms, but the downloaded torrent files are being downloaded to a datastore on the “naulty-datastore” dataset.

However, I’ve noticed poor performance when playing videos off this dataset.

Also, when I run gstat - it seems to show that one drive is running extra hot - i.e., rather than spreading writes across all 8 drives, the torrent seems to be writing only to one drive? Can that be right?

Code:

dT: 1.025s  w: 1.000s
 L(q)  ops/s    r/s   kBps   ms/r    w/s   kBps   ms/w   %busy Name
    0      0      0      0    0.0      0      0    0.0    0.0| nvd0
    0      0      0      0    0.0      0      0    0.0    0.0| nvd1
    0      0      0      0    0.0      0      0    0.0    0.0| nvd1p1
    0      1      1     16   13.2      0      0    0.0    1.3| ada0
    0      0      0      0    0.0      0      0    0.0    0.0| gptid/23aa0022-78f1-11e9-bb8f-ac1f6b4072fc
    0      1      1     16   19.5      0      0    0.0    1.9| ada1
    0      1      1     16   13.2      0      0    0.0    1.3| ada0p1
    0      1      1     20   16.5      0      0    0.0    1.6| ada2
    0      1      1     16   13.2      0      0    0.0    1.3| gptid/1f335dba-665a-11e9-a056-ac1f6b4072fc
    0      1      1     16   19.5      0      0    0.0    1.9| ada1p1
    0      0      0      0    0.0      0      0    0.0    0.0| ada3
    0      1      1     16   19.5      0      0    0.0    1.9| gptid/22682cff-665a-11e9-a056-ac1f6b4072fc
    0      1      1     20   16.6      0      0    0.0    1.6| ada2p1
    0      1      1     20   16.6      0      0    0.0    1.6| gptid/259baf96-665a-11e9-a056-ac1f6b4072fc
    0      1      1     20   19.0      0      0    0.0    1.9| ada4
    0      0      0      0    0.0      0      0    0.0    0.0| ada3p1
    0      0      0      0    0.0      0      0    0.0    0.0| gptid/28cd6540-665a-11e9-a056-ac1f6b4072fc
   10      5      4    242   1640      1      4   1844  182.8| ada5
    0      1      1     20   19.0      0      0    0.0    1.9| ada4p1
    0      1      1     20   19.0      0      0    0.0    1.9| gptid/2bfb1f24-665a-11e9-a056-ac1f6b4072fc
    0      1      1     20   15.7      0      0    0.0    1.5| ada6
    6      4      3    242   1562      1      4   1844  182.8| ada5p1
    0      1      1     16   10.3      0      0    0.0    1.0| ada7
    0      0      0      0    0.0      0      0    0.0    0.0| da0
    6      4      3    242   1562      1      4   1844  182.8| gptid/2f332199-665a-11e9-a056-ac1f6b4072fc
    0      1      1     20   15.7      0      0    0.0    1.5| ada6p1
    0      1      1     16   10.3      0      0    0.0    1.0| ada7p1
    0      1      1     20   15.7      0      0    0.0    1.5| gptid/327a1be9-665a-11e9-a056-ac1f6b4072fc
    0      0      0      0    0.0      0      0    0.0    0.0| da0p1
    0      0      0      0    0.0      0      0    0.0    0.0| da0p2
    0      1      1     16   10.3      0      0    0.0    1.0| gptid/35b92bc9-665a-11e9-a056-ac1f6b4072fc
    0      0      0      0    0.0      0      0    0.0    0.0| gptid/351d2de8-63e6-11e9-bc5a-ac1f6b4072fc

 

[Jessep]

Recommended vdev layout is Raid-Z2 or better for drive sizes over 2TB.

 

[victorhooi]

I’m guessing that’s for data-integrity reasons, right? As in, in case of two drives failing, or an additional drive failing whilst rebuilding?

Or do you think it’s related to this hot disk issue as well?

 

[MikeyG]

Is this only while playing those video sourced from torrents? Or does it show up during other kinds of activity as well? Are you moving the torrents to another dataset after download?