Sanity Check: Is my proposed change to my storage pool correct?

[surfrock66]

I have a FreeNAS box that HAD 4TB of storage, 2 2TB drives, in a ZFS pool using ZFS I think 18 (or 15, the old one that came with FreeNAS 8.0). I just upgraded to FreeNAS 8.3.1 and migrated my ZFS pool to v28 successfully, I'm very happy.

I just got 2 new 3TB drives, and want to configure this as a Mirrored setup with 5TB of mirrored storage. I do not have anywhere close to enough space to throw this stuff elsewhere to backup in case of catastrophe, so I'm trying to do this live. I need to sanity check what I'm trying to do before I do it. I've read the documentation thoroughly but will never discount the fact that I don't know where I'm being an idiot.

I've read this: http://docs.oracle.com/cd/E19082-01/817-2271/gazgw/
And this: http://doc.freenas.org/index.php/Volumes

My plan, and what I'm asking is, is this the correct command set and order to run?

Add the 2 new 3TB drives as a fresh new mirrored pool (call it sr66-nas1), then move 3TB of the data to that, then the remainder should be enough to stash on another computer, blow away the original pool (sr66-nas) and add those 2 2TB drives to extend the sr66-nas1 mirrored pool, then add it back. These are the commands I think will do that:

Existing drives: ada2 (2TB), ada3 (2TB), ada0 (3TB), ada1 (3TB)

1. zpool create sr66-nas1 mirror ada0 ada1
2. Share the new pool in NFS Shares, mount it, and copy 3TB of data to it from the old share, copy remaining data off the old share onto other computers.
3. zpool destroy sr66-nas
4. zpool add sr66-nas1 ada2
5. zpool attach sr66-nas1 ada2 ada3

So...Am I nuts? Please advise.

 

[cyberjock]

You aren't nuts. Everything sounds fine.

Do keep in mind that with mirrors you do have a redundant copy, but if one disk fails and the other has even 1 bad sector the potential exists to lose data(how much depends on alot of factors). For this reason I never recommend anything less than RAIDZ2.

 

[surfrock66]

Thanks for the confirmation.

Given my setup (2 2TB drives, 2 3TB drives, the first 2 2TB drives in a pool with about 3.1TB of data) if I tried to set up a raidz2 (which I researched but looked like it was more than I needed) what would be the total storage, still 5TB? And what would be the command order to that? Could I still do it live?

 

[cyberjock]

If you did a RAIDZ2(I started to type that but realized your potential issue) you'd be forcing all of the drives on the vdev to the smallest size(2TB) so you'd really have only 4TB usable.

Keep in mind the zpools should never be filled beyond 90%. Even then, some people recommend 80%.

 

[surfrock66]

Ok. I'll go with the mirrored solution...it's an archive of hard to find japanese MMA, plus a whole bunch of raw video since I did production for small local MMA promotions...nothing terribly critical but I want it mirrored as a little extra protection.

Thanks!

 

[Stephens]

As an avid MMA fan, here's hoping you don't lose it. I'm sure you know already, but RAID isn't backup. You might want to copy some of the most critical/rare stuff elsewhere, such as DVD-R backup, or even just dumping it to an external drive.

 

[paleoN]

1. zpool create sr66-nas1 mirror ada0 ada1
2. Share the new pool in NFS Shares, mount it, and copy 3TB of data to it from the old share, copy remaining data off the old share onto other computers.
3. zpool destroy sr66-nas
4. zpool add sr66-nas1 ada2
5. zpool attach sr66-nas1 ada2 ada3

I would swap the last two commands to this one:

Code:

zpool add sr66-nas1 mirror ada2 ada3

 

[surfrock66]

I would swap the last two commands to this one:

Code:

zpool add sr66-nas1 mirror ada2 ada3

So...I'm trying this command (It took a long time to do the transfers out, but now sr66-nas1 is full to 80% of stuff, then everything else is on my desktop. I get the following:

Code:

[root@freenas ~]# zpool add sr66-nas1 mirror ada2 ada3                          
invalid vdev specification                                                      
use '-f' to override the following errors:                                      
mirror contains devices of different sizes

Is it trying to mirror the original mirror, or adding a new mirror to the pool?

 

[paleoN]

Is it trying to mirror the original mirror, or adding a new mirror to the pool?

It's adding a new mirror vdev to the pool which will be striped with the existing vdev. It appears to be complaining that ada2 & ada3 are not the same size?

Code:

camcontrol identify /dev/ada2

camcontrol identify /dev/ada3

Might shed some light.

In this case you can rerun the command with the -f assuming those are the drives you want to use.

 

[surfrock66]

It's adding a new mirror vdev to the pool which will be striped with the existing vdev. It appears to be complaining that ada2 & ada3 are not the same size?

Code:

camcontrol identify /dev/ada2

camcontrol identify /dev/ada3

Might shed some light.

In this case you can rerun the command with the -f assuming those are the drives you want to use.

That's the trick...that I asked instead of enforcing -f. It needed to be ada3 and ada4. The OS is on an 80GB old IDE drive which is ada2.

Code:

[surfrock66@freenas] /# camcontrol identify /dev/ada0
pass0: <WDC WD30EZRX-00DC0B0 80.00A80> ATA-9 SATA 3.x device
pass0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)

protocol              ATA/ATAPI-9 SATA 3.x
device model          WDC WD30EZRX-00DC0B0
firmware revision     80.00A80
serial number         WD-WMC1T2736354
WWN                   50014ee00389eccf
cylinders             16383
heads                 16
sectors/track         63
sector size           logical 512, physical 4096, offset 0
LBA supported         268435455 sectors
LBA48 supported       5860533168 sectors
PIO supported         PIO4
DMA supported         WDMA2 UDMA6 

Feature                      Support  Enabled   Value           Vendor
read ahead                     yes	yes
write cache                    yes	yes
flush cache                    yes	yes
overlap                        no
Tagged Command Queuing (TCQ)   no	no
Native Command Queuing (NCQ)   yes		32 tags
SMART                          yes	yes
microcode download             yes	yes
security                       yes	no
power management               yes	yes
advanced power management      no	no
automatic acoustic management  no	no
media status notification      no	no
power-up in Standby            yes	no
write-read-verify              no	no
unload                         no	no
free-fall                      no	no
data set management (TRIM)     no
[surfrock66@freenas] /# camcontrol identify /dev/ada1
pass1: <WDC WD30EZRX-00DC0B0 80.00A80> ATA-9 SATA 3.x device
pass1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)

protocol              ATA/ATAPI-9 SATA 3.x
device model          WDC WD30EZRX-00DC0B0
firmware revision     80.00A80
serial number         WD-WMC1T2783897
WWN                   50014ee6addc2a78
cylinders             16383
heads                 16
sectors/track         63
sector size           logical 512, physical 4096, offset 0
LBA supported         268435455 sectors
LBA48 supported       5860533168 sectors
PIO supported         PIO4
DMA supported         WDMA2 UDMA6 

Feature                      Support  Enabled   Value           Vendor
read ahead                     yes	yes
write cache                    yes	yes
flush cache                    yes	yes
overlap                        no
Tagged Command Queuing (TCQ)   no	no
Native Command Queuing (NCQ)   yes		32 tags
SMART                          yes	yes
microcode download             yes	yes
security                       yes	no
power management               yes	yes
advanced power management      no	no
automatic acoustic management  no	no
media status notification      no	no
power-up in Standby            yes	no
write-read-verify              no	no
unload                         no	no
free-fall                      no	no
data set management (TRIM)     no
[surfrock66@freenas] /# camcontrol identify /dev/ada2
pass2: <WDC WD181AA 29.05T29> ATA-4 device
pass2: 66.700MB/s transfers (UDMA4, PIO 8192bytes)

protocol              ATA/ATAPI-4
device model          WDC WD181AA
firmware revision     29.05T29
serial number         WD-WM9160058044
cylinders             16383
heads                 16
sectors/track         63
sector size           logical 512, physical 512, offset 0
LBA supported         35419104 sectors
LBA48 not supported       
PIO supported         PIO4
DMA supported         WDMA2 UDMA4 

Feature                      Support  Enabled   Value           Vendor
read ahead                     yes	yes
write cache                    yes	yes
flush cache                    no	no
overlap                        no
Tagged Command Queuing (TCQ)   no	no
Native Command Queuing (NCQ)   no
SMART                          yes	yes
microcode download             yes	yes
security                       no	no
power management               yes	yes
advanced power management      no	no
automatic acoustic management  no	no
media status notification      no	no
power-up in Standby            no	no
write-read-verify              no	no
unload                         no	no
free-fall                      no	no
data set management (TRIM)     no
[surfrock66@freenas] /# camcontrol identify /dev/ada3
pass3: <ST32000542AS CC34> ATA-8 SATA 2.x device
pass3: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes)

protocol              ATA/ATAPI-8 SATA 2.x
device model          ST32000542AS
firmware revision     CC34
serial number         5XW17Z7A
WWN                   5000c5002a79380a
cylinders             16383
heads                 16
sectors/track         63
sector size           logical 512, physical 512, offset 0
LBA supported         268435455 sectors
LBA48 supported       3907029168 sectors
PIO supported         PIO4
DMA supported         WDMA2 UDMA6 
media RPM             5900

Feature                      Support  Enabled   Value           Vendor
read ahead                     yes	yes
write cache                    yes	yes
flush cache                    yes	yes
overlap                        no
Tagged Command Queuing (TCQ)   no	no
Native Command Queuing (NCQ)   yes		32 tags
SMART                          yes	yes
microcode download             yes	yes
security                       yes	no
power management               yes	yes
advanced power management      yes	no	0/0x00
automatic acoustic management  yes	no	0/0x00	254/0xFE
media status notification      no	no
power-up in Standby            yes	no
write-read-verify              yes	no	0/0x0
unload                         no	no
free-fall                      no	no
data set management (TRIM)     no
[surfrock66@freenas] /# camcontrol identify /dev/ada4
pass4: <ST32000542AS CC34> ATA-8 SATA 2.x device
pass4: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes)

protocol              ATA/ATAPI-8 SATA 2.x
device model          ST32000542AS
firmware revision     CC34
serial number         5XW182YN
WWN                   5000c5002a718fe0
cylinders             16383
heads                 16
sectors/track         63
sector size           logical 512, physical 512, offset 0
LBA supported         268435455 sectors
LBA48 supported       3907029168 sectors
PIO supported         PIO4
DMA supported         WDMA2 UDMA6 
media RPM             5900

Feature                      Support  Enabled   Value           Vendor
read ahead                     yes	yes
write cache                    yes	yes
flush cache                    yes	yes
overlap                        no
Tagged Command Queuing (TCQ)   no	no
Native Command Queuing (NCQ)   yes		32 tags
SMART                          yes	yes
microcode download             yes	yes
security                       yes	no
power management               yes	yes
advanced power management      yes	no	0/0x00
automatic acoustic management  yes	no	0/0x00	254/0xFE
media status notification      no	no
power-up in Standby            yes	no
write-read-verify              yes	no	0/0x0
unload                         no	no
free-fall                      no	no
data set management (TRIM)     no

- - - Updated - - -

Ok, one more thing...sorry I'm asking on like each command...I'm a bit paranoid, and like I said, I only kind of know what I'm doing...

Got this command set. My instinct is to reboot and try it again:

Code:

[surfrock66@freenas] /# zpool add sr66-nas1 mirror ada3 ada4
cannot add to 'sr66-nas1': no such pool or dataset
[surfrock66@freenas] /# zpool list
NAME        SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOT
sr66-nas1  2.72T  2.16T   568G    79%  1.00x  ONLINE  /mnt

 

[paleoN]

Ok, one more thing...sorry I'm asking on like each command...I'm a bit paranoid, and like I said, I only kind of know what I'm doing...

Better to ask than have to destroy the pool and start over. Those 2TB drives appear to be 512b native which means you will want to gnop them first to match the ashift=12 existing vdev. Given that you are running 8.3.1 you also can extend the existing mirror from the GUI checking the Force 4096 bytes sector size to do this for you. Set the swap to 0 under Advanced Settings ahead of time if you don't want a swap partition.

Got this command set. My instinct is to reboot and try it again:

That is a bit odd. I take it a zpool status -v looks normal?

 

[surfrock66]

That is a bit odd. I take it a zpool status -v looks normal?

Yup, take a look:

Code:

[surfrock66@freenas] /# zpool status -v
  pool: sr66-nas1
 state: ONLINE
  scan: none requested
config:

	NAME                                            STATE     READ WRITE CKSUM
	sr66-nas1                                       ONLINE       0     0     0
	  mirror-0                                      ONLINE       0     0     0
	    gptid/3059a414-9c76-11e2-8297-0030678932f1  ONLINE       0     0     0
	    gptid/316366e4-9c76-11e2-8297-0030678932f1  ONLINE       0     0     0

errors: No known data errors

I'll try to extend them using the gui, unless anyone has other thoughts. The stuff you listed above is voodoo to me, if there's a command set that'll do it I'm happy to try it.

- - - Updated - - -

One last sanity check...according to the wiki:

to extend a ZFS mirror, add the same number of drives. The resulting striped mirror is a RAID 10.

So when I'm in the volume manager, choose my 2 disks, choose the correct volume to extend, choose ZFS, force 4096...am I doing group type mirror or stripe?

 

[titan_rw]

So when I'm in the volume manager, choose my 2 disks, choose the correct volume to extend, choose ZFS, force 4096...am I doing group type mirror or stripe?

Mirror. When you choose volume to extend, freenas stripes the new vdev into the existing pool. Choosing mirror ensures the vdev that gets striped in has redundancy.

If you chose stripe, you'd have a mirror vdev, and a stripe vdev, both striped together. Definitely not recommended.