Best way to swap from RAIDZ1 to RAIDZ2

[Stuggi]

Hi everybody,

I just got my FreeNAS up and running (HP DL320G6, E5502, 18 GB PC3-10600R) after half a year of research, and still I made a somewhat boneheaded mistake, I set up RAIDZ1 without thinking! I'm a server/network admin by day, and we regularly use RAID5 with smaller (sub 1TB) SAS drives (URE rates of 10^15 or 10^16), so I didn't think much of it, but with "normal" drives this might be an issue.

So, current setup is 2 1TB Red drives and 2 250GB "used" HP SATA drives in RAIDZ1. I started out testing the NAS with what I had laying around (2x160GB and 2x250GB), and when it got serious about an hour in I resilvered the vDev (or maybe it's the Pool that gets resilvered?) with the two 1TB drives.

Now, I have 2 4TB Reds in the mail, and 2 more in the pipeline for next month, so before I get too much data in the pool I would like to swap out the vDev for a RAIDZ2 from the 1TB and 4TB Reds, netting me 2TB of usable space ATM and 8TB after the upgrade.

Now the question becomes how to offload the data while I work on the pool. I have a VMware host with a 2 TB datastore sitting pretty next to the FreeNAS, so I can set up just about anything on that to take the data from the FreeNAS. What I would like to accomplish is something where I can basically just transfer out the whole pool, delete it from the FreeNAS and copy it back in again, without having to fix a whole lot of stuff, like my Plex Jail and what have you. Can this be done easily and how? From what I can gather it's quite easy to do this if you have something that runs ZFS that you can backup to and from, but how do I set this up with virtual HDDs in VMware? (I know the right way would be to pass the drives through to the VM directly, but I don't want to break my ESXi environment if I don't have to.)

Quite long for a first post, but I like to jump into things head first (which is why I need to do this in the first place) :)

Cheers,
Sebastian

 

[depasseg]

I think this might help: https://forums.freenas.org/index.ph...te-data-from-one-pool-to-a-bigger-pool.40519/ For the replication, I would suggest using the CLI to replicate your data to a file that resides on the datastore (mount using nfs or send over ssh) re-swizzle the pool, and then replicate back from the file.

 

[Stuggi]

Okay, that looks simple enough, I gather that in my case I would skip the renaming step and just set up a new pool on the new RAIDZ2 with the same name and then copy everything back?
Would it perhaps be a better idea to just get a suitable USB drive and connect that to the FreeNAS temporaily? The whole pool is only about 300 GB used out of 750 GB, so it not that much data in the end.

 

[depasseg]

Yes, bothof those sound right. There have been some issues in the past with USB though. I'm not sure what they are, but you can try to make the initial copy to the USB drive and see how it goes. For best results, the USB drive should be added as a new pool using the freenas GUI. You won't be able to write to a windows/ntfs formatted drive.

 

[Stuggi]

Yeah, I was thinking that, then I would not have to mess with getting FreeNAS to run properly on VMDKs.
Is there any way to turn of all the shenanigans in ZFS that needs access to the drives proper so that it would run all right on VMware with virtual disks? Or should I just go with NFS if I go down that route?

 

[depasseg]

I'm sorry, but I don't understand what you mean.

 

[Stuggi]

Ah, as I've understood it, ZFS need direct access to the drives, otherwise you risk data corruption since it can get confused about what's actually written to the drives themselves. What I was wondering was if it's possible to set up ZFS in any way, for example one vdev per virtual disk, so that you don't risk data corruption. If that's possible I could just set up another FreeNAS in a VM and replicate everything over to that instead.
From what I can find online, there's a similar problem with USB drives and ZFS, since the USB interface doesn't give ZFS "direct access" to the drives themselves, so that would need solving as well.

This might just be overthinking on my part, I come from an enterprise school-of-thought, where data integrity is key, and since I'm not overly familiar with ZFS I might be obsessing a bit too much about everything I read. :)

 

[depasseg]

Ahh, I see. Yes, you could install a freenas VM and replicate to that. But you can also replicate ZFS filesystem to a file located on an NFS share for instance. Or you could format the USB drive as ZFS and replicate there.

 

[TAC]

Timing is perfect on this question!

I also have a FreeNAS (9.3) system running RAIDZ1 (three 3TB drives) and have decided to move to RAIDZ2. As I sit here looking at my six brand new HGST 0S03664 4TB drives I wonder the most efficient way to move my 'stuff'.

My data (movies/music/TV) isn't really problem in that I can move that temporarily to a Windozs machine (if needed). What I'm really interested in maintaining are my jails and the configurations of all the plugins (PlexMediaServer, SABnzbd, Sonarr, and CouchPotato). I pretty much have these dialed in and would hate to have to reinstall them from scratch.

Since my MoBo (Supermicro X10SLL-F, 16GB) has only 6 SATA (two 3.0, four 2.0) ports I was wondering if I should just get a PCI-E 8 port SATA 3.0 card and then start the process with all 9 drives in my main machine.

Another option would be to setup another temporary FreeNAS server with some drives I have laying around and use this box to hold my temporary data/files/jails.

As I'm pretty much a noob trying to move from RAIDZ1 to RAIDZ2 as efficiently (painless) as possible any comments/suggestions would be much appreciated.


-TAC

 

[depasseg]

The same theory applies. Figure out where you can replicate to (depends on how much data you have and how much space you have).

 

[Stuggi]

I went through with the move, using a 2 TB USB drive as a temporary pool to hold my data! Thanks depasseg for all the help!
I did a short write-up on exactly what I did over on my blog, I'd be really grateful if you could have a look at it and point out if I did something stupid unnecessary!
http://grumpytechie.net/2016/02/06/freenas-migrating-a-raid-z-vdev-to-a-raid-z2/

 

[Yatti420]

FYI in the future you should move Z2 to Z2 to Z2 etc.. Never leave yourself in a situation with no backup or adequate redundancy.. Doing it through VMware is even worse idea - per the freenas rules.. For me I just setup both servers side by side and fire the data across.. Aka z2 (or z1 since you error-ed) to z2..

 

[depasseg]

Just a couple comments:

2) I'm pretty sure the System drive isn't read only, and you can place the system dataset on it (that's what I'm doing). You don't need a separate USB drive.

6) I would recommend not running anything from the USB drive and instead just using the same "migrate" snapshot. But that's just me.

I think the caveat about the jails was due to the fact that you overwrote the second "tank" instance without exporting and importing. You overwrote the pool that Freenas was aware of, but didn't let it get a chance to re-inventory (for lack of a better term) the datasets you copied into it.

Otherwise it looks good! Congrats

 

[Stuggi]

FYI in the future you should move Z2 to Z2 to Z2 etc.. Never leave yourself in a situation with no backup or adequate redundancy.. Doing it through VMware is even worse idea - per the freenas rules.. For me I just setup both servers side by side and fire the data across.. Aka z2 (or z1 since you error-ed) to z2..

Yeah, I know, but since this was non-essential data that I had copies of laying around, I didn't bother setting up a second server. If I would have had a large enough server laying around I would have definitely just gone ahead and temporarily installed FreeNAS on it.

 

[Stuggi]

Just a couple comments:

2) I'm pretty sure the System drive isn't read only, and you can place the system dataset on it (that's what I'm doing). You don't need a separate USB drive.

6) I would recommend not running anything from the USB drive and instead just using the same "migrate" snapshot. But that's just me.

I think the caveat about the jails was due to the fact that you overwrote the second "tank" instance without exporting and importing. You overwrote the pool that Freenas was aware of, but didn't let it get a chance to re-inventory (for lack of a better term) the datasets you copied into it.

Otherwise it looks good! Congrats

2) I went by this article, https://www.pretzellogix.net/2015/02/04/how-to-get-freenas-9-3-to-spin-down-your-idle-hard-drives/ and some other source that states that the installation drive is mounted read-only, but I can't find it now.

6) Okay, I didn't actually modify the data at all on the USB drive, the second snapshot was just a "belts-and-suspenders" approach. So I could just have copied the same snapshot back?

Okay, so I should have detached the pool after copying everything back, export it and import it? I read through your guide before attempting this, but I couldn't really figure out what the export step did exactly. :)

 

[depasseg]

2). This was fixed last year. https://www.pretzellogix.net/2015/0...spin-down-your-idle-hard-drives/#comment-1707

6). Yep. That's what I did.

The import process lets freenas go through the datasets and import what's there. In your case I think the fact that you overwrote the jails dataset meant that even though they were in the dataset, freenas want really aware that they were there. Just a guess though because my jails were OK.

 

[Stuggi]

2) Okay, missed that comment. :)

6) Do I need to detach it in GUI, then run the export command, then import it through the GUI, or is export and detach through the GUI the same thing? Aka. could I just do GUI Detach and then GUI Import? Just checking since in your tutorial, you run GUI Detach, CLI Import to rename the pools, CLI Export, and then GUI Import.

 

[depasseg]

2) Okay, missed that comment. :)

6) Do I need to detach it in GUI, then run the export command, then import it through the GUI, or is export and detach through the GUI the same thing? Aka. could I just do GUI Detach and then GUI Import? Just checking since in your tutorial, you run GUI Detach, CLI Import to rename the pools, CLI Export, and then GUI Import.

Sorry about the confusion, since you weren't renaming, detach and import would be fine.