Register for the iXsystems Community to get an ad-free experience and exclusive discounts in our eBay Store.

Help planning my upgrade

Status
Not open for further replies.

BBarker

Member
Joined
Aug 7, 2015
Messages
114
I am upgrading my system this week and would like some input as to the best way to transfer the data that is on my current system which is in my signature. I am running FreeNAS-9.10-STABLE-201605021851 (35c85f7) and everything is working well.

I will be changing enough of the system where if I want, I can bench build the new system with the new drives and do a complete new install of FreeNAS 9.10 with two 6TB drives in a mirrored pool and an SSD for my jails and .system files. I saw an online tutorial regarding setting up FreeNAS 9.10 in this fashion and if anyone has the link handy, I'd appreciate if you could post it here as I am having some trouble locating it again.

Once it is up and running, is it possible to install the current pool of two 3TB mirrored drives from my current system into the new system and then transfer just the data? Also, once it is done, I'd like to wipe the old drives and create a new pool with them for additional storage. My original plan was to just swap the old drives with the new one at a time and then replace the motherboard with the new components when the drive swap was complete but the idea of a new install seems appealing and maybe less problematic.
 

DrKK

FreeNAS Generalissimo
Joined
Oct 15, 2013
Messages
3,630
First of all, I am required to say this: AMD APU-based FreeNAS build? ewwwwwwwwwwwwwwwwwww.

Second of all, you have Seagate Barracuda ST3000DM001. The correct storage configuration with this model hard drive is as deep in a landfill as possible. See, for example: https://www.backblaze.com/blog/3tb-hard-drive-failure/

Third of all, I don't know if I mentioned (oh wait, I just did---but it bears repeating)....You have Seagate Barracuda ST3000DM001's. You should take a croquet mallet, and smash smash smash smash smash those drives as soon as possible. Here is a YouTube video that demonstrates the accepted technique.


Fourth of all, if I understand you correctly, you want to sort of temporarily mount the old 2x3TB pool alongside the 2x6TB pool to copy the data. That is quite reasonable and easy to do. Make sure you properly detach (i.e., "export") the old pool from your FreeNAS, which you can do in the "STORAGE" screen. (This will be easiest if your old and new pools do NOT have the same name). Make your new boot installation for your 2x6TB system, configure it however you like. Attach the two 3TB drives to the motherboard and reboot. Now, you can "import" the 3TB pool, and make whatever copies you want, then you can detach/export the old pool and remove the drives.

Fifth of all, I still think the best way to do this is to replace the 3TB drives by the 6TB drives one-at-a-time. Your situation is precisely why this functionality exists in ZFS in the first place, don't know why you would want to circumvent it.
 

BBarker

Member
Joined
Aug 7, 2015
Messages
114
First of all, I am required to say this: AMD APU-based FreeNAS build? ewwwwwwwwwwwwwwwwwww.

Second of all, you have Seagate Barracuda ST3000DM001. The correct storage configuration with this model hard drive is as deep in a landfill as possible. See, for example: https://www.backblaze.com/blog/3tb-hard-drive-failure/

Third of all, I don't know if I mentioned (oh wait, I just did---but it bears repeating)....You have Seagate Barracuda ST3000DM001's. You should take a croquet mallet, and smash smash smash smash smash those drives as soon as possible. Here is a YouTube video that demonstrates the accepted technique.


Fourth of all, if I understand you correctly, you want to sort of temporarily mount the old 2x3TB pool alongside the 2x6TB pool to copy the data. That is quite reasonable and easy to do. Make sure you properly detach (i.e., "export") the old pool from your FreeNAS, which you can do in the "STORAGE" screen. (This will be easiest if your old and new pools do NOT have the same name). Make your new boot installation for your 2x6TB system, configure it however you like. Attach the two 3TB drives to the motherboard and reboot. Now, you can "import" the 3TB pool, and make whatever copies you want, then you can detach/export the old pool and remove the drives.

Fifth of all, I still think the best way to do this is to replace the 3TB drives by the 6TB drives one-at-a-time. Your situation is precisely why this functionality exists in ZFS in the first place, don't know why you would want to circumvent it.
Your reply is very much appreciated but still left me unsure on whether your are, or are not, a fan of Seagate drives and AMD processors. Can you please clarify this for me?

All kidding aside......the original AMD build was from spare parts I had on hand to try it out. I love how the system works in all respects and will be upgrading my motherboard, processor, RAM and power supply to parts found in the recommended hardware sticky this week. I do have to say that unlike others, I have had great success with AMD and Seagate builds. Just lucky I guess.

I was thinking a new build with the new hardware would be preferable since I will be making some changes as to where I will keep my jails by using an SSD. In the scenario where I can export from the old to import to the new, this action won't transfer any confuguration settings from the old system but will allow me to transfer my media files only?
 

Ericloewe

Not-very-passive-but-aggressive
Moderator
Joined
Feb 15, 2014
Messages
16,252
Your reply is very much appreciated but still left me unsure on whether your are, or are not, a fan of Seagate drives and AMD processors. Can you please clarify this for me?

All kidding aside......the original AMD build was from spare parts I had on hand to try it out. I love how the system works in all respects and will be upgrading my motherboard, processor, RAM and power supply to parts found in the recommended hardware sticky this week. I do have to say that unlike others, I have had great success with AMD and Seagate builds. Just lucky I guess.

I was thinking a new build with the new hardware would be preferable since I will be making some changes as to where I will keep my jails by using an SSD. In the scenario where I can export from the old to import to the new, this action won't transfer any confuguration settings from the old system but will allow me to transfer my media files only?
Does anything in your current configuration specifically bother you?
If not, you can just move the boot device over and redo the networking settings.

If you want to start over, just do a fresh install and then import the pool (and do the switcheroo or copying over to it).

Fifth of all, I still think the best way to do this is to replace the 3TB drives by the 6TB drives one-at-a-time. Your situation is precisely why this functionality exists in ZFS in the first place, don't know why you would want to circumvent it.
Well, if all the new drives are available, zfs send/receive would clear much of whatever fragmentation the original pool might have. That might be interesting in some cases.

For reference, if you're going to try the "copy stuff over" approach, to rename pools, export them and then import them with
Code:
zpool import oldname newname
 

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
In short:

Seagate make good drives (usually) but the ST3000DM001 is awful.

AMD ECC support is crappy and you really want ECC RAM to work to protect your data.
 

BBarker

Member
Joined
Aug 7, 2015
Messages
114
Nothing in my current configuration bothers me and quite frankly has been working very well. It was always going to be a temporary setup anyway as I explored FreeNAS and now that I want to continue to use it, I will change over to more "forum friendly" approved hardware (meaning ECC RAM and Intel processor). That and the fact I want to move my Jails to an SSD makes me think it would be nice to start fresh. Besides, I have already promised the components now being used to a church member that needs a pc and is a little down on their luck right now. Pretty sure they won't be as concerned about AMD and Seagate components.....lol.
 

DrKK

FreeNAS Generalissimo
Joined
Oct 15, 2013
Messages
3,630
Well, if all the new drives are available, zfs send/receive would clear much of whatever fragmentation the original pool might have. That might be interesting in some cases.
Do we know that the upgrade-in-place procedure doesn't do this? Does it in fact make a block-by-block copy, or does it unroll the filesystem on file at a time? If the latter, that would seem to clear fragmentation, no? Just curious.
 

Ericloewe

Not-very-passive-but-aggressive
Moderator
Joined
Feb 15, 2014
Messages
16,252
Do we know that the upgrade-in-place procedure doesn't do this? Does it in fact make a block-by-block copy, or does it unroll the filesystem on file at a time? If the latter, that would seem to clear fragmentation, no? Just curious.
Hmm, it's an interesting question.

I'm fairly certain it can't be a straight copy from the old drive - that sounds like it would break atomicity.
At the same time, I'm not sure that the new drive can actually end up defragmented - if it were possible, it sounds like defrag would be rather simple, but we know that would require Block Pointer Rewrite.
 

DrKK

FreeNAS Generalissimo
Joined
Oct 15, 2013
Messages
3,630

BBarker

Member
Joined
Aug 7, 2015
Messages
114
Here, by the way, is the recommended technique for handling Seagate 3TB Barracudas, this is the DrKK-certified algorithm:

https://www.youtube.com/watch?v=XKhghdIGiLI

That's too much work.......

Back to topic. If I export the pool, build the new system, install FreeNAS 9.10 with the new drives and then import the old pool, do I use my windows pc to see the two pools and transfer files?
 

Ericloewe

Not-very-passive-but-aggressive
Moderator
Joined
Feb 15, 2014
Messages
16,252
That's too much work.......

Back to topic. If I export the pool, build the new system, install FreeNAS 9.10 with the new drives and then import the old pool, do I use my windows pc to see the two pools and transfer files?
Or you can be a bit smarter about it and log in via SSH, run a recursive snapshot of the whole pool, and zfs send it over to the new pool.
 

BBarker

Member
Joined
Aug 7, 2015
Messages
114
Or you can be a bit smarter about it and log in via SSH, run a recursive snapshot of the whole pool, and zfs send it over to the new pool.
Tutorial?

Will this just handle the files and not have anything to do with configuration? My old system had jails installed in the same pool as my media files and the new system is moving them to their own pool on a SSD.
 

Ericloewe

Not-very-passive-but-aggressive
Moderator
Joined
Feb 15, 2014
Messages
16,252
Tutorial?
I don't have one. I suggest you read the man page for ZFS.

Will this just handle the files and not have anything to do with configuration?
It has nothing to do with the config.

My old system had jails installed in the same pool as my media files and the new system is moving them to their own pool on a SSD.
You'll want to replicate the jails dataset to the new location and/or delete it, depending on whether you want to start over.
 

BBarker

Member
Joined
Aug 7, 2015
Messages
114
Would this be a valid way to move the data?

cp -R /mnt/OLDPool/Storage/data/* /mnt/NEWPool/raid

It's about 2.6Tb
 
Last edited:

Ericloewe

Not-very-passive-but-aggressive
Moderator
Joined
Feb 15, 2014
Messages
16,252
That won't preserve datasets and may present additional oddities.

If you want a proper copy of the whole layout, use zfs send/receive.
 

BBarker

Member
Joined
Aug 7, 2015
Messages
114
That won't preserve datasets and may present additional oddities.

If you want a proper copy of the whole layout, use zfs send/receive.
Sorry about being so dense about this, I've read information on using zfs send / receive and still find it a little confusing. Let me post what I think I need to do and get some input before I actually try and move the data.
 

Ericloewe

Not-very-passive-but-aggressive
Moderator
Joined
Feb 15, 2014
Messages
16,252

BBarker

Member
Joined
Aug 7, 2015
Messages
114

BBarker

Member
Joined
Aug 7, 2015
Messages
114
zfs send pool/fs@a | ssh host zfs receive poolB/received/fs@a
zfs
send -i a pool/fs@b | ssh host zfs receive poolB/received/fs

I am assuming that "pool" is the current pool I am using and that "poolB" will be the pool I am sending the data to?

Do I have to have a recent snapshot of my active pool to use this function?
Also, do I need to rename "pool" to my current designation: mnt/barker_1/media and "poolB" to the whatever I designate it to be?

Here is a screen shot of my current system volume.

Untitled.jpg
 

BBarker

Member
Joined
Aug 7, 2015
Messages
114
Been reading for hours......

System is up and running. I have created a new pool that is currently empty and can now import my old pool using the import volume feature. Once that is done, take a recursive snaphot of the old pool and then SSH into the system with putty and type this command from the root:

zfs send -R OLDPool@snapshot_transfer | zfs receive -Fdu NEWPool

Will this transfer all my data and datasets to the new pool?
 
Status
Not open for further replies.
Top