Upgrading 9.10.2 to 11.2 with a fresh install, what about my existing pool?

[Luddite Technologist]

Greetings all. I have been happily running FreeNAS for about five years with no problems. I am getting ready to 'upgrade' my current box and although I have searched through the forums, I can't find the answer to my specific question. If I have missed it, please give me the appropriate amount of grief and point me to the correct thread.

First of all, my FreeNAS box is for home use and serves the dual purpose of providing a convenient and pretty-darn-safe home storage solution for movies, family photos, and all the other digital junk that accumulates over the years, as well as giving me the opportunity to 'play' in a non-critical environment for my own personal enjoyment and keep my skills fresh. Everything important gets backed up off-site as well as in the cloud.

My current system:

• FreeNAS 9.10.2 U6
• Supermicro X11SSM-F (I upgraded after my ASRock C2750D4I died)
• Intel Xeon 1245 V6
• 2 x MICRON 16GB ECC UDIMM (MTA18ASF2G72AZ-2G3B1)
• 4 x WD RED 4TB in a RAID-Z1 configuration
• 2 x Kingston DataTraveler SE9 G2 16GB USB drives (boot + mirror)

I have purchased four new WD RED 6TB drives, downloaded a fresh copy of FreeNAS 11.2 U7, prepared the installation media (USB drive) and I am ready to roll! Here is where my question starts.

I want to start fresh. A clean install of FreeNAS. I don't like the decisions I made years ago when I first set up my system and I have bungled around in there sufficiently that I don't even know at this point what I have unwittingly 'broken'. I don't care about any of the current configurations, plugins, jails, or anything else, just the data. Plus, it has been years since I set this whole thing up and I want to re-learn how to do things from scratch because at this point, I have forgotten!

Problem is, I am not sure what the best way to do this upgrade is. The tl;dr version of my goal here is that I want a fresh FreeNAS system incorporating the new additional storage I have purchased without losing any data on my current pool. What seems to me to be the best, based on what I have read, would be this:

1. Remove the USB boot drive(s) and four HDD's currently in the system, put them aside.
2. Install the four new HDD's, new USB boot drives
3. Install a clean copy of FreeNAS 11.2
4. Configure the new HDD's into a RAID-Z1 pool
5. Setup any plugins and fun bits.
6. Go get a cup of coffee

Now, with the new system up and running happily:

1. Put the four old HDD's back in the system
2. Import (?) the old pool as a temporary pool rather than directly adding it into the primary pool (this is the only way to preserve the data on the old pool, correct?)
3. Move the data that I want to keep from the temporary pool into the primary pool on the new HDD's
4. Kill the temporary pool and wipe the four old HDD's
5. Add the four old HDD's as a new vdev to extend the primary pool
6. Go get another cup of coffee and a piece of cake (I deserve a reward, right?)

Can anyone see any issues with following this plan? Is there a better way? Are there pitfalls to this that I should be aware of? Will the 9.10.2 configured pool be seen by the new 11.2 system? Any suggestions on a good celebratory cake?

Thanks in advance!

 

[Alecmascot]

looks ok but...

A raidz1 pool is sub-optimal for 6TB drives. Raidz2 would be better.
USB boot drives are a no-no now. Use an SSD.

 

[Jasse Jansson]

Boot SSD's uses valuable and limited number of SATA ports.
RaidZ2 is only viable if you have a lot of SATA ports.
I have booted from mirrored USB sticks for years with very little trouble.
PS.
SSD drives dies too.
DS.

 

[Jessep]

What case are you using (how many bays, ports)? There isn't a reason to remove the drives, just unplug if you are concerned.

Likely you can just use new boot media, do the install, import your old pool, create a new pool on new drives, and then coffee.

Don't forget to do all the testing on the new drives before putting them into service, bad blocks etc.
https://www.ixsystems.com/community/threads/uncle-festers-freenas-beginners-guide.76703/

I second use RaidZ2 and SDD boot.

 

[garm]

Boot SSD's uses valuable and limited number of SATA ports.

My HBA supports 256 drives, SATA ports are a non issue

 

[Luddite Technologist]

looks ok but...

A raidz1 pool is sub-optimal for 6TB drives. Raidz2 would be better.
USB boot drives are a no-no now. Use an SSD.

You are referring to the danger in losing the pool during resilvering after losing a drive? Yeah, I know. It is the trade off between the pain of having to restore everything from backups vs. (a lot) less available space for usage. In my case, since this is not critical data and if it takes a week (or month) to restore, sucks but I will accept the risk to get the extra space.

If you are referring to something else instead... do share.

 

[Luddite Technologist]

What case are you using (how many bays, ports)? There isn't a reason to remove the drives, just unplug if you are concerned.

I second use RaidZ2 and SDD boot.

Case is Fractal Design Node 804. A pretty little thing that makes very little noise and the aesthetic does not bother my wife (a big plus). The case can handle more than the eight drives I have, especially if we are talking about an extra SSD (or two), but the motherboard only has eight SATA ports AFAIK so unless I wanted to do something with an expansion card... mirrored USB boot drives it is. I always buy them four at a time and have the extra two ready to go in a pinch.

Don't forget to do all the testing on the new drives before putting them into service, bad blocks etc.
https://www.ixsystems.com/community/threads/uncle-festers-freenas-beginners-guide.76703/

Good looking out. I forgot to add that step to the list!

 

[Jasse Jansson]

What case are you using (how many bays, ports)? There isn't a reason to remove the drives, just unplug if you are concerned.

Likely you can just use new boot media, do the install, import your old pool, create a new pool on new drives, and then coffee.

The server I'm messing with now (U7) is a HP Microserver gen8 maxed out with 5 SATA HHD's.
The 5:th one is in an external case on top of the server with the SATA cable going out the back ut the HP.
I'm boot from 2 USB sticks so I can have those 5 disks for my data.
Boot SSD is not an option here and I have had 2 SSD failures in the last year in other computers at home.
RaidZ2 with 4 disks ??? Won't happen here.

 

[Jasse Jansson]

My HBA supports 256 drives, SATA ports are a non issue

Some people have other things in their house then just servers.

 

[garm]

I’m using a Dell T20 tucked away under my desk.. it has 4 SATA ports and a LSI 9211, I have 7 disks..

 

[Chris Moore]

Any suggestions on a good celebratory cake

Did you get to the cake part yet?

 

[Alecmascot]

The server I'm messing with now (U7) is a HP Microserver gen8 maxed out with 5 SATA HHD's.
The 5:th one is in an external case on top of the server with the SATA cable going out the back ut the HP.
I'm boot from 2 USB sticks so I can have those 5 disks for my data.
Boot SSD is not an option here and I have had 2 SSD failures in the last year in other computers at home.
RaidZ2 with 4 disks ??? Won't happen here.

I used one of these to fit an ssd into a maxed out pair of micro servers and now they are running in two Dell T110 II.
Been running for 4 years now.

mSATA 3 III SSD & SATA 3 III to PCI-E PC Expansion Card Converter Adapter Board | eBay

Extend SATA 3.0+mSATA data interfaces, and support more storage devices. Port :SATA3.0+Msata. Support IDC/AHCI Model. Support for Win7,Win8,win 10 Win XP, Vista. We are very glad to help you resolve the problem.

www.ebay.co.uk

 

[blueether]

1. Put the four old HDD's back in the system
2. Import (?) the old pool as a temporary pool rather than directly adding it into the primary pool (this is the only way to preserve the data on the old pool, correct?)
3. Move the data that I want to keep from the temporary pool into the primary pool on the new HDD's
4. Kill the temporary pool and wipe the four old HDD's
5. Add the four old HDD's as a new vdev to extend the primary pool

I don't understand why you have to do it that way?

I would just import the old pool and then add the 4 new drives as a new vdev to the old pool as you are keeping to raidz1 on both vdevs or am I missing something?

 

[Chris Moore]

I don't understand why you have to do it that way?

I would just import the old pool and then add the 4 new drives as a new vdev to the old pool as you are keeping to raidz1 on both vdevs or am I missing something?

The advantage of moving the data to the new drives is that, theoretically, the new drives will be faster. If you just add a vdev to an existing pool, the data remains where it is. The method illustrated is all about moving the data to the newer, larger drives and then putting the old drives back in the pool as additional storage. After the second vdev is added to the pool, subsequent writes will be spread across both vdevs, but the old data will remain on the new drives unless it is modified.

 

[Chris Moore]

Any suggestions on a good celebratory cake?

If you want to spread the "old" data between the two vdevs, you could just add the new vdev to the pool and then copy all the data to a new directory. Because of the way ZFS works, when you copy the data, those new writes will be spread across both vdevs utilizing all eight drives instead of just four. After that copy is completed, which can be done internal to the NAS, not over the network, just delete the old location of the data to recover the disk space. You can use Midnight Commander to make this copy happen if you are not familiar with using the command line.

 

[Chris Moore]

Here is a quick demo of Midnight Commander if you want to take a look at it.

Demo of Midnight Commander

Midnight Commander is a text utility that is integrated with many Linux and Unix operating systems. This demo is being done with FreeNAS, which runs on BSD ...

m.youtube.com

 

[garm]

If you want to spread the "old" data between the two vdevs, you could just add the new vdev to the pool and then copy all the data to a new directory. Because of the way ZFS works, when you copy the data, those new writes will be spread across both vdevs utilizing all eight drives instead of just four. After that copy is completed, which can be done internal to the NAS, not over the network, just delete the old location of the data to recover the disk space. You can use Midnight Commander to make this copy happen if you are not familiar with using the command line.

Another way is to replace the old drives with the new ones and then add the old drives back as a new vdev ;)

 

[Chris Moore]

Another way is to replace the old drives with the new ones and then add the old drives back as a new vdev ;)

True. Many ways to approach it, but that might be slower because of doing four separate resilver operations.

 

[garm]

True. Many ways to approach it, but that might be slower because of doing four separate resilver operations.

True, but might be safer if you are unfamiliar with FreeBSD file operations

 

[Luddite Technologist]

If you want to spread the "old" data between the two vdevs, you could just add the new vdev to the pool and then copy all the data to a new directory. Because of the way ZFS works, when you copy the data, those new writes will be spread across both vdevs utilizing all eight drives instead of just four. After that copy is completed, which can be done internal to the NAS, not over the network, just delete the old location of the data to recover the disk space. You can use Midnight Commander to make this copy happen if you are not familiar with using the command line.

I have made some progress and have the shiny-new 11.2 up and running with both the new pool and the old pool side-by-side. I think I have most of the config done so I am now playing with the zfs send/receive (just learned about that one, pretty cool) to move datasets from one pool to the other, which allows we to reorganize the data directly into the 'new' structures.

So far so good, but if I get stymied, I will take a look at Midnight Commander and see if that doesn't make things easier. My biggest challenge right now is patience! I have a couple of datasets that I need to copy from one pool to the other that are 1-4Tib and that just takes time! I copied a few smaller ones already, just to make sure I understood the process, so I think I got it right... but only time will tell!

Big thanks for all the help and suggestions to everyone here. I have been lurking for a number of years in the forums, but until now I didn't have the opportunity to see first-hand the great community here!