Keep FreeNAS or move out?

[AdrianB1]

I am using FreeNAS for about 5 years and I am quite happy with it. But I want to consolidate a bit and I am not sure what is the best option.

I have 3 older and weaker PC's doing FreeNAS and Hyper-V; the FreeNAS box had 5 disks in raidz. All the FreeNAS data is personal, except for a backup of a couple of VM's used for work.

I want to put everything in a single computer to do both and move it (and the UPS) to the basement to free up my work area, get rid of lots of noise and heat. I need to use Windows & Hyper-V for several reasons related to work and licensing, so I am considering:

1. Use Hyper-V and some separate storage (M.2) to boot and keep the VM's and build a FreeNAS VM to manage the existing disks. I am concerned about 2 things:
a) I have no PCIe controller, in the current box the disks are connected to the integrated SATA ports from the chipset on the motherboard (old Intel platform). Will it FreeNAS properly? Current disks are physical, in the VM with no PCI pass-thru ... ?
b) What are there risks related to migrating FreeNAS to a VM? I will take a backup of the most important data I have, but I have no easy way to take a backup of everything.

2. User Hyper-V, virtualize FreeNAS, use the FreeNAS storage for the rest of the VM's. Is it realistic and wise? I expect to loose a lot of performance (versus nVME) and gain redundancy (the nVME storage has backups, today on FreeNAS, but no redundancy).

3. Use Hyper-V, wipe the FreeNAS disks, create a Windows RAID5 array, copy everything back, stay on Windows. Not a fan of this option, but I seriously consider it as a simpler option at the end of the day. I will have to find a way to back up the existing data, it is difficult but not impossible.

I am cooking this for a while, I already have the new computer for the new setup (Ryzen 2700, 32 GB of RAM, another 32 GB if needed, a 500GB nVME SSD) and I plan to do the switch next weekend. I am missing the information to decide between the 3 options, I would prefer the first one if possible but I will go with any.

 

[joeschmuck]

Some of the risks I see is your new platform from an ECC perspective but if you have been running FreeNAS this long you should already be aware of it.

I don't know how Hyper-V works for FreeNAS but I know ESXi work great, but if you search the internet you will find tutorials on putting FreeNAS on Hyper-V. I understand the reason to consolidate as well.

You didn't mention the capacity of your five hard drives but you did state you have been running it for 5 years in a RAIDZ configuration. My assumption is that these drives are either beyond warranty or about to exceed it. My advice is to purchase a new set of drives, maybe go for a RAIDZ2 configuration, and create a new FreeNAS on Hyper-V, then copy your data over.

Here at these forums we will not push you either direction unless there is some obvious reason. The main reason for me is data reliability that I get with ZFS over any other format, and RAID5 or RAID6 just doesn't have it.

Good luck on your decision.

 

[sretalla]

a) I have no PCIe controller, in the current box the disks are connected to the integrated SATA ports from the chipset on the motherboard (old Intel platform). Will it FreeNAS properly? Current disks are physical, in the VM with no PCI pass-thru ... ?

This is your biggest point of concern and controller cards are cheap (I have seen Chris Moore recommending ones on ebay for under $30). You should get one and use it as part of the plan.

 

[jgreco]

a) I have no PCIe controller, in the current box the disks are connected to the integrated SATA ports from the chipset on the motherboard (old Intel platform). Will it FreeNAS properly? Current disks are physical, in the VM with no PCI pass-thru ... ?

So you probably really do have what appears to be a PCIe controller from the system's point of view. PCH SATA controllers are typically eligible for PCIe passthru.

 

[AdrianB1]

Than you for the answers. Let me address a few:
- the disks were replaced once, they are fairly new; the capacity and model are irrelevant to my dilemma, this is why I did not mention it
- today FreeNAS is not virtualized, so it can use directly the disks via the PCH controller. In a virtualized environment I am afraid the integrated controller will not be available for pass-thru; I cannot check without actually installing Hyper-V and attaching a disk, but chances are pretty low: on the 2 Hyper-V machines it does not work. They are older models of Intel and AMD hardware, the VT-D and IOMMU in BIOS are enabled, I have no high hope it will be different now.
I will look for a PCIe controller, but I am living in Eastern Europe and for eBay delivery is usually more expensive than the merchandise.

 

[jgreco]

Install ESXi on a thumb drive and check that way. No impact on your system. Bonus: you get experience with a better quality hypervisor.

 

[AdrianB1]

I have installed a test VM with FreeNAS under Hyper-V and so far it works just fine with a pair of mirrored disks. I will continue testing the rest of the week and this weekend I will move the physical FreeNAS machine in a virtual one. The test VM I will transfer to a physical one and keep it offline (shut down) as a data backup for a while. I will update on this Hyper-V experience in a few weeks when I will have some relevant time with it.

 

[AdrianB1]

More than a month later, I can confirm everything is working perfectly. Based on the current experience I will probably leave it running for the foreseeable future, as a small home NAS it is enough for me.

 

[AdrianB1]

Several months later: it simply works.

I had that setup running since June with no problems. This week I replaced the disks by building a new instance on a temporary computer, also on Hyper-V, copying everything from one to another and then putting the new disks in the old computer. I encountered a few problems:

1. The source computer froze several times, about once for every 2 TB copied. It is the computer I used since June, it was stable all this time, until I did this massive copy. The only wild guess is that some counter in the NIC driver get to overflow and freezes the Windows Server OS. Strange thing is that the computer I copied to is using exactly the same NIC, a discrete Intel Desktop CT model, that one never crashed. In my regular use of that Windows Server computer I regularly restart it for Windows patches, maybe this is why it did not crash until now.
2. When I tried to move the new FreeNAS instance that I created on the target computer after I copied the content, it did not work: migrating a working FreeNAS from one computer to another under Hyper-V does not work in this specific case - using physical disks. I did not attempt to do an online migration, that makes no sense, just copy the boot disk from one computer to another and move the physical disks. It started, recognized the pool, then frozen in different ways. I did a reset (option 10, I think, in the post-boot console menu) that restored some functionality, but not good enough. I reinstalled FreeNAS in 3 minutes and it was all fine. Not worth investigating because it is not a realistic use case.
3. The dashboard hangs a lot. Most of the time I see the System Information box, then everything is loading. Some times I see the Network info to CPU usage info boxes. The CPU temperature never works, this is probably because the virtualization, and the CPU average, I have no clue for that.
4. SMART is not working. This probably an effect of virtualization, no idea if there is any solution for that until the disk driver from Microsoft will include SMART pass-thru capability.

Other than these problems that are specific just to my configuration, I am more than happy. The host is a 8 CPU lower power (65W) Ryzen 2700 with 64 GB of RAM, FreeNAS has 2 CPU's and 16 GB allocated and 3 data disks in raid-z. The large file copy was ~ 110 MB/sec in a physical machine, now it is just ~100 MB/sec, no point to complain because this configuration is very clean and very flexible, the 10% performance loss is worth paying.

 

[blueether]

Hi, I'm glad it all works (mostly). As @jgreco has said the bits that you seem to have had trouble with just work under esxi (and proxmox) I've moved my testing server between esxi/proxmox/bare-metal several times and have had no issues. I would try a cheep HBA and pass that through to enable SMART tests to give more peace of mind

 

[1kokies]

Several months later: it simply works.

I had that setup running since June with no problems. This week I replaced the disks by building a new instance on a temporary computer, also on Hyper-V, copying everything from one to another and then putting the new disks in the old computer. I encountered a few problems:

1. The source computer froze several times, about once for every 2 TB copied. It is the computer I used since June, it was stable all this time, until I did this massive copy. The only wild guess is that some counter in the NIC driver get to overflow and freezes the Windows Server OS. Strange thing is that the computer I copied to is using exactly the same NIC, a discrete Intel Desktop CT model, that one never crashed. In my regular use of that Windows Server computer I regularly restart it for Windows patches, maybe this is why it did not crash until now.
2. When I tried to move the new FreeNAS instance that I created on the target computer after I copied the content, it did not work: migrating a working FreeNAS from one computer to another under Hyper-V does not work in this specific case - using physical disks. I did not attempt to do an online migration, that makes no sense, just copy the boot disk from one computer to another and move the physical disks. It started, recognized the pool, then frozen in different ways. I did a reset (option 10, I think, in the post-boot console menu) that restored some functionality, but not good enough. I reinstalled FreeNAS in 3 minutes and it was all fine. Not worth investigating because it is not a realistic use case.
3. The dashboard hangs a lot. Most of the time I see the System Information box, then everything is loading. Some times I see the Network info to CPU usage info boxes. The CPU temperature never works, this is probably because the virtualization, and the CPU average, I have no clue for that.
4. SMART is not working. This probably an effect of virtualization, no idea if there is any solution for that until the disk driver from Microsoft will include SMART pass-thru capability.

Other than these problems that are specific just to my configuration, I am more than happy. The host is a 8 CPU lower power (65W) Ryzen 2700 with 64 GB of RAM, FreeNAS has 2 CPU's and 16 GB allocated and 3 data disks in raid-z. The large file copy was ~ 110 MB/sec in a physical machine, now it is just ~100 MB/sec, no point to complain because this configuration is very clean and very flexible, the 10% performance loss is worth paying.

thanks for the update and it's interesting, i am thinking to move from esxi to hyper-V, however the FreeNAS box remains physical.

 

[AdrianB1]

New update after FreenNAS updates: the dashboard seems to be fine now, I did not encounter problems with the current build (11.3-U2.1 ).

SMART is kind of working, I am running Seatools (for the Seagate drives) in the host and that is good enough for the moment.

I had a problem with the network after I added another Intel dual-NIC in the host, FreeNAS network stopped working and I had to reinstall it, but now it is working fine. No idea why the host NIC affected the FreeNAS networking, I just changed the virtual switch.

Overall, I would not hesitate to run FreeNAS under Hyper-V. I did use a second one on a different host (an i3 7100) with some older disks as a backup for the main NAS, no problems there either. I am waiting for Ryzen 4000/B550 to do some changes in the host (the current B450 motherboard has a very limited nVME + SATA and PCIe capacity), but it will take a few months till that update.

 

[AdrianB1]

New update.

1. The problem with the reboots mentioned in the post #9 is definitely linked to the Intel Desktop CT adapter; I am using a dual port Intel NIC and I had no problems since.
2. The dashboard works fine since some FreeNAS update, it works very well with TrueNAS.
3. I moved the disks to a secondary TrueNAS backup server and put some larger disks that I found too late they are SMR; the performance is bad, much worse that the old disks.
4. I am really happy with TrueNAS on HyperV, no problems in > 18 months on 2 servers (the backup one is shut down most of the time, but it is still running for data sync from time to time).