Impressed with FreeNAS... But...

[SubnetMask]

So I played with FreeNAS a long time ago as a temp solution to help me figure a way to migrate from one standalone VMWare host to another, and it was ok - it did its job (this was several years ago - don't recall what version - And it was virtualized). Fast forward to this year, I got FreeNAS running on a re-tasked Barracuda 490 backup device for essentially the same purpose, with the long term goal of using it as a B2D backup location - the problem with that device being it didn't have enough RAM (or the correct type) to support the most recent version (8GB ECC minimum), and whether or not the Opteron based barracuda device would even support ECC (it WOULD support up to 8GB) was an open question (The ECC memory I tried didn't work, but it was all registered ECC memory - unregistered ECC might have worked). So I loaded an older version up an created a UFS volume to handle the task. It handled it admirably (the only exception being when it lost power unexpectedly and it spent the better part of a day re-syncing - everything DRAAAAAGED during this time).

So once that was done, I ditched the Barracuda box and got another one that has 24GB ECC RAM (The Barracuda box was free), dual quad core procs, the works. I set it up with the same drives running ZFS and it really runs really well. I was really impressed with how well it ran the VM load I put on my 4Gb FC Promise SAN while I was migrating data off it to remove my VMS5 datastores and re-create them as VMFS6 datastores. It really gave me thoughts on the idea of replacing my Promise NAS with FreeNAS. But there's one thing that squashes that thought before it gets to take off: Redundancy.

FreeNAS has redundancy with disks, just like pretty much any other SAN or disk array, and depending on the hardware, power supplies, but that's really it. A CPU, MB, or maybe even RAM failure will bring it down hard - not keep going and just alert you that 'I HAVE A SERIOUS PROBLEM', while still serving up data. With my setup as it is, a LOT would have to go wrong at the same time to bring my environment down: multiple simultaneous disk failures (which really can happen with anything, if generally unlikely), multiple controller failures (HIGHLY unlikely), multiple power supply failures (again, highly unlikely), and multiple total host failures, either due to multiple simultaneous power supply failures (HIGHLY unlikely) or multiple system board failures (highly unlikely). Again, a LOT would have to go wrong at the same time to take it down.

But if I switched to FreeNAS, all it would take to bring everything down is a single motherboard failure (or CPU and maybe even RAM - if a CPU dies hard, generally, the entire system goes with it). Granted, out of all the possible failures, the disk failures are, by far, the most likely, but I still prefer to keep as much redundancy as I can to prevent almost anything other than a power failure from bringing my environment down. As it stands now, I could have a host totally fail, AND a controller on my SAN fail, AND a power supply on my SAN fail AND a power supply on my remaining host fail AND a disk (or multiple, depending on if they were part of the same disk array or not - as it stands, I could tolerate two failures in my RAID50 array if they were not on the same 'spindle' AND one on my RAID5 data array at the same time) fail all at the same time (Odds of all that happening? Yeah, I'll likely win the $4B powerball jackpot before that all happens) and I would still be able to continue working.

That being said, is there any thought to being able to set up FreeNAS with multiple controllers connecting to a common disk pool (think Compellent SC030, SC040, SC8000, SC9000, or even better, IMO, re-task Compellent SC4020 or SCv2000/SCv2020 with FreeNAS)? The concept is obviously possible - Compellent's been doing it for quite some time, originally using 'commodity' SuperMicro hardware, and now with the SC8000's and SC9000's, Dell PER720 and R730 hardware, respectively. Based on my current experience with FreeNAS, I would rather love to be able to set up a new SAN running FreeNAS and dual controllers, such as the cheapest SC4020, SCv2000 or SCv2020 box I could find (obviously I wouldn't care about licenses as long as the hardware was ok). The SC4020 and (as far as I know) SCv2x00 units are all Intel based, so *technically* they should be able to boot FreeNAS - the biggest problem most likely being the two controllers talking to and working with each other.

Personally, if I could source the cheapest Compellent I could find (Probably ideally a SCv2000, due to the 3.5" disk bays) and load it's controllers up with a HA setup of FreeNAS, with the ability to add more SC200 enclosures (adding another 12 3.5" bays) and/or SC220 (adding 24 2.5" bays), or, for that matter, if because it was no longer running Dell/Compellent software, whatever SAS JBOD enclosure I wanted, to further expand storage, I would really be hard pressed not to move over to FreeNAS. But the current single point of failure is something I really don't want to put into play when I have a highly redundant solution in play now.

Thoughts from the crowd?

 

[Stux]

High Availability is a key differentiator between the commercial TrueNAS product and the free FreeNAS version from ixSystems.

FreeNAS supports multi pathing to sas drives.

 

[SubnetMask]

High Availability is a key differentiator between the commercial TrueNAS product and the free FreeNAS version from ixSystems.

FreeNAS supports multi pathing to sas drives.

Yes, I've read that - but at that (price) point, why would I go with TrueNAS over Compellent? The starting price point for the TrueNAS (I believe I read $24k {ish}?) vs the Compellent SC4020, or possibly an even bigger hit on the TrueNAS, the SCv2x00, why? Yes, FreeNAS, and by extension based on its origins, TrueNAS, based on my recent experience, is a really good product, no doubt, but no offense, if I had the money to spend on TrueNAS, I'd buy a Compellent... Heck - I could buy an unlicensed Compellent and re-license it through Dell with full CoPilot support for WAAAAAAAAAAAAAAYYYY less than TruNAS and still get an enterprise grade highly redundant and highly expandable SAN (Yes, I recently re-licensed an unlicensed Compellent device, so yes, I have an idea what it costs).

I'm not an Enterprise customer. I'm an 'IT Nut' that for whatever reason, likes playing with IT (And I work in IT so I use my home environment to expand my knowledge relating to my job). And there is no middle ground for people like me. I either have to get lucky in acquiring the hardware such as I have for good prices, or settle for cheap junk that could fail me with a single failure - maybe I'm a purist, maybe I'm a nut, whatever the case may be, I don't want a single PCB to take my entire environment down. The 'joke' at my last job was that my personal environment was far better than that of any of our customers. At my current job, that's technically still true (or I believe true for the vast majority of our customers from what I gather - I don't deal with our customers in the same capacity as I did at my previous job, so I don't have a good 'bead' on all of our customers), but the environments of ours that I manage far exceed my own.

I've been working with Compellents for a few years and functionally, FreeNAS is really good, based on my recent experience, but frankly, from a management perspective, Compellent blows it away seven ways to Sunday. There really is no comparison. So again, $ for $, why should I (or anyone) choose TrueNAS over Compellent? I really have no idea what IX's support is like, but in my experience, Compellent's CoPilot support has been ridiculously, freaking, amazingly awesome (My account rep was giving me prophecies of fire, brimstone and monkeys with swords if I tried to set the array up myself - the problems I encountered were related to the fact that I wasn't setting it up the way you're SUPPOSED to because at the time I didn't have any other option (long story, not related in any way to Dell, CoPilot or their support). They have really gone way beyond what they technically had to do for me and been incredibly happy and helpful while doing so (Not related to my production Compellent - Literally had NO problems there whatsoever, but related to setting up our re-licensed Compellent).

If FreeNAS could give a similar level of redundancy, again, I'd most likely go to it. But if I'm all go, no quit, have to re-mortgage my house to accomplish the same thing? I'm going Compellent.... Sorry, but sometimes the truth hurts.

I'm not saying TrueNAS is junk - but four weeks ago (heck - even a week ago), I had never heard of it (yes, I'd heard of FreeNAS, but not TrueNAS). I've known of EMC, EqualLogic, Compellent, Promise, NetAPP, StorageWorks, QNap and more than likely a few I'm forgetting at this point in time, for a very long time, And I'm sure there are MANY others like me. If we weren't considering my personal environment, why would I, as an enterprise or SMB customer, even consider TrueNAS against much larger, much more well known names at similar price points? You want to get your name out there and get your foot in the door, starting with the smallest businesses with the intention of growing to be much bigger? Drop your starting price point for TrueNAS. Big time. And should support be needed? Own it. Blow away any expectations the customer may have had. *10 at least.

 

[gpsguy]

I used to manage a dual controller Compellent SAN. It was rock solid and tech support was top notch. But the hardware and ongoing maintenance was costly.

One can't expect iXsystems to provide this functionality in FreeNAS for FREE.

While many FreeNAS users are enthusiasts who use it at home, some accept the risks and use it in a business environment. As @Stux said one could look at TrueNAS.

 

[Stux]

Suffice to say, if you want true HA then FreeNAS explicitly does not support that, and I've provided the business case why.

Personally I like to understand the business model behind a free product, and understanding that basically means putting up with the risk of downtime that a motherboard failure can potentially offer.

Oh well.

Your choice.

 

[Chris Moore]

But if I switched to FreeNAS, all it would take to bring everything down is a single motherboard failure (or CPU and maybe even RAM - if a CPU dies hard, generally, the entire system goes with it). Granted, out of all the possible failures, the disk failures are, by far, the most likely, but I still prefer to keep as much redundancy as I can to prevent almost anything other than a power failure from bringing my environment down. As it stands now, I could have a host totally fail, AND a controller on my SAN fail, AND a power supply on my SAN fail AND a power supply on my remaining host fail AND a disk (or multiple, depending on if they were part of the same disk array or not - as it stands, I could tolerate two failures in my RAID50 array if they were not on the same 'spindle' AND one on my RAID5 data array at the same time) fail all at the same time (Odds of all that happening? Yeah, I'll likely win the $4B powerball jackpot before that all happens) and I would still be able to continue working.

You are being really silly. That level of redundancy in a home environment? Really? The thing about FreeNAS is ZFS. ZFS is the bomb. That is why you want FreeNAS. If you don't know that, why are you even here? I had a SAS HBA in one of my FreeNAS systems die and it took the array offline. I replaced the controller and remounted the zpool and all my data was there where I left it. No data loss, no recovery. It was just there without fault. There, you wanted a reason. If you want redundant controllers, you will have to go to TrueNAS and it costs because the hardware costs and they charge for the software and support just like any other company would that is selling a NEW hardware solution.

 

[Chris Moore]

PS. If you want high availability and ZFS, you can use GlusterFS as a way of pooling the storage of multiple systems to get the level of redundancy you are talking about.

 

[garym]

You are being really silly. That level of redundancy in a home environment? Really? The thing about FreeNAS is ZFS. ZFS is the bomb. That is why you want FreeNAS. If you don't know that, why are you even here? I had a SAS HBA in one of my FreeNAS systems die and it took the array offline. I replaced the controller and remounted the zpool and all my data was there where I left it. No data loss, no recovery. It was just there without fault. There, you wanted a reason. If you want redundant controllers, you will have to go to TrueNAS and it costs because the hardware costs and they charge for the software and support just like any other company would that is selling a NEW hardware solution.

I have to agree, ZFS is the important factor for data protection. The next would be a good backup device running on a ZFS file system.
I would replace the word "silly" with "addicted". I know that SubnetMask would call it passion for what he wants and does and tries to justify his desires, I see the signs in what he wrote. No different than being in love or addicted to drugs, results are the same.
SubnetMask, reread your post, and see if you see the signs. Learn to control your desires and don't let technology control you before things get out of control. Sorry, I had to say it.

 

[SubnetMask]

Really, lol? If you guys seem to think that wanting controller redundancy, something my now EOL Promise array can do just fine, is 'silly' or 'addicted'... yeah... I won't be taking any advice from either of you. Addicted might be mortgaging my house to get the latest and greatest storage arrays and servers, not getting used equipment as cheap as I can that does what I need (and want) it to. The drives in my vTrak cost more than the array did - So hypothetically, had I gone with FreeNAS way back instead of the Promise, I would have either not saved much going FreeNAS, or I would have spent MORE to go FreeNAS to get the enclosure and supporting hardware, with no controller redundancy (and what would have been plenty adequate for FreeNAS back then doesn't meet the minimum requirements for ZFS, so I would most likely have had to buy hardware all over again just to upgrade FreeNAS).

So, well, whatever...

 

[Chris Moore]

It handled it admirably (the only exception being when it lost power unexpectedly and it spent the better part of a day re-syncing - everything DRAAAAAGED during this time).

You would never have had a problem like that with FreeNAS. If you are a fan of the hardware you have. Why are you looking around for another solution? You are extolling the virtues of hardware that costs $50k or more and you have to buy a license to use it. My two FreeNAS servers cost me about $2000 to setup, not counting drives.

and what would have been plenty adequate for FreeNAS back then doesn't meet the minimum requirements for ZFS, so I would most likely have had to buy hardware all over again just to upgrade FreeNAS

It doesn't matter what hardware you buy, it must be replaced eventually.

 

[SubnetMask]

You would never have had a problem like that with FreeNAS. If you are a fan of the hardware you have. Why are you looking around for another solution? You are extolling the virtues of hardware that costs $50k or more and you have to buy a license to use it. My two FreeNAS servers cost me about $2000 to setup, not counting drives.

It doesn't matter what hardware you buy, it must be replaced eventually.

That WAS FreeNAS.

I'm not really looking for another solution. It's more along the lines of I was going to use my old Drobo to migrate the VMs from an old VM host to a new one at my mothers house, but it just would NOT cooperate with VMFS - just went all kinds of sideways every time I formatted it, so I ended up using the old Barracuda 490 appliance I was scrapping from work with an older version of FreeNAS (had to run UFS because it only had 4GB of non-ECC RAM), and aside from that resync after the power loss, I was quite surprised at how well it ran. The last time I had monkeyed with FreeNAS was a few years ago and I set it up in a VM for essentially the same purpose - it worked, but it wasn't stellar - not terribly surprising considering it was running in a VM.

FreeNAS does offer a feature or two that the older Promise doesn't, such as VMWare Hardware acceleration, but there's really nothing wrong with my promise and I'm not burning to replace it (if the right thing comes along for the right price, I might think about it). The only 'issue' with the Promise is drive selection - I can put pretty much any drive 2TB or less in it no problem, but over 2TB (4TB max) there is a very small list of drives that'll work (any drive will 'work', but only the first 2TB will be recognized). I had to get specific 4TB SAS HGST drives.

 

[Chris Moore]

older version of FreeNAS (had to run UFS because it only had 4GB of non-ECC RAM), and aside from that resync after the power loss, I was quite surprised at how well it ran.

You should not base any opinion of FreeNAS on any version old enough that it allowed you to use UFS. It is actually a totally different operating system now.

The last time I had monkeyed with FreeNAS was a few years ago and I set it up in a VM

You will also not get a true representation of FreeNAS from an instance running in virtualization.

FreeNAS is very good and only getting better and if you use the recommended hardware instead of some ancient scrap, you will be pleased with how well and reliably it performs.