Deemphasising FreeBSD?

[Basil Hendroff]

Throwing this in the mix https://www.servethehome.com/ixsystems-starts-de-emphasizing-freebsd-for-truenas-scale-out-project/.

How do you feel about this?

 

[proto]

we will die with systemd, finally.

 

[morganL]

It was an article written without talking to iXsystems.... we don't agree with it.

Our intention is to become multi-OS with TrueNAS software residing on both FreeBSD and Linux. Improvements to TrueNAS will generally apply to both OSes. However, the scale-out feature set uses Linux tools and hence the decision.

The work to make OpenZFS 2.0 consistent between FreeBSD and Linux was critical to enabling this multi-OS capability.

 

[Basil Hendroff]

Time will tell. I think it's a move that's welcome and definitely needed for TrueNAS to grow its user base. The strength of TrueNAS is in storage. Personally, however, I feel the dependency on volunteers to maintain packages and ports is FreeBSD's Achilles heel. Application builds that are readily available on Linux are awaiting porting or packaging on FreeBSD, or, will never be ported. It may be limiting the adoption of TrueNAS. This alone, I suspect, will, in time, steer some members away from running TrueNAS on FreeBSD to running it on Linux, but at the same time, targets a previously untapped audience within the Linux community.

 

[proto]

Extending the user base is right after all and in fact that article on STM is a little too pessimistic.

 

[Ericloewe]

To me, it feels like Linux "won" a few years ago, for some definition of that and to some extent.

That's not to say we don't need FreeBSD or that people should jump ship. It means that being different for the sake of being different will not help anyone, and keeping things compatible across operating systems makes sense.

 

[ornias]

I feel this is a shamefull quality article and you guys should give Patrick a call.
This is not the quality we should expect from STH and contains a lot of "If we read between the lines here " crap.

I also think this author missed the memo:
Most scale- solutions are either not available on FreeBSD or in significant different state qualitywise.

The comparison with Proxmox also shows bad research on the part of the author IMHO:
Proxmox is very vocal in not supporting (and not going to support) Gluster and Application containers (such as docker), A solution using Gluster on top of ZFS natively supporting Docker containers, is a totally different beasty.

 

[JoshDW19]

I feel this is a shamefull quality article and you guys should give Patrick a call.
This is not the quality we should expect from STH and contains a lot of "If we read between the lines here " crap.

I also think this author missed the memo:
Most scale- solutions are either not available on FreeBSD or in significant different state qualitywise.

The comparison with Proxmox also shows bad research on the part of the author IMHO:
Proxmox is very vocal in not supporting (and not going to support) Gluster and Application containers (such as docker), A solution using Gluster on top of ZFS natively supporting Docker containers, is a totally different beasty.

I think you hit the nail on the head ornias. It seems like the article was written on a bit of a pessimistic slant and with a lot of assumptions. Who knows if that was intentional or just a lack of understanding. We did reach out to try and help them understand why Linux is necessary for this type of solution. Unfortunately, it doesn't seem like there was much interest in our actual plans for the future. Hopefully, given time, they'll be able to better understand our trajectory.

We still love FreeBSD and we're excited to expand our product lineup with TrueNAS SCALE. The point isn't to make anyone feel alienated, just to create another Open Source solution as an option for our community and business to build on.

 

[HoneyBadger]

Unfortunately, it doesn't seem like there was much interest in our actual plans for the future.

"Top 10 reasons why sites write clickbait - Number 4 will SHOCK YOU!" seems like an accurate representation; controversy and arguments drive clicks.

One could make the same (poorly backed) argument that OpenZFS is "de-emphasizing Linux" by integrating with the FreeBSD stream. It's not a preference of one over another, it's becoming platform-agnostic, which is rarely if ever a bad thing.

And TrueNAS SCALE isn't going to target the same audience as the "storage-focused" TrueNAS; I don't see the former replacing the latter.

 

[ornias]

Unfortunately, it doesn't seem like there was much interest in our actual plans for the future. Hopefully, given time, they'll be able to better understand our trajectory.

Lets be pessimistic for a second myself:
I do see what IX tries to be doing here, its call: "Hedging their bets". If FreeBSD declines more, they can easily move to Linux this way and continue without much commerical interuption.

But:
Even being that pessimistic: You can't blindly say that hedging one's bets means someone is planning to move away.
- Do I personally(!) see IX moving away from FreeBSD in 10 years? With the current road FreeBSD is going, Yes. But that does not mean they would do so without regret or need.


Back to normal me:
It's actually bloody good bussiness practice to BOTH open a new product (which increases sales) AND hedge your bets in case your other product starts to loose potential. Heding requires Linux and the New (scale-out) solution requires Linux. So win-win bussiness wise.

When SCALE launches, I don't expect enterprises to jump on it right away, it would still take a year or 2 of traction with the tech community to get enterprises interested. For that very reason I don't expect many SCALE appliances at launch right away.

After it is gaining traction I expect more of the non-HA enterprise solutions being converted to SCALE offerings. Because on those platforms SCALE would just simply offer more bang-for-bug and IX would be able to also sell those same hardware systems as scale-out solutions.

After about 5 years after SCALE launch I would expect 3 catagories of hardware products from IX:
1. Enthousiast NAS devices without HA running TrueNAS Core
2. Medium bussiness HA storage using TrueNAS Enterprise
3. High density storage without HA, but with TrueNAS SCALE.

Why wouldn't they move the other offerings?
TrueNAS Core:
*BSD is known to be very solid without intervention. It's ideal for 24-7 consumer devices like NAS's in my opinion. But also: It's a giant userbase to get feedback on the enterprise solution. There is no storage solution that has so many users on different kinds of hardware, it generate A LOT of usefull feedback and thus stability which would also be usefull for the Enterprise offering.

TrueNAS Enterprise:
Trust, Enterprises trust the FreeBSD based offering and would expect both support and development of the platform. Breaching said trust without good reasoning, would costs IX a SIGNIFICANT amount of money.

IF FreeBSD starts to decline (and thats the IF):
If after 10 years we see FreeNAS failing, IX can firstoff easily scrap TrueNAS Core and replace it with a SCALE-based offering. Enterprise would take some time, (due to service life) but about 5 years later we would see Enterprise being mostly replace by SCALE-based solutions too.


Conclusion:
With above analysis I don't see any reason to expect IX Systems is currently already planning to move away from FreeBSD. It would objectively not make any sense.

 

[potatohead]

Would it be to late to consider RancherOS, instead of Debian, for TrueNAS Scale? Lots of advantages and less polarizing to many.

 

[ornias]

Would it be to late to consider RancherOS, instead of Debian, for TrueNAS Scale? Lots of advantages and less polarizing to many.

Yes, thats too late by now.
They are already developing it for months. They are certainly not going to throw away those amounts of money due to a single firstposter on the forums. (Nothing personal, sometimes the facts are just harsh;) )

But thats not the only thing:
The philosophies aren't compatible. RancherOS runs everything in Dockers. Such a solution isn't compatible with the design IX has made: A cross-platform middleware, with a cross platform GUI. They spend about 2 years now on slowly rewriting their middleware to support this, they are not likely to do this again because someone want's it to be based on RancherOS and in that case they would need a completely seperate version for FreeBSD-based operatingsystems.

Ergo.
Aint gonna happen.

 

[morganL]

Would it be to late to consider RancherOS, instead of Debian, for TrueNAS Scale? Lots of advantages and less polarizing to many.

It was necessary to choose one platform and focus tools and testing on that. Debian is decided. We do want containers and VMs supported (KVM). The plan is to enable Kubernetes, without being too prescriptive about which version. Rancher should be supportable. ...especially if you'd like to contribute.

 

[ornias]

The plan is to enable Kubernetes

Thanks for finally giving some insight in the containerisation platform of choice :)
Are you guys actually going to try and simplify k8s deployments? that would be awesome indeed :)

 

[electromagnetic.jolt]

I'm happy about Linux support but aren't FreeBSD jails more secure than Docker containers? A few months ago I attempted using Docker for the first time, and I got all my containers working, but Kubernetes was super confusing to me. And it appeared Docker wasn't the best container solution, I saw podman being preferred, but there were less tutorials setting it up with Kubernetes. I just remember how frustrating it was recreating containers constantly getting testing the docker compose scripts I had. I could take a look at it again but I like the setup I have now with FreeNAS. I like how easy it is to manage the jails, and the security is a plus.

 

[morganL]

I'm happy about Linux support but aren't FreeBSD jails more secure than Docker containers? A few months ago I attempted using Docker for the first time, and I got all my containers working, but Kubernetes was super confusing to me. And it appeared Docker wasn't the best container solution, I saw podman being preferred, but there were less tutorials setting it up with Kubernetes. I just remember how frustrating it was recreating containers constantly getting testing the docker compose scripts I had. I could take a look at it again but I like the setup I have now with FreeNAS. I like how easy it is to manage the jails, and the security is a plus.

Yes, Jails and Plugins have advantages. TrueNAS CORE will continue to support them on a FreeBSD base.

 

[Basil Hendroff]

I'm happy about Linux support but aren't FreeBSD jails more secure than Docker containers?

Yes, Jails and Plugins have advantages. TrueNAS CORE will continue to support them on a FreeBSD base.

I'm at a crossroad myself. Both jails and containers provide OS-level virtualisation. The table in the link provides a useful comparison. The FreeNAS thread Jail vs Docker? also provides a useful (but slightly out-of-date) perspective. For me, there are two key docker traits that make them attractive:

1. Containers are readily available for applications I'm still waiting to be ported to FreeBSD.
2. Keeping independent containers up-to-date is straightforward and easy to automate.

For example, while I still use Resilio Sync in an iocage jail, I now wonder whether I wait for the latest package to be available (it's only just made it to Fresh Ports) or move to use a current release (available four weeks ago) in a container? I'm still able to use FreeNAS for Sync storage. It's tempting.

My current thinking is, 'Where I can afford some downtime, containerise it; where I feel I need to be closer to the hardware and not as dependent on virtualisation layers (I run Docker in an Ubuntu VM), stick with jails'.

 

[jgreco]

I'm happy about Linux support but aren't FreeBSD jails more secure than Docker containers? A few months ago I attempted using Docker for the first time, and I got all my containers working, but Kubernetes was super confusing to me. And it appeared Docker wasn't the best container solution, I saw podman being preferred, but there were less tutorials setting it up with Kubernetes. I just remember how frustrating it was recreating containers constantly getting testing the docker compose scripts I had. I could take a look at it again but I like the setup I have now with FreeNAS. I like how easy it is to manage the jails, and the security is a plus.

The main problem with both Linux containers and FreeBSD jails is that the default configuration for these is to place an entire OS filesystem tree within the ${thing}. This is the only way that the average user can easily make use of ports or packaging systems that make these things "easy" to deploy. Unfortunately, doing this introduces a copy of /bin/sh into the environment, which is a typical vector that skript kiddiez use to break into and take over UNIX-type systems (look at "stack smash exploits" for example).

You get substantially better resilience against breakins when you design a jail to only have needed things in it. This starts by NOT extracting a FreeBSD tarball inside. A very basic example might be configuring PostgreSQL. A simple install of OpenSSL and PostgreSQL can result in a decent jailed pgsql that has no /bin/sh exposure. It also has a very small footprint.

The problem is that it is difficult to do, and even more difficult to do *well*. Modern software systems can be extremely complex. To do a webhosting-quality Apache server stack with PHP and enough stuff to count as "usable" requires nearly a hundred things to be built and set up inside the jail, which is a daunting task if you are just some random person who merely wants a webserver up and running.

The other problem is that a lot of containerized and Kubernetes design is driven by ideologies that simply focus on getting a job done with as little attention to detail as possible. You have some carp-arse "compose" script that just barely makes a passably functional thing that pays no attention to issues such as security, access control, or any of the finer points. SQL with default passwords and wide open access, etc. This is "crappy DevOps." This is encouraged by both cloud computing, where resources are often placed in semirandom locations on the Internet, making typical basic security practices such as ACL's and firewalls impractical.

I've spent a fair portion of my career writing a more-hardened version of FreeBSD and jails to run on it in order to provide services to Internet Service Providers and other Internet-exposed businesses. I've been doing jails since PHK announced the feature, and the vast, vast majority of them have been built from the ground up from source packages. It's not for everybody, or, even, for most people. I kinda wish it was. We'd have a lot more security if there were fewer attack vectors.

Edit: It's worth noting that the theoretical ability to write /bin/sh-less jails applies to containers as well.

 

[jgreco]

Wow. Came in to five likes on that post just overnight. Makes me pleased to know that some people agree with me on some parts of that. Now leaves me wondering which parts. ;-)

I'm not sure, but is it worth asking this... is either iXsystems or the community interested in the possibility of this kind of thing being made available for FreeNAS? I haven't really looked too closely at this, but these jails are highly compartmentalized and self-contained, though they do their own jail management. Mmm.

 

[Ericloewe]

I think that's something that would go along very well with iocage. Having a base system is obviously useful, but minimizing the attack surface is always good.

One-size-fits-all solutions are more of a -fits-one thing, but even a write-up of the procedure for one of your examples would be very interesting. A lot of it would also translate over to Linux in concept, I imagine.