Lower RAM requirement for RAID1?

[John Nas]

Only the data is copied. There is no reason to resilver unused space of the disk.

But let's say you have been running the system for 6 months, and have accrued 3TB of data on 6TB drive(s). In any week you might only add 50GB of new data/changes. Would the resilver write 3TB while incremental/delta backups (like rsync) would only perform 50GB's of writes?

Sure, this would work, but I think @Arwen was talking about using 3-way mirrors, not 4-way, but mirrors can be n-way and the more disks in the mirror, the more read capacity.

Do you mean with only two drives in the mirror and the third at the offsite location? The three way mirror only coming together when you're updating the offsite drive?

 

[Chris Moore]

In any week you might only add 50GB of new data/changes. Would the resilver write 3TB while incremental/delta backups (like rsync) would only perform 50GB's of writes?

There is some possibility that you could add a drive back to a mirror, if it was the right drive set within the pool, and ZFS might recognize the disk and merely update it with the changes because the drive would still have the same name and data that it had when it was last in the pool. This happens sometimes, automatically when communication to a drive is lost temporarily and then restored. If the drive was not recognized as having been a member of the pool, it would be completely wiped and the data would be rewritten from scratch. In that event, an incremental backup would allow for less write to the drive.

Do you mean with only two drives in the mirror and the third at the offsite location? The three way mirror only coming together when you're updating the offsite drive?

That is what I understood, but you can do it the way you described also. It is a workable situation. The more drives you have, the larger the chassis required and the more power consumed.
Are you still questioning the memory requirements?

 

[Chris Moore]

Any advice on how a user should think about that minimum?

You should avail yourself of the documentation. ZFS on Linux may not "require" as much RAM to start, but no matter the operating system, ZFS needs as much RAM as you can give it because it uses RAM for cache. We already told you that and pointed you at the documentation. I am confused by why you are still asking. The minimum is the bare minimum, that isn't hard to understand, more than the minimum is better and the rule of thumb is 1GB of memory for 1TB of storage, but never less than the minimum.

Hardware Requirements
http://www.freenas.org/hardware-requirements/

Did you read the manual?
http://doc.freenas.org/11/freenas.html

 

[Stux]

There is some possibility that you could add a drive back to a mirror, if it was the right drive set within the pool, and ZFS might recognize the disk and merely update it with the changes because the drive would still have the same name and data that it had when it was last in the pool. This happens sometimes, automatically when communication to a drive is lost temporarily and then restored. If the drive was not recognized as having been a member of the pool, it would be completely wiped and the data would be rewritten from scratch. In that event, an incremental backup would allow for less write to the drive.

That is what I understood, but you can do it the way you described also. It is a workable situation. The more drives you have, the larger the chassis required and the more power consumed.
Are you still questioning the memory requirements?

I do actually do this all the time. One of my systems boots baremetal sometimes, and ESXi sometimes... when its bare metal its missing the ESXI boot mirror drive.

it constantly moans about being degraded... when I reboot to ESXI it quickly re-silvers just the changes...

Not sure if its a good mode to keep a production pool in though.

 

[Arwen]

Any advice on how a user should think about that minimum? I understand you should use no less, but is it now regarded as perfectly sufficient for a NAS up to a certain size or merely as the absolutely most bare-bones acceptable? Also any ideas, as has already been discussed here a little bit, if it's applicable to ZFS in general (ZoL etc) or more specific to the FreeNas software?
...

If it's just basic storage, then 8GB is fine, even with lots of large disks.

However, if you have jail(s), VM(s), require repetitive reads, use deduplication or use zVols for network attached VMs, then more memory is not just desirable, it can be necessary.

The reason we can't answer your question perfectly, is that we are outsiders to your needs, budget and skill level. Even if you told us all those things, it still would not matter because of the fact that you are not buying a pre-assembled computer or NAS for a reason.

...
What is the cross-pollination like between ZoL and ZFS on FreeBSD/FreeNas? I know there's a license issue but does a feature like this, when added to one, tend to make it's way over to the other codebase?
...

It takes a bit of time, (months, sometimes 1 or 2 years), for features to move between the OpenZFS sub-units, (Solaris work-alikes, FreeBSD, Linux & MacOS). But it does happen.

There is ZERO license issues inside of OpenZFS. Every developer agrees in writing to allow cross-pollination. Generally just using the CDDL license like the original source code.

Note that in some regards, ZFS on Linux project has more activity. Used to be Solaris work-alikes and FreeBSD were the mainstays.

 

[HoneyBadger]

However, if you have jail(s), VM(s), require repetitive reads, use deduplication or use zVols for network attached VMs, then more memory is not just desirable, it can be necessary.

If anyone posts about using deduplication on a system with 8GB of RAM, my only response will be as follows

Show : Contains naughty language

 

[John Nas]

I am confused by why you are still asking. The minimum is the bare minimum, that isn't hard to understand, more than the minimum is better and the rule of thumb is 1GB of memory for 1TB of storage, but never less than the minimum.

Apologies, I'm a noob and you are all very helpful and experienced, so I was taking the opportunity to try and grok the why rather than just get the what. I didn't mean in any way to imply the opinions behind ECC or limit of 8GB was negotiable. As stated previously in this thread I was a previous nas4free experimenter using their recommendation of 2GB, which they upped to 8GB when they changed versions. So I find understanding what lies behind a recommendation, what factors affect it, how changes in those factors might change that recommendation in the future (e.g. the aforementioned step from 6GB to 8GB with FreeNas) is as important as just knowing the number.

I only ask about minimum, since I find the term can be subjective in terms of standards of performance. I understand it means the lowest possible, but when it comes to games and software minimum specs, for some cases that means the application will run perfectly well but could run much better, for others it means it will technically load but be so slow to be unusable. For example: Photoshop min is Core 2, 2Ghz, 2GB ram, win 7, but I would expect such a machine to be a *severe* challenge to work with. To call that a minimum is technically accurate but functionally disingenuous.

Intuitively, for me, the RAM dependency of ZFS doesn't make sense with regards to the core functionality that I imagine attracts many first time users. So while I come up to speed, understanding why it exists feels important with regards to what ZFS is, how it works, and how it differentiates itself from other filesystems and earlier versions of itself. More reading is indeed required :)

 

[John Nas]

Not sure if its a good mode to keep a production pool in though.

Is there a reason why you do it? Do you find it more convenient or performant compared to the alternatives?

 

[HoneyBadger]

Apologies, I'm a noob and you are all very helpful and experienced, so I was taking the opportunity to try and grok the why rather than just get the what.

The core part to really internalize is this:

The 8 GB minimum is a FreeNAS thing, not a ZFS thing.

You can absolutely run ZFS from a hand-rolled FreeBSD, Solaris-like, or Linux installation, on (significantly) less memory. Performance will be subpar, but it will work, run, and most importantly be stable and not cause your pool to eat itself.

FreeNAS, on the other hand, is an appliance that comes with its own daemons, services, processes, etc, that demand more memory. As such, it has its own recommended minimums, and based on previous history they should be strongly adhered to, lest you put your data at risk.

 

[gpsguy]

So I find understanding what lies behind a recommendation, what factors affect it, how changes in those factors might change that recommendation in the future (e.g. the aforementioned step from 6GB to 8GB with FreeNas) is as important as just knowing the number.

IIRC, the recommendation to increase the minimum RAM requirement for ZFS on FreeNAS was made 6+ years ago. Maybe with one of the later 8.x versions.

The active forum members would see a lot of users show up with all sorts of random problems when running with less than 8GB RAM. Assuming the pool could be mounted, upgrading to 8GB resolved the issues.

Will FreeNAS run with less than 8GB? Maybe. But, don't come back crying for help if you lost your pool after running it on 2GB RAM.

 

[danb35]

Will FreeNAS run with less than 8GB? Maybe. But, don't come back crying for help if you lost your pool after running it on 2GB RAM.

The thing that can be frustrating (and somewhat deceptive) is that FreeNAS will, in fact, run on considerably less than 8 GB under many circumstances. Until it doesn't any more.

 

[Arwen]

If I remember correctly, Solaris 10 update 6 or later, (the ones that supported install root on ZFS), required a minimum of 1GB.
If you used UFS, it was 512MB for x86/x64, (and I don't remember if SPARC had minimum for UFS).

I use ZFS on Linux with my personal laptop, both boot pool and dumping ground pool. The laptop has 2GB of memory. Works
fine, and was reasonably fast, (until Intel's Meltdown feature dropped performance by perhaps 20% to 40%, depending on app).