About pool encryption

[Heracles]

After offering support in the forum for a little while, I am baffled at how many people have pool encryption turned ON in their pool. Considering how often pool encryption turns to a self-inflicted ransomware, I wish to offer some input about that functionality and remember people how high risk and low benefit that feature can be.

For pool encryption like for every encryption, the security is entirely provided by the key. This is the golden rule of cryptography : No crypto solution can provide a security better than the security applied to its key. It is true for FreeNAS and ZFS.

So when doing pool encryption, how secure is the key ? In fact, not much... The key is clear text in the filesystem and protected only by access control to that file.

That creates the first limitation : as long as the drives are part of the system, their content it not protected by anything else than access control at filesystem level. Considering that their filesystem is already protected by such access control, the net value is a plain big ZERO.

If the access is remote, it will be served by the system. If access is granted to the data, they will be decrypted without any question. If access is denied, encryption is not helping because data are not accessed anyway.

If the access is local to the system, the key will be as easily accessible as the rest of the data. Because the key has no extra security, it will not provide any extra security to the data,

The only way to find a benefit is to separate the key that is in the system and the drive it encrypts. But even here, it is not such a great plus.

Considering how ZFS spreads the data over its drives, no single drive contains an intelligible part of the data. Except for a pool with a single mirror or a single drive, one can not recover anything intelligible from a single ZFS drive. There is a reason why there is no file recovery tools for ZFS. It is way too cryptographic by itself to extract anything meaningful if the pool is not working by itself. Add to that that usually, a drive is removed from a system when it starts failing, the probability of recovering anything from that drive is already close to 0. As for the swap space, that one is already encrypted anyway.

So the situation is:
No protection at all for any remote access to the system.
No protection at all for any physical access to the system.
No protection at all for the swap space because that one is already encrypted.
Low protection for the content of a drive once that drive is removed from the system.

On the other side, pool encryption is very high risk for the legitimate owner :
--Must do extra backups (and by the way, these backed up keys also reduce the encryption's security by increasing the chance for an intruder to get a copy of the key)
--Must do extra procedures during regular operations like disk replacement, pool migration, OS re-install and more.

At the end, if everything is done perfectly about it, pool encryption will provide almost no benefit. If done wrong, it will quickly ruins the entire pool.

If data need encryption at rest, it is a million time better to do it at application level instead of filesystem level. Many applications offer a much better key management solution. Such an encryption will not have any effect on ZFS and the pool itself, so no increased risk when managing FreeNAS. Considering that usually it is not 100% of the data that requires encryption, the risk of an encryption problem will be limited to what was encrypted instead of the entire pool.

I would say that pool encryption is like deduplication :
you almost certainly don't need it even if you think you do.
Almost certainly, it will not do you any good, even if you imagine it will.
Almost certainly, it will easily and quickly betray you down the road, ruining what could have been a very good solution in every other aspect.

 

[danb35]

you almost certainly don't need it even if you think you do.

Agreed.

Almost certainly, it will not do you any good, even if you imagine it will.

Partially agree. It will benefit if the disks are separated from the boot device. You can dispose of disks without having to worry about wiping them (which could be a benefit if the disk just dies, and you're concerned about a highly-motivated attacker reading data directly from them). But (IMO) the benefit is minimal, particularly compared with the risk.

Almost certainly, it will easily and quickly betray you down the road, ruining what could have been a very good solution in every other aspect.

I think "almost certainly" overstates the risk (just as the "RAID5 is dead" crowd do), but yes, there are lots of ways it could blow up on you, and a fairly narrow range of risks it would protect against. I think native ZFS encryption may address a lot of the ways that FreeNAS' implementation is so fragile; hopefully we'll see that before too long.

 

[kingc]

So when doing pool encryption, how secure is the key ? In fact, not much... The key is clear text in the filesystem and protected only by access control to that file.

That creates the first limitation : as long as the drives are part of the system, their content it not protected by anything else than access control at filesystem level. Considering that their filesystem is already protected by such access control, the net value is a plain big ZERO.

If the access is local to the system, the key will be as easily accessible as the rest of the data. Because the key has no extra security, it will not provide any extra security to the data,

Not necessarily - this is why FreeNAS supports a pool encryption passphrase. Both the key and the passphrase are then required to decrypt the volume master key. So, the system is ultimately as secure as the passphrase, as one would expect.

 

[Heracles]

So, the system is ultimately as secure as the passphrase, as one would expect.

The passphrase is far to be such a silver bullet. The first consequence is that with a passphrase, the pool will not mount by itself at boot. To require a manual intervention at boot is a major drawback for many systems.

Also, you made here the very mistake I hoped to help people avoid... You said that the --SYSTEM-- is ultimately as secure as the passphrase. This is completely wrong.

1-Pool encryption does nothing at all for the --SYSTEM--.
2-Passphrase or not, encryption is of no use once the pool is mounted, so when system is online

When done properly, pool encryption make data recovery from ZFS drive impossible.
Without pool encryption, data recovery from ZFS drives is already almost impossible without a complete set of drives.

Doing pool encryption is to take a gigantic risk for basically such a small benefit that it is almost non-existing...

 

[kingc]

Haha, ok then.

The passphrase is far to be such a silver bullet. The first consequence is that with a passphrase, the pool will not mount by itself at boot. To require a manual intervention at boot is a major drawback for many systems.

You're moving the goal posts. First, you stated that encryption has no value because the key is not protected. Then, once I point out that the key can be protected, that's too inconvenient?

Also, you made here the very mistake I hoped to help people avoid... You said that the --SYSTEM-- is ultimately as secure as the passphrase. This is completely wrong.

Read again - this time in context. The previous sentence is rather important:

Both the key and the passphrase are then required to decrypt the volume master key.

This is the system I was referring to. System as in "method", not system as in "operating system".

2-Passphrase or not, encryption is of no use once the pool is mounted, so when system is online

It's not intended to be. LUKS, Bitlocker, Veracrypt, FileVault all fall in this category. Oh, and native ZFS encryption has just added to ZoL. Interesting how implementations across ALL major operating systems are of "no use".

Without pool encryption, data recovery from ZFS drives is already almost impossible without a complete set of drives.

You're fixated on one scenario. The fact that it's near impossible to recover data from a striped pool of disks (ZFS and normal RAID) has nothing to do with the value of encryption.

Advising FreeNAS users to understand the risks associated with enabling encryption is an admirable thing (especially since the UI/UX is pretty terrible in this area). I'd rather someone uses encryption and manages the risks (incidentally, the same risks exist in your recommended "application level" approach) if they value security.

Cheers.

 

[Heracles]

LUKS, Bitlocker, Veracrypt, FileVault all fall in this category.

Indeed, all of these encryption tools but VeraCrypt are of the same low benefit. For one to get pass Bitlocker on a stolen computer, he only needs to wait next month's patch Tuesday for a brand new remote exec against the system. Using it, he will enter the system which will have self decrypt for the case. And that is only if the computer is already perfectly patched up, which is probably not...

The thing that put Veracrypt in a different category is that it is not exclusively designed for full drive-level encryption. Its hidden container is also something different when compared to the others.

the same risks exist in your recommended "application level" approach) if they value security.

About that, I mentioned that often these applications offer a much easier and better key management process. Also, it is way easier to encrypt only a small subset of the data and so will expose you only to loosing these data and not everything. But should you do it all wrong, indeed you can create as much damage. The difference is that pool encryption can not damage only part of the data, it will damage the entire pool. With a more difficult key management, the probability of such damage is also higher. Much higher damage and higher probability gives you a much much higher risk with pool encryption.

You're fixated on one scenario. The fact that it's near impossible to recover data from a striped pool of disks (ZFS and normal RAID) has nothing to do with the value of encryption.

There is a reason to fix on that single scenario : It is the only scenario where the pool level encryption is of any use, passphrase or not. Even with a passphrase, once someone has physical access to the system where a passphrase is hard required at every boot, it is easy to steal that password at the same occasion. So because pool encryption has not other benefit, there is no other scenario to talk about.

 

[Arwen]

I wrote a blog about ZFS on Linux native encryption;

OpenZFS on Linux encryption – Now testing on a Linux kernel near you

In my opinion, file system encryption is not what some people think it is. For example, someone here on the forums insisted he needed FreeNAS boot pool encryption too. Since FreeNAS does not support that function, and probably never will, we pointed him else where.

Others think it will protect them from hackers. Really? Whence the pool & it's datasets are mounted, encryption might not as well exist. Snapshots can actually protect data from ransomware. But, data theives? Nope, (unless there are physical data theives that still the hardware and power it down in the process).

In the end, pool or dataset encryption will only do what other at rest encryption will do, protect data when it's not powered up.

 

[PhiloEpisteme]

After offering support in the forum for a little while, I am baffled at how many people have pool encryption turned ON in their pool. Considering how often pool encryption turns to a self-inflicted ransomware, I wish to offer some input about that functionality and remember people how high risk and low benefit that feature can be.

While in general I think that many people on the forums do not give encryption the proper respect it deserves, I think the risks and benefits are somewhat individual. For example, a person who keeps regular backups of all keys stored securely and keeps one on-site and one off-site backup of data is at significantly lower risk of losing their data to encryption that someone who set up their system years ago and forgot they used encryption and thus aren't sure if their keys are up-to-date or not.

I'm not chiming in to ruffle feathers but just to suggest that it is very hard to judge what benefits someone else may gain from encryption because their situation and data may be different and something you don't expect. I think it is a great idea to help make it clear to people what encryption in FreeNAS protects (data at rest) and what it does not protect. Knowing that, understanding the risks, and appreciating the extra steps required to properly manage an encrypted pool I think is enough to let a person decide for themselves whether the risks vs benefits result in net positive gains or not for them.

 

[Heracles]

Hey Philo,

I think the risks and benefits are somewhat individual.

to suggest that it is very hard to judge what benefits someone else may gain from encryption because their situation and data may be different and something you don't expect

In fact, not that much. Here is an easy way to illustrate why :

Consider that tomorrow, you suffer a security incident. You have no choice about it, an incident will happen. Luckily for you, you can choose which incident you will have to face.

Incident No1 : As a consequence of the incident, someone got access to your data, but after the incident, you too still have them.
Incident No2 : Nobody accessed your data despite that incident, but after it, you do not have them either.

Which incident will you face ? For well over 99% of the cases, to keep access to your own data is a better ending. First, because not that many data are that confidential and second, even the ones that are so confidential still worth more to you after disclosure than not having them anymore at all. For that 99% of cases, Availability is more important than Confidentiality.

There are --VERY-- rare cases where that situation is not true, but so few! One of such case would be a military defence plan. Better for you to react with an improvised defence plan than to react according to a plan the enemy knows about. But for basically everything else, better to keep your data even should it means that someone else learned about them.

a person who keeps regular backups of all keys stored securely and keeps one on-site and one off-site backup of data

That is all the problem : you have to pay a very high price to get pool encryption right...

FreeNAS protects (data at rest)

..and FreeNAS is meant never to let its data at rest. FreeNAS is meant to be always On. So the very high price is paid for a very low value.

So in fact, encryption is a million time more of a risk than a security for 99% of the cases.

Nice to exchange with you,

 

[jgreco]

..and FreeNAS is meant never to let its data at rest. FreeNAS is meant to be always On.

That's not what "data at rest" means. Data at rest is an industry term to refer to data on an underlying storage device. If I shut off your FreeNAS box and pull the disks, and can make sense of the data, then your "data at rest" is unencrypted.

This is a common problem in virtualization environments and certain other environments. For example, I have an automated VM build and configuration tool that powers a large number of VM's. This tool has a lot of host configuration information that I wouldn't like to be leaked to an intruder at a cloud provider, so I assume that the underlying storage service is compromised, and use GELI encryption to encrypt the data at rest. When the build tool is booted, it requires an administrator to provide the encryption passphrase that then decrypts the encryption key, because that simply isn't available anywhere online.

Many industries are required to take various steps to protect their computing systems and their data. PCI-DSS is a classic example, where encryption is mandated. The idea is that you want to make it less likely that a lost HDD results in a big crisis.

https://www.consumerreports.org/data-theft/the-data-breach-next-door/
https://www.careersinfosecurity.com/hard-drives-lost-affecting-nearly-1-million-a-8829
https://www.greatfallstribune.com/s...warned-data-breach-hard-drive-theft/97896768/
https://www.pcworld.com/article/302...eaches-and-whats-being-done-to-stop-them.html
https://www.hipaajournal.com/phi-76000-texas-patients-stolen-hard-drive/
https://fox8.com/2013/11/04/university-hospitals-hard-drive-stolen-with-patient-information/
https://www.scmagazine.com/home/opi...-contained-unencrypted-data-on-3500-students/

These are all cases where the easier case of "not" encrypting their data was chosen, and there are thousands more.

In any case, I'm closing this thread because it's feeling like it has become less of a discussion of merits and more about entrenched opinions, in which case I think all the major points have already been made. FreeNAS does not force you to encrypt your pool. If you encrypt your pool and you later come to regret it because you didn't read all the docs or do all the right things, that's tragic, but at this point the deed is done.