Arwen:
YOU PROBABLY ARE CORRECT e.g. removing disk in ZFS array, wipe it, and then resilver it back – the empty space should be in fact empty (I mention wipe meaning that every block, on the disk, is written over, more than once, before resilvering – wiping a disk, by writing over each block, can be done multiple times – wiping a disk is a whole thread by itself e.g. random writes, DOD standards, disk type, etc.).
ZFS & resilver, I had seen where the FreeNAS literature (online manual) mentions that ZFS only copies the blocks that are in use during resilver as opposed to most hardware raids where they have to copy every block – making ZFS faster and is also interruptible where ZFS resumes where it left off should an interruption occur – great benefits for ZFS for sure and I think your reasons for asking why removing / wiping / resilvering would not work – again here I say you are probably or for certain correct.
I was thrown off by discussions of the development of ZFS itself – new feature flags, commands, etc. and if this ZFS development is affecting any parts of the resilver process where the general notes would not state - specifically what is getting resilvered in regard to the blocks & latest features? If the general notes on ZFS, taken at face value, then your statement is correct – a resilvered disk would have fresh empty space and only bring back the blocks that are in use.
I even read recently where the ZFS developers want to add a feature to allow recovering deleted datasets – not sure how they would even start such a project but there are discussions on this as a future enhancement to ZFS but the function would probably have to be enabled before the dataset was deleted – kind of like a dataset snapshot if you will. Just saying, ZFS is very complicated.
My suggestion here, for making the empty space unrecoverable, on an existing NON-encrypted dataset, was to create a new dataset then either use ZFS send/receive or rsync to copy the data then delete the original dataset. This does not write over the empty space but makes any kind of recovery, on the original deleted dataset, almost impossible and as you mention the longer the system runs the empty space gets written over (I could tell by that statement you know a lot about this subject)
From what I have read if you delete a file from a ZFS dataset the file most likely can be recovered as opposed to making a new dataset, moving the data, then deleting the original dataset – the latter making recovery near impossible – at least I have not read where anybody was able to recover data from a deleted dataset - & I have read some pretty complicated attempts at doing so (fortunately I have never needed such a method but there have been users that accidentally deleted a dataset they wanted back and spent much time trying – most of us have redundant backups, etc.).
Also, I am not sure but would guess that ZFS send / recv only copies data as well to the new dataset – ZFS send / receive is the fastest way, I have found, to transfer data between datasets.
RE: Encryption – yes even the original poster of this message knows that encryption is the best method for protecting data (the poster even says so) and we all agree but the poster wanted to know the status of the empty space & the historical deleted data and how to insure it cannot be recovered on a NON-Encrypted dataset – I myself have been curious about this for some time.
Heck, data erasure / clearing / wiping has been a serious issue for people who store data and for operating systems that write directly to the disk blocks, there are 100s of utilities and even the DOD lists many random write methods acceptable for disposing of hard drives. SSD drives are another animal. Of course, ZFS datasets are totally different and you cannot just write over the physical blocks directly e.g. the reason for this thread – I realize most people do not care but when you get in the weeds this subject can be quite challenging.
In Closing per your comment:
Detaching, Wiping & resilvering each disk probably will leave all the empty space truly empty if ZFS performs as stated in the FreeNAS manual – meaning resilver only brings back blocks that are in use reducing the time to rebuild the vdev & thereby leaving the empty space truly empty.
A bit risky to be detaching disks and resilvering and I am not encouraging anybody in doing this – you do not want to lose your array should another disk or two fail while doing such a procedure
This is a conversation on the status of the empty space, in a NON-encrypted ZFS dataset, where files have been deleted – can they be recovered or what can be done so that the deleted files, in the empty space, cannot be recovered – in my opinion a very complicated subject.