Considering the following, two years after the NAS goes live, is it true that starting with a 2x4TB mirror would actually be safer than a 4x2TB zraid2? / will be on a satadom, backed up nightly.
1. ZFS shows increasing checksum errors for a drive
2. You do validate your replacement 4TB drive as non-defective with a smart short and conveyance test, then run badblocks, then a long smart test
3. A few days later, the checksum errors are getting serious; errors appear in the smart output
4. You can finally attach the 4TB drive as a mirror, making it a 3-way mirror.
5. It resilvers very quickly (someone please provide some hard numbers)
5q. How safe is your data during the resilver?
6. After 40 days of no problems
7. You RMA the drive, get the new one, validate it, and add it as the second mirror
It would seem that drive failure becomes an opportunity for adding redundancy. Chances are you'll get the RMA drive before the upper limit for early hard drive mortality, and can continue with a 3-way mirror. Finally, in a case with room for only four hard drives, this path provides a path for future expansion without replacing existing drives.
Compared this raidz2 scenario:
1. ZFS shows increasing checksum errors for a drive
2. You do validate your replacement 2TB drive as non-defective
3. A few days later, the checksum errors are getting serious; errors appear in the smart output
4. Finally, you physically replace the failing 2TB drive
5. It resilvers slowly in comparison to a mirror (please provide some hard numbers)
5a. While resilvering, a drive completely dies
5b. Now you have zraid1-level parity with 2TB drives under the stress of a resilver
5c. The dying drive slows down the resilver
5d. You can't add parity to a zraid2
In effect, doesn't this mean that when a drive is slowly dying, and then a second drive dies during the resilver, that a 4x2TB zraid2 is more dangerous than a 3-way 4tb mirror resilver in progress?
I've tried to isolate these scenarios from my Linux experience, but my tendency is to believe that a resilver is harder on the drives than a scrub, and that a zraid2 resilver places the drives in a state of greater stress/potential failure than a mirror resilver.