Evi Vanoost
Explorer
- Joined
- Aug 4, 2016
- Messages
- 91
I have the following pool, made up out of 36 4TB hard drives. It's a backup pool so it receives ZFS streams. It is getting near to full (73%) but this morning, I received messages that the backups had failed (most likely because they couldn't receive the massive data stream) and then there is this error.
There seems to be only 4 checksum errors on a single drive, they're in RAIDZ2 so I don't understand why it would report massive data corruption. Does ZFS not handle the drives running full? If so, that is really, really bad.
There seems to be only 4 checksum errors on a single drive, they're in RAIDZ2 so I don't understand why it would report massive data corruption. Does ZFS not handle the drives running full? If so, that is really, really bad.
Code:
pool: Backup state: ONLINE status: One or more devices has experienced an error resulting in data corruption. Applications may be affected. action: Restore the file in question if possible. Otherwise restore the entire pool from backup. see: http://illumos.org/msg/ZFS-8000-8A scan: resilvered 0 in 0h25m with 0 errors on Sun Jun 11 03:27:43 2017 config: NAME STATE READ WRITE CKSUM Backup ONLINE 0 0 0 raidz2-0 ONLINE 0 0 0 gptid/4a03c5a7-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 0 gptid/4a902559-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 0 gptid/4b1a5dc7-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 0 gptid/4b9e58e2-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 0 gptid/4c20c17b-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 0 gptid/63a6b1b9-b582-11e6-a19e-003048f15c54 ONLINE 0 0 0 gptid/05a36a2c-c934-11e6-b962-003048f15c54 ONLINE 0 0 0 gptid/4dc3255f-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 0 gptid/4e53bf06-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 0 gptid/4ee405e5-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 0 gptid/4f730710-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 0 gptid/500440ad-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 0 raidz2-1 ONLINE 0 0 0 gptid/50ee8494-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 0 gptid/51790b55-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 0 gptid/5202efc1-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 0 gptid/528ee10a-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 0 gptid/531f8b1a-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 0 gptid/53ace5a0-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 0 gptid/544886ba-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 4 gptid/54e25596-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 0 gptid/557ea40a-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 0 gptid/5616cb43-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 0 gptid/56af5efc-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 0 gptid/574b5f60-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 0 raidz2-2 ONLINE 0 0 0 gptid/583af6e1-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 0 gptid/58d02102-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 0 gptid/59698b70-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 0 gptid/59fbeaf8-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 0 gptid/5a8f22f5-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 0 gptid/5b21f5d8-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 0 gptid/5bb3b29d-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 0 gptid/5c4abf1e-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 0 gptid/5cd6d8ab-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 0 gptid/5d616609-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 0 gptid/5de5c744-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 0 gptid/5e6f1c39-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 0 logs gptid/60814fd1-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 0 cache gptid/5f72a8b4-1e8f-11e6-ad7f-003048f15c54 ONLINE 0 0 0 errors: Permanent errors have been detected in the following files: <metadata>:<0x0> /var/db/system/update/base-os-9.10.2-U5-3c693eb3ffa689371074d2ddaab93d59.tgz <0x9f0e>:<0x0> <0x9f0e>:<0x59f217> Backup/.system/syslog-5d784362f850449eb03d556b65045a45:<0x55> /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/cpu-6/cpu-interrupt.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/geom_stat/geom_bw-da20.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/geom_stat/geom_ops-ada1p2.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/geom_stat/geom_bw-da7p1.eli.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/geom_stat/geom_ops_rwd-ada1p2.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/geom_stat/geom_busy_percent-da7p1.eli.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/geom_stat/geom_ops_rwd-da21p1.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/cpu-10/cpu-interrupt.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/disk-da12/disk_octets.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/geom_stat/geom_ops-da8p1.eli.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/zfs_arc_v2/gauge_arcstats_raw_cp-c.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/disk-da13/disk_octets.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/geom_stat/geom_busy_percent-da8p1.eli.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/geom_stat/geom_ops-da21.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/geom_stat/geom_ops-da9p1.eli.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/geom_stat/geom_bw-da21.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/geom_stat/geom_ops-da0p2.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/geom_stat/geom_ops_rwd-da0p2.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/disk-da16/disk_octets.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/cpu-7/cpu-idle.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/geom_stat/geom_busy_percent-da21.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/cpu-8/cpu-idle.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/geom_stat/geom_ops-da1p1.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/geom_stat/geom_latency-da10p1.eli.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/zfs_arc_v2/gauge_arcstats_raw_size-data_size.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/geom_stat/geom_busy_percent-da10p1.eli.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/cpu-13/cpu-idle.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/geom_stat/geom_ops_rwd-da11p1.eli.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/geom_stat/geom_ops-da1p2.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/geom_stat/geom_bw-da11p1.eli.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/zfs_arc_v2/gauge_arcstats_raw_l2-l2_misses.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/geom_stat/geom_ops_rwd-da1p2.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/geom_stat/geom_latency-da11p1.eli.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/geom_stat/geom_busy_percent-da11p1.eli.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/disk-da21/disk_octets.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/zfs_arc_v2/gauge_arcstats_raw_l2-l2_feeds.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/disk-da22/disk_octets.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/disk-da23/disk_octets.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/geom_stat/geom_busy_percent-da13p1.eli.rrd /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/geom_stat/geom_queue-da14p1.eli.rrd ... /var/db/system/rrd-5d784362f850449eb03d556b65045a45/zombie.rcbi.rochester.edu/geom_stat/geom_busy_percent-da6p1.eli.rrd <0xffffffffffffffff>:<0x1725a5c> <0xffffffffffffffff>:<0x1725a5f> pool: freenas-boot state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM freenas-boot ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 ada0p2 ONLINE 0 0 0 ada1p2 ONLINE 0 0 0