Corrupted Files?

[creative_username]

I have noticed that some of my video files are corrupted where the last 1/3rd of the video is gone and it reverts the player if I move the play time to that portion of the video. This isn't common - I have noticed it on two files now out of hundreds that i've looked through. I doubt they were originally corrupted like that and was wondering what I should do?
I've checked to make sure that ECC is working on my system, I believe all my drives are healthy (testing now), and I run a scrub every month.

System:
8x Seagate 6TB Desktop Drive (non-smr) raid z2
32GB ECC RAM
intel i3-6100
supermicro mobo

 

[SweetAndLow]

zfs doesn't just allow files to be corrupted. If something like that happens it usually tells you in some way or another. What is the output of zpool status(post with output in code tags)?

 

[creative_username]

zfs doesn't just allow files to be corrupted. If something like that happens it usually tells you in some way or another. What is the output of zpool status(post with output in code tags)?

[root@freenas ~]# zpool status
pool: StoragePool
state: ONLINE
scan: scrub repaired 0 in 28h18m with 0 errors on Tue Apr 12 04:37:14 2016
config:

NAME STATE READ WRITE CKSUM
StoragePool ONLINE 0 0 0
raidz2-0 ONLINE 0 0 0
gptid/4118a6e6-ada1-11e5-84be-0025905d6dcc ONLINE 0 0 0
gptid/ec3e41b1-ad4b-11e5-9fa8-0025905d6dcc ONLINE 0 0 0
gptid/07bc5745-acea-11e5-8793-0025905d6dcc ONLINE 0 0 0
gptid/5d87e1c5-ac8c-11e5-bc77-0025905d6dcc ONLINE 0 0 0
gptid/fcae77ac-ac18-11e5-ace5-0025905d6dcc ONLINE 0 0 0
gptid/9c287209-ab91-11e5-aa1f-0025905d6dcc ONLINE 0 0 0
gptid/7eedf25a-ab2d-11e5-ba85-0025905d6dcc ONLINE 0 0 0
gptid/004b7556-aaba-11e5-85ac-0025905d6dcc ONLINE 0 0 0

errors: No known data errors

pool: freenas-boot
state: ONLINE
scan: scrub repaired 0 in 0h0m with 0 errors on Mon Apr 11 03:45:02 2016
config:

NAME STATE READ WRITE CKSUM
freenas-boot ONLINE 0 0 0
gptid/77520838-a69e-11e5-8f06-d0509976a6c1 ONLINE 0 0 0

errors: No known data errors

 

[styno]

How do these files gets onto the machine? This sounds like a file is not completely transferred when picked up by a process that moves them around too early.

 

[russnas]

yes as above you need to check the original file that was copied to freenas,it can occur if you have downloaded online and not verified.

 

[SweetAndLow]

Most likely bad source file. After verifying the source you can look at other areas where problems could occur.

 

[Starpulkka]

So you checked that ecc is working server, how? did you see on logs or did you do single bit injection and multi bit's injection and get proper email and system halt?
Have you checked on your local machine that is memory ok, better yet use ecc memory on local machine if you dont need inject trainers to games.(remember that you can have ecc corrected notice on windows in middle of a gaming, so if you on a critical tournament remember take ecc off on bios)

How did you copy data to server, as basic to data copying is you first check sender machine that it is healty no mem errors etc. and make a md5 sum or multipar check list files what you going to send to server and again check that md5 or par check files just was created are showing ok. (making par files you have to have corruption free memory on send machine) and then youshould check that server machine is ok (no corruption or man in middle hacks etc.) after checkin that server is ok you start copying files. When files are copied to server you issue order on server machine to check that md5 or par sum on files that did files travel unaltered. (if you dont know how type commands on server you can also use windows client on server to check those files..) After you get confirmation all files ok then you should be good if not, multipar can heal those files.

Losting big part from videofile does indeed sound that file transfer was interrupted, or hdd and NFTS have lost some bits on your original video file and happily serves you a corrupt video file without telling you.. if you lose few bits you only see some artifacts on frames.

 

[David Dyer-Bennet]

Do you take periodic snapshots? If so you can go back to the oldest one and see if the file was also damaged then; if not clearly something happened while on the server, but if it's bad in the oldest snapshot (if you have them), that would suggest that perhaps it got damaged on the way in. For that matter, do you have clear memories of playing the last third of the damaged files ever? That would also be suggestive (not as good as evidence that can be exhibited, but still useful).

Since you're doing scrubs, the files should be very VERY safe once they get correctly onto the server. I've seen so many cases of people doing initial experiments with scratch systems, getting errors, and eventually proving to themselves that ZFS is finding and reporting errors nothing else has ever found with their particular scrap-heap systems. Keeping separate checksums for each data block (not just relying on the hardware checksums) is a huge step up.

None of which is to make any absolute claims that ZFS is perfect or the data entrusted to is incorruptible. I don't know any actual cases that didn't involve hardware failures in excess of the redundancy on that pool, though. It's the best I've used or read about for the kind of server I build and operate (home and underfunded small business), but it's a human product like everything else, and humans screw up.

 

[creative_username]

Thanks for the replies everyone. A lot of the files were mass downloaded and transferred over lan without me checking them so it is possible they were corrupt before hand. I have so much data that it is impossible for me to check all of the files, I just happen to stumble across the bad ones when I do. I'll try the md5 method that was mentioned to confirm if there is something going on. It just makes me really paranoid that some files have been corrupted, especially since I have a lot of files that would be difficult to replace.
I tested ecc by using memtest86 paid version; I don't know how to test it while freenas is running.
Would having little free space cause any issues like mine? I'm at 96% usage right now.

 

[SweetAndLow]

you probably don't want to go over 90% usage. Going higher should not cause corruption but things can randomly fail because certain writes to disk will just fail.