Files appearing corrupt CIFS

[alieg]

Until a few days ago everything was fine with playing video files on my android, ipad & on my pc, accessing & opening files etc... but now something has happened & I'm not sure what or why

I'm getting errors such as
Slow loading of directory contents in windows explorer
Invalid directory when I try to access a folder
I couldn't copy large files on or off

These have since not been so much of an issue since I reset to defaults, but it's still not working as before
But I still get errors such as
make sure you're connected to the network location
network location no longer available
video files sometimes won't play at all or past a certain point, file damaged/invalid etc...
good image files I've copied don't appear correctly will have areas of grey like it's been corrupted - only after a reboot, the areas of grey changed

My directory permissions are 777
2x 2TB HDD RAID1
FreeNAS-8.3.1-RELEASE-p2-x64 (r12686+b770da6_dirty)
I've reset to factory defaults
I only have CIFS on as sharing
I've tested files that I know played fine before & now it either says the file is damaged or incomplete, or it will play for a few mins & stop
I've tried playing one of these same files today & it went past the position where it stopped playing before with a slight skip & then stopped a little further on.

I also keep getting "sorry an error occurred" in the GUI

I set the volumes up as ZFS because I knew I was going to upgrade the RAM to 4GB - which is the max the mobo could take, but it seemed to perform better with 1GB. I tried replacing with the old RAM, but I got an insufficient memory error so I can't test it - the only thing I can think of is that the new RAM is PC2-6400, whereas the pc specs say "PC2-4200 (533MHz) or PC2-5300 (667MHz) memory supported"- but the new RAM should just operate at 667MHz rather than the 800MHz - I don't see how it's a problem, but it's the only thing I can think of that's changed.

I'm thinking it's either a problem with CIFS or the network card in the NAS, it was all working fine a few days ago & started being a problem with slow loading times in windows explorer & some images failing to show & videos stopping. It's really driving me insane!

 

[joeschmuck]

Run a RAM test on your system. Uses MemTest86+ and run it for 3 complete passes. If you have bad RAM you could and likely will corrupt all your files on your system during a scrub. Maybe I should ask you if you have ECC RAM.

 

[alieg]

Run a RAM test on your system. Uses MemTest86+ and run it for 3 complete passes. If you have bad RAM you could and likely will corrupt all your files on your system during a scrub. Maybe I should ask you if you have ECC RAM.

I've run MemTest86+ 3 times & Memtest86 just the once, tested both drives for errors (ViVard) & they're all reporting fine - it's Non-ECC RAM - the model is Kingston KTD-DM8400C6/2G & the old pc is a Lenovo J115 7388, I'm pretty sure it doesn't support ECC anyway

my impression is that the files themselves aren't actually corrupt, sometimes they'll show errors at different places or show errors straight after I copy the file to the NAS - some video file that stopped at a certain point will play past it after a reboot.
It doesn't seem to discriminate by file size, so it's not like it's having problems reading files over xMb, in a sequence of .tiff's, all of the same size, some will display fine, others will have missing bits or won't be displayed. The missing bits will be the same on both my android & pc, but the affected files seem to change each time.

Also no difference if I'm on WiFi or LAN

 

[joeschmuck]

Well you do know the recommended minimum RAM is 6GB, right? I'm glad your current RAM appears fine. So we are not dealing with files corruption, that is good. 4GB of RAM should run stable however I can't express strongly enough that before you sink too much money into a system (like I did) that you seriously think about a system that runs ECC RAM. I'm going to lose about $300.00 when I replace my MB, CPU, and RAM with a system which has ECC RAM.

Things that you might look at on your system since you said it was working fine are... How full are the drives? ZFS changes once you have less than 10% free space available. You also didn't list your system description which is really required to help out further. And 2 drives as a RAIDZ1, you should shoot for a mirror since that is basically the same thing. Well that is my opinion.

 

[alieg]

the drives have 1.25TB / 1.8TB free - so about 500GB used - sorry it is a mirror not RAIDZ1
Yeah I did read about RAM requirements, of course, the more the better, I went for 4GB because it's should be stable & also because it's the max the mobo could take - from the reports, not all the RAM is being used - I've disabled snapshot tasks - any extra processes that could have been causing a problem or using RAM

I have now managed to re-install the previous 2x 512MB sticks of RAM & it's fine, but I'd really like to have the 4GB...
hmmm, maybe a BIOS compatibility with the new RAM then

 

[joeschmuck]

I don't see why the RAM would be an issue since you said you ram MEMTEST86+ for 3 passes. Let me clarify, three passes takes a very long time, overnight generally. Just making sure you were not looking at three individual tests of the overall suite vice three passes of the entire suite. So if you didn't run it overnight, do that no matter how many tests are completed, this will test the RAM very well and if you have a problem, it will show up.

 

[alieg]

I tested the system out earlier with 1 stick of 2GB ram - all fine, the other 2GB stick - all fine, 2GB + 512MB - all fine, both 2GB sticks - files start not displaying after a few mins of use.
I did MemTest until it said it had passed - about 4 hours each, for 3 separate tests, I'll give it another couple of tests overnight then

 

[joeschmuck]

There could be a RAM compatibility issue but I'd think that would be found with MemTest. Yes run it for 24 hours. And you are letting it start automatically, right? Not selecting any specific tests?

The other thing to consider is your power supply and while you are accessing data that your PS can't handle it. It kills me that I asked you to list your system hardware but you haven't yet.

 

[alieg]

Apart from the new RAM & HDD's, and minus card reader & disk drive, it's still a standard Lenovo J115 7388-21G
http://support.lenovo.com/en_GB/downloads/detail.page?DocID=PD011020#pow

NVIDIA C51G core chipset
AMD Athlon 64 X2 3800+ 2.0GHz
PC2-4200 (533MHz) or PC2-5300 (667MHz) memory supported
Two DDR2 SDRAM UDIMM sockets
Supports a maximum memory up to 4GB
1.8V, 240-pin, (DDR2 SDRAM)
Integrated Broadcom Gigabit Ethernet with Broadcom 5787 Ethernet controller
250 Watt or 280 Watt supply with variable fan speed
110 Volt or 220 Volt operation controlled by a switch on the back of the unit

FreeNAS-8.3.1-RELEASE-p2-x64 on Kingston DT101 4GB USB
2x 2TB WD20EAUS mirrored
1.25 TB free
2x Kingston KTD-DM8400C6/2G
PC2-6400
800MHz
CL6

It passed the 24hr memtest - but after a bit of further digging I found that there is a reported BIOS issue with 2GB RAM sticks, I managed to update the BIOS to the latest version - I'm not sure if that worked as there is a warning that the dhclient failed to start & an IP is not assigned - it just goes to 0.0.0.0

I can't quite see how it's related to the BIOS update, I tried a fresh FreeNAS install on a different USB & the same error occured

with 2x 512MB RAM & 1x 2GB + 1x 512MB RAM - it connects
2x 2GB RAM it doesn't connect

 

[joeschmuck]

Assuming it's your BIOS, try to reset your BIOS to default settings, if you haven't already done that. Before saving the BIOS settings make sure you have AHCI set if you had it set previously.

 

[alieg]

yep that restored the connection at least, but the the files still look corrupt

 

[joeschmuck]

Corrupt or they are just not playing/streaming/copying well?

Just to go over it one more time... You created a clean installation of FreeNAS and did not restore any settings? Not an upgrade or anything. And you should auto-import your pool, set up the shares, take it slow adding to your setup and test before moving on the the next thing, assuming it works with a minimalistic setting.

You backed up your important data? If not then install the original RAM and do that.

Last thing to try, install FreeNAS 9.1.0 RC2. See if that makes a difference, and again from scratch.

If none of this works then I really don't know what to tell you.

 

[cyberjock]

It could be that the system is choking due to insufficient RAM and the server is not able to stream the data at the proper speed and the media player is showing partial corruption of the video stream. Many media players like VLC will show some visual corruption if the stream is stopped and then later restored. This could be giving the false representation that data is corrupt.

Everyone knows you shouldn't use ZFS with less than 6GB of RAM minimum....

 

[alieg]

The files play/display fine when anything other than when both 2GB sticks are in together. When both 2GB sticks are in, it seems to be ok for about 2mins where everything acts normal, then the errors appear. This correlates with an increase in RAM usage here at 21:17 before it levels off again at 21:19



Images have black lines/blocks through them/grey sections/"Windows Photo viewer can't display this image because it is too large, damaged or corrupt", videos will either play a few seconds & freeze/playback stops(exits the player)/missing index/won't start/"File damaged or corrupt", copies won't start - it will stay at 0bytes & can't calculate how big the files are, opened documents have sometimes been blank - just a lot of typical symptoms of corrupt files.

It's not a streaming issue

Tried on both clean install & factory reset, didn't restore settings or upgrade.
Auto-imported volume, only have CIFS running
Data is backed up
Yep it's a minimal setup, volume + datasets + network interface + CIFS, no plugins, no tasks, just admin user, 777 permissions, anonymous CIFS login
plus DVD & card readers have been removed - so they're not drawing any power,
Newest BIOS, after I reset it, all I did was change the boot order again & enabled Hard Drive timeout.

& I'm testing the files mostly on my laptop & android, & they both see corruption in the same places & no corruption with the old RAM & combination of 2GB + 512MB

FreeNAS 9.1.0 RC2 still has the same problem

hmmm... well thanks anyway for your help Joeschmuck, I think what I'll have to do is use the 2GB + 512MB RAM combination & see how it goes.

 

[cyberjock]

Few ideas... kind of grasping at straws to help you...

-Have you tried a new USB stick with a fresh install of FreeNAS? Many people have had very weird results when their USB stick starts having problems. I'm thinking you did this by trying FreeNAS 9.1-RC2.

-Have you tried copying a large file(4GB should be okay) then doing an MD5sum of the file on the server and the desktop for comparison. This would at least confirm corruption. You can use the command md5 <filename> from the CLI to do the MD5 of the file on the server.

-The RAM speeds shouldn't cause a problem, but we all know how that goes.

-If you are 100% sure the RAM is good via a RAM test, what happens if you do a zpool scrub? Note that if your RAM is bad and you do a scrub you risk trashing the pool. A scrub can be initiated with the command zpool scrub <zpoolname> and the scrub can be monitored with zpool status.

There's the possibility your hardware is failing and that's causing your corruption. Kind of sucks that the problems are nondescript though.

 

[paleoN]

-The RAM speeds shouldn't cause a problem, but we all know how that goes.

-If you are 100% sure the RAM is good via a RAM test, what happens if you do a zpool scrub?

I've personally seen RAM tested with memtest86+ for about 36 hours with no errors, but the machine locked up. Hard reset and FreeNAS failed to boot. Second hard reset and FreeNAS failed earlier in boot-up. Third reset and the system would no longer even post. Dropped the memory speed down a notch and everything was peachy. In that instance the problem was caused by the automated memory timing settings in the BIOS. Setting the timings manually allowed it to work at it's rated speed.

 

[joeschmuck]

I've personally seen RAM tested with memtest86+ for about 36 hours with no errors, but the machine locked up. Hard reset and FreeNAS failed to boot. Second hard reset and FreeNAS failed earlier in boot-up. Third reset and the system would no longer even post. Dropped the memory speed down a notch and everything was peachy. In that instance the problem was caused by the automated memory timing settings in the BIOS. Setting the timings manually allowed it to work at it's rated speed.

I've never heard of that, very interesting. The whole time it's sounded like a RAM issue to me and I hope the OP can manually set the speed.

 

[paleoN]

I've never heard of that, very interesting. The whole time it's sounded like a RAM issue to me and I hope the OP can manually set the speed.

One should keep in mind that memtest can only validate RAM as bad. Which it's quite excellent at. You would need a hardware tester to validate as good and even then... Still I was quite surprised at the sequence of events myself. I would typically expect memtest86+ to be having errors in such circumstances.

I've been reading the thread and concur with your conclusion, most likely a RAM issue. Possibly the motherboard itself rather than the memory sticks.

 

[alieg]

I've been a bit busy & haven't had time to tinker with it too much over the past week - I've kept the 2560MB RAM in to give it some time & see if any problems occur, which they haven't - I did a md5 on some large files & zfs scrub (with the working RAM) - there were no errors & the md5's matched, so the files are definitely not corrupt

Unfortunately I can't see anywhere to set the speed manually in the BIOS, I did notice under the system settings that when the odd sticks are in the speed is registered as 400MHz, whereas when both 2GB's are in it registers at the full 800MHz - I assumed it would have been capped to 667MHz

Before updating the BIOS I tried a couple of OS's, Ubuntu 13.04, FatDog64, Win7, WinXP, they all froze up in about the same amount of time with the 4GB RAM & worked with the 2560MB RAM

For now I think the only thing I can do is leave the 2GB + 512MB in place