Permissions Issue For Large Files Only (6GB+)

[solarisguy]

Ask a Windows sharing specialist.

 

[Sonny Williams]

I can find no other users Windows with this online and I have never had this issue with any other NAS. Seeing as it happens with all of my windows machines, and only with FreeNAS and only with 1 file type, it seems exponentially unlikely that it is a problem on my side. Plus, I do not seem to be the only person having this issue with FreeNAS. I am happy to continue testing on my side, bit from experience, it just seems so unlikely.

 

[solarisguy]

It is possible that the problem is caused by some configuration setting of your FreeNAS server. However, in my opinion, the problem is inherently related to Windows permissions, thus someone with Windows experience has to look at the permissions and tell you (us) what is wrong.

 

[anodos]

My issues seem to be getting more strange. Now, from time to time, I get permission errors simply by trying to drag something into my CIFS share. However, these are failing immediately. I rebooted the NAS, and then I updated it to the latest version. I don't have any errors or warnings in my NAS alerts.

When I drag a file over, it says "Destination Folder Access Denied. I can create files in here and add folders manually, but when I drag one over it just says permission denied. Then, out of nowhere, it will just magically let me drag and drop files. Then, after some seemingly random amount of time, it goes back to permission denied.

still sounds like you have permissions type set to "Unix". Since you recreated your share, please post an updated debug file.

 

[Sonny Williams]

I verified that the storage was set to use Windows permissions. I don't see anywhere in the actual CIFS share to set permissions. I have attached the full debug log again, so perhaps you can tell me if I need to set the permissions to Windows somewhere else also.

 

[Sonny Williams]

It's driving me nuts. I cannot even save files to this Freenas server anymore. I try to copy them, and it just fails. I have 3.6TB of data in here, and I don't have that kind of space just lying around to move it off to and recreate so I'm becoming frustrated at what to do. :(

 

[Sonny Williams]

Well, I fixed that issue that was cropping up obnoxiously. I changed the owner on the Volume to be my user account, and not the root user. Once that was done, I was able to save stuff to it again. I was realizing that this issue was seemingly ONLY affecting my Windows 10 machines. All other machines could save to it without issue. Upgrading my couple of media machines to try and resolve the "TS" issue apparently created this new issue with the root user owner Permissions Denied error.

Anywho, the "TS" files are still having the same very strange issue from any Windows machine. It copies to 100% then starts over at 0% and when it reaches 100% again for a second time, it deletes the file from the share and gives a Permission Denied error. However, as pointed out once previously, I can stop it at 99% and use this file from any machine on the network on this share, but if I let it get to 100% it dies.

 

[anodos]

Well, I fixed that issue that was cropping up obnoxiously. I changed the owner on the Volume to be my user account, and not the root user. Once that was done, I was able to save stuff to it again. I was realizing that this issue was seemingly ONLY affecting my Windows 10 machines. All other machines could save to it without issue. Upgrading my couple of media machines to try and resolve the "TS" issue apparently created this new issue with the root user owner Permissions Denied error.

Anywho, the "TS" files are still having the same very strange issue from any Windows machine. It copies to 100% then starts over at 0% and when it reaches 100% again for a second time, it deletes the file from the share and gives a Permission Denied error. However, as pointed out once previously, I can stop it at 99% and use this file from any machine on the network on this share, but if I let it get to 100% it dies.

Sorry. I forgot to reply to this thread. Did you check for alternate datastreams?

Try the following:

• Delete your share config.
• Reset permissions type on your main dataset (/mnt/rz1) to "Unix" through the webgui
• Create a new dataset. For instance, /mnt/rz1/media, configure it with "windows" permissions type, and set it so that it is owned by "sonny:wheel"
• Create a new CIFS share pointing at /mnt/rz1/media. Check the box "apply default permissions" and hit "OK".

If the above procedure doesn't work, try installing freenas on a fresh USB stick, importing your pool, and creating the share again.

 

[solarisguy]

@Sonny Williams, as anodos has suggested, please check for alternate data streams (ADS)

Sorry. I forgot to reply to this thread. Did you check for alternate data streams? [...]

URL http://blogs.technet.com/b/askcore/archive/2013/03/24/alternate-data-streams-in-ntfs.aspx lists two tools you can use to check for ADS.

 

[Sonny Williams]

So... After all this time, I think I found a solution. It does not appear to have anything to do with the size or the file format. Something I think we all already knew. This appears to be a DRM problem. I bought and installed Aimersoft Video Converter, and it allowed me to remove the DRM protection, while keeping everything else the same. It is still a TS file, it is still 15420kbps video bit rate and still 7.5GB in size. The only difference is that it removed the DRM. I can now copy this file to Freenas. I assumed this was the case, but didn't want to buy this piece of software to figure it out. This also explains why I can copy this exact file to Freenas from a Mac machine, as the Mac doesn't seem to care as much about the Copy Protection flags, I suppose!?!?

Any thoughts on this?

 

[silentflight15]

I can verify everything described in this thread. I had two .ts files from NextPVR that would complete, restart, request admin approval, and ultimately hang/fail. I successfully transferred all kinds of other file types, bigger and smaller in size as well. I finally tried converting to an h.264 .mp4 using Handbrake and that obviously transferred to my Freenas share just fine.

That was all, of course, during many hours spent frustrated before finding this thread after the fact...

 

[solarisguy]

It would be nice to know which part of Windows is responsible for that silent DRM fiasco. And have instructions here handy (on how to check the limits, DRM etc.). Otherwise with the average media file size growing and growing we would encounter more often people blaming FreeNAS for Windows features...

 

[Klontje]

I read through this topic and someone suggested renaming the ts files to something else. Did you try this and did it help or not? I couldn't find that anywhere in the thread. Another thing worth trying; put the failing .ts file in a zip/tar/etc and isn't recognised as a .ts file anymore and see what happens.

 

[anodos]

It would be nice to know which part of Windows is responsible for that silent DRM fiasco. And have instructions here handy (on how to check the limits, DRM etc.). Otherwise with the average media file size growing and growing we would encounter more often people blaming FreeNAS for Windows features...

I believe FreeNAS defaults to using the vfs object "streams_xattr" to handle alternate datastreams. The samba documentation states "Please note that most file systems have severe limitations on the size of xattrs. So this module might work for applications like IE that stores small zone information in streams but will fail for applications that store serious amounts of data in ADSs."

As far as I can tell, the best alternative for handling large ADS is to use the "streams_depot" VFS object. This option saves the ADS information in a hidden directory in the root of the share (rather than as a filesystem extended attribute) thereby avoiding the size limitation. https://www.samba.org/samba/docs/man/manpages/vfs_streams_depot.8.html
Note that this feature is still marked as experimental.

<snip old irrelevant information>

<edit 11/02/2017>
I would personally avoid using streams_depot. FreeNAS 11.1 will have a patch that enables writing large extended attributes. I've tested up to 255 MB in size.

 

[danruehle]

Pretty sure this has absolutely nothing to do with DRM. I could not copy an AVI file created with a digital camcorder to my FreeNAS device. It was very frustrating. I was attempting to copy a 5.5GB file. Robocopy would get to 99.9% of the copy and then fail saying "Access is denied." and retry. It wouldn't say which side, source or destination had the problem though. I did some searching and found https://forums.freenas.org/index.ph...with-alternative-data-stream-avi-files.24166/. It states that files with an alternate file stream greater than 64KB will have issues. I used powershell and this command "get-item -Path .\MyRecordedVideoFile.avi -stream *" to view the streams and their sizes, the alternate stream was 190KB. I used streams to delete the extra stream via this command "streams -d .\2008-05-10-MyRecordedVideoFile.avi" which removed the alternate stream and the file copied fine. I would guess that the alternate stream contained some start/stop points where I stopped and started recording again. Either way, it wasn't necessary and removing the alternate stream made the file copy over fine. I know its been a while, but figured I would put this here in case someone else hits this and pain overcomes their head.

 

[anodos]

Pretty sure this has absolutely nothing to do with DRM. I could not copy an AVI file created with a digital camcorder to my FreeNAS device. It was very frustrating. I was attempting to copy a 5.5GB file. Robocopy would get to 99.9% of the copy and then fail saying "Access is denied." and retry. It wouldn't say which side, source or destination had the problem though. I did some searching and found https://forums.freenas.org/index.ph...with-alternative-data-stream-avi-files.24166/. It states that files with an alternate file stream greater than 64KB will have issues. I used powershell and this command "get-item -Path .\MyRecordedVideoFile.avi -stream *" to view the streams and their sizes, the alternate stream was 190KB. I used streams to delete the extra stream via this command "streams -d .\2008-05-10-MyRecordedVideoFile.avi" which removed the alternate stream and the file copied fine. I would guess that the alternate stream contained some start/stop points where I stopped and started recording again. Either way, it wasn't necessary and removing the alternate stream made the file copy over fine. I know its been a while, but figured I would put this here in case someone else hits this and pain overcomes their head.

Wow! Thanks for posting about this. Documentation about what software / vendors are actually using ADS is very sparse. Can you please post the make / model of the digital camcorder? :D

 

[anodos]

Update for those concerned about this issue. In FreeNAS 11.1 samba will be patched to support large ADS's (I've tested up to ~200 MB in size). Thanks Timur!

 

[ShimadaRiku]

Update for those concerned about this issue. In FreeNAS 11.1 samba will be patched to support large ADS's (I've tested up to ~200 MB in size). Thanks Timur!

Is it fixed yet in 11.1 or 11.2? I'm still on 9.3

Just stumbled on this problem too while trying to use NextPVR with my HDHomeRun tuner. The TS file had ADS metadata and for the longest time confused why video files above 1.5GB was having issues. Ended up just removing streams_xattr from the VFS object of my CIFS share.

 

[anodos]

Is it fixed yet in 11.1 or 11.2? I'm still on 9.3

Just stumbled on this problem too while trying to use NextPVR with my HDHomeRun tuner. The TS file had ADS metadata and for the longest time confused why video files above 1.5GB was having issues. Ended up just removing streams_xattr from the VFS object of my CIFS share.

It's fixed in both 11.1 and 11.2. I'd hold off on upgrading until the next 11.2 U release (it will be soon). This is because we are bringing in Samba 4.9, which has _significant_ performance improvements. When it comes time to upgrade, you should back up your config file and your jails (if you have any). There is a huge delta between 9.3 and 11.2.

 

[ShimadaRiku]

It's fixed in both 11.1 and 11.2. I'd hold off on upgrading until the next 11.2 U release (it will be soon). This is because we are bringing in Samba 4.9, which has _significant_ performance improvements. When it comes time to upgrade, you should back up your config file and your jails (if you have any). There is a huge delta between 9.3 and 11.2.

9.3 to 11.x is it best to fresh install from scratch or upgrade? Currently virtualized in ESXI 6.5, passthrough, encrypted drives.