Permissions Issue For Large Files Only (6GB+)

[baltilion]

All,

I've got FreeNas 9.3 running, the FreeNas box is set up as domain controller, Active Directory is bound to this domain, and everything seems to be working great, including 105-125MB/sec transfers across a 1Gb network.

But I have one big problem. Whenever I copy a file larger than about 6GB from Windows to the FreeNas share I get this strange behavior. First the file will appear to be copying, speeds ranging between 105-125 MB/sec (1Gb network). Next the status bar goes back to the beginning and the file starts copying again but at a slower speed. Finally, I get a prompt telling me "You'll need to provide administrator permission to copy this folder". If I click continue it errors out.

This only happens for files over about 6GB. I copied a 5.25GB file and it worked perfectly (along with many smaller files). I've tried setting owner as root, group as domain/domain admins. I've tried setting owner as domain/administrator and group as domain/domain users. I've tried setting permissions as Windows and as Unix. I've tried copying to the share folder and tried copying to a sub folder. Tried copying to a Raid Z2 dataset then tried with striped data set (thinking the problem was a limitation with Raid Z2). Tried with three different releases of FreeNas (a June release, 8/25 and 9/2). Tried without using the domain and signing as root. Tried two different computers copying the file. Nothing seems to help.

Is there some strange permission or setting somewhere that I'm missing for files over 6GB?

FreeNas is running on a desktop PC, Asrock Z97 Pro board, i3-4370 CPU, 16GB RAM, 4x Toshiba 5GB drives.

Thanks in advance for your help!

 

[SweetAndLow]

I'll start out by saying your hardware is a very poor choice for FreeNAS. But it does at least meet the minimum spec. Have you made any modifications to your FreeNAS install? Maybe changed tunables or adding auxiliary parameters? Also can you copy that 6GB file to anything on the network? maybe it is a client problem?

 

[anodos]

Increase CIFS logging verbosity and post relevant log entries from /var/log/samba4 and enclose in [ code ] tags.

Note that the samba project strongly recommends against using the same samba instance as an AD domain controller and as a file server. It is likely that your problem stems from this configuration.

 

[baltilion]

Thanks for the replies.

SweetAndLow, the install is pretty much out of the box. No tunables, no additional parameters, etc. I can copy the 6GB file to an existing Windows Server 2008 share without issue, and I'm getting this odd behavior with two different clients. And I agree with you the hardware is not server grade, but this box will have two users at most for home use.

Andos, I tried something else. I set up FreeNas on another box... this time with 500GB drives in case there is an issue with large hard drives. I did not set up as a domain controller, but I did set up permissions, CIFS share, etc. I'm noticing the same behavior, smaller files copy fine, but 6TB+ does the same thing. I did set the logging, got two files, one had 10k+ lines in it, not sure about the other one but it was also huge.

It just seems really odd that I'm only seeing issues with big files... everything else works flawlessly and fast. Could there be some sort of file quota somewhere that needs to be configured?

Thanks.

 

[anodos]

Thanks for the replies.

SweetAndLow, the install is pretty much out of the box. No tunables, no additional parameters, etc. I can copy the 6GB file to an existing Windows Server 2008 share without issue, and I'm getting this odd behavior with two different clients. And I agree with you the hardware is not server grade, but this box will have two users at most for home use.

Andos, I tried something else. I set up FreeNas on another box... this time with 500GB drives in case there is an issue with large hard drives. I did not set up as a domain controller, but I did set up permissions, CIFS share, etc. I'm noticing the same behavior, smaller files copy fine, but 6TB+ does the same thing. I did set the logging, got two files, one had 10k+ lines in it, not sure about the other one but it was also huge.

It just seems really odd that I'm only seeing issues with big files... everything else works flawlessly and fast. Could there be some sort of file quota somewhere that needs to be configured?

Thanks.

Post /etc/local/smb4.conf.
Most likely not a samba specific issue. Do some network tests with iperf. Post full hardware specs. Perhaps try directly connecting client to server and using known-good cable.

 

[Sonny Williams]

Were you able to figure this out? I am having the same exact issue, but on massively stronger hardware. I already have all my stored data in my Freenas running a Raidz and then backed up offsite as well. I was trying to set it up as my recording repository for my TV, but most of the HD 1 hour long shows are 6GB+. When I try to do a test copy, it goes all the way to 100% and then tells me that I need permission from the Freenas\<username> before continuing, and I can click Try Again or Cancel. If I click Try Again it just keeps repeating the prompt, but if I click cancel it just deletes the file. It literally only happens with these '.ts' 6gb+ files. I can take the HD half hour long shows that run about 3-4GB in size and move them over without any issue at all.

My setup is also using Windows based permissions inside the CIFS. BTW, I wasn't bragging about my hardware being stronger. I was merely pointing out that it doesn't seem to be your hardware causing the issue.

Build FreeNAS-9.3-STABLE-201509282017
Platform Intel(R) Xeon(R) CPU E5-2420 0 @ 1.90GHz (6 Cores Hyper-Threaded)
Memory 16312MB

 

[anodos]

Were you able to figure this out? I am having the same exact issue, but on massively stronger hardware. I already have all my stored data in my Freenas running a Raidz and then backed up offsite as well. I was trying to set it up as my recording repository for my TV, but most of the HD 1 hour long shows are 6GB+. When I try to do a test copy, it goes all the way to 100% and then tells me that I need permission from the Freenas\<username> before continuing, and I can click Try Again or Cancel. If I click Try Again it just keeps repeating the prompt, but if I click cancel it just deletes the file. It literally only happens with these '.ts' 6gb+ files. I can take the HD half hour long shows that run about 3-4GB in size and move them over without any issue at all.

My setup is also using Windows based permissions inside the CIFS. BTW, I wasn't bragging about my hardware being stronger. I was merely pointing out that it doesn't seem to be your hardware causing the issue.

Build FreeNAS-9.3-STABLE-201509282017
Platform Intel(R) Xeon(R) CPU E5-2420 0 @ 1.90GHz (6 Cores Hyper-Threaded)
Memory 16312MB

Post /etc/local/smb4.conf.
Most likely not a samba specific issue. Do some network tests with iperf. Post full hardware specs. Perhaps try directly connecting client to server and using known-good cable.

 

[Sonny Williams]

Post /etc/local/smb4.conf.
Most likely not a samba specific issue. Do some network tests with iperf. Post full hardware specs. Perhaps try directly connecting client to server and using known-good cable.

Ya, I read that up above. However, I can move 3TB of data across so I am certain that it isn't an issue with the network. What more are you looking for on my hardware specs? I posted what I had in regard to Freenas Version, CPU, and Memory. Seeing as I have no issues with anything other than these specific '.ts' 6GB+ files, AND it doesn't crash while moving the file, only after the file is completely done, I am just at a loss. I did, however, verify that I can move other very large files to this server. I found a 17GB file in there that is in the '.mkv' container.

Anywho, I am a very basic user of Freenas. I am mostly a GUI user, so I am not sure how to pull that file for you. I was actually under the impression that Samba was like an Active Directory thing, to which I am not using. This is my home, and I only have 1 user setup on the NAS. Since you are asking for it, I assume that is an incorrect thought.

 

[anodos]

Ya, I read that up above. However, I can move 3TB of data across so I am certain that it isn't an issue with the network. What more are you looking for on my hardware specs? I posted what I had in regard to Freenas Version, CPU, and Memory. Seeing as I have no issues with anything other than these specific '.ts' 6GB+ files, AND it doesn't crash while moving the file, only after the file is completely done, I am just at a loss. I did, however, verify that I can move other very large files to this server. I found a 17GB file in there that is in the '.mkv' container.

Anywho, I am a very basic user of Freenas. I am mostly a GUI user, so I am not sure how to pull that file for you. I was actually under the impression that Samba was like an Active Directory thing, to which I am not using. This is my home, and I only have 1 user setup on the NAS. Since you are asking for it, I assume that is an incorrect thought.

CIFS = Samba
Click on "system -> "advanced" -> "save debug" and post resulting file here.

 

[Sonny Williams]

I have the file. This doesn't expose any passwords or password hashes does it? There are too many files to go through each one to verify. Do you only want the dump.txt under the \ixdiagnose\fndebug\CIFS??

 

[SweetAndLow]

If any system you use can tell you your password you need to stop using that system. Passwords are one way and it shouldn't be possible to get your actual password. There isn't really that many files and you can look through them in about 5min. But I don't think there is anything private in there. It does have a unique identifier but that isn't really used for much.

 

[Sonny Williams]

I'm not quite sure I could scrutinize them in 5 minutes. There are 44 files total and 21 files under the fndebug folder. Some of them have more than 1500 lines. Very easy to miss something with that many files to go through. In any case, I usually scrutinize programs to death when I start using them and I have run across ways to snag password hashes out of programs before. They are encrypted versions of the passwords, but I always worry that if they are in there then the people that programmed the creation of those password hashes could reverse engineer them. Honestly, I was also looking to make sure my public IP wasn't anywhere in these files also, as there is absolutely no need for it to be in the debug logs also. Luckily, I didn't find it anywhere.

I suppose when you spend your time hacking programs, you get used to being paranoid about giving people logs. lol. However, I did go through every line of the CIFS dump.txt and it is perfectly safe to share. I'm not a huge fan of sharing, with the entire public of people who may ever see this thread, the 2 networking folders that expose my internal IP and IP structure. That is why I was asking if he would just want the dump.txt folder under the CIFS, or if there were specific files he wants.

Heck, I'd be okay with messaging the entire to him somehow, but I didn't want to put all of that info up for everyone to snag.

 

[cyberjock]

No. It's safe to post unless you consider your network config to be a secret (ie the ip your server has and the ip of your clients, if they happen to be in the logs).

 

[Sonny Williams]

I apologize for the delay. I appreciate the help so far and completely forgot to come back and upload the log. I have, however, uploaded them now. Thank you. I duplicated my issue again twice just before dumping the logs this time.

 

[solarisguy]

@Sonny Williams, at what time did you start the transfer ? When did the transfer fail?

Did you try to replicate your issue with a quality Intel network card?

P.S.
Why a USB hub...?

 

[Sonny Williams]

I apologize, but what USB hub? I have no USB devices in this machine. Also, this does not appear to be an issue with the network. It just doesn't seem to like these particular files. I can move all other files to this machine without any issues. I can also move these files to other Windows and Linux machines without any issues. It just seems to be these ".ts" files into FreeNas CIFS.

Also, I tried the transfers about 5 minutes before I posted the logs and it fails right as the copy is finishing each time. So, once it hits 0 seconds, it fails with a permissions issue.

 

[solarisguy]

Forget the USB angle for a moment. My USB comment is unrelated to your issue.

I looked at your logs, and I was trying to correlate your transfer to any obvious entry, but nothing was meaningful to me. (On my 9.3, I do not get messages like you do, so it is possible that you have some trivial configuration issue.)

 

[Sonny Williams]

So, the forum was down for a couple days, and it looks like we lost our last 2 posts. I don't have a linux machine here to try to copy to FreeNAS, however, I took the opportunity to copy the file to a Mac, and then copy from the Mac to the Freenas and it moved without an issue.

Any '.ts' file less than 4gb in size goes fine. I can copy the files from Windows to Windows and to Mac and to Linux without issues. I can copy very large files of all types to and from the FreeNAS server without issue, including files much larger than 6gb in size, as long as they aren't a '.ts' file. However, it does appear that when I move these particular files from a Windows machine to the FreeNAS box, I get this weird permissions issue.

 

[solarisguy]

So, the forum was down for a couple days, and it looks like we lost our last 2 posts. I don't have a linux machine here to try to copy to FreeNAS, however, I took the opportunity to copy the file to a Mac, and then copy from the Mac to the Freenas and it moved without an issue.

Any '.ts' file less than 4gb in size goes fine. I can copy the files from Windows to Windows and to Mac and to Linux without issues. I can copy very large files of all types to and from the FreeNAS server without issue, including files much larger than 6gb in size, as long as they aren't a '.ts' file. However, it does appear that when I move these particular files from a Windows machine to the FreeNAS box, I get this weird permissions issue.

That is why I started thinking that Windows has some special treatment for TS files...

 

[Sonny Williams]

It is a digital recording file and is recorded in the format that it is being received. I was hoping to find a way to change the container, but apparently it comes in that way. I'll upgrade from windows 7 to windows 10 later tonight and see if that fixes my issue.