Backup Exec Job Failing with Access Denied

[mikebob]

We have a FreeNAS server which I occasionally like to backup to tape using BackupExec. The backup has worked in the past but now fails with "Access Denied." The job backs up data and I can restore that data, I just get a generic access denied; it doesn't tell me what was denied access. Two things have changed since the last successful backup, the Backup Exec version, and the FreeNAS version. So I setup some VM's to try and play with the different versions. I went through different versions of BackupExec (versions 16 and 20) and FreeNAS (version 11.1 and 11.2), and nailed down the upgrade that makes the job fail was from FreeNAS 11.2U1 to 11.2U2. I can fully replicate this. I have worked with Veritas on this, and they cant find anything on their end. I've added the FreeNAS to AD vs passthrough, and still the same issue. I've tried enabling the SMB1 support after the 11.2U2 update, which didn't help. I believe there is a different SAMBA version in 11.2U2, maybe that has something to do with it.

Any ideas what might have changed, or is there some special setting I'm missing?

 

[dlavigne]

Were you able to resolve this?

 

[mikebob]

Were you able to resolve this?

no not yet, are you having the problem too?

 

[dlavigne]

If updating to 11.2-U5 doesn't resolve the issue, please create a ticket at bugs.ixsystems.com and post the issue number here.

 

[mikebob]

NAS-102720

 

[anodos]

NAS-102720

Hi Mike in the debug you posted the SMB cilent was requesting an unsupported SMB2 ioctl. I added support for it and posted a recompiled smbd binary in your jira ticket for testing.

 

[RickRepsher]

I know this is an old post, but I am experiencing the same issue. I have reverted back to 11.1 and am in the process of testing our backup jobs. Was this resolved, and if so.... how?

Thanks!
Rick

 

[anodos]

I know this is an old post, but I am experiencing the same issue. I have reverted back to 11.1 and am in the process of testing our backup jobs. Was this resolved, and if so.... how?

Thanks!
Rick

No. I need more information. A pcap of the behavior would be a good place to start.

 

[RickRepsher]

Sorry. I'm not familiar with pcap. Steer me in the direction you are looking for....

Rick

 

[anodos]

Sorry. I'm not familiar with pcap. Steer me in the direction you are looking for....

Rick

You can generate pcap by first running command ifconfig to figure out name of your interface (ex. "igb0"). Once you have done this, you can get a pcap using tcpdump with the command tcpdump -i igb0 -w /tmp/smb.pcap -p -s0 host <ip of client> and port 445. When you're done, hit ^c (control c). Do this if it fails quickly, if you're transferring multiple GB of data before failure, you'll probably run out of space on the tmpfs filesystem. Otherwise, you can run wireshark on the windows client.

 

[RickRepsher]

Ok... I see how that works. But now a couple more questions. Do you want me to run this on both 11.1 and 11.2 for comparison? I can set up a job to try and move the same amount of data under both versions. Also, how do I pull the smb.pcap file from the system? Sorry for all of the questions, but I'm new to the backend of the system.

 

[anodos]

Ok... I see how that works. But now a couple more questions. Do you want me to run this on both 11.1 and 11.2 for comparison? I can set up a job to try and move the same amount of data under both versions. Also, how do I pull the smb.pcap file from the system? Sorry for all of the questions, but I'm new to the backend of the system.

Enable SSH and use winscp to copy it off. Comparitive pcaps would be good.

 

[RickRepsher]

Ok.... I created a small test backup job, and ran it with both 11.1 and 11.2.

11.2 gives me the following error:

>>>>>>>>>>>>>>>>>>>
(Server: "XXXX-MyServer") (Job: "Test Backup") Test Backup -- The job failed with the following error: Access is denied.

Access denied to directory \.
Error reading security data stream.
<<<<<<<<<<<<<<<<<<<

I have attached the two pcap files as a single zip.

I am doing the testing on our replication machine, so I will continue to run 11.1, but if you need me to help with testing, I can flip flop the versions.

Thank you!

 

[RickRepsher]

Update:

I created a machine in our VMware environment to test with. This morning I tested a small backup job on 11.1-U7, which worked fine. I shut the machine down and made a snapshot of it. I then restarted the machine, changed the train to 11-Nightlies, and updated to 11.3. The backup job failed after the update.

I am going to supply the pcap files from both jobs to my contact in tech support, and he has assured me that he will pass them on.

Thank you!

 

[Ronnie Ferrell]

Is there a resolution for this. I upgraded this week from 11.1U7 to 11.3U1 and getting the same issue as OP. When I test the backup credentials in Backup Exec 2012, I get success to the FreeNAS box, but not to the shared volumes on the NAS. See attached image.

 

[RickRepsher]

I was not able to resolve the issue, but to be fair.... I haven't tested it lately. I set up a test machine in VMware, so it was relatively easy to flip back to the previous version. Maybe once the world gets back to normal I'll give it another shot.

 

[anodos]

My vague recollection of looking at backup exec pcaps was that the application first tries to communication with a custom RPC server (which we don't have because they haven't written an agent for FreeBSD), then it appears to fall back to trying to send transactional NTFS ioctls over SMB (which doesn't work because that's a very windows-specific part of the win32 API), then it spins for a while and gives up (11.2+) or succeeds (11.1 or earlier). If I have time I'll look through the pcaps again and maybe something will click.

 

[RickRepsher]

I am currently running 11.1-U7.1 without issues. I would have to recreate my test system... it got wiped from VMware, but that's not a big deal. If you need me to recreate it, I'd be happy to test again. I was trying the nightlies.

Rick

 

[Ronnie Ferrell]

I'm backing back down to 11.1-U7 tonight so that I can pull backups. Once the world gets back to normal I'll give it another go.

 

[js_level2]

Everyone,

I am working with a Veritas employee to identify and correct this issue. Right now all information is internal, so there is no external links to provide. I'll update this thread once we have this figured out. As we have working code to reproduce this issue, this is likely to be fixed sooner than later. I can't make any promises though, as the developers still have to look at the code Veritas provided.