CIFS slow to delete?

[VAHokies]

I've got a CIFS share setup. And some directories have thousands of files. When I try to delete the whole directory via the command line it churns away but then I get 'The directory is not empty'. It is able to delete some files/directories but seems like since so many it gets tripped up. And I'm thinking it is because things are slow so it is almost like the process is too fast and chokes.
Here is my dos command line code:
del /f/s/q %1 > nul
rmdir /s/q %1


FWIW - I can delete same directory manually from Windows Explorer and it works fine.
Also, I can use 'rm -rf' from Freenas shell and that works just fine.

Any ideas?

thanks

 

[gpsguy]

Thiumbs.db might be getting in the way of your DOS command.

You can see them with: dir /ah /a thumbs.db

Oops, should have been: dir /ah /s thumbs.db

Sent from my phone (fixed typo)

 

[VAHokies]

I double-checked and no thumbs.db files.
And I've replicated the folder on a NTFS disk and I can delete things from DOS command fine.

 

[cyberjock]

What files aren't getting deleted? For example, ome Windows programs will install a program to make thumbnails of movies, but the actual database of thumbnails is stored locally. So they won't get deleted because they'll be in use, but you won't have thumbs.db either. Welcome to Windows!

The bottom line is that most likely the problem is that the files are in use somewhere. So your goal should be to figure out what is keeping those files open. There's some commands you can run from the CLI to list open files and who has them open, and those might help. I don't know those commands or I'd provide them, but a google search has found them for others in the past. When you delete stuff from the CLI you are bypassing the samba locks. Since you are able to delete them from the CLI its pretty reasonable to assume that it must not be a local file system problem. So start checking out your remote systems to see what is going on.

 

[gpsguy]

The command should have been: dir /ah /s thumbs.db

I mistyped it on my phone yesterday. Or, you could do: dir /ah /s *.* to search for all the hidden files.

I double-checked and no thumbs.db files.

 

[VAHokies]

I don't think the files are locked because when when I try to repeat the process the number of files it deletes varies.

Ok, I tried another test. I created a single directory with a variety of files only (no subfolders). It had 390 files - none of which are read-only or hidden. On this directory I ran DEL *.* it returned no errors but 274 files remained so it only deleted 116 files. I kept running DEL *.* until it deleted all files - it took four runs to finally delete all files- see below.

390 files
274 files remaining after run 1
149 files remaining after run 2
25 files remaining after run 3
0 files remaining after run 4


I copied the same 390 files to a NTFS drive and then ran DEL *.* and as expected it deleted all files in one try.

Any ideas?

 

[cyberjock]

I'm really not sure what value there is from the test you just ran. If I remember correctly, del *.* stops on file lock errors for network shares. So if it starts deleting them in alphabetical order and gets to "C" and "Charlie and the Chocolate Factory" is currently being thumbnailed because its new you might only lose 30 files. Then next time it might go to "S" for Star Trek because you recently put a Star Trek movie in there. Surprise, a different number of files were deleted, we got something that kinda resembles your results, and you proved nothing except you happened to delete files in order.

And that's not to say that you didn't have 274 files locked at the time of the first run. Samba, last time I read about it, opportunistically removes locks. That is, it doesn't aggressively unlock files. It unlocks files when CPU usage permits. But, when a user attempts to lock a file that is already locked there is a check to verify that the lock is valid with the other user still. If not, its freed, then immediately relocked with the other user.

I'm really not seeing how this is scientific at all or even useful diagnostically for determining your problem. You should look for diagnostic information related to Samba and see if you can actually prove that the files were locked or not. Doing anything on the client itself is a waste of time at this point since the issue is manifesting itself on the client. Clearly this is something to do with the relationship between the client and server and not a server problem by itself. If it were a server problem itself there would be lots of people with your problem, and you're the first I've heard of this year.

 

[VAHokies]

I'm not trying to run a diagnostic by doing DEL *.* but rather just trying to show a concrete example of the problem.
I'll try to look into the locks but seems odd that I can't delete from the command line but I can delete the same files via Windows Explorer just fine albeit a little slower. Almost seems like the command line delete is running faster than CIFS can keep up so it trips over some files in the process.

I'll keep digging.
thx

 

[cyberjock]

I'm not trying to run a diagnostic by doing DEL *.* but rather just trying to show a concrete example of the problem.
I'll try to look into the locks but seems odd that I can't delete from the command line but I can delete the same files via Windows Explorer just fine albeit a little slower. Almost seems like the command line delete is running faster than CIFS can keep up so it trips over some files in the process.

I'll keep digging.
thx

Ah, ok. I thought you were trying to do it for diagnostic reasons.

Windows Explorer does some things different than the command line. When I examined this in XP years ago XP was locking the file, then deleting it, then doing something to try to flush the disk cache before the next delete command would begin. Conversely the del command would just basically try to delete the entry from the file system with no regards to locking the file or trying to flush the disk. So if you have to delete 10k worth of files, del was faster.

Don't confuse the local locking with the Samba locking. They are different. But, if you are deleting from a network share than the local locking will result in the file being locked in Samba as well.

My understanding of the research I did years ago was that del wasn't really designed for ntfs, and wasn't ever corrected to be protective of the file system. It was left to the command interpreter to make things behave(or not behave). Real mess is what it turned out to be. Most people, if they ran a command to delete 10k files and 5k were left after a sudden loss of power would find it acceptable and just continue the deletion anyway. So not a big deal if it didn't get to complete.

I will say that stuff deleted FAR faster in Linux, even in the GUIs! Perhaps even as fast as del.

 

[John M. Długosz]

Here is my dos command line code:
del /f/s/q %1 > nul
rmdir /s/q %1

I've used the replacement command shell 4DOS, then 4NT, which became Take Command, for what seems like forever.

Code:

del /sxyqz dirname

really really does the trick, no residue, no backtalk.