Slow deletes from FreeNAS CIFS share.

mangyDOG

Cadet
Joined
Mar 29, 2018
Messages
4
I have build a FreeNAS based storage server which is working very well providing file shares to my various windows based computers, except when it comes to deleting (or cutting & pasting) files from the NAS.

Here are a series of screenshots to illustrate the issue:

Small file copy from NAS to PC:
FreeNAS_SmlFileCopy.jpg


Small file CUT from NAS to PC:
FreeNAS_SmlFileCut.jpg


File Delete from NAS:
FreeNAS_SmlFileDelete.jpg


The NAS is connected via 10Gb ethernet and when I use a windows computer with a NVMe drive I have transferred large files to and from it at around 800MB/s. The only issue I have is when it comes to removing files from the NAS share (either delete or cut) then it just tanks. If use the console and delete files directly it only takes a few seconds to complete, but that is not a solution I can use for normal operations.

The NAS is currently running 11.1-U2

Hardware consists of:
AMD Ryzen5 1600 six core CPU
32Gb Unbuffered ECC RAM
2*Sandisk USB thumb drives for OS
8*Seagate enterprise 3Tb 7200rpm SATA hard-drives (single ZFS-Z2 volume with LZ4 compression)
1 * Samsung 960EVO 250Gb NVMe SSD (ARC)
1 * Intel P3600 400Gb PCIe SSD (L2ARC)
1 * Intel X520-DA2 with intel fibre transceiver.


I look forward to any assistance :smile:
regards,
mangyDOG.
 

DrKK

FreeNAS Generalissimo
Joined
Oct 15, 2013
Messages
3,630
Your system seems like the posterchild for a system that should not have an L2ARC. Are you sure you need one?

Also, is this purely a SMB/CIFS issue? If you go to the command line and delete the 20000 files with a simple "rm *" (or whatever), does it still operate this slowly?
 

mangyDOG

Cadet
Joined
Mar 29, 2018
Messages
4
Your system seems like the posterchild for a system that should not have an L2ARC. Are you sure you need one?

Also, is this purely a SMB/CIFS issue? If you go to the command line and delete the 20000 files with a simple "rm *" (or whatever), does it still operate this slowly?
1) I haven't tried the system without the L2ARC cache drive, is there a process to remove the drive?

2) RM from the command line works as expected with all files deleted in a few seconds.

Thanks
 

DrKK

FreeNAS Generalissimo
Joined
Oct 15, 2013
Messages
3,630
Deleting 20000 files from Windows SMB/CIFS, even on a local drive, takes quite a bit of time in many cases, as does transfering a whole boatload of small files. I don't know what to tell you. But if the problem disappears at the command line, and when using, say, FTP to transfer files, then the issue is the general craptasticness of SMB.

Unless someone else has a more constructive idea?
 

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
Are the 20k items in the same folder?
 

Mirfster

Doesn't know what he's talking about
Joined
Oct 2, 2015
Messages
3,215
IMHO everything from the GUI in Windows is slow. Have you tried simply doing this stuff from within a CMD Window?

Like using RoboCopy or even RD (to delete entire directories, subfiles and subfolders)?

Also, FWIW (and if my old mind is recalling correctly); when you "Copy and Paste" the file's permissions are inherited from the destination. However, when you "Cut and Paste" Windows will try to maintain the file's perms on the destination. Not 100% if that is a possible issue, but figured I would toss that out there as well.
 

mangyDOG

Cadet
Joined
Mar 29, 2018
Messages
4
Are the 20k items in the same folder?
One top level folder and multiple sub-folders.

The screen shots are just an example I used. The issue persists with all delete operations. If there are only ten files in a folder, it will still take over 10 seconds (< 1 file delete operation per second) to delete the folder.

I am aware that deletes across the network using SMB are slow, but on my much older NAS units (Synology RAID6 with EXT4 and QNAP RAID6 with EXT3) I get at least >50 delete operations per second and I was expecting that with the one based on FreeNAS.
 

mangyDOG

Cadet
Joined
Mar 29, 2018
Messages
4
IMHO everything from the GUI in Windows is slow. Have you tried simply doing this stuff from within a CMD Window?

Like using RoboCopy or even RD (to delete entire directories, subfiles and subfolders)?.

Using RM from the FreeNAS console works as expected with large numbers of files deleted within a few seconds. I just tested using a Windows command prompt "del * /s /q" and it was running at about 1 delete per second.
 

TwistMaitre

Cadet
Joined
May 22, 2019
Messages
1
Did you ever find a solution to this mangyDOG? I have a a pretty similar system and an identical problem.
 

anodos

Sambassador
iXsystems
Joined
Mar 6, 2014
Messages
9,543
From MacOS client (macbook pro with gigabit ethernet) on FreeNAS master, I can delete a directory containing 8200 objects in about 14 seconds, which seems acceptable. When I have time I'll take a look at deletions on 11.2.
 

anodos

Sambassador
iXsystems
Joined
Mar 6, 2014
Messages
9,543
I changed defaults for 11.3 to improve delete speeds in MacOS. The slowness is a direct result of the type of network traffic generated by the MacOS client. Try following: (1) create dataset that is case-insensitive (2) enable ixnas and remove zfsacl and zfs_space from VFS objects (3) set share auxiliary parameter "case sensitive = true". Restart samba and re-do your test.
 

kurbis

Cadet
Joined
Sep 25, 2019
Messages
5
I changed defaults for 11.3 to improve delete speeds in MacOS. The slowness is a direct result of the type of network traffic generated by the MacOS client. Try following: (1) create dataset that is case-insensitive (2) enable ixnas and remove zfsacl and zfs_space from VFS objects (3) set share auxiliary parameter "case sensitive = true". Restart samba and re-do your test.
Thank you for the reply. 1. I have windows clients, not MacOS. This OS have same problems? How can i create new pool without removing old pool? 2. When i removed zfacl and zfs_space my cifs permissions stopped work :(
 

kurbis

Cadet
Joined
Sep 25, 2019
Messages
5
I found resolution fo me! Just disable acl_tdb and acl_xattr . I hope it will not destroy my ntfs permissions on the folders :)
 

anodos

Sambassador
iXsystems
Joined
Mar 6, 2014
Messages
9,543
I found resolution fo me! Just disable acl_tdb and acl_xattr . I hope it will not destroy my ntfs permissions on the folders :)
acl_xattr does not work on FreeBSD (on ZFS). acl_tdb works except when it's crashing the box. Better not to use those ever and just stick to zfsacl or ixnas.
 
Top