Hi all, I'm new to this thread, but I have the same problem: directory bowsing of a big directory is very slow when accessed via CIFS (via windows client or via commandline smbclient from ubuntu). Here is my configuration:
- The directory is composed of 19300 subdirectories
- Freenas is 8.2.0-RELEASE-p1-x64
- Hardware: HP Proliant Microserver N40L with 8GB RAM
adding "max protocol = SMB2" did not solve the problem.
Browsing the directory takes 5 minutes via CIFS, but it takes only few seconds with ls -la lauchend internally in a ssh console.
I played around with top, truss and zpool iostat.
The only process in freenas doing some activity during a smbclient "dir" is smbd (for a maximum of 2 or 3% of cpu load!). The disk activity LED is always on.
truss -D -p _pid_of_smbd_ reports the following
Code:
0.019411824 stat("Lavori/gamatec 40+15x34 men 80 gns D 01-03-2010",{ mode=drwxrwxr-x ,inode=84164,size=5,blksize=4096 }) = 0 (0x0)
0.128582195 extattr_get_file(0x803cbee50,0x1,0x1692a7b,0x0,0x0,0x0) = 56 (0x38)
0.000044977 extattr_get_file(0x803cbee50,0x1,0x1692a7b,0x7fffffffdae0,0x100,0x0) = 56 (0x38)
0.000011245 fcntl(13,F_SETLKW,0x7fffffffd9f0) = 0 (0x0)
0.000010755 fcntl(13,F_SETLKW,0x7fffffffda50) = 0 (0x0)
0.305608883 stat("Lavori/lani 32+13x39 jnx milano 20-03-2012",{ mode=drwxrwxrwx ,inode=233551,size=3,blksize=4096 }) = 0 (0x0)
1.183616283 extattr_get_file(0x803cbee50,0x1,0x1692a7b,0x0,0x0,0x0) = 56 (0x38)
0.000062089 extattr_get_file(0x803cbee50,0x1,0x1692a7b,0x7fffffffdae0,0x100,0x0) = 56 (0x38)
0.000020534 fcntl(13,F_SETLKW,0x7fffffffd9f0) = 0 (0x0)
0.000011733 fcntl(13,F_SETLKW,0x7fffffffda50) = 0 (0x0)
...
0.012985868 stat("Lavori/tavns & co. 25+11x31 kissa 18-06-2007",{ mode=drwxrwxr-x ,inode=46253,size=6,blksize=4096 }) = 0 (0x0)
0.000053778 extattr_get_file(0x803cb0050,0x1,0x1692a7b,0x0,0x0,0x0) = 56 (0x38)
0.000082622 extattr_get_file(0x803cb0050,0x1,0x1692a7b,0x7fffffffdae0,0x100,0x0) = 56 (0x38)
0.000017111 fcntl(13,F_SETLKW,0x7fffffffd9f0) = 0 (0x0)
0.000013689 fcntl(13,F_SETLKW,0x7fffffffda50) = 0 (0x0)
3.287608551 stat("Lavori/winnen-pa- 41+12x35 jnx maruneo PA 08-03-2010",{ mode=drwxrwxr-x ,inode=2801,size=3,blksize=4096 }) = 0 (0x0)
0.000050355 extattr_get_file(0x803c0a910,0x1,0x1692a7b,0x0,0x0,0x1) = 56 (0x38)
0.000062578 extattr_get_file(0x803c0a910,0x1,0x1692a7b,0x7fffffffdae0,0x100,0x1) = 56 (0x38)
...
0.000088978 stat("Lavori/herzannma 20+10x27 raxele 20-11-2009",{ mode=drwxrwxr-x ,inode=94037,size=4,blksize=4096 }) = 0 (0x0)
0.010090668 extattr_get_file(0x803c9cfd0,0x1,0x1692a7b,0x0,0x0,0x1) = 56 (0x38)
2.044919815 extattr_get_file(0x803c9cfd0,0x1,0x1692a7b,0x7fffffffdae0,0x100,0x1) = 56 (0x38)
0.000035200 fcntl(13,F_SETLKW,0x7fffffffd9f0) = 0 (0x0)
0.000019555 fcntl(13,F_SETLKW,0x7fffffffda50) = 0 (0x0)
As you can see, there are simple syscall (like "stat" or extattr_get_file) that sometimes takes up to 2-3 seconds for some subdirectories. These subdirs only contains 1 to 5 files, usually large 5 to 10MB each one.
But the most weird thing is that... the system is WRITING to the filesystem at 3MBytes/sec ??? This test is done without any client connected to CIFS or other protocols, and zfs writings starts exactly when I type "dir" on smbclient and stops when dir is done.
Code:
[root@pyprel] ~# zpool iostat 1
capacity operations bandwidth
pool used avail read write read write
---------- ----- ----- ----- ----- ----- -----
Volume1 3.06T 2.37T 42 82 193K 632K
Volume1 3.06T 2.37T 0 525 0 4.09M
Volume1 3.06T 2.37T 7 287 63.9K 1.84M
Volume1 3.06T 2.37T 0 409 0 3.09M
Volume1 3.06T 2.37T 3 480 20.0K 3.71M
Volume1 3.06T 2.37T 0 324 0 2.54M
Volume1 3.06T 2.37T 0 313 0 2.40M
Volume1 3.06T 2.37T 4 321 40.0K 2.11M
Volume1 3.06T 2.37T 0 505 0 3.90M
Volume1 3.06T 2.37T 0 478 0 3.69M
Volume1 3.06T 2.37T 0 294 0 2.25M
Volume1 3.06T 2.37T 0 261 0 2.04M
Volume1 3.06T 2.37T 0 328 0 2.17M
Volume1 3.06T 2.37T 0 322 7.99K 2.36M
Volume1 3.06T 2.37T 0 420 0 3.28M
Volume1 3.06T 2.37T 0 593 0 4.64M
Any idea ?
I hope this could also help in solve other people's directory browsing problems :)
Thank you
Giovanni
EDIT and edit again:
I disabled "Support DOS File Attributes" into the CIFS options and the directory browsing is now VERY FAST. Note that "EA support" was already disabled.
After disabilng "Support DOS File Attributes", I lost all the icons that were associated to some directories: icons are configured in a Desktop.ini file that needs READONLY attribute on the parent dir to be used by explorer.
So the real question are now a bit different
a) why is freenas so slow when "Support DOS File Attributes" is enabled
b) Why zpool iostat reports write activity ?
Thank you again
EDIT/UPDATE:
I removed "atime" from zfs, and now browsing from CIFS is faster again with "Support DOS File Attributes".
So... should I think zfs is not caching atime updates ?