CIFS Gigabit LAN speed slow freenas 9.1

[Verwooks]

Hey, Im new to freenas and these forums. But I will try to make a decent thread as a freebsd newbie.

I just wanna use my Freenas as a personal fileserver on LAN for backup, media sharing and transmission.

Im getting 25-35 megabyte transfer speeds using windows exlorer on windows 7 and windows 8 clients through cifs shares. Sometimes I can see 110+ mb per second for 10 seconds, then suddenly a drop in speed to around 20-25. Im testing transferring a large file, 4 gigabyte.
This happens whenever i transfer from a shared jail folder or outside jail.

Eks: http://ge.tt/6tsFpkq/v/0?c

Using FTP tranfers from nas to workstation I get 112-115 megabytes per second constantly.

Using FTP tells me:
Disk read/write is OK
NIC config is OK
CPU is OK
USB flashdrive can't be bottlenecking?

I have tested transferring files from windows 7 to my windows 8 workstation with good speeds.

I have tried different configs and NIC on both workstations.
To mention some
Enabling jumbo packets
Disabling large send offload v2 ipv4.
++
Without luck, but settings are back at default.

Tried realtek and Intel NIC.

What kind of speeds can I expect in CIFS shares? Im sure I can get more if I tweak config? Is this a cache problem?

Will CIFS use more cpu than FTP? The problem resides somewhere. Cacti statistics never show CPU load above 50%.

I think the problem resides with CIFS and not hardware capabilites here.

How can I solve this? Transfer speeds from 70 megabytes+ should be possible?

Please tell if you need more info.

Below is config and hardware specs.

CIFS GUI Config
Enabled:
Lokal master
Time server for domain
EA Support
Support Dos file attributes
Unix Extensions
Enable AIO
Zeroconf share discovery
Hostnames lookup

Not adjusted Min and max AIO write/read size.

Hardware Freenas Box:
Freenas 9.1 x64
EVGA 680i SLI Motherboard
Intel(R) Core(TM)2 Duo CPU E6750 @ 2.66GHz
4 gig Corsair DDR2 ram
3 tb WD red drive
Freenas is running on a 4 gigabyte USB flash drive (Im not sure how this flash usb drive is performancewise, and if its usb2 or usb1.)

Network:
CAT5e network cables (tried swapping cables as well)
Cisco SB 100 Series 5-port Gigabit

Windows workstations:
They are new, and my newest have 12gb ram, 2 ssd disks and a Quad core @4,6 ghz. on z77 chipset.

ifconfig output:


nfe0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=c2099<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWTSO,LINKSTATE>
ether 00:04:4b:05:21:94
inet 192.168.10.109 netmask 0xffffff00 broadcast 192.168.10.255
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nfe1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=c219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO,LINKSTATE>
ether 00:04:4b:05:21:95
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet autoselect (none)
status: no carrier
ipfw0: flags=8801<UP,SIMPLEX,MULTICAST> metric 0 mtu 65536
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x6
inet 127.0.0.1 netmask 0xff000000
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 02:3b:39:7b:24:00
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: epair1a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 9 priority 128 path cost 2000
member: epair0a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 8 priority 128 path cost 2000
member: nfe0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 3 priority 128 path cost 20000
epair0a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 02:09:11:00:08:0a
inet6 fe80::9:11ff:fe00:80a%epair0a prefixlen 64 scopeid 0x8
inet 0.0.0.0 netmask 0xff000000 broadcast 255.255.255.255
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active
epair1a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 02:88:29:00:09:0a
inet6 fe80::88:29ff:fe00:90a%epair1a prefixlen 64 scopeid 0x9
inet 0.0.0.0 netmask 0xff000000 broadcast 255.255.255.255
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active

HERE IS A PARM OF MY smb.conf

testparm /usr/local/etc/smb.conf
Load smb config files from /usr/local/etc/smb.conf
max_open_files: increasing sysctl_max (11095) to minimum Windows limit (16384)
rlimit_max: increasing rlimit_max (11095) to minimum Windows limit (16384)
Processing section "[NAS]"
Loaded services file OK.
Server role: ROLE_STANDALONE
Press enter to see a dump of your service definitions

[global]
dos charset = CP437
server string = FreeNAS Server
map to guest = Bad Password
obey pam restrictions = Yes
smb passwd file = /var/etc/private/smbpasswd
private dir = /var/etc/private
syslog only = Yes
max log size = 10
max protocol = SMB2
max xmit = 65535
time server = Yes
deadtime = 15
hostname lookups = Yes
load printers = No
printcap name = /dev/null
disable spoolss = Yes
dns proxy = No
idmap config * : backend = tdb
create mask = 0666
directory mask = 0777
aio read size = 8192
aio write size = 8192
directory name cache size = 0
store dos attributes = Yes
strict locking = No

[NAS]
path = /mnt/WDRED3TB
read only = No
inherit acls = Yes
inherit owner = Yes
veto files = /.snap/.windows/.zfs/
map archive = No
map readonly = no
vfs objects = zfsacl
nfs4:chown = yes
nfs4:acedup = merge
nfs4:mode = special



diskinfo -t ada0

[root@NAS] ~# diskinfo -t ada0
ada0
512 # sectorsize
3000592982016 # mediasize in bytes (2.7T)
5860533168 # mediasize in sectors
4096 # stripesize
0 # stripeoffset
5814021 # Cylinders according to firmware.
16 # Heads according to firmware.
63 # Sectors according to firmware.
WD-WCC1T1271692 # Disk ident.

Seek times:
Full stroke: 250 iter in 8.367368 sec = 33.469 msec
Half stroke: 250 iter in 5.686325 sec = 22.745 msec
Quarter stroke: 500 iter in 9.827521 sec = 19.655 msec
Short forward: 400 iter in 3.618002 sec = 9.045 msec
Short backward: 400 iter in 2.710177 sec = 6.775 msec
Seq outer: 2048 iter in 0.180063 sec = 0.088 msec
Seq inner: 2048 iter in 0.150705 sec = 0.074 msec
Transfer rates:
outside: 102400 kbytes in 0.723809 sec = 141474 kbytes/sec
middle: 102400 kbytes in 0.860068 sec = 119060 kbytes/sec
inside: 102400 kbytes in 1.426941 sec = 71762 kbytes/sec


smartctl -a -q noserial /dev/ada0


smartctl 6.1 2013-03-16 r3800 [FreeBSD 9.1-STABLE amd64] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family: Western Digital Red (AF)
Device Model: WDC WD30EFRX-68AX9N0
Firmware Version: 80.00A80
User Capacity: 3,000,592,982,016 bytes [3.00 TB]
Sector Sizes: 512 bytes logical, 4096 bytes physical
Device is: In smartctl database [for details use: -P show]
ATA Version is: ACS-2 (minor revision not indicated)
Local Time is: Sun Sep 1 19:48:48 2013 CEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

Warning! SMART Attribute Data Structure error: invalid SMART checksum.
Warning! SMART Attribute Thresholds Structure error: invalid SMART checksum.
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
See vendor-specific Attribute list for marginal Attributes.

General SMART Values:
Offline data collection status: (0x00) Offline data collection activity
was never started.
Auto Offline Data Collection: Disabled.
Total time to complete Offline
data collection: ( 0) seconds.
Offline data collection
capabilities: (0x00) Offline data collection not supp orted.
SMART capabilities: (0x0000) Automatic saving of SMART data i s not implemented.
Error logging capability: (0x00) Error logging supported.
General Purpose Logging supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_ FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 0
3 Spin_Up_Time 0x0027 179 178 021 Pre-fail Always - 6025
4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 13
5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0
7 Seek_Error_Rate 0x002e 100 253 000 Old_age Always - 0
9 Power_On_Hours 0x0032 100 100 000 Old_age Always - 242
10 Spin_Retry_Count 0x0032 100 253 000 Old_age Always - 0
11 Calibration_Retry_Count 0x0032 100 253 000 Old_age Always - 0
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 13
192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 10
193 Load_Cycle_Count 0x0032 200 200 000 Old_age Always - 2
194 Temperature_Celsius 0x0022 117 115 000 Old_age Always - 33
196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0
197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 0
198 Offline_Uncorrectable 0x0030 100 253 000 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0
200 Multi_Zone_Error_Rate 0x0008 100 253 000 Old_age Offline - 0
1 Raw_Read_Error_Rate 0x037b 000 001 051 Pre-fail Always FAILI NG_NOW 0

Warning! SMART ATA Error Log Structure error: invalid SMART checksum.
SMART Error Log Version: 1
No Errors Logged

Warning! SMART Self-Test Log Structure error: invalid SMART checksum.
SMART Self-test log structure revision number 1
No self-tests have been logged. [To run self-tests, use: smartctl -t]


Selective Self-tests/Logging not supported

 

[cyberjock]

Well, you're using old hardware and only 4GB of RAM. If you read the manual it will tell you that with less than 8GB of RAM ZFS is unreliable, and less than 4GB of RAM you will lose prefetch(which tanks performance).

If you aren't using UFS, don't expect record setting performance. It has no advanced features that ZFS has to get amazing performance.

I have no idea what program you are using to copy files to/from the server. But if that isn't the actual copying feature that is part of Windows Explorer I wouldn't use it. One guy I helped spent 3+ hours trying to troubleshoot weird slow performance. Turns out that TurboCopy(or whatever 3rd party program he was using) that claimed to provide faster network transfers was entirely responsible for the problems.

 

[Verwooks]

Using windows explorer, nothing 3rd part.

And it seems like 2 of the harddrives on my computer couldn't handle more than like average 60 megabytes when I tested with HD tune. I get around 100mb/s copying from and to SSD disks.

So i kinda screwed up, but found the cause at least.

Im gonna look into UFS anyway. Don't have any old DDR2 ram laying around.

 

[Yatti420]

If you can't get 10-16 GB miniumum in I wouldn't use ZFS.. I have a second server that needs to have everything pulled and a new UFS mirror created as its 32 bit and just can't handle a ZFS mirror with speed..

Edit: In fact I'd probably limit 32 bit to UFS.. Not sure why ZFS is allowed on a 32bit implementation..

 

[HAL 9000]

If you read the manual it will tell you that with less than 8GB of RAM ZFS is unreliable, and less than 4GB of RAM you will lose prefetch(which tanks performance)

So why FTP transfer is unaffected?

Using FTP tranfers from nas to workstation I get 112-115 megabytes per second constantly.

 

[cyberjock]

So why FTP transfer is unaffected?

Does FTP pre-fetch its own data? Does Samba? Does NFS?

Does FTP need lots of RAM to function? Does Samba? Does NFS?

The short is you aren't going to get much help or sympathy with any performance issues in this forum if you are choosing to use 1/2 of the recommended minimum RAM for ZFS.

 

[Ryback]

Don't have any old DDR2 ram laying around.

I happened to pick up an old computer yesterday with ddr2 ram. I'm only planning on using the case and am happy to send you the RAM if you want it, and live somewhere in Europe (due to shipping cost). I'm not sure how gigs it is. Probably 2x2 or 2x4 non ECC. The computer was scrapped because the mobo was fried and the RAM might be bad too. But if you still want it, send me your address on a PM. Just don't blame me if your data gets corrupted because of the RAM :p

 

[Ryback]

I happened to pick up an old computer yesterday with ddr2 ram. I'm only planning on using the case and am happy to send you the RAM if you want it, and live somewhere in Europe (due to shipping cost). I'm not sure how gigs it is. Probably 2x2 or 2x4 non ECC. The computer was scrapped because the mobo was fried and the RAM might be bad too. But if you still want it, send me your address on a PM. Just don't blame me if your data gets corrupted because of the RAM :p

 

[robdeep]

I can sympathize with the original post, although I have been running 16gb of RAM in my FreeNas box since day one. The best I've been able to get when copying via CIFS/SMB ~60mb/sec, although it fluctuates. FTP transfers reliably saturate the 1 gig ethernet interface going to the box. If anyone is still offering feedback, I would love to hear it. My buddy just thinks its due to all the overheard of copying via CIFS.

 

[Yatti420]

Other than ram.. (I put as much as I could in my NAS box DDR3)..

One of the biggest things I've noticed is size of pool -How much of is is consumed? Under 10% free? Expect drastic performance cuts.. These are only single drives.. ZFS doesn't like not having some free space.. I've noticed speed decreases since updating a few drivers on my clients and moving to latest release.. It's still fast for me.. Fast clients go 80-110Mb/s no problems..

 

[cyberjock]

I can sympathize with the original post, although I have been running 16gb of RAM in my FreeNas box since day one. The best I've been able to get when copying via CIFS/SMB ~60mb/sec, although it fluctuates. FTP transfers reliably saturate the 1 gig ethernet interface going to the box. If anyone is still offering feedback, I would love to hear it. My buddy just thinks its due to all the overheard of copying via CIFS.

If you are having a problem you should make a new thread, include hardware and software, etc. As is, you've made one post and the only things I know:

1. You have FreeNAS
2. It's slow with CIFS but fast with FTP.
3. You have Gb ethernet.

Not much to go on.... but make a new thread if you are having problems.

 

[robdeep]

If you are having a problem you should make a new thread, include hardware and software, etc. As is, you've made one post and the only things I know:

1. You have FreeNAS
2. It's slow with CIFS but fast with FTP.
3. You have Gb ethernet.

Not much to go on.... but make a new thread if you are having problems.

Sorry, I wasn't sure what would be appropriate.

I have a AMD-E350 embedded system. 4, 3tb WD Red Drives. I've read that the 4k sector size may help with my issue. I was not aware of that detail when I first formatted my pool with only the 512k sector size.

 

[gpsguy]

With that processor, "~60mb/sec" is the probably the best you'll see with CIFS.

Get a fast CPU (and new motherboard), if you want faster speeds.

I have a AMD-E350 embedded system.

 

[robdeep]

Thanks. After all I've tried, I think that's an accurate assessment.

 

[gpsguy]

If you look at the thread regarding hardware suggestions you'll see that a fast processor is recommended for use with Samba.

I'm in the same boat you are - but I knew it going in. Given my needs, I'm okay with the performance.

 

[cyberjock]

.. and I said to make a new thread... /sigh. I'm going to close this thread as the original poster's discussion ended over 45 days ago.