Read Performance

[Tookerder]

I'm trying to diagnose read performance issues on my FreeNAS. Setup is:

• Supermicro X10 motherboard
• Intel(R) Core(TM) i3-4160 CPU
  
• 32GB ECC RAM
• 2 x 2TB, 2 X 3TB, and 2 x 4TB RAID10 (I believe) - a stripe of three mirrored pairs?
• Jails and two VMs are on a single 500GB SDD that I have replicated snapshots send to main pool each night
• FreeNAS 9.10.1-U4 (reluctant to update since breaks VMs, and I can't remember how I fixed it last time, lol!)

To test my read/write performance I have been using the following command on a dataset with compression turned off:

Code:

dd if=/dev/zero of=testfile bs=1024000 count=50000

for write
and

Code:

dd if=testfile of=/dev/zero bs=1024000 count=50000

for read

My average write performance is around 250 MB/s and average read is around 130 MB/s. My disks are all under 50% full. I can't understand for the life of me the read performance is so much slower than the write performance. I just want to figure out if there is something wrong with my setup.
I've searched the forums and cannot find the answer. I'm sorry if I missed it and thanks for your help FreeNAS champs!

Happy to post any more details as needed.

 

[dlavigne]

Were you able to figure out the reason for this?

 

[Tookerder]

Nope.

 

[Evilpog]

I'm having a similar problem. Write performance is around 250 MB/s (slow for my rig), but read is around 85 MB/s. FreeNAS is virtualized in my case on ESXi 6.5 w/ a hard RAID 10 underneath. I know the RAID 10 is not recommended, but its the hardware given to me so to hedge I do backups daily to external encrypted drives and offsite to AWS S3 w/ versioning enabled.

Code:

xxxx@freenas-node-a:/mnt/Store/xxxx % sudo dd if=/dev/zero of=testfile bs=1024000 count=50000
50000+0 records in
50000+0 records out
51200000000 bytes transferred in 180.478684 secs (283690012 bytes/sec)
xxxx@freenas-node-a:/mnt/Store/xxxx % sudo dd if=testfile of=/dev/zero bs=1024000 count=50000
50000+0 records in
50000+0 records out
51200000000 bytes transferred in 629.344948 secs (81354431 bytes/sec)
xxxx@freenas-node-a:/mnt/Store/xxxx % sudo dd if=testfile of=/dev/zero bs=1024000 count=50000
50000+0 records in
50000+0 records out
51200000000 bytes transferred in 468.401269 secs (109307987 bytes/sec)
xxxx@freenas-node-a:/mnt/Store/xxxx %

I've bench marked other VMs performance they clock around 800 MBs/S reads and 400 MBs/S reads so I know its not the underlying hardware and the RAID is reporting green across the board. Even benchmarked with Windows 2016 Storage Server with comparable results so its definitely something about my FreeNAS 9.10's configuration. Interesting side note is 1st file transfer via CIF is slow, but 2nd will hit around 500 MBs/S and sometimes faster (Yes, my NICs are trunked at layer 2 w/ LACP).



Thoughts? Specs below:

Code:

Hardware/Hypervisor
ESXi 6.5
Dell PowerEdge R710
2 Physical Xeons w/ 6 cores apiece
128 GBs RAM
H700 RAID Card
4x 1 Gbps NICs

FreeNAS VM
2 4 Core vCPUs
16 GBs RAM
6 Disks
	1 External USB (Encrypted)
	1 Boot Disk (FreeNAS Boot)
	4 Storage Disk (Striped)

 

[Holt Andrei Tiberiu]

I'm trying to diagnose read performance issues on my FreeNAS. Setup is:

• Supermicro X10 motherboard
• Intel(R) Core(TM) i3-4160 CPU
  
• 32GB ECC RAM
• 2 x 2TB, 2 X 3TB, and 2 x 4TB RAID10 (I believe) - a stripe of three mirrored pairs?
• Jails and two VMs are on a single 500GB SDD that I have replicated snapshots send to main pool each night
• FreeNAS 9.10.1-U4 (reluctant to update since breaks VMs, and I can't remember how I fixed it last time, lol!)

To test my read/write performance I have been using the following command on a dataset with compression turned off:

Code:

dd if=/dev/zero of=testfile bs=1024000 count=50000

for write
and

Code:

dd if=testfile of=/dev/zero bs=1024000 count=50000

for read

My average write performance is around 250 MB/s and average read is around 130 MB/s. My disks are all under 50% full. I can't understand for the life of me the read performance is so much slower than the write performance. I just want to figure out if there is something wrong with my setup.
I've searched the forums and cannot find the answer. I'm sorry if I missed it and thanks for your help FreeNAS champs!

Happy to post any more details as needed.

Please go in freenas, select storage, select your pool and down on the page select "volume status"
and poste that pic here because i do not understand what type of volume you have created from thoese disk's
also please specify each hdd model.

 

[Holt Andrei Tiberiu]

I'm having a similar problem. Write performance is around 250 MB/s (slow for my rig), but read is around 85 MB/s. FreeNAS is virtualized in my case on ESXi 6.5 w/ a hard RAID 10 underneath. I know the RAID 10 is not recommended, but its the hardware given to me so to hedge I do backups daily to external encrypted drives and offsite to AWS S3 w/ versioning enabled.

Code:

xxxx@freenas-node-a:/mnt/Store/xxxx % sudo dd if=/dev/zero of=testfile bs=1024000 count=50000
50000+0 records in
50000+0 records out
51200000000 bytes transferred in 180.478684 secs (283690012 bytes/sec)
xxxx@freenas-node-a:/mnt/Store/xxxx % sudo dd if=testfile of=/dev/zero bs=1024000 count=50000
50000+0 records in
50000+0 records out
51200000000 bytes transferred in 629.344948 secs (81354431 bytes/sec)
xxxx@freenas-node-a:/mnt/Store/xxxx % sudo dd if=testfile of=/dev/zero bs=1024000 count=50000
50000+0 records in
50000+0 records out
51200000000 bytes transferred in 468.401269 secs (109307987 bytes/sec)
xxxx@freenas-node-a:/mnt/Store/xxxx %

I've bench marked other VMs performance they clock around 800 MBs/S reads and 400 MBs/S reads so I know its not the underlying hardware and the RAID is reporting green across the board. Even benchmarked with Windows 2016 Storage Server with comparable results so its definitely something about my FreeNAS 9.10's configuration. Interesting side note is 1st file transfer via CIF is slow, but 2nd will hit around 500 MBs/S and sometimes faster (Yes, my NICs are trunked at layer 2 w/ LACP).



Thoughts? Specs below:

Code:

Hardware/Hypervisor
ESXi 6.5
Dell PowerEdge R710
2 Physical Xeons w/ 6 cores apiece
128 GBs RAM
H700 RAID Card
4x 1 Gbps NICs

FreeNAS VM
2 4 Core vCPUs
16 GBs RAM
6 Disks
	1 External USB (Encrypted)
	1 Boot Disk (FreeNAS Boot)
	4 Storage Disk (Striped)

Freenas in VM works, but performance will be slower, you can lock the vm's resources, so ESXI won't put that vm in resource standby, meaning, if a VM is not drawing constant performance, ESXI will not alocate resource to it, like CPU power, storage bandwith, memory, etc....

Also you cold flash that H700 to it mode and make a real storage out of it.

http://www.xtremesystems.org/forums...(Dell-H700-LSI-9260-IBM-M5015-Intel-RS2BL080)

 

[Tookerder]

Please go in freenas, select storage, select your pool and down on the page select "volume status"
and poste that pic here because i do not understand what type of volume you have created from thoese disk's
also please specify each hdd model.

View attachment 15726

View attachment 15727

Thanks and apologies for the delay in posting this. Here are the requested items: