Posted on reddit with no real responses yet. Thought I would just go straight to the source.
I have a dell r720 with esxi and freenas 11.3 u2 with a passthrough LSI 9207-8e IT flash to an sc200 array. The esxi host has one data store with 16 900gb 10k in raid 10 (hardware h710). The array has 12 new 600gb SAS 15k drives. Freenas picks them up no problem and multipath worked right out of the gate. Freenas has 50GB of memory allocated to it.
Set up a pool with 6 vdev mirrors (2 drives each) in freenas and started SMB. Connected to it with an ubuntu VM on the same host and vswitch. Only getting ~115mb/s no matter the file size in each direction read or write to or from. Same file, no matter the size (20mb-3gb file) from an ubuntu to ubuntu VM through SMB is at least 135-140 mb/s on the same vswitch. All VMs are using vmx3 network interfaces. Other VMs that are running are basically idle appliances. These VMs should be getting full overhead access
Iperf3 from an ubuntu to freenas is anywhere from 8.5-12gbit/s during the test. From freenas to ubuntu is very consistent at 19gbit/s. From ubuntu to ubuntu it varies from 7.5-14.8gbit/s during the test and yet these are where I get my fastest transfers. I optioned out TSO and LRO on freenas and maybe picked up ~5mb/s to where I am now so no real gains there. No encryption and default compression on the pool. Even doing multiple copy overs of the same file I can see that the ZFS cache is storing it and growing but I still only max out at ~125mb/s from ~115mb/s(pre-cache) outbound.
I thought I be getting way faster speeds than this? I can connect to the freenas SMB via a win10 station over the network, saturate the physical 1g line @ 115mb/s, and get the same speeds I do with a VM running on the same host and vSwitch as freenas in on with WAY more headroom available
I tested with an iScsi on freenas to a windows VM on the same host I can get ~350mb/s first transfer and 700mb/s+ on follow up copies once its in cache. I really need SMB to work though as multiple machines will be connected. Thanks all!
I have a dell r720 with esxi and freenas 11.3 u2 with a passthrough LSI 9207-8e IT flash to an sc200 array. The esxi host has one data store with 16 900gb 10k in raid 10 (hardware h710). The array has 12 new 600gb SAS 15k drives. Freenas picks them up no problem and multipath worked right out of the gate. Freenas has 50GB of memory allocated to it.
Set up a pool with 6 vdev mirrors (2 drives each) in freenas and started SMB. Connected to it with an ubuntu VM on the same host and vswitch. Only getting ~115mb/s no matter the file size in each direction read or write to or from. Same file, no matter the size (20mb-3gb file) from an ubuntu to ubuntu VM through SMB is at least 135-140 mb/s on the same vswitch. All VMs are using vmx3 network interfaces. Other VMs that are running are basically idle appliances. These VMs should be getting full overhead access
Iperf3 from an ubuntu to freenas is anywhere from 8.5-12gbit/s during the test. From freenas to ubuntu is very consistent at 19gbit/s. From ubuntu to ubuntu it varies from 7.5-14.8gbit/s during the test and yet these are where I get my fastest transfers. I optioned out TSO and LRO on freenas and maybe picked up ~5mb/s to where I am now so no real gains there. No encryption and default compression on the pool. Even doing multiple copy overs of the same file I can see that the ZFS cache is storing it and growing but I still only max out at ~125mb/s from ~115mb/s(pre-cache) outbound.
I thought I be getting way faster speeds than this? I can connect to the freenas SMB via a win10 station over the network, saturate the physical 1g line @ 115mb/s, and get the same speeds I do with a VM running on the same host and vSwitch as freenas in on with WAY more headroom available
I tested with an iScsi on freenas to a windows VM on the same host I can get ~350mb/s first transfer and 700mb/s+ on follow up copies once its in cache. I really need SMB to work though as multiple machines will be connected. Thanks all!
