@solarisguy, I have checked the Procurve switch and saw no communication errors whatsoever but noticed that while all ports were configured for "auto" speed, in the status page, 3 out of 24 were reported as 100Mbps while the others were reported as 1000Mbps as they should. On the 3 ones reported as 100Mbps, 2 of them are connecting IPMI cards which I *believe* cant do Gigabit anyways (to be confirmed). The remaining 100Mbps port is the one that connects the freenas server to the network and acc. to the Supermicro site, the port should be Gigabit capable so is the switch...
Obviously I tried to manually set the port's speed to 1000Mbps in the switch's Web interface, and after a few seconds noticed that the speed in the "reports" page of freeNAS went up to 400Mbps...
So 2 questions:
1. Why is using Auto setting on the switch does not use native hardware capability? (in my case Gigabit)
2. I have plenty free ports on the switch, and another unused ports on the freenas server (the server has 4 ports, 2 onboard LAN, one IPMI card with a dedicated port and another LAN port that I believe is shareable...) Could I use something like LAGG to create redundant/additional communication channel between the switch and the freenas server? Would it help? Right now, 7 virtual machines are using the server for storage, while 3 physical clients are using it also for read/write intensive applications...
Also to answer you: No VLANs and the freenas server is directly connected to the procurve switch (so are the other machines).
@rs225, "when multiple NFS clients were communicating over a ZFS dataset" I strongly believe that this is what I am doing... On the dataset I created, I have several NFS mount points that I share with Linux VM's. I am not using freenas to store VM's.
PS: Sorry for the delay in responding to you guys