SO. MUCH. WIN.
I wasn't expecting to do this when I built this server, but this FreeNAS box just went from great to #$@!* amazing for an additional investment of $79.
I noted in post #3 that benchmarking this box is fairly pointless because the gigabit connection is an obvious bottleneck - you can transfer files at 110 MB/s (effectively filling the gig link) and the box is just yawning. Using link aggregation (multiple gigabit links) won't make anything faster for a single client because each session will only use one link. The only real option to increase single client speed is 10-gigabit ethernet. I assumed that this would be cost-prohibitive, and for many use cases it still is. But here's the thing - 10G
switches are cost prohibitive, and many clients aren't equipped to use them anyway. Ten gig
NICs, on the other hand, can be gotten cheaply - far more cheaply than I expected. In my home, if I'm doing any heavy-duty work or transferring a lot of data (Blu-ray rips for example), I'm using my primary PC, which is sitting right next to the FreeNAS server. In fact you can see a picture of it in post #3.
I realized that if I could have a 10 Gbps path to FreeNAS for just this one PC, it would be enormously useful. For that, you don't need a switch. You only need two 10G NICs and a way to connect them. After looking around Ebay, I discovered that Chelsio S310E-CR NICs are dirt cheap - seriously, $25 or so! These are supported by FreeNAS and have an SFP+ slot on the card. Ten gig connections often use SFP+ transceivers with optics and fiber, but there's a cheaper way for short runs. You can buy a copper twinax cable with SFP+ modules on each end that plugs straight into these cards. I got a Tripp Lite N280-01M-BK cable off Ebay for $20. By the time I added shipping, my total cost for 2 cards and the twinax cable was $79.
The NICs arrived today so I installed them into the FreeNAS server and my Windows 7 box. The install was practically effortless. FreeNAS immediately recognized the card and gave me a new cxgb0 interface. I downloaded the Windows drivers from
Chelsio's support site and installed them with no trouble. You do have to manually configure both NICs when doing this. These NICs are not part of your regular network - you are creating a very small second network with only 2 machines on it. I arbitrarily chose to use a 192.168.250.0 subnet for this network, with FreeNAS on 192.168.250.1 and my Windows PC on 192.168.250.2. I then disconnected my existing CIFS drives in Windows (which were using my main 192.168.1 network). I mapped new drives via \\192.168.250.1 so that they would actually use the 10G NIC. Time for testing!
I'll cut to the chase - I can now copy files between my PC and FreeNAS at up to 500 MB/s (4 Gbps). This is a huge improvement over what I had, I couldn't be happier. Considering that the SATA3 link to the SSD in my desktop maxes out at 6 Gbps, I'm not at all surprised that the transfer is only 4 Gbps. Don't expect a full 10 Gbps if you do this.
Two caveats if you are thinking about replicating this. First, the S310E-CR is basically an obsolete card. It works fine in FreeNAS 9.3 and Windows 7 x64, but I cannot guarantee that it works with Windows 8 or 10. Second, don't think you are going to get 500 MB/s if your client is using a spinning drive. The only reason this works is that I have a fast SSD on the client side, and 8 spinning drives feeding me data (at ~70 MB/s each) on the server side.
I'll try to update this post over the weekend with a couple of photos.