Configuring Gigabit LAN and MTU

[Laziali]

Hello,

I just added a gigabit nic to my nas, and i was reading the documentation and trying to figure out how to get things working properly. My conclusion was to activate the MTU 9000 option.

My first guessing is to know if the jumbo pack is supported on my new nic (ENCORE ENLGA-1320), and yes, is supported.

My second guessing is to find out how to activate this feature, and i found this on the documentation: in networking, options, add the value "mtu 9000".

I wanted to test it copying a big file, just to ensure that everything was ok, and i copied a 2gb zip file from the nas to my computer.....veeeeeeeery fast!. Now i tried backwards, copying the same file from my computer to the nas....dead slow. WHY?????????????

I just simply read many forums and found nothing.....if anyone here knows how to solve this, i will appreciate the help.

Thanks,

Jose.

 

[louisk]

So, in order to take advantage of jumbo frames, you need to have the following:
freenas configured
switch configured
clients configured

Did you get all of the above done? My guess is that your client is still using an MTU of 1500.

 

[Laziali]

Thanks for answering. I just wanted my nas to work faster over the lan, thats all. I have a small network controlled by a Dlink DIR-655 gigabit router, with a Dlink DGS-1005D connected to it (here's where the nas is attached to). All the computers have gigabit nics. I read that the only way to make it work is changing the MTU to 9000.

Now, answering your question, the only place where i configured the MTU 9000 was on the Freenas.

 

[louisk]

If you don't configure the client to use jumbo frames, it won't. I don't know of any system (server, switch, etc..) that will auto-detect the ability to use jumbo frames and do it. It needs to be explicitly configured from one to the other.

 

[joeschmuck]

A few comments which might help...

1) Typically in my experience an MTU of 5000 or so is a happy value for both fast large files and reasonable small file transfers however on my current system I am running the default MTU of ~1500 (see my throughput below). My network setup is a Gigabit network switch to which my NAS and main computer is connected, I have a DIR-825 Router which ONLY routes my wireless and internet traffic, not transfers between my computer and NAS.
2) You didn't list any specs of your hardware so things could be slow on the NAS or things could be slow on your computer.
3) You didn't specify your transfer rates. I get ~95MB/sec Writing to the NAS and ~85MB/sec Reading from the NAS for large files transfers of ~5MB in size.

Please post your specs and transfer speeds other than fast or slow. I doubt changing the MTU will amount to any huge speed increase. We will help if we can.

 

[jgreco]

Just curious, joeschmuck, what hardware do you have that supports jumbo frames but slows down if you use a more normal size like 9000?

In the typical usage scenario for jumbo, it's best to pick the largest number that will work across all gear, which is typically 9000, but if you're sure you won't be adding other gear at a later date, might be as high as 16K. There are also drivers (and cards) with reasons to use a size that's less than a page (4096), but I cannot think of any reason 5000 would be a good choice.

Normally, a driver will pick a bucket that's appropriate for a frame, so "reasonable small file transfers" shouldn't really vary depending on MTU. However, there have been various combinations of drivers and cards where jumbo didn't work well due to sizes coded into the driver; I'm not sure all of those have been resolved but I don't see any current notes on that, and I know that it was being worked-on several years ago.

The benefits of jumbo frames have dwindled somewhat with the advent of better interrupt handling/polling, hardware offload of processing some parts of the network stack, etc. Jumbo is something I'd have argued for strongly a decade ago, or maybe even five years ago, but at this point the payoff is marginal compared to the hassle, and many networks have decided to just stick with 1500. However, in the meantime, the hassle has typically been reduced too, which is why it's interesting that you would pick some odd number for jumbo...

 

[joeschmuck]

When I was mucking around with jumbo frames a few years ago for my local network (at home) I found transfer speeds for large file to not increase above ~5000 MTU (depends on the hardware if it's 4096, 5K, or something similar, no one seems to remember 1024 = 1K and they seem to use 1000 = 1K, drives me nuts) and small file transfer to be impacted. To be honest, when I sync my music and photos, or other large groups of small files the transfer rates dropped using large jumbo frames. At the present time I'm running ~1500 MTU size throughout my network and I get very good results. I think some of that has to do with using good network switches and I do not use my Wireless Router other than it sits between my network switch and my cable modem. Well it has my network printer connected to it so there is some slow traffic other than internet that can go there. I do not test throughput via wireless because I'd go nuts trying to optimize it. It is what it is and does what it needs to do, no complaints.

If a person is moving large file sizes all the time, those jumbo frames of 9000 or 16K, or whatever might be appropriate but if you have a network that is prone to lost packets/collisions, those large MTUs could cause a slow down.

In a home system I don't think you need jumbo frames however if you are buying new equipment, I'd ensure jumbo frame support is there as it might talk to the quality of the component. Buy true network switches, not HUBs. Wire your network to where you are transferring your data through switches, not a Router.

So:
1) Connect your internet to your Router
2) Connect you Router to a Network Switch
3) Connect all your computers to the Network Switch (including your FreeNAS)
4) If you have long runs in your house, use an additional Network Switch to connect two or more areas.

At my home I have an 8 port switch in my computer room, two computers and my FreeNAS is connected to it. One additional line goes to my basement.

In the basement I have a 5 port switch, it runs one line to my attic and to my bedroom for DirecTv, One line to my living room to a third 5 port network switch. One line to a D-Link NAS, also in the basement. Leaving 1 port open for my FreeNAS box once I'm done setting it up.

In the living room I have the 5 port network switch which feeds two DirecTv boxes, a Blu-Ray player, and a second wireless router which I use only as a second wireless access point (I have a large house).

I have found out that a good network switch and how you connect them is the difference from an okay system to a great system.

Also, I'm not a networking expert. If you wanted to talk submarines and missile systems and had the proper clearance and need to know, I'm your guy, but networking was a trial and error thing for me, and a lot of reading.

 

[jgreco]

Hubs effectively don't exist anymore; the GigE and faster specs don't even allow for them, and the few 100Mbps ones that existed were all sufficiently gimpy and switch chipsets were sufficiently cheap that even low end consumer CPE mostly offers either silicon or software switched 100Mbps ports. Some of us old-timers still have 10Mbps hubs laying around or even in-service for OOB networks and the like. I think I ditched our last 100Mbps hub many years ago.

However, it is a fine point that the quality of gear varies widely, and in particular, switches are often a place where vendors make compromises. You also need to remember that every device in the broadcast domain needs to support AND BE CONFIGURED FOR jumbo frames: you cannot just have a few machines doing it, things won't work right. That puts the home user and the hobbyist at a bit of a disadvantage. You cannot easily have an ad-hoc network (not *wireless* ad-hoc, but ad-hoc in the *unplanned* and *it just evolved* sense) that supports jumbo. For example, assuming that your wireless router is in the same broadcast domain (think: "same network") as the rest of your jumbo devices, you could be adversely impacting performance if the "wireless router" is actually a small Linux computer with a few cheap 100Mbps ethernet ports built in, which many of them are! Your jumbo-configured PC may try to send a packet out to the Internet, and it may exceed whatever size the shoddy CPE ethernet chip can handle, and then things get ... oddly slow.

For a home user, this means buying gear with quality ethernet interfaces, and probably a cheap managed switch, and then putting the wireless stuff and cable modem behind something like a pfSense box, so that you have one network that's all jumbo. Then your good pfSense router will do fragmentation for you, and life is theoretically going to be pretty good.

But quite frankly, despite having a network architecture here that is specifically designed to segregate SAN/NAS traffic onto separate networks, I've been finding it hard to find the time to go and set a flag day to "go jumbo." I suppose I should, and then go report back on what sort of a difference it makes...

 

[louisk]

I know its off topic, and slightly nit-picky, but the wireless portion of APs and Routers are hubs.

 

[joeschmuck]

I don't feel a home user should have the need to go out and replace all their components just to get Jumbo frames. Replace the components when you can afford them. But there are good deals on network switches periodically and they don't cost very much and I'm not talking about purchasing corporate level equipment but those devices aren't much more money if that is what you want. As a side note, I'd love to have a 10Gb Ethernet system but I can't afford that nor do I honestly have the need for one but it would be cool to have it. When I need to replace my components due to failure, I plan to purchase 10Gb compatible devices, not because I need them, but because I'm curious how fast that would be for my home network.

I know we got off topic in this thread some but I hope this helped the OP.

 

[jgreco]

Well, if you want to be slightly nit-picky, wireless is only kinda-sorta hubbish. I wonder if anyone is doing jumbo over wireless.

 

[louisk]

Interesting question. I'll have to poke and see which vendors support JoW.

I don't use jumbo, and I'm getting ~600Mbit on/off my FreeNAS. I wouldn't see why you would need jumbo frames for a home senario.

 

[jgreco]

I don't feel a home user should have the need to go out and replace all their components just to get Jumbo frames.

You may not feel that fact, but the technical realities stand between your wish and actual reality. You may be able to cobble together something that seems to work with jumbo without actually ensuring that the entire broadcast domain supports and is configured for jumbo, but it is pretty much just asking for random unexplained brokenness in your networking that will bite you at some future point. It's more insidious because some hardware will even seem to work with jumbo when it isn't configured to, but the drivers may screw up some of the data because they're not initialized properly for the task.

You're much better off finding a way to make it work legitimately and correctly. At a bare minimum, you need a switch that supports jumbo. These are relatively cheap these days! Consider for example the Netgear GS108T, which supports jumbo and vlans. While I haven't tried it with this switch, theoretically you can configure your servers and devices with two vlans on a single ethernet port, one configured for internal jumbo NAS access, one configured for Internet access. It's a $90 switch. It isn't too onerous these days to get a jumbo-capable setup, it's just ironic that it is less meaningful than it used to be.