How to configure 2 NICs

[Eli Singer]

I have FreeNAS running on a Supermicro X11SSH-CTF motherboard which has 2 10Gb ports.
Up untill now i had 1 port connected to a 1Gb router.
I just bought a 10Gb card for my main machine which currently has a 1Gb port also connected to the router.
I want to connect the second FreeNAS port to the new 10Gb card on the main machine so that information between the two would run using those ports and not through the router.

Would love to get some educating posts\tutorials on the matter.

Thanks,

Eli

 

[danb35]

I want to connect the second FreeNAS port to the new 10Gb card on the main machine

Easy. Assign the second FreeNAS port, and the 10GbE card on the main machine, different static IPs on a separate subnet than you're using for the first FreeNAS port.

 

[Eli Singer]

Easy. Assign the second FreeNAS port, and the 10GbE card on the main machine, different static IPs on a separate subnet than you're using for the first FreeNAS port.

Can you point me to some more detailed information on how to do that?

Thanks,

Eli

 

[Eli Singer]

How do i make the FreeNAS and my main machine talk only using the 10Gb connection while both of them are also connected to the router?
Currently it seem to still go through the router which limits the speed to 1Gb...

Eli

 

[Johnnie Black]

What I do is add the 10GbE FreeNAS IP to the hosts file, e.g.:

192.168.0.10 yourfreenasservername

If you're using Windows you need to open notepad as admin.

 

[Eli Singer]

What I do is add the 10GbE FreeNAS IP to the hosts file, e.g.:

192.168.0.10 yourfreenasservername

If you're using Windows you need to open notepad as admin.

What does it actually do?
I still need the windows machine to use the regular 1Gb port for connecting to the network, i just want to have it communicate with the FreeNAS on the 10Gb.

Eli

 

[Johnnie Black]

I still need the windows machine to use the regular 1Gb port for connecting to the network, i just want to have it communicate with the FreeNAS on the 10Gb.

That's exactly what it will do.

 

[Eli Singer]

Well, i must have done something wrong cause it's still uses the 1Gb NIC.
This is my configuration:

FreeNAS IP
LAN 1 (ix0 Interface) Connected to router
IP: 192.168.10.21
Subnet: 255.255.255.0

LAN 2 (ix1 Interface) Connected to 10Gb card on main computer
IP: 192.168.9.20
Subnet: 255.255.255.0

Gateway (router)
192.168.10.1

On my Windows machine i have the 10Gb NIC configured for:
IP: 192.168.9.25
Subnet: 255.255.255.0

I tried adding to the hosts file the FreeNAS IP that is connected to it directly:
192.168.9.20 freenas
Tried also
192.168.9.20 freenas.local

But it still uses just the 1Gb NIC.

What am i doing wrong?

Eli

 

[Eli Singer]

Well, i just deleted the entries from the host file and it suddenly seems to work!
i get around 460MB/s writing to the FreeNAS from a Sandisk Ultra II 960GB SSD on the Windows machine and around 240MB/s the other way around.
I actually thought that reading from the FreeNAS and writing to the SSD would be faster but it still a lot better than the 110MB/s i used to get...

Thanks for all the advice!

If you have an idea on how to improve the 240MB/s number it would be great :)

Eli

 

[chris crude]

From your Windows machine, can you connect to your FreeNAS at
LAN 2 (ix1 Interface) Connected to 10Gb card on main computer
IP: 192.168.9.20
Subnet: 255.255.255.0

 

[chris crude]

You connected before i pressed send!
As speed is concerned, your signature says Z2 pool I dont think you get much faster without setting up you pool with mirrors.

 

[Eli Singer]

You connected before i pressed send!
As speed is concerned, your signature says Z2 pool I dont think you get much faster without setting up you pool with mirrors.

But how come i get x2 faster writing speeds to the FreeNAS than reading from it?

 

[Chris Moore]

These two videos (taken together) give a really good introduction to what you need to know:

10Gb Home Network (P1) - Introduction - https://youtu.be/MgNpI6VAAhI

10Gb Home Network (P2) - Peer to Peer - https://youtu.be/KHiucyRYX_w

This guy does some testing that is interesting, but some of his conclusions are just wrong because he is a learner, not an expert and he is mistaken about how some things work.

10gb Peer to Peer With Your FreeNAS Server - https://youtu.be/8f5cuzcNrLo

FreeNAS Raid Performance Comparison - https://youtu.be/GGd-DiDnLpw

 

[chris crude]

But how come i get x2 faster writing speeds to the FreeNAS than reading from it?

Because the Z2 pool can read nearly as fast as the SSD can write, but cannot write as fast as an SSD can read?

 

[Chris Moore]

Would love to get some educating posts\tutorials on the matter.

Please note my post above. The videos will probably answer many of your questions.

 

[Eli Singer]

These two videos (taken together) give a really good introduction to what you need to know:

10Gb Home Network (P1) - Introduction - https://youtu.be/MgNpI6VAAhI

10Gb Home Network (P2) - Peer to Peer - https://youtu.be/KHiucyRYX_w

This guy does some testing that is interesting, but some of his conclusions are just wrong because he is a learner, not an expert and he is mistaken about how some things work.

10gb Peer to Peer With Your FreeNAS Server - https://youtu.be/8f5cuzcNrLo

FreeNAS Raid Performance Comparison - https://youtu.be/GGd-DiDnLpw

I watched all videos you linked. In the last video he suggested that the low writing speeds to the Z2 array is due to the FreeNAS CPU who is the bottleneck.
I wonder if that's true or is it what you referred to as a mistake from his part.
It's interesting because i have a pretty similar setup in my FreeNAS of 6 Disks in Z2 so i'd like to know if my CPU is holding me back or not.

Eli

 

[Chris Moore]

I watched all videos you linked. In the last video he suggested that the low writing speeds to the Z2 array is due to the FreeNAS CPU who is the bottleneck.
I wonder if that's true or is it what you referred to as a mistake from his part.
It's interesting because i have a pretty similar setup in my FreeNAS of 6 Disks in Z2 so i'd like to know if my CPU is holding me back or not.

Eli

The limit is two things.
First, the number of drives. Each drive has a mechanical limit of how fast it can either read or write data. So, if you have 6 drives and 2 of them are just for parity, that gives you the throughput of the remaining 4 to work with. There is some overhead involved with the process. As an example. let's say that each drive is rated to write 130MB/s, your actual speed would be closer to 100MB/s, and the array might be able to manage 400MB/s. That's theory but it usually goes a little slower. So, in the same pool, if you add a second (matching) vdev, it doubles the speed. That's what I have, 2 vdevs, 6 drives each, using RAID-z2. I get speeds between 600 & 800 MB/s. I should say, I get that with large files.
The exception to this is where you have many, small file. There is proportionally more file overhead with small files, so they go slow. The only way to overcome this is with more vdevs. The theory is, more vdevs is generally faster.
I have a server that I manage at work that has 4 vdevs and it is great, even for 10GB network, with large files but it still slows with the small files.

Sent from my SAMSUNG-SGH-I537 using Tapatalk

 

[chris crude]

I just copy-pasted a 20GB Windows Image Backup from my Samsung 850EVO SSD on WIN10 client, direct connected with Intel 10GB NIC to Z1 pool of my FreeNAS (I know everyone says i shouldnt use Z1 for 5x3TB disks, i have backups) and got 250-260MB/s. Changed the name of the file and then copy-paste from Z1 FreeNAS back to client and got 240-250MB/s. Both speeds are slower than rated for SSD because i'm at the limit for my Z1 pool.
Chose different 15GB Windows Image and went from my 4x4TB RAID10 (HighPoint Raid card) on WIN10 client to FreeNAS got 150-180MB, Changed the name of the file and then copy-paste back from FreeNAS to WIN10 spiked at 400 at first but quickly leveled off and sustained 190MB/s.
What does all this mean? I only have basic pools so they arent keeping up with my SSD, and even slower with cheap RAID card even in RAID10.