Please help - 10gb-10gb-1gb setup without 10gb switch

Joined
Nov 11, 2014
Messages
1,174
Thanks everyone, I got it to work - everything is perfect.
One more question though.
Just like you suggested, my setup is:
192.168.1.13 - 1Gb adapter connected to a 1Gb switch, ip is given by DHCP
192.168.0.100 - 10Gb - I manually set this ip.
My workstation connected to the switch by 1Gb nic, and directly to freenas by 10gb. CIRF shares, I'm using windows 7.

Everything works like it should - I'm getting gigabit speeds then I connect to freenas by typing //192.168.1.13, and if I go with //192.168.0.100 speeds are 5x higher. So it's good.
Question is - I just realized if I type //freenas it will automatically choose 10GB ip. Is that normal? Can I map network drives using //freenas, or should I still force //192.168.0.100?

Can you share what kind of speeds you are getting on 10Gb connection(Windows 7 with CIFS share) ?
 

VolumeTank

Dabbler
Joined
Dec 23, 2018
Messages
38
Hey everyone this is an old thread but seems useful to me I'm trying to do something similar bur I'm not sure of something. In my case I use an iMac as workstation the easiest way to ad 10GB support to a Mac is with this
Sonnet Echo Express SE I Thunderbolt 3 to Full-Height/Half-Length PCIe Card Expansion System
and this
Sonnet 1-Port Solo 10G Ethernet PCI Express 3.0 Card
which I got that part clear. Now on the freeNAS box I'm not sure if this card
ASUS XG-C100C 10GBase-T PCIe Network Adapter
would be support by BSD? Would that be ok? Or a different NIC for the FreeNAS is recommended?
 

VolumeTank

Dabbler
Joined
Dec 23, 2018
Messages
38

Chris Moore

Hall of Famer
Joined
May 2, 2015
Messages
10,080
Thanks. So there is no way to setup a bridge on the freenas server?
Not in the GUI which means that you shouldn't try to make it work.
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
Not in the GUI which means that you shouldn't try to make it work.

And that's a healthy warning. You can definitely make this work, but it won't be a supported configuration, and has some caveats. I once described how to do this, IIRC. Maybe even in this thread. The bridging stuff is pretty well baked into FreeBSD so this isn't hard to do.
 

studiomtn

Cadet
Joined
Jul 8, 2019
Messages
7
Basically I can't spend money on 10gb switch because I only need 1 of the machines connected by 10Gb.
Below is a quick doodle to explain what I want to achieve. My main workstation is used for 3D, video editing, after effects, nuke, etc. It would really benefit from 10 Gb connection. Other few computers are used for basic graphic design and 1GBs is enough.
Is it possible to setup freenas to act like a switch? Sorry I have zero FreeBSD experience, would really appreciate is someone can point me at right direction.
View attachment 5703

Hi, I am trying to do this same setup and was wondering how you manually configured the 10gbe connection?
 

VolumeTank

Dabbler
Joined
Dec 23, 2018
Messages
38
Hi, I am trying to do this same setup and was wondering how you manually configured the 10gbe connection?

Do you have the what you need already? Are you connecting to your freenas, on windows or mac?
 

studiomtn

Cadet
Joined
Jul 8, 2019
Messages
7
Do you have the what you need already? Are you connecting to your freenas, on windows or mac?
Everything's been up and running for a few months already on a 1gbe connection to multiple computers running Mac OS X. I just wanted to connect one computer(mac pro with 10gbe thunderbolt adapter) to the unused 10gbe port of the freenas.
 

VolumeTank

Dabbler
Joined
Dec 23, 2018
Messages
38
Everything's been up and running for a few months already on a 1gbe connection to multiple computers running Mac OS X. I just wanted to connect one computer(mac pro with 10gbe thunderbolt adapter) to the unused 10gbe port of the freenas.


I'm not sure which 10gbe thunderbolt adapter you have. But you need like the Sonnet Echo Express SE I plus NIC that goes inside of the Sonnet Echo Express SE I. Something like the Sonnet Presto Dual or a cheaper one will be the SuperMicro X540 or maybe even one of the same that you have install on your FreeNAS box. Also there is a few adapter like Sonnet Solo 10 which I didn't like.

My connection goes as follow:

SuperMicro x540 on FreeNAS box >
Network > Interface >
IP Address: 10.10.10.1 (needs to be different from your default connection to your router)
Subnet: 24
Options: mtu 9000
Screen Shot 2019-07-08 at 5.27.55 PM.png


On my iMac
Sonnet Presto Dual Install on Sonnet Echo Express SE I >
Connect to iMac via Thunderbolt 3 to Thunderbolt 2 adapter >
System Preference > Network > ( make sure driver is install and find the Thunderbolt connection)
Select connection >
Configure IPv4: Manually
IP Address: 10.10.10.2
Subnet Mask: 255.255.255.0 > Apply!!!
Screen Shot 2019-07-08 at 5.36.26 PM.png

With Thunderbolt connection selected >
Advance > Hardware >
Configure: Manually
Speed: 10GB-T
Duplex: full-duplex
MTU: Jumbo (900) > ok > Apply!!!
Screen Shot 2019-07-08 at 5.39.20 PM.png


From there would be to connect to server on mac via Finder or Go > Connect to server and put your credentials form FreeNAS using:
10.10.10.1 (FreeNAS 10Gb NIC Adress) >
user:
password:

That's how I connect mine, I'm pretty sure that's how must peer to peer connection on Mac's goes. Unless you have a 10Gbe switch( I don't think so). Which is the reason I use RJ45 instead of SFP+ because soon I will purchase a 10Gbe switch with enough ports to connect all my computers. MacBook Pro( 2017) iMac 27(Late 2015) 2 custom built pc, on for my kids gaming and the other for workstation and streaming.

Like I said I'm not sure which adapter you have!!! GOOD LUCK!
 

Attachments

  • Screen Shot 2019-07-08 at 5.27.55 PM.png
    Screen Shot 2019-07-08 at 5.27.55 PM.png
    426 KB · Views: 300
  • Screen Shot 2019-07-08 at 5.36.26 PM.png
    Screen Shot 2019-07-08 at 5.36.26 PM.png
    241.3 KB · Views: 272
  • Screen Shot 2019-07-08 at 5.39.20 PM.png
    Screen Shot 2019-07-08 at 5.39.20 PM.png
    1.3 MB · Views: 297
Last edited:

studiomtn

Cadet
Joined
Jul 8, 2019
Messages
7
Thanks @VolumeTank I have the Sonnet Solo adapter and not sure whats brand 10gig card is inside the FreeNAS. Currently all the 10 computers in my office are hooked up using a 1gbe connection on a 1gbe switch to the NAS. I just wanted one computer to have a direct peer to peer connection to the 10gbe port on the FreeNAS as well at the same time. Thanks for the info, I will try this out once my CAT6 cables come in.
 

VolumeTank

Dabbler
Joined
Dec 23, 2018
Messages
38
Thanks @VolumeTank I have the Sonnet Solo adapter and not sure whats brand 10gig card is inside the FreeNAS. Currently all the 10 computers in my office are hooked up using a 1gbe connection on a 1gbe switch to the NAS. I just wanted one computer to have a direct peer to peer connection to the 10gbe port on the FreeNAS as well at the same time. Thanks for the info, I will try this out once my CAT6 cables come in.

You're welcome, hope you get everything working!
Is a huge different in data transfer. I had the Sonnet Solo and returned, not that is bad (it does get pretty HOT), it use an Aquantia AQC-107S controller. The Sonnet Presto Dual use Intel X550 and is just a $100 more plus an extra port. The only downside is that in order to use with mac computers you need The Sonnet Echo Express or any other Thunderbolt 3 to PCIe Card Expansion System. Make sure the NIC on your FreeNAS is not Aquatina controller, I think that Aquatina doesn't have driver yet for FreeBSD. Do some research just in case. But the Sonnet Solo connected into the mac will work, I did got it to work connecting my MacBook Pro with the Sonnet Solo, to the FreeNAS with an Intel X540 Controller NIC.
 

studiomtn

Cadet
Joined
Jul 8, 2019
Messages
7
Hi @VolumeTank everything works likes you said. I ran into a minor issue. So prior to making this work I had the FreeNAS hooked up onto a 1gbe network switch which serves to 10 computers in the office also providing internet. The issue now is that on the single computer that now has a 10gbe connection straight to the FreeNAS, also has a 1gbe connection to the switch. So now I have to manually activate the 10gbe connection in mac os X network panel each time I need to mount the FreeNAS shared drive. If not then my computer picks the 1gbe connection. If I disconnect the 1gbe connection then I do not have internet. Hope that makes sense. I am trying to figure out a solution. in the image below is the network panel. "ethernet" is the 1gbe connection to the FreeNAS and internet. The "thunderbolt" is the 10gbe connection to the FreeNAS only.
 

Attachments

  • Screen Shot 2019-07-10 at 7.09.06 PM.png
    Screen Shot 2019-07-10 at 7.09.06 PM.png
    111.2 KB · Views: 277
  • freenas.jpg
    freenas.jpg
    68.9 KB · Views: 266

Constantin

Vampire Pig
Joined
May 19, 2017
Messages
1,828
I’d consider getting a inexpensive mikrotik switch with 2 SFP+ ports and hook everything into that - your station @ 10, FreeNAS box @ 10, and everything else at 1Gbe. See Mikrotik CSS326-24G-2S+RM 24 port Gigabit Ethernet switch with two SFP+ ports

https://www.amazon.com/dp/B0723DT6MN

As an example (have to edit this later, sorry, site does not render properly in iOS safari)
I use fiber to connect to the switch from my Mac and a twinax DAC to connect the switch to the FreeNAS.

The switch is fanless, uses little power, and costs less than $150. The twinax is usually cheap, ditto for fiber connections. I hope you’re not relying on copper 10Gbe as that’s pretty expensive re SFP+ transceivers.
 

studiomtn

Cadet
Joined
Jul 8, 2019
Messages
7
I’d consider getting a inexpensive mikrotik switch with 2 SFP+ ports and hook everything into that - your station @ 10, FreeNAS box @ 10, and everything else at 1Gbe. See Mikrotik CSS326-24G-2S+RM 24 port Gigabit Ethernet switch with two SFP+ ports

https://www.amazon.com/dp/B0723DT6MN

As an example (have to edit this later, sorry, site does not render properly in iOS safari)
I use fiber to connect to the switch from my Mac and a twinax DAC to connect the switch to the FreeNAS.

The switch is fanless, uses little power, and costs less than $150. The twinax is usually cheap, ditto for fiber connections. I hope you’re not relying on copper 10Gbe as that’s pretty expensive re SFP+ transceivers.

I would do that, except the position I am in and its current setup, I pretty much am unable to acquire/purchase any more equipment or what you recommend. And if I did it will be a major logistical problem getting it setup for reasons that are not under my control.
 

Constantin

Vampire Pig
Joined
May 19, 2017
Messages
1,828
Got it, I was just going for the least difficult solution. Well, then I'd go for the solutions mentioned at the beginning of this thread. Either
  • Use multiple connections, assign static IP addresses in a separate subnet for the 10GbE stuff from server to workstation, etc.; or
  • enable bridging in the NAS as discussed later
Each approach has advantages and disadvantages.
 

VolumeTank

Dabbler
Joined
Dec 23, 2018
Messages
38
Hi @VolumeTank everything works likes you said. I ran into a minor issue. So prior to making this work I had the FreeNAS hooked up onto a 1gbe network switch which serves to 10 computers in the office also providing internet. The issue now is that on the single computer that now has a 10gbe connection straight to the FreeNAS, also has a 1gbe connection to the switch. So now I have to manually activate the 10gbe connection in mac os X network panel each time I need to mount the FreeNAS shared drive. If not then my computer picks the 1gbe connection. If I disconnect the 1gbe connection then I do not have internet. Hope that makes sense. I am trying to figure out a solution. in the image below is the network panel. "ethernet" is the 1gbe connection to the FreeNAS and internet. The "thunderbolt" is the 10gbe connection to the FreeNAS only.

Remember there are 2 different IP’s on your Mac finder sidebar on network will show the 1gbe connection. The 10gbe connection, you would have to do that manually. If you connected the 1gbe manually juste delete that one. You don’t need to have both connected, you could if you want. Also to avoid that problem once you are connected via 10gbe go to settings and add the network drive as a login startup item.

I also, Isee that your subnet is set to 255.255.0.0 if that’s the one you going to use, make sure you use that on frenas for the 10gbe card as well.

The solution that @Constantin refer is also great. Only minor problem is that you would need the sfp+ to RJ45 adapter. The price for everything would be the same as getting a 10gbe RJ45 switch. ASUS have one for $245 with 2 10gbe ports. The other solution he refer(enable bridging) I never tried it, it does sound interesting.
 

VolumeTank

Dabbler
Joined
Dec 23, 2018
Messages
38
Hi @VolumeTank everything works likes you said. I ran into a minor issue. So prior to making this work I had the FreeNAS hooked up onto a 1gbe network switch which serves to 10 computers in the office also providing internet. The issue now is that on the single computer that now has a 10gbe connection straight to the FreeNAS, also has a 1gbe connection to the switch. So now I have to manually activate the 10gbe connection in mac os X network panel each time I need to mount the FreeNAS shared drive. If not then my computer picks the 1gbe connection. If I disconnect the 1gbe connection then I do not have internet. Hope that makes sense. I am trying to figure out a solution. in the image below is the network panel. "ethernet" is the 1gbe connection to the FreeNAS and internet. The "thunderbolt" is the 10gbe connection to the FreeNAS only.

If your main NIC 1gbe is for example 192.168.0.4/ for the FreeNAS. Than it should not be any problem with the 10gbe NIC 10.10.10.2 connection. I’m kind of lost on why it does that.
 

studiomtn

Cadet
Joined
Jul 8, 2019
Messages
7
If your main NIC 1gbe is for example 192.168.0.4/ for the FreeNAS. Than it should not be any problem with the 10gbe NIC 10.10.10.2 connection. I’m kind of lost on why it does that.
If for any reason I unmount the shared drive over the 10gbe connection and mount it again the finder will choose to mount over the 1gbe connection. (i ran speed tests to confirm) So everytime I want to mount the shared drive over 10gbe I have to go to the network panel and turn off the 1gbe network, then mount over 10gbe and then turn on 1gbe so I can have internet access and access to other computers on the network. Its kinda of annoying and wish I didnt have to do that. below I made a diagram of my setup. IF I cut off the 1gbe connection then I loose internet and access to other computers in the building. @Constantin we have a switch with 4 sfp+ ports but we are unable to run cables to the other side of the building or purchase sfp+ to rj45 adapters.
 

Attachments

  • setup.jpg
    setup.jpg
    159.2 KB · Views: 405

VolumeTank

Dabbler
Joined
Dec 23, 2018
Messages
38
Base on your diagram is the same way mines connected. That must be something on the mac's pro or something. Can you share a a screen shot from your freenas interface set up on the GUI. Also you have 2 mac's pro? If so remember that every port would need its own IP.

In your case I would setup like this:

FreeNAS 10Gbe 1 -> 10.10.10.1
Mac Pro 1 -> 10.10.10.2

FreeNAS 10Gbe 2 -> 25.25.25.1
Mac Pro 2 -> 25.25.25.1

The 10Gbe NIC's should not collide with the NIC's 1Gbe because is not connected to the network. I would share a screen shot of mine.
 
Top