AOC-STGN-i2S Intel 82599ES 10GbE not detected

[Indra Pramana]

Dear all,

I am running FreeNAS-8.3.1-RELEASE-p2-x64 (r12686+b770da6_dirty) on a SuperMicro SSG-6047R-E1R24L 4U server, and it's running fine with its 1 GBps NIC card. Recently I tried to add new 10 GBps NIC card: AOC-STGN-i2S from SuperMicro, which is running Intel 82599ES chipset, and the NIC is not detected by the system.

FreeBSD version:

FreeBSD cs-nas-01 8.3-RELEASE-p7 FreeBSD 8.3-RELEASE-p7 #1 r249203M:
Sat Apr 6 09:28:27 PDT 2013
root@build.ixsystems.com:/tank/home/jpaetzel/fn8.3/freenas/os-base/amd64/tank/home/jpaetzel/fn8.3/freenas/FreeBSD/src/sys/FREENAS.amd64 amd64

Understand that I need to load the "ixgbe" driver so I tried to load the driver via Tunables: System > Tunables > Add Tunables:

Variable: if_ixgbe_load
Value: YES
Enabled: True

However the NIC is still cannot be detected.

The NIC card comes with the CD which contain drivers for FreeBSD: ixgbe-2.5.12.tar.gz but I can't copy the file to the system because it's marked read-only. I tried to copy the file to one of the ZFS volume (which I have write access) but I am still not able to compile because it require kernel source to compile.

[root@cs-nas-01 /mnt/vol1/ixgbe/ixgbe-2.5.12/src]# make
"/usr/share/mk/bsd.kmod.mk", line 12: "can't find kernel source tree"

Any advice how can I make the NICs detected by the system? Below is the pciconf -lv output:

http://pastebin.com/HH0QSQce

Below is the dmesg output:

http://pastebin.com/rHipnaQq

ifconfig result shows that the 10G NIC card is not detected, only the 1G NIC card (igb0/1/2/3 ports) is detected:

http://pastebin.com/8JebAZeR

Looking forward to your advices.

Thank you.

 

[cyberjock]

Put a ticket in at support.freenas.org. I'd include that tar.gz file on the ticket. With a little luck someone will stick that driver in the next FreeNAS build and it'll be supported out of the box with FreeNAS 9.1.

Edit: Wait a minute.. that's a linux driver, isn't it? FreeBSD isn't Linux...

Another Edit: The latest driver version seems to be 2.5.8.. https://downloadcenter.intel.com/Detail_Desc.aspx?agr=Y&DwnldID=14688

It will have to be compiled which may prove difficult if you're new to FreeNAS. I'll submit the ticket with the link. I'll provide the link in a few minutes.

 

[cyberjock]

Ticket created:

https://support.freenas.org/ticket/2451#ticket

 

[Indra Pramana]

Hi cyberjock,

Good day to you, and thank you for your reply. :) And many thanks for your help in creating the support ticket, greatly appreciated.

I got the file: ixgbe-2.5.12.tar.gz from the drivers CD that comes with the NIC card, and the README file mention that it's for FreeBSD not for Linux.

"FreeBSD Driver for Intel(R) Ethernet 10 Gigabit PCI Express Server Adapters"

You are right that the latest version available on the Intel website is 2.5.8, I suspect the one which comes with the CD (2.5.12) is a newer version but was not updated/published on Intel website.

Would it be possible for me to provide the version 2.5.12 of the driver instead to be added for the out-of-the-box FreeNAS 9.1? :) I can upload the file here or send to you if needed.

Looking forward to your reply, thank you.

Cheers.

 

[cyberjock]

You can link it to here. I think it would be better to stick with Intel's officially released driver. It could be that your version was removed because it was broken or something. Without a list of the changes and such, I'd just stick with the download from the Intel website.

 

[Indra Pramana]

Hi cyberjock,

Good day to you, and thank you for your reply.

Understood, I agree with your suggestion. Let us just stick with the download from the Intel website.

Do you think how soon that someone will look into the support ticket and perform the compilation, and whether it will be in time for the stable version of 9.1.0 to be released? Any idea on when will 9.1.0 come out from its current RC (pre-production) phase to the stable (production) release?

Looking forward to your reply, thank you.

Cheers.

 

[cyberjock]

It really depends on alot of factors. Considering how close they are to RELEASE it might not make it in time for this next version. But you never know, someone could get bored and take care of it. This is one of those times that you could have done it yourself if you had the experience. :P We'll just have to wait and see.

 

[Indra Pramana]

Hi cyberjock,

Good day to you, and thank you for your e-mail.

Is it confirmed that this is something that only FreeNAS team can do? I mean, it's confirmed that it's not technically possible for a user (like me) to compile the driver myself and load it to the existing setup that I have (existing 8.3.1 version) without having for the driver to be included on the next release, correct?

I am thinking of setting up a FreeBSD server (with exact version of what FreeNAS 8.3.1 is using) and compile it there. But is there a way for me to copy the compiled driver into the FreeNAS system, considering that the filesystem is read-only? Not too sure whether it'll work.

Looking forward to your reply, thank you.

Cheers.

 

[cyberjock]

It's totally possible for you to compile the driver yourself. However its not trivial to do, especially to someone that has never done it before. If you do compile it to a .ko file you can easily copy it to your FreeNAS system. The FreeNAS FAQ includes steps to make the file system writable. Do keep in mind that each time FreeNAS is updated you would have to recompile the drivers for the new FreeBSD version, any upgrade will overwrite the old files, etc.

 

[Indra Pramana]

Dear cyberjock and all,

I have upgraded my FreeNAS to the latest version: FreeNAS-9.1.0-RC2-x64 and the upgrade was successful. The new FreeNAS version uses a newer version of FreeBSD, which is: FreeBSD 9.1-STABLE.

I setup a server using the same FreeBSD OS version 9.1, and I manage to compile the kernel manually. I then copied the if_ixgbe.ko file into /boot/kernel directory of the FreeNAS server and have done 3 options below, but all fails to load the module with the same error message:

1. Change the value for if_ixgbe_load on /boot/defaults/loader.conf from "NO" to "YES":
if_ixgbe_load="YES"

2. Add the above into Tunables, which in turn will add the same entry on /boot/loader.conf.local

3. Add ixgbe_load="YES" into /boot/loader.conf

All fails to load the driver module upon boot-up with these error messages noted during boot-up and on dmesg:

===
module_register: module pci/ixgbe already exists!
Module pci/ixgbe failed to register: 17
===

Can anyone advise what is the best method to load/register the driver module? Or do I have to recompile the kernel? The problem is that the kernel sources (normally under /usr/src) is not installed on the FreeNAS system.

Looking forward to your reply, thank you.

Cheers.

 

[cyberjock]

That sounds like there is an ixgbe driver already loading by the FreeNAS kernel so yours won't load. I'm not sure how to fix that easily. I know how I fixed it months ago when I was playing with drivers... I had to recompile FreeNAS without that driver so I could inject my own without the driver "already existing". Took something like 6 hours to compile on my dual processor Xeon machine. Was shocked at how long it took.

 

[Indra Pramana]

Hi cyberjock,

Good day to you, and thank you for your e-mail.

Yes, it seems that there is an existing ixgbe driver, which might be an earlier version of the driver which cannot detect my 10G NIC, inside the kernel. This prevents my compiled kernel (the if_ixgbe.ko file) from being loaded.

How did you recompile the FreeNAS kernel? I can't seem to find the kernel sources available on the FreeNAS system which I installed. Appreciate if you can guide me on how to recompile the kernel. :)

Looking forward to your reply, thank you.

Cheers.

 

[cyberjock]

I used the guide that starts at http://forums.freenas.org/threads/g...part-1-building-a-freebsd-virtual-machine.48/. I'm not sure if that guide is still valid since we use github now. I haven't tried to compile FreeNAS in quite some time, but that guide really is an "idiots guide". ;)

Edit: Part 2 - http://forums.freenas.org/threads/guide-how-to-build-freenas-part-2-install-and-build-freenas.50/

 

[Indra Pramana]

Hi cyberjock,

Good day to you, and thank you for your reply.

I tried to follow the instruction, but after running the svn, I noticed that the guidance is for building FreeNAS 8.*. Is there any guidance for building FreeNAS 9.1?

Looking forward to your reply, thank you.

Cheers.

 

[cyberjock]

Nope. :(

 

[paleoN]

Is there any guidance for building FreeNAS 9.1?

Except, for this. That may be a bit outdated. Be sure to check the README on GIT which may be up to date.

 

[Indra Pramana]

Dear cyberjock,

Intel just released the latest version of the driver: 2.5.15 on their website.

I have tried to:
- Recompile the FreeBSD kernel to use the latest version of the driver;
- Rebuild FreeNAS using the above method you have given; It's really taking a lot of time!
- Reinstall my FreeNAS system using the ISO file generated by the above rebuild process.

However, the 10G NICs are still not being detected! :( I tried to add into /boot/loader.conf, /boot/defaults/loader.conf even manually using kldload also cannot. :(

What else can I do?

 

[cyberjock]

LOL. I guess Intel's website is a little slow. :P

Assuming you properly added the driver and such, it should work. There's no reason for it not to unless you recompiled it wrong somehow. Keep in mind those instructions may not be up to date, so you may be unwittingly doing something wrong. I've found that there's very little information on how to compile FreeBSD(and naturally FreeNAS) if you don't know jack about squat. I'm somewhat in that boat, and I'd love a step-by-step guide for compiling FreeNAS from scratch, how to make and test changes(good example... updating a driver), how to make that into a "diff" file, and then how to put that stuff on git. I know I could provide stuff(especially driver updates) for the FreeNAS project, but as I'm clueless and I don't have the know-how to identify what I should do from what I shouldn't, how to identify that an error really means something bad vice something to ignore(for example.. getting a noDHCPServerfound error when you aren't connected to a network is safe to ignore), etc I can't provide you with any more assistance from what I have already.