10 Gbe nic

[liquidice]

First off, I apologize if this has been answered already, but I have been looking through the forums and have not found an answer that has worked or that I understood.

I am using freenas 9.2.1.7 booting from a jump drive and am trying to get 10 gig working. I have tried 3 different nics in all available pci-e ports and none of them are showing up. I tried a qlogic qle8152, a chelso i-cant-remember-the-model-number, and an intel x520-sr2 (which is what is currently installed). All three work in multiple other computers and are on the freebsd supported nic list.

I'm not entirely sure what to look at next so if anyone can provide troubleshooting steps, I would appreciate it. Thanks!

 

[mjws00]

How bout a list of the actual hardware mobo, ram etc as per the forum rules above and dmesg output in code tags.

 

[liquidice]

mobo is an Asus P9X79 Deluxe with 64 GB ram (don't know speed) , Intel Core i7-3820 CPU @ 3.60GHz, various HDDs (mix between Hitachi and Seagate 4T drives, no other pci cards. ifconfig and dmesg are below. dmesg doesn't show anything.

Code:

[root@freenas] ~# ifconfig
em0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
  options=42098<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWTSO>
  ether c8:60:00:cb:13:1c
  inet 192.168.1.106 netmask 0xffffff00 broadcast 192.168.1.255
  nd6 options=9<PERFORMNUD,IFDISABLED>
  media: Ethernet autoselect (1000baseT <full-duplex>)
  status: active
ipfw0: flags=8801<UP,SIMPLEX,MULTICAST> metric 0 mtu 65536
  nd6 options=9<PERFORMNUD,IFDISABLED>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
  options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
  inet6 ::1 prefixlen 128
  inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
  inet 127.0.0.1 netmask 0xff000000
  nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
  ether 02:1d:62:db:13:00
  nd6 options=1<PERFORMNUD>
  id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
  maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
  root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
  member: epair0a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
  ifmaxaddr 0 port 7 priority 128 path cost 2000
  member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
  ifmaxaddr 0 port 1 priority 128 path cost 20000
epair0a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
  options=8<VLAN_MTU>
  ether 02:44:25:00:07:0a
  nd6 options=1<PERFORMNUD>
  media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
  status: active
[root@freenas] ~# dmesg
[root@freenas] ~#

 

[liquidice]

in pasting that, i just noticed the "epair0a" shows 10 gig. what is that?

 

[mjws00]

The epair interface typically shows up with jails and vlans. Maybe someone can chime in with a deep explanation. We need a physical interface, and we want to figure out what the OS sees. Also which card is currently installed?

Output of 'cat /var/run/dmesg.boot'

 

[liquidice]

the intel x520-sr2 is currently installed.

Code:

[root@freenas] ~# cat /var/run/dmesg.boot
Copyright (c) 1992-2013 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
    The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 9.2-RELEASE-p10 #0 r262572+4fb5adc: Wed Aug  6 17:07:16 PDT 2014
    root@build3.ixsystems.com:/fusion/jkh/921/freenas/os-base/amd64/fusion/jkh/921/freenas/FreeBSD/src/sys/FREENAS.amd64 amd64
gcc version 4.2.1 20070831 patched [FreeBSD]
CPU: Intel(R) Core(TM) i7-3820 CPU @ 3.60GHz (3708.04-MHz K8-class CPU)
  Origin = "GenuineIntel"  Id = 0x206d7  Family = 0x6  Model = 0x2d  Stepping = 7
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x1fbee3bf<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX>
  AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
  AMD Features2=0x1<LAHF>
  TSC: P-state invariant, performance statistics
real memory  = 60129542144 (57344 MB)
avail memory = 58068054016 (55378 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <ALASKA A M I>
FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s) x 2 SMT threads
cpu0 (BSP): APIC ID:  0
cpu1 (AP): APIC ID:  1
cpu2 (AP): APIC ID:  2
cpu3 (AP): APIC ID:  3
cpu4 (AP): APIC ID:  4
cpu5 (AP): APIC ID:  5
cpu6 (AP): APIC ID:  6
cpu7 (AP): APIC ID:  7
WARNING: VIMAGE (virtualized network stack) is a highly experimental feature.
ACPI Warning: FADT (revision 5) is longer than ACPI 2.0 version, truncating length 268 to 244 (20110527/tbfadt-320)
ioapic0 <Version 2.0> irqs 0-23 on motherboard
ioapic1 <Version 2.0> irqs 24-47 on motherboard
kbd1 at kbdmux0
cryptosoft0: <software crypto> on motherboard
aesni0: <AES-CBC,AES-XTS> on motherboard
padlock0: No ACE support.
acpi0: <ALASKA A M I> on motherboard
ACPI Error: [RAMB] Namespace lookup failure, AE_NOT_FOUND (20110527/psargs-392)
ACPI Exception: AE_NOT_FOUND, Could not execute arguments for [RAMW] (Region) (20110527/nsinit-380)
acpi0: Power Button (fixed)
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
cpu2: <ACPI CPU> on acpi0
cpu3: <ACPI CPU> on acpi0
cpu4: <ACPI CPU> on acpi0
cpu5: <ACPI CPU> on acpi0
cpu6: <ACPI CPU> on acpi0
cpu7: <ACPI CPU> on acpi0
attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
Event timer "RTC" frequency 32768 Hz quality 0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 950
Event timer "HPET" frequency 14318180 Hz quality 550
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
acpi_ec0: <Embedded Controller: GPE 0x1e> port 0x62,0x66 on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> irq 27 at device 1.0 on pci0
pci3: <ACPI PCI bus> on pcib1
pcib2: <ACPI PCI-PCI bridge> irq 33 at device 2.0 on pci0
pci1: <ACPI PCI bus> on pcib2
vgapci0: <VGA-compatible display> port 0xe000-0xe07f mem 0xfa000000-0xfaffffff,0xc0000000-0xcfffffff,0xd0000000-0xd1ffffff irq 32 at device 0.0 on pci1
pci1: <multimedia, HDA> at device 0.1 (no driver attached)
pcib3: <ACPI PCI-PCI bridge> irq 41 at device 3.0 on pci0
pci2: <ACPI PCI bus> on pcib3
pci0: <base peripheral> at device 5.0 (no driver attached)
pci0: <base peripheral> at device 5.2 (no driver attached)
pcib4: <ACPI PCI-PCI bridge> irq 16 at device 17.0 on pci0
pci4: <ACPI PCI bus> on pcib4
pci0: <simple comms> at device 22.0 (no driver attached)
em0: <Intel(R) PRO/1000 Network Connection 7.3.8> port 0xf040-0xf05f mem 0xfb600000-0xfb61ffff,0xfb628000-0xfb628fff irq 18 at device 25.0 on pci0
em0: Using an MSI interrupt
em0: Ethernet address: c8:60:00:cb:13:1c
ehci0: <EHCI (generic) USB 2.0 controller> mem 0xfb627000-0xfb6273ff irq 23 at device 26.0 on pci0
usbus0: EHCI version 1.0
usbus0 on ehci0
pci0: <multimedia, HDA> at device 27.0 (no driver attached)
pcib5: <ACPI PCI-PCI bridge> irq 17 at device 28.0 on pci0
pci5: <ACPI PCI bus> on pcib5
pcib6: <ACPI PCI-PCI bridge> irq 16 at device 28.1 on pci0
pci6: <ACPI PCI bus> on pcib6
ahci0: <ASMedia ASM1061 AHCI SATA controller> port 0xd050-0xd057,0xd040-0xd043,0xd030-0xd037,0xd020-0xd023,0xd000-0xd01f mem 0xfb500000-0xfb5001ff irq 17 at device 0.0 on pci6
ahci0: AHCI v1.20 with 2 6Gbps ports, Port Multiplier supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich1: <AHCI channel> at channel 1 on ahci0
pcib7: <ACPI PCI-PCI bridge> irq 18 at device 28.2 on pci0
pci7: <ACPI PCI bus> on pcib7
pci7: <serial bus, USB> at device 0.0 (no driver attached)
pcib8: <ACPI PCI-PCI bridge> irq 19 at device 28.3 on pci0
pci8: <ACPI PCI bus> on pcib8
pci8: <serial bus, USB> at device 0.0 (no driver attached)
pcib9: <ACPI PCI-PCI bridge> irq 17 at device 28.4 on pci0
pci9: <ACPI PCI bus> on pcib9
siis0: <SiI3132 SATA controller> port 0xc000-0xc07f mem 0xfb284000-0xfb28407f,0xfb280000-0xfb283fff irq 16 at device 0.0 on pci9
siisch0: <SIIS channel> at channel 0 on siis0
siisch1: <SIIS channel> at channel 1 on siis0
pcib10: <ACPI PCI-PCI bridge> irq 19 at device 28.7 on pci0
pci10: <ACPI PCI bus> on pcib10
ahci1: <Marvell 88SE9130 AHCI SATA controller> port 0xb040-0xb047,0xb030-0xb033,0xb020-0xb027,0xb010-0xb013,0xb000-0xb00f mem 0xfb110000-0xfb1107ff irq 19 at device 0.0 on pci10
ahci1: AHCI v1.20 with 8 6Gbps ports, Port Multiplier not supported
ahci1: quirks=0x800<ALTSIG>
ahcich2: <AHCI channel> at channel 0 on ahci1
ahcich3: <AHCI channel> at channel 1 on ahci1
ahcich4: <AHCI channel> at channel 2 on ahci1
ahcich5: <AHCI channel> at channel 3 on ahci1
ahcich6: <AHCI channel> at channel 4 on ahci1
ahcich7: <AHCI channel> at channel 5 on ahci1
ahcich8: <AHCI channel> at channel 6 on ahci1
ahcich9: <AHCI channel> at channel 7 on ahci1
ehci1: <EHCI (generic) USB 2.0 controller> mem 0xfb626000-0xfb6263ff irq 23 at device 29.0 on pci0
usbus1: EHCI version 1.0
usbus1 on ehci1
pcib11: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci11: <ACPI PCI bus> on pcib11
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
ahci2: <Intel Patsburg AHCI SATA controller> port 0xf090-0xf097,0xf080-0xf083,0xf070-0xf077,0xf060-0xf063,0xf020-0xf03f mem 0xfb625000-0xfb6257ff irq 20 at device 31.2 on pci0
ahci2: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported
ahcich10: <AHCI channel> at channel 0 on ahci2
ahcich11: <AHCI channel> at channel 1 on ahci2
ahcich12: <AHCI channel> at channel 2 on ahci2
ahcich13: <AHCI channel> at channel 3 on ahci2
ahcich14: <AHCI channel> at channel 4 on ahci2
ahcich15: <AHCI channel> at channel 5 on ahci2
pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
pcib12: <ACPI Host-PCI bridge> on acpi0
pci255: <ACPI PCI bus> on pcib12
pci255: <base peripheral> at device 8.0 (no driver attached)
pci255: <base peripheral> at device 8.3 (no driver attached)
pci255: <base peripheral> at device 8.4 (no driver attached)
pci255: <base peripheral> at device 9.0 (no driver attached)
pci255: <base peripheral> at device 9.3 (no driver attached)
pci255: <base peripheral> at device 9.4 (no driver attached)
pci255: <base peripheral> at device 10.0 (no driver attached)
pci255: <base peripheral> at device 10.1 (no driver attached)
pci255: <base peripheral> at device 10.2 (no driver attached)
pci255: <base peripheral> at device 10.3 (no driver attached)
pci255: <base peripheral> at device 11.0 (no driver attached)
pci255: <base peripheral> at device 11.3 (no driver attached)
pci255: <base peripheral> at device 12.0 (no driver attached)
pci255: <base peripheral> at device 12.1 (no driver attached)
pci255: <base peripheral> at device 12.6 (no driver attached)
pci255: <base peripheral> at device 12.7 (no driver attached)
pci255: <base peripheral> at device 13.0 (no driver attached)
pci255: <base peripheral> at device 13.1 (no driver attached)
pci255: <base peripheral> at device 13.6 (no driver attached)
pci255: <base peripheral> at device 14.0 (no driver attached)
pci255: <dasp> at device 14.1 (no driver attached)
pci255: <base peripheral> at device 15.0 (no driver attached)
pci255: <base peripheral> at device 15.1 (no driver attached)
pci255: <base peripheral> at device 15.2 (no driver attached)
pci255: <base peripheral> at device 15.3 (no driver attached)
pci255: <base peripheral> at device 15.4 (no driver attached)
pci255: <base peripheral> at device 15.5 (no driver attached)
pci255: <base peripheral> at device 15.6 (no driver attached)
pci255: <base peripheral> at device 16.0 (no driver attached)
pci255: <base peripheral> at device 16.1 (no driver attached)
pci255: <base peripheral> at device 16.2 (no driver attached)
pci255: <base peripheral> at device 16.3 (no driver attached)
pci255: <base peripheral> at device 16.4 (no driver attached)
pci255: <base peripheral> at device 16.5 (no driver attached)
pci255: <base peripheral> at device 16.6 (no driver attached)
pci255: <base peripheral> at device 16.7 (no driver attached)
pci255: <base peripheral> at device 17.0 (no driver attached)
pci255: <base peripheral> at device 19.0 (no driver attached)
pci255: <dasp> at device 19.1 (no driver attached)
pci255: <dasp> at device 19.4 (no driver attached)
pci255: <dasp> at device 19.5 (no driver attached)
pci255: <base peripheral> at device 19.6 (no driver attached)
acpi_button0: <Power Button> on acpi0
ichwd0 on isa0
orm0: <ISA Option ROMs> at iomem 0xce800-0xd2fff,0xd3000-0xd5fff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
wbwd0: DevID 0xc3 DevRev 0x33, will not attach, please report this.
coretemp0: <CPU On-Die Thermal Sensors> on cpu0
est0: <Enhanced SpeedStep Frequency Control> on cpu0
p4tcc0: <CPU Frequency Thermal Control> on cpu0
coretemp1: <CPU On-Die Thermal Sensors> on cpu1
est1: <Enhanced SpeedStep Frequency Control> on cpu1
p4tcc1: <CPU Frequency Thermal Control> on cpu1
coretemp2: <CPU On-Die Thermal Sensors> on cpu2
est2: <Enhanced SpeedStep Frequency Control> on cpu2
p4tcc2: <CPU Frequency Thermal Control> on cpu2
coretemp3: <CPU On-Die Thermal Sensors> on cpu3
est3: <Enhanced SpeedStep Frequency Control> on cpu3
p4tcc3: <CPU Frequency Thermal Control> on cpu3
coretemp4: <CPU On-Die Thermal Sensors> on cpu4
est4: <Enhanced SpeedStep Frequency Control> on cpu4
p4tcc4: <CPU Frequency Thermal Control> on cpu4
coretemp5: <CPU On-Die Thermal Sensors> on cpu5
est5: <Enhanced SpeedStep Frequency Control> on cpu5
p4tcc5: <CPU Frequency Thermal Control> on cpu5
coretemp6: <CPU On-Die Thermal Sensors> on cpu6
est6: <Enhanced SpeedStep Frequency Control> on cpu6
p4tcc6: <CPU Frequency Thermal Control> on cpu6
coretemp7: <CPU On-Die Thermal Sensors> on cpu7
est7: <Enhanced SpeedStep Frequency Control> on cpu7
p4tcc7: <CPU Frequency Thermal Control> on cpu7
(noperiph:siisch1:0:-1:-1): rescan already queued
Timecounters tick every 1.000 msec
ipfw2 (+ipv6) initialized, divert enabled, nat enabled, default to accept, logging disabled
DUMMYNET 0xfffffe000c260c00 with IPv6 initialized (100409)
load_dn_sched dn_sched RR loaded
load_dn_sched dn_sched WF2Q+ loaded
load_dn_sched dn_sched FIFO loaded
load_dn_sched dn_sched PRIO loaded
load_dn_sched dn_sched QFQ loaded
usbus0: 480Mbps High Speed USB v2.0
usbus1: 480Mbps High Speed USB v2.0
ugen0.1: <Intel> at usbus0
uhub0: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus0
ugen1.1: <Intel> at usbus1
uhub1: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
ada0 at siisch1 bus 0 scbus3 target 0 lun 0
ada0: <ST4000DM000-1F2168 CC52> ATA-8 SATA 3.x device
ada0: Serial Number Z300GYNA
ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada0: Command Queueing enabled
ada0: 3815447MB (7814037168 512 byte sectors: 16H 63S/T 16383C)
ada0: quirks=0x1<4K>
ada0: Previously was known as ad10
ada1 at ahcich2 bus 0 scbus4 target 0 lun 0
ada1: <HGST HDS724040ALE640 MJAOA580> ATA-8 SATA 3.x device
ada1: Serial Number PK2334PBJMG1MT
ada1: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada1: Command Queueing enabled
ada1: 3815447MB (7814037168 512 byte sectors: 16H 63S/T 16383C)
ada1: Previously was known as ad12
ada2 at ahcich3 bus 0 scbus5 target 0 lun 0
ada2: <Hitachi HDS724040ALE640 MJAOA3B0> ATA-8 SATA 3.x device
ada2: Serial Number PK1310PAG5U3XJ
ada2: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada2: Command Queueing enabled
ada2: 3815447MB (7814037168 512 byte sectors: 16H 63S/T 16383C)
ada2: Previously was known as ad14
ada3 at ahcich10 bus 0 scbus12 target 0 lun 0
ada3: <Hitachi HDS724040ALE640 MJAOA250> ATA-8 SATA 3.x device
ada3: Serial Number PK2310PAG053SM
ada3: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada3: Command Queueing enabled
ada3: 3815447MB (7814037168 512 byte sectors: 16H 63S/T 16383C)
ada3: Previously was known as ad28
ada4 at ahcich11 bus 0 scbus13 target 0 lun 0
ada4: <Hitachi HDS724040ALE640 MJAOA250> ATA-8 SATA 3.x device
ada4: Serial Number PK2311PAG09ULM
ada4: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada4: Command Queueing enabled
ada4: 3815447MB (7814037168 512 byte sectors: 16H 63S/T 16383C)
ada4: Previously was known as ad30
ada5 at ahcich12 bus 0 scbus14 target 0 lun 0
ada5: <Hitachi HDS724040ALE640 MJAOA3B0> ATA-8 SATA 3.x device
ada5: Serial Number PK1331PAGEB58S
ada5: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada5: Command Queueing enabled
ada5: 3815447MB (7814037168 512 byte sectors: 16H 63S/T 16383C)
ada5: Previously was known as ad32
ada6 at ahcich13 bus 0 scbus15 target 0 lun 0
ada6: <ST4000DM000-1F2168 CC52> ATA-8 SATA 3.x device
ada6: Serial Number Z300GQ73
ada6: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada6: Command Queueing enabled
ada6: 3815447MB (7814037168 512 byte sectors: 16H 63S/T 16383C)
ada6: quirks=0x1<4K>
ada6: Previously was known as ad34
ada7 at ahcich14 bus 0 scbus16 target 0 lun 0
ada7: <ST4000DM000-1F2168 CC52> ATA-8 SATA 3.x device
ada7: Serial Number Z300GZBB
ada7: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada7: Command Queueing enabled
ada7: 3815447MB (7814037168 512 byte sectors: 16H 63S/T 16383C)
ada7: quirks=0x1<4K>
ada7: Previously was known as ad36
ada8 at ahcich15 bus 0 scbus17 target 0 lun 0
ada8: <Hitachi HDS724040ALE640 MJAOA3B0> ATA-8 SATA 3.x device
ada8: Serial Number PK1331PAG8LHZS
ada8: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada8: Command Queueing enabled
ada8: 3815447MB (7814037168 512 byte sectors: 16H 63S/T 16383C)
ada8: Previously was known as ad38
pass3 at ahcich9 bus 0 scbus11 target 0 lun 0
pass3: <Marvell 91xx Config 1.01> Removable Processor SCSI-0 device
pass3: Serial Number HKDP221516WL
pass3: 150.000MB/s transfers (SATA 1.x, UDMA4, ATAPI 12bytes, PIO 8192bytes)
SMP: AP CPU #1 Launched!
SMP: AP CPU #7 Launched!
SMP: AP CPU #3 Launched!
SMP: AP CPU #2 Launched!
SMP: AP CPU #6 Launched!
SMP: AP CPU #5 Launched!
SMP: AP CPU #4 Launched!
Swap zone entries reduced from 7111631 to 4741087.
uhub0: 2 ports with 2 removable, self powered
uhub1: 2 ports with 2 removable, self powered
ugen0.2: <vendor 0x8087> at usbus0
uhub2: <vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr 2> on usbus0
ugen1.2: <vendor 0x8087> at usbus1
uhub3: <vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr 2> on usbus1
Root mount waiting for: usbus1 usbus0
uhub2: 6 ports with 6 removable, self powered
uhub3: 8 ports with 8 removable, self powered
ugen1.3: <Dell> at usbus1
ukbd0: <Dell Dell USB Keyboard, class 0/0, rev 1.10/3.01, addr 3> on usbus1
kbd2 at ukbd0
Root mount waiting for: usbus1
ugen1.4: <SMI Corporation> at usbus1
umass0: <SMI Corporation USB DISK, class 0/0, rev 2.10/11.00, addr 4> on usbus1
umass0:  SCSI over Bulk-Only; quirks = 0x4000
umass0:19:0:-1: Attached to scbus19
da0 at umass-sim0 bus 0 scbus19 target 0 lun 0
da0: <SMI USB DISK 1100> Removable Direct Access SCSI-6 device
da0: Serial Number 1114282799999999
da0: 40.000MB/s transfers
da0: 7500MB (15360000 512 byte sectors: 255H 63S/T 956C)
da0: quirks=0x3<NO_SYNC_CACHE,NO_6_BYTE>
ugen1.5: <vendor 0x0cf3> at usbus1
Trying to mount root from ufs:/dev/ufs/FreeNASs1a [ro]...
GEOM_RAID5: Module loaded, version 1.1.20130907.44 (rev 5c6d2a159411)
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
GEOM_ELI: Device ada0p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:     Crypto: hardware
GEOM_ELI: Device ada2p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:     Crypto: hardware
GEOM_ELI: Device ada3p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:     Crypto: hardware
GEOM_ELI: Device ada4p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:     Crypto: hardware
GEOM_ELI: Device ada5p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:     Crypto: hardware
GEOM_ELI: Device ada6p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:     Crypto: hardware
GEOM_ELI: Device ada7p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:     Crypto: hardware
GEOM_ELI: Device ada8p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:     Crypto: hardware
GEOM_ELI: Device ada1p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:     Crypto: hardware
[root@freenas] ~#

 

[mjws00]

Well, I was hoping we'd spot some sort of error or something interesting in there. The ixgbe driver is supported, so I don't know why it isn't being detected. Is it possible it needs to be manually loaded? i.e if_ixgbe_load="YES"

Maybe @cyberjock or one of the bsd wizards will have an idea. At the very least we've given them some info to start with.

 

[cyberjock]

mobo is an Asus P9X79 Deluxe with 64 GB ram (don't know speed) , Intel Core i7-3820 CPU @ 3.60GHz, various HDDs (mix between Hitachi and Seagate 4T drives, no other pci cards. ifconfig and dmesg are below. dmesg doesn't show anything.

Hate to break it to you, but that mobo isn't server-grade. There's a chance there's some obscure incompatibility with that board and FreeBSD as a result. :(

Entries like this:

1. pci255: <base peripheral> at device 8.0 (no driver attached)
2. pci255: <base peripheral> at device 8.3 (no driver attached)
3. pci255: <base peripheral> at device 8.4 (no driver attached)
4. pci255: <base peripheral> at device 9.0 (no driver attached)
5. pci255: <base peripheral> at device 9.3 (no driver attached)
6. pci255: <base peripheral> at device 9.4 (no driver attached)
7. pci255: <base peripheral> at device 10.0 (no driver attached)
8. pci255: <base peripheral> at device 10.1 (no driver attached)
9. pci255: <base peripheral> at device 10.2 (no driver attached)
10. pci255: <base peripheral> at device 10.3 (no driver attached)
11. pci255: <base peripheral> at device 11.0 (no driver attached)
12. pci255: <base peripheral> at device 11.3 (no driver attached)
13. pci255: <base peripheral> at device 12.0 (no driver attached)
14. pci255: <base peripheral> at device 12.1 (no driver attached)
15. pci255: <base peripheral> at device 12.6 (no driver attached)
16. pci255: <base peripheral> at device 12.7 (no driver attached)
17. pci255: <base peripheral> at device 13.0 (no driver attached)
18. pci255: <base peripheral> at device 13.1 (no driver attached)
19. pci255: <base peripheral> at device 13.6 (no driver attached)
20. pci255: <base peripheral> at device 14.0 (no driver attached)
21. pci255: <dasp> at device 14.1 (no driver attached)
22. pci255: <base peripheral> at device 15.0 (no driver attached)
23. pci255: <base peripheral> at device 15.1 (no driver attached)
24. pci255: <base peripheral> at device 15.2 (no driver attached)
25. pci255: <base peripheral> at device 15.3 (no driver attached)
26. pci255: <base peripheral> at device 15.4 (no driver attached)
27. pci255: <base peripheral> at device 15.5 (no driver attached)
28. pci255: <base peripheral> at device 15.6 (no driver attached)
29. pci255: <base peripheral> at device 16.0 (no driver attached)
30. pci255: <base peripheral> at device 16.1 (no driver attached)
31. pci255: <base peripheral> at device 16.2 (no driver attached)
32. pci255: <base peripheral> at device 16.3 (no driver attached)
33. pci255: <base peripheral> at device 16.4 (no driver attached)
34. pci255: <base peripheral> at device 16.5 (no driver attached)
35. pci255: <base peripheral> at device 16.6 (no driver attached)
36. pci255: <base peripheral> at device 16.7 (no driver attached)
37. pci255: <base peripheral> at device 17.0 (no driver attached)
38. pci255: <base peripheral> at device 19.0 (no driver attached)

might be a clue. But as that mobo is basically guaranteed to have stuff on-board that isn't supported there's no way to know for 100% certainty without deep level debugging if your 10Gb NIC is on there.

The only good answer I have is to get proper hardware and see if it works on there. Unfortunately, the Intel X520 should "just work" and since it's not I tend to think hardware incompatibility because of your choice of motherboard.

 

[mjws00]

That's what I was afraid of. Was hoping I was missing something. Thanks for popping by.

 

[liquidice]

Bummer. Thanks for the info though. I am curious, however; I have another workstation (running windows) with a P8Z77-V Premium motherboard that all three cards work in. I am under the assumption that they were the same class of board in their respective generations. Do you think is is more of a mobo to nic incompatibility or is FreeBSD part of the issue? I know you can't know all of the 10 gig nics out there, but if you had to guess, would you think there is one that would work in the current setup?

 

[mjws00]

BSD is definitely the problem. It's a combination of chipsets and components that is giving us grief, imho. The cards would be happy under windows, likely under linux. Maybe under bsd 10.1 or something...? x79 didn't get a lot of love, but I think they handled the basics? I have the pro version of the board, it was a fail at all but windows ;) But it was a new release when I was testing, so I think they've made progress.

 

[jgreco]

BSD is definitely the problem.

I'd be hesitant to make such a blanket statement. Prosumer hardware is funny stuff - sometimes a slot won't be able to support anything but graphics cards, for example. But in the specific case of the X520, the driver issues with that haven't been hammered out until, I believe, 9.2.1.8?

The failure of other cards, such as a Chelsio, to work, suggest that there's a possibility that the slot itself is incompatible; you might want to try installing your cards in a different PCIe slot and see if that changes anything.

You can use "pciconf -l" to find out what the system sees out there in PCIe land, this also gives us some clues as to other things.

Looking for messages in the dmesg output is also the right thing to do, look especially for things that are different when you move or swap cards.

x79 didn't get a lot of love, but I think they handled the basics? I have the pro version of the board, it was a fail at all but windows ;)

That kind of thing is sadly all too common in the PC world. Hardware gets hammered out that is "just good enough" to support Windows, and Windows has a lot of workarounds for brokenness in it. FreeBSD and Linux don't always have that level of tolerance for broken hardware because someone with broken hardware who actually has the skill and desire has to determine what's wrong and then figure out the correct fix, which happens somewhat less often in the land of free software than it does when the mainboard manufacturer is involved.

 

[liquidice]

thanks for the info. i will try a different pcie slot. i guess i shouldn't have high hopes though. maybe getting a 4x 1 gig port card and bonding the interfaces might server me better? would i need to port channel them at the switch then?

Code:

[root@freenas] ~# pciconf -l
hostb0@pci0:0:0:0:      class=0x060000 card=0x84ef1043 chip=0x3c008086 rev=0x07 hdr=0x00
pcib1@pci0:0:1:0:       class=0x060400 card=0x84ef1043 chip=0x3c028086 rev=0x07 hdr=0x01
pcib2@pci0:0:2:0:       class=0x060400 card=0x84ef1043 chip=0x3c048086 rev=0x07 hdr=0x01
pcib3@pci0:0:3:0:       class=0x060400 card=0x84ef1043 chip=0x3c088086 rev=0x07 hdr=0x01
none0@pci0:0:5:0:       class=0x088000 card=0x84ef1043 chip=0x3c288086 rev=0x07 hdr=0x00
none1@pci0:0:5:2:       class=0x088000 card=0x84ef1043 chip=0x3c2a8086 rev=0x07 hdr=0x00
ioapic0@pci0:0:5:4:     class=0x080020 card=0x84ef1043 chip=0x3c2c8086 rev=0x07 hdr=0x00
pcib4@pci0:0:17:0:      class=0x060400 card=0x84ef1043 chip=0x1d3e8086 rev=0x06 hdr=0x01
none2@pci0:0:22:0:      class=0x078000 card=0x84ef1043 chip=0x1d3a8086 rev=0x05 hdr=0x00
em0@pci0:0:25:0:        class=0x020000 card=0x849c1043 chip=0x15038086 rev=0x06 hdr=0x00
ehci0@pci0:0:26:0:      class=0x0c0320 card=0x84ef1043 chip=0x1d2d8086 rev=0x06 hdr=0x00
none3@pci0:0:27:0:      class=0x040300 card=0x84d81043 chip=0x1d208086 rev=0x06 hdr=0x00
pcib5@pci0:0:28:0:      class=0x060400 card=0x84ef1043 chip=0x1d108086 rev=0xb6 hdr=0x01
pcib6@pci0:0:28:1:      class=0x060400 card=0x84ef1043 chip=0x1d128086 rev=0xb6 hdr=0x01
pcib7@pci0:0:28:2:      class=0x060400 card=0x84ef1043 chip=0x1d148086 rev=0xb6 hdr=0x01
pcib8@pci0:0:28:3:      class=0x060400 card=0x84ef1043 chip=0x1d168086 rev=0xb6 hdr=0x01
pcib9@pci0:0:28:4:      class=0x060400 card=0x84ef1043 chip=0x1d188086 rev=0xb6 hdr=0x01
pcib10@pci0:0:28:7:     class=0x060400 card=0x84ef1043 chip=0x1d1e8086 rev=0xb6 hdr=0x01
ehci1@pci0:0:29:0:      class=0x0c0320 card=0x84ef1043 chip=0x1d268086 rev=0x06 hdr=0x00
pcib11@pci0:0:30:0:     class=0x060401 card=0x84ef1043 chip=0x244e8086 rev=0xa6 hdr=0x01
isab0@pci0:0:31:0:      class=0x060100 card=0x84ef1043 chip=0x1d418086 rev=0x06 hdr=0x00
ahci2@pci0:0:31:2:      class=0x010601 card=0x84ef1043 chip=0x1d028086 rev=0x06 hdr=0x00
none4@pci0:0:31:3:      class=0x0c0500 card=0x84ef1043 chip=0x1d228086 rev=0x06 hdr=0x00
vgapci0@pci0:1:0:0:     class=0x030000 card=0x13103842 chip=0x0a6510de rev=0xa2 hdr=0x00
none5@pci0:1:0:1:       class=0x040300 card=0x13103842 chip=0x0be310de rev=0xa1 hdr=0x00
ahci0@pci0:6:0:0:       class=0x010601 card=0x84b71043 chip=0x06121b21 rev=0x01 hdr=0x00
none6@pci0:7:0:0:       class=0x0c0330 card=0x84881043 chip=0x10421b21 rev=0x00 hdr=0x00
none7@pci0:8:0:0:       class=0x0c0330 card=0x84881043 chip=0x10421b21 rev=0x00 hdr=0x00
siis0@pci0:9:0:0:       class=0x010600 card=0x31321095 chip=0x31321095 rev=0x01 hdr=0x00
ahci1@pci0:10:0:0:      class=0x010601 card=0x84381043 chip=0x91301b4b rev=0x11 hdr=0x00
none8@pci0:255:8:0:     class=0x088000 card=0x84ef1043 chip=0x3c808086 rev=0x07 hdr=0x00
none9@pci0:255:8:3:     class=0x088000 card=0x84ef1043 chip=0x3c838086 rev=0x07 hdr=0x00
none10@pci0:255:8:4:    class=0x088000 card=0x84ef1043 chip=0x3c848086 rev=0x07 hdr=0x00
none11@pci0:255:9:0:    class=0x088000 card=0x84ef1043 chip=0x3c908086 rev=0x07 hdr=0x00
none12@pci0:255:9:3:    class=0x088000 card=0x84ef1043 chip=0x3c938086 rev=0x07 hdr=0x00
none13@pci0:255:9:4:    class=0x088000 card=0x84ef1043 chip=0x3c948086 rev=0x07 hdr=0x00
none14@pci0:255:10:0:   class=0x088000 card=0x84ef1043 chip=0x3cc08086 rev=0x07 hdr=0x00
none15@pci0:255:10:1:   class=0x088000 card=0x84ef1043 chip=0x3cc18086 rev=0x07 hdr=0x00
none16@pci0:255:10:2:   class=0x088000 card=0x84ef1043 chip=0x3cc28086 rev=0x07 hdr=0x00
none17@pci0:255:10:3:   class=0x088000 card=0x84ef1043 chip=0x3cd08086 rev=0x07 hdr=0x00
none18@pci0:255:11:0:   class=0x088000 card=0x84ef1043 chip=0x3ce08086 rev=0x07 hdr=0x00
none19@pci0:255:11:3:   class=0x088000 card=0x84ef1043 chip=0x3ce38086 rev=0x07 hdr=0x00
none20@pci0:255:12:0:   class=0x088000 card=0x84ef1043 chip=0x3ce88086 rev=0x07 hdr=0x00
none21@pci0:255:12:1:   class=0x088000 card=0x84ef1043 chip=0x3ce88086 rev=0x07 hdr=0x00
none22@pci0:255:12:6:   class=0x088000 card=0x84ef1043 chip=0x3cf48086 rev=0x07 hdr=0x00
none23@pci0:255:12:7:   class=0x088000 card=0x84ef1043 chip=0x3cf68086 rev=0x07 hdr=0x00
none24@pci0:255:13:0:   class=0x088000 card=0x84ef1043 chip=0x3ce88086 rev=0x07 hdr=0x00
none25@pci0:255:13:1:   class=0x088000 card=0x84ef1043 chip=0x3ce88086 rev=0x07 hdr=0x00
none26@pci0:255:13:6:   class=0x088000 card=0x84ef1043 chip=0x3cf58086 rev=0x07 hdr=0x00
none27@pci0:255:14:0:   class=0x088000 card=0x84ef1043 chip=0x3ca08086 rev=0x07 hdr=0x00
none28@pci0:255:14:1:   class=0x110100 card=0x84ef1043 chip=0x3c468086 rev=0x07 hdr=0x00
none29@pci0:255:15:0:   class=0x088000 card=0x84ef1043 chip=0x3ca88086 rev=0x07 hdr=0x00
none30@pci0:255:15:1:   class=0x088000 card=0x84ef1043 chip=0x3c718086 rev=0x07 hdr=0x00
none31@pci0:255:15:2:   class=0x088000 card=0x84ef1043 chip=0x3caa8086 rev=0x07 hdr=0x00
none32@pci0:255:15:3:   class=0x088000 card=0x84ef1043 chip=0x3cab8086 rev=0x07 hdr=0x00
none33@pci0:255:15:4:   class=0x088000 card=0x84ef1043 chip=0x3cac8086 rev=0x07 hdr=0x00
none34@pci0:255:15:5:   class=0x088000 card=0x84ef1043 chip=0x3cad8086 rev=0x07 hdr=0x00
none35@pci0:255:15:6:   class=0x088000 card=0x84ef1043 chip=0x3cae8086 rev=0x07 hdr=0x00
none36@pci0:255:16:0:   class=0x088000 card=0x84ef1043 chip=0x3cb08086 rev=0x07 hdr=0x00
none37@pci0:255:16:1:   class=0x088000 card=0x84ef1043 chip=0x3cb18086 rev=0x07 hdr=0x00
none38@pci0:255:16:2:   class=0x088000 card=0x84ef1043 chip=0x3cb28086 rev=0x07 hdr=0x00
none39@pci0:255:16:3:   class=0x088000 card=0x84ef1043 chip=0x3cb38086 rev=0x07 hdr=0x00
none40@pci0:255:16:4:   class=0x088000 card=0x84ef1043 chip=0x3cb48086 rev=0x07 hdr=0x00
none41@pci0:255:16:5:   class=0x088000 card=0x84ef1043 chip=0x3cb58086 rev=0x07 hdr=0x00
none42@pci0:255:16:6:   class=0x088000 card=0x84ef1043 chip=0x3cb68086 rev=0x07 hdr=0x00
none43@pci0:255:16:7:   class=0x088000 card=0x84ef1043 chip=0x3cb78086 rev=0x07 hdr=0x00
none44@pci0:255:17:0:   class=0x088000 card=0x84ef1043 chip=0x3cb88086 rev=0x07 hdr=0x00
none45@pci0:255:19:0:   class=0x088000 card=0x84ef1043 chip=0x3ce48086 rev=0x07 hdr=0x00
none46@pci0:255:19:1:   class=0x110100 card=0x84ef1043 chip=0x3c438086 rev=0x07 hdr=0x00
none47@pci0:255:19:4:   class=0x110100 card=0x84ef1043 chip=0x3ce68086 rev=0x07 hdr=0x00
none48@pci0:255:19:5:   class=0x110100 card=0x84ef1043 chip=0x3c448086 rev=0x07 hdr=0x00
none49@pci0:255:19:6:   class=0x088000 card=0x84ef1043 chip=0x3c458086 rev=0x07 hdr=0x00

 

[jgreco]

There 's no reason to expect another network card to work any better (slower is just slower). The damn thing about this business though is that you actually need to try things to know for sure.

The 10G is a lot easier in many ways. For link aggregation you have work on both the switch and the server, plus load balancing does not always work out the way you'd like.

 

[mjws00]

If I had the hardware in front of me. I'd fire up a linux livecd, and also a FreeBSD 10.1 and see if the card "just worked". I don't know what has been fixed but the chipset is "old" now.

I tried to run ESXI on it, and it wasn't worth the fight. But Win7 Pro with VMWare workstation has been very stable, a couple reboots by choice in over 2.5 years. In hindsight I would have bought something else, but it was the best board I could get locally at the time. 64GB and a 3820 was pretty formidable in its day. I have twins and they were temperamental early on.

Now I'd punt it on ebay and just grab a Supermicro. I blame cyberjock and jgreco completely for that. Or just run windows ;).