SOLVED FreeNAS 9.3 FC Fibre Channel Target Mode DAS & SAN

Holt Andrei Tiberiu

Contributor
Joined
Jan 13, 2016
Messages
129
QLE2462 is a 4 GBS card, 400 mb+ is a maximum, so my question is, what is the problem? or you are using 2 x QLE2462 ?

1 Gigabit is 125 megabyte.

On FC you can use 80% of the total bandwith, the rest 20% is rezerved for comunication.

Your values are correct for 1 x 4GBS connection.
 

Letni

Explorer
Joined
Jan 22, 2012
Messages
63
Hi Mav,
Thanks again for helping me setup fiber channel storage years ago. I'm back with a quick question.
The QLE2462 4Gb HBAs I have are maxing out on FC at 410.91MB/s read consistently.
Using iSCSI vdev extent with two striped SSDs. (Testing on initiator with Crystal Disk Mark on bare-metal Windows 7)

Created a jail to run Bonnie++ which outputs...
610m on Sequential Output and 945m on Sequential Input

I had a spare mobo so I swapped it out - CPU and RAM power now doubled and I'm getting the same results. Phenom 980 3.7GHz 16GB RAM.

Also seeing someone else online with this HBA model get maxed out at 400MBps too.

Could this be a driver issue or just a bad HBA?
As others have stated, 400 MByte/Second is the MAX of a single FC connection.. Luckily FC is NOT like IP/Ethernet in you can easily "combine" the connection and gain speed directly for the same data channel.. This is known as multi-pathing (ie adding paths for redundancy and speed).

Look into Multipathing drivers on your host (the machine receiving the disks from FreeNAS). If you connect 2 x cables with the same volumes being presented down to the host, and properly multi path, you will easily increase the disk transfer speeds..

In VMware (there is native multipathing software and configuration built in), I use the QLE2462 and plug in both FC ports from FreeNAS to my ESXi. As long as I configure the round-robin (a parameter of the multipathing software) on each volume, I get effectively 800 MByte/Second transfer to each disk.. It scales up (though realistically 4-8 paths is the practical limit)
 

Holt Andrei Tiberiu

Contributor
Joined
Jan 13, 2016
Messages
129
I confirm what Letni wrote.
Also for 8 GBs, i used it in the past. Achieved over 1400 meg/sec with 2 x 8 GBS QLE Cards.
 

sfcredfox

Patron
Joined
Aug 26, 2014
Messages
340
Anyone using FC target mode with version 11.3?

I can't find any specific posts about 11.3 and FC. One would assume that it still supports ISP2432 based cards, but I always like to see if anyone has personal experience.

My system specifically is providing shared storage to ESXi 6.5, so I'm curious if anyone is personally using 11.3 with FC to ESXi 6.5?
 

Holt Andrei Tiberiu

Contributor
Joined
Jan 13, 2016
Messages
129
It works, tha same way as in the other versions.

I have a few clients using it, i can confirm 11.3u1 also
 

shistory

Cadet
Joined
Jun 28, 2020
Messages
4
FreeNAS-11.3-U3.2
I can't find Qlogic ISP 2532,What shall I do?


root@freenas[~]# dmesg | grep isp
vgapci0: <VGA-compatible display> port 0x1070-0x107f mem 0xe8000000-0xefffffff,0xfe000000-0xfe7fffff irq 16 at device 15.0 on pci0
isp0: <Qlogic ISP 2532 PCI FC-AL Adapter> port 0x6000-0x60ff mem 0xfd2fc000-0xfd2fffff irq 16 at device 0.0 on pci5
isp0: Chan 0 setting role to 0x2
isp1: <Qlogic ISP 2532 PCI FC-AL Adapter> port 0x6400-0x64ff mem 0xfd2f8000-0xfd2fbfff irq 17 at device 0.1 on pci5
isp1: Chan 0 setting role to 0x2
 

shistory

Cadet
Joined
Jun 28, 2020
Messages
4
FreeNAS-11.3-U3.2
I can't find Qlogic ISP 2532,What shall I do?


root@FreeNAS[~]# dmesg | grep isp
vgapci0: <VGA-compatible display> port 0x1070-0x107f mem 0xe8000000-0xefffffff,0xfe000000-0xfe7fffff irq 16 at device 15.0 on pci0
isp0: <Qlogic ISP 2532 PCI FC-AL Adapter> port 0x6000-0x60ff mem 0xfd2fc000-0xfd2fffff irq 16 at device 0.0 on pci5
isp0: Chan 0 setting role to 0x2
isp1: <Qlogic ISP 2532 PCI FC-AL Adapter> port 0x6400-0x64ff mem 0xfd2f8000-0xfd2fbfff irq 17 at device 0.1 on pci5
isp1: Chan 0 setting role to 0x2
@mav@
 

Holt Andrei Tiberiu

Contributor
Joined
Jan 13, 2016
Messages
129
you should reset the cards in thei'r bios to factory default.
Also do a FW update to the latest Version..
 

mav@

iXsystems
iXsystems
Joined
Sep 29, 2011
Messages
1,428
I can't find Qlogic ISP 2532,What shall I do?
My crystal ball is not functioning to guess what problem you may have or even what are you trying to achieve without any input.
The role 0x2 means initiator. The recommendation for target is to set role to 0 (disabled) via loader.conf tunable and then enable target mode through the `ctladm` when system is booted. At least first step seems like not done. So please, help everybody and read available materials first.
 

orion88

Cadet
Joined
Jul 19, 2021
Messages
2
Hi,

I made the configuration from the guide on the TrueNas 12.0-U4 version and I find that everything works as planned.

I shared the disk space with FC cards for hosts running vmware esxi 6.5 and from vcenter I configured the resource shared with FC technology and everything worked fine.

Thanks for the instructions.

Greetings
 

sfcredfox

Patron
Joined
Aug 26, 2014
Messages
340
@orion88,
You sir, read my mind. Thanks.
 

orion88

Cadet
Joined
Jul 19, 2021
Messages
2
Hi,

I made the configuration from the guide on the TrueNas 12.0-U6 version and I find that everything works as planned.

I used an FC QLogic QLE2564 8G card and it works fine.
I shared the disk space with FC cards for hosts running vmware esxi 6.5 and from vcenter I configured the resource shared with FC technology and everything worked fine.
I also plan to test the configuration on Proxmox.

Greetings
 

sfcredfox

Patron
Joined
Aug 26, 2014
Messages
340
Hi,

I made the configuration from the guide on the TrueNas 12.0-U6 version and I find that everything works as planned.

I used an FC QLogic QLE2564 8G card and it works fine.
I shared the disk space with FC cards for hosts running vmware esxi 6.5 and from vcenter I configured the resource shared with FC technology and everything worked fine.
I also plan to test the configuration on Proxmox.

Greetings
Has anyone tried the same scenario on TrueNAS scale? I have not come across any guides yet. Tunables turn into something in Debian I assume?
 
Joined
Apr 26, 2015
Messages
320
I enjoyed FC on freenas and trusnas for many years without problems.
I've moved on to 10GbE instead but I'd love to keep the configuration.
Is there a way to backup just that config?
 

philxor

Cadet
Joined
Apr 29, 2023
Messages
5
Sorry for bringing up an old thread, but trying to get this working and having some difficulty. I have a QLE 2562 based card in my server with 4 8G ports. I'm using ESXI and passing just one of the ports to a TrueNAS

Most of the output on TrueNAS seems okay and I believe the SAN switches are setup correctly, but I can't see the LUN I've created on another ESX host connected via FCoE.

ctlfe_onoffline: isp0 current WWNN 0x20000024ff780ec0
ctlfe_onoffline: isp0 current WWPN 0x21000024ff780ec0
ctlfe_onoffline: isp0 (path id 3) target role enable succeeded

root@truenas[~]# ctladm portlist -i
Port Online Frontend Name pp vp
0 NO camsim camsim 0 0 naa.5000000d8a27f701
Target: naa.5000000d8a27f700
1 YES ioctl ioctl 0 0
2 YES tpc tpc 0 0
3 YES camtgt isp0 0 0 naa.21000024ff780ec0
Target: naa.20000024ff780ec0
4 YES iscsi iscsi 1 1 iqn.2005-10.org.freenas.ctl:ronfc,t,0x0001
Target: iqn.2005-10.org.freenas.ctl:ronfc

root@truenas[~]# ctladm portlist -l
Port Online Frontend Name pp vp
0 NO camsim camsim 0 0 naa.5000000d8a27f701
All LUNs mapped
1 YES ioctl ioctl 0 0
All LUNs mapped
2 YES tpc tpc 0 0
All LUNs mapped
3 YES camtgt isp0 0 0 naa.21000024ff780ec0
All LUNs mapped
4 YES iscsi iscsi 1 1 iqn.2005-10.org.freenas.ctl:ronfc,t,0x0001
No LUNs mapped

I noticed on the SAN switch (Cisco MDS) the port shows up like this, but thought it would be scsi-fcp:target?
0x790000 N 21:00:00:24:ff:78:0e:c3 (Qlogic) scsi-fcp:init

I can't seem to get into the QLogic BIOS on this UCS server to check any potential settings there.

Any help is appreciated.
 

Simon Sparks

Explorer
Joined
May 24, 2016
Messages
57

Simon Sparks

Explorer
Joined
May 24, 2016
Messages
57
sorry for hijacking the post a little bit. I have no previous experiences of FC and FC switch. I have did what Aran said in the post but i have no idea what i should do with my brocade switch.
can anyone tell the basic flows/steps ? such as setup zone? and etc
I believe I can look up the cli guides online.
I couldn't use the ezswitch software because some java related issue.
Download the OLD java then use the JAVA control panel to disable your NEWest JAVA version and enable the old one then launch the FC switch management application.
 

philxor

Cadet
Joined
Apr 29, 2023
Messages
5
@philxor have you followed all the screenshots in my post


You need to add tunables and also an init script.

Hi, yes I've followed all the instructions in the post and that seems to be working okay. Thanks for the detailed post. If I reboot TrueNAS I see the same output via ctladm. If I change the role in the tunables it reflects the change in role on the card. I'm still going through the SAN switch configuration to make sure I'm not missing something and also directly attaching another port on the server to the QLogic HBA to rule that out.
 

Simon Sparks

Explorer
Joined
May 24, 2016
Messages
57
Which switch do you have?

I have the Cisco MDS9148 so I can help with the config of that if that helps you.

In a perfect world you have a zone with contains a SINGLE fibre channel target device and a SINGLE fibre channel initiator.

However it is acceptable to have a zone with contains a SINGLE fibre channel target device and a MULTIPLE fibre channel initiators.

e.g. assuming you have a QLC2694 fibre channel card in your TrueNAS

Zone 1 - TrueNAS - FC WWPort 1 and 4 x ESXi Hosts x 2 FC Ports
Zone 2 - TrueNAS - FC WWPort 2 and 4 x ESXi Hosts x 2 FC Ports
Zone 3 - TrueNAS - FC WWPort 3 and 4 x ESXi Hosts x 2 FC Ports
Zone 4 - TrueNAS - FC WWPort 4 and 4 x ESXi Hosts x 2 FC Ports

therefore each zone will contain 9 entries, 1 FC target and 8 FC initiators.
 
Top