Hello
This is my first time setting up a FreeNAS/TrueNAS. I run into a weird issue that I cannot find answers anywhere.
My hardware:
Dell PowerEdge R720XD chassis
2x Intel Xeon E5 2667 v2
256GB of RAM
12x 4TB SAS at front, 1x Samsung SSD at rear flex.
1x H310 mini mono
1x H310 Full size
Everything else is just basic R720XD hardware (I believe it's 12th gen).
My problem is that the the system does not allow me to boot from H310 Full size.
Background:
I purchased this server not only for FreeNAS/TrueNAS, but also for virtualization on Proxmox. The plan is to install Proxmox on the one SSD at rear flex and run a FreeNAS VM. And then pass the front 12 SAS drives to the FreeNAS VM. I specifically picked this CPU because it has the highest frequency among the same generation. So it is very useful for a lot of other virtualization tasks besides FreeNAS.
You might wonder why I have two HBA controllers. At first I did not know H310 mini and H310 are different. I proceeded to flash H310 mini with the H310 instruction and ended up bricking it. After I learned my mistake, I quickly purchased a H310, along with a USB eeprom programmer to fix H310 mini. Eventually I have both HBA flashed to IT mode. Both of them are tested working.
I proceeded with H310 mini to setup Proxmox and TrueNAS core. But then I realized that I need to pass through the HBA to TrueNAS for best performance. Being a noob, I did just that. And Voilà! My proxmox crashed because it lost access to the one and only HBA.
Well I still have a full size H310. So I think maybe I can connect both HBA to the server, and connect the SSD to only the H310 full size, and leave the rest to H310 mini.
However, no matter what I do, the system won't recognize H310 as a bootable device. It can only boot from H310 mini. I checked BIOS settings. All PCI-E slots are enabled to be bootable. Both H310 and H310 mini are recognized by BIOS. If I boot to a iso (through USB or virtual disk), I can use
I tried to connect the whole back plane to H310 full size. Does not boot.
I tried to connect only the SSD to H310 full size through cable. Does not boot. System shows a error saying a cable connected to wrong port.
I also taped B5 and B6 on H310. Nothing changed.
I booted to a live OS to check whether H310 is working properly for all of the above configurations. The OS can recognize both the H310 and the hard drive attached.
I believe my H310 full size is working properly, but the system refuse to recognize it as a bootable device. I even successfully installed proxmox on the SSD that was connected to H310 while the back plane with the SAS drives connected to H310 mini. It means that both HBA can operate together.
Please provide me some thoughts. I have exhausted all my research.
P.S. I was able to have a TrueNAS core VM working by passing through hard drives instead of HBA. According to my research, this is not the best practice. I achieved this set up with H310 mini.
This is my first time setting up a FreeNAS/TrueNAS. I run into a weird issue that I cannot find answers anywhere.
My hardware:
Dell PowerEdge R720XD chassis
2x Intel Xeon E5 2667 v2
256GB of RAM
12x 4TB SAS at front, 1x Samsung SSD at rear flex.
1x H310 mini mono
1x H310 Full size
Everything else is just basic R720XD hardware (I believe it's 12th gen).
My problem is that the the system does not allow me to boot from H310 Full size.
Background:
I purchased this server not only for FreeNAS/TrueNAS, but also for virtualization on Proxmox. The plan is to install Proxmox on the one SSD at rear flex and run a FreeNAS VM. And then pass the front 12 SAS drives to the FreeNAS VM. I specifically picked this CPU because it has the highest frequency among the same generation. So it is very useful for a lot of other virtualization tasks besides FreeNAS.
You might wonder why I have two HBA controllers. At first I did not know H310 mini and H310 are different. I proceeded to flash H310 mini with the H310 instruction and ended up bricking it. After I learned my mistake, I quickly purchased a H310, along with a USB eeprom programmer to fix H310 mini. Eventually I have both HBA flashed to IT mode. Both of them are tested working.
I proceeded with H310 mini to setup Proxmox and TrueNAS core. But then I realized that I need to pass through the HBA to TrueNAS for best performance. Being a noob, I did just that. And Voilà! My proxmox crashed because it lost access to the one and only HBA.
Well I still have a full size H310. So I think maybe I can connect both HBA to the server, and connect the SSD to only the H310 full size, and leave the rest to H310 mini.
However, no matter what I do, the system won't recognize H310 as a bootable device. It can only boot from H310 mini. I checked BIOS settings. All PCI-E slots are enabled to be bootable. Both H310 and H310 mini are recognized by BIOS. If I boot to a iso (through USB or virtual disk), I can use
lspci | grep LSI
to see both HBAs and use lsscsi
to see all drives attached to them. To make it less confusing. I also removed H310 mini leaving only H310 full size. The system simply does not recognize H310 full size as a bootable device.I tried to connect the whole back plane to H310 full size. Does not boot.
I tried to connect only the SSD to H310 full size through cable. Does not boot. System shows a error saying a cable connected to wrong port.
I also taped B5 and B6 on H310. Nothing changed.
I booted to a live OS to check whether H310 is working properly for all of the above configurations. The OS can recognize both the H310 and the hard drive attached.
./sas2flash -c 0 -list
shows this result. It seems some information is N/A. Is that normal? I lost my original SAS address, so I just made up one.I believe my H310 full size is working properly, but the system refuse to recognize it as a bootable device. I even successfully installed proxmox on the SSD that was connected to H310 while the back plane with the SAS drives connected to H310 mini. It means that both HBA can operate together.
Please provide me some thoughts. I have exhausted all my research.
P.S. I was able to have a TrueNAS core VM working by passing through hard drives instead of HBA. According to my research, this is not the best practice. I achieved this set up with H310 mini.