Areca 1280ML-24 IRQ storm and CCB time out

[cyberjock]

I have an Areca 1280ML-24 with the latest firmware (1.49 12/2/2010). I have started experimenting with ESXi and I have the card on a slot that is not behind the NF200 chip so that I can support PCI passthrough.

First I experimented with the 8.3.1 nightly. That didn't go too well. So I went to 8.3-RELEASE. Now I have different problems, but still problems I can't resolve by searching.

Note that if I boot from FreeNAS 8.3 on a USB key I do not have any of these issues. I can issue SMART tests and obtain SMART info. Everything seems to work fine.. if I use USB. If I use ESXi 5.1 then things change.

Currently the system has a 250GB drive hooked up to my motherboard's Intel SATA port as the boot drive. I also have a 250GB drive on my Areca as a test drive to see if I can get proper passthrough. The controller is in RAID mode with the one attached hard drive setup in passthrough mode. I gave the VM 6GB of RAM while I'm testing, then I'll up it to at least 10GB.

During bootup the card is properly identified and the driver loads (arcmsr0). Then everything starts going wrong...

Code:

arcmsr_dr_handle: Target=0, lun=0, Plug-IN!!!
run_interrupt_driven_hooks: still waiting after 60 seconds for xpt_config
run_interrupt_driven_hooks: still waiting after 120 seconds for xpt_config
arcmsr0: scsi id 0 lun 0 cmd=0x12 srb='0xffffff8000176000' ccb command time out!
arcmsr0: scsi id 1 lun 0 cmd=0x12 srb='0xffffff8000176260' ccb command time out!
arcmsr0: scsi id 2 lun 0 cmd=0x12 srb='0xffffff80001764c0' ccb command time out!
arcmsr0: scsi id 3 lun 0 cmd=0x12 srb='0xffffff8000176720' ccb command time out!
arcmsr0: scsi id 4 lun 0 cmd=0x12 srb='0xffffff8000176980' ccb command time out!
arcmsr0: scsi id 5 lun 0 cmd=0x12 srb='0xffffff8000176be0' ccb command time out!
arcmsr0: scsi id 6 lun 0 cmd=0x12 srb='0xffffff8000176e40' ccb command time out!
arcmsr0: scsi id 7 lun 0 cmd=0x12 srb='0xffffff80001770a0' ccb command time out!
arcmsr0: scsi id 8 lun 0 cmd=0x12 srb='0xffffff8000177300' ccb command time out!
arcmsr0: scsi id 9 lun 0 cmd=0x12 srb='0xffffff8000177560' ccb command time out!
arcmsr0: scsi id 10 lun 0 cmd=0x12 srb='0xffffff80001777c0' ccb command time out!
arcmsr0: scsi id 11 lun 0 cmd=0x12 srb='0xffffff8000177a20' ccb command time out!
arcmsr0: scsi id 12 lun 0 cmd=0x12 srb='0xffffff8000177c80' ccb command time out!
arcmsr0: scsi id 13 lun 0 cmd=0x12 srb='0xffffff8000177ee0' ccb command time out!
arcmsr0: scsi id 14 lun 0 cmd=0x12 srb='0xffffff8000178140' ccb command time out!
arcmsr0: scsi id 15 lun 0 cmd=0x12 srb='0xffffff80001783a0' ccb command time out!
arcmsr0: scsi id 17 lun 0 cmd=0x12 srb='0xffffff8000178600' ccb command time out!
run_interrupt_driven_hooks: still waiting after 180 seconds for xpt_config
run_interrupt_driven_hooks: still waiting after 240 seconds for xpt_config
arcmsr0: scsi id 0 lun 0 cmd=0x12 srb='0xffffff8000178860' ccb command time out!
arcmsr0: scsi id 1 lun 0 cmd=0x12 srb='0xffffff8000178ac0' ccb command time out!
arcmsr0: scsi id 2 lun 0 cmd=0x12 srb='0xffffff8000178d20' ccb command time out!
arcmsr0: scsi id 3 lun 0 cmd=0x12 srb='0xffffff8000178f80' ccb command time out!
arcmsr0: scsi id 4 lun 0 cmd=0x12 srb='0xffffff80001791e0' ccb command time out!
arcmsr0: scsi id 5 lun 0 cmd=0x12 srb='0xffffff8000179440' ccb command time out!
arcmsr0: scsi id 6 lun 0 cmd=0x12 srb='0xffffff80001796a0' ccb command time out!
arcmsr0: scsi id 7 lun 0 cmd=0x12 srb='0xffffff8000179900' ccb command time out!
arcmsr0: scsi id 8 lun 0 cmd=0x12 srb='0xffffff8000179b60' ccb command time out!
arcmsr0: scsi id 9 lun 0 cmd=0x12 srb='0xffffff8000179dc0' ccb command time out!
arcmsr0: scsi id 10 lun 0 cmd=0x12 srb='0xffffff800017a020' ccb command time out!
arcmsr0: scsi id 11 lun 0 cmd=0x12 srb='0xffffff800017a280' ccb command time out!
arcmsr0: scsi id 12 lun 0 cmd=0x12 srb='0xffffff800017a4e0' ccb command time out!
arcmsr0: scsi id 13 lun 0 cmd=0x12 srb='0xffffff800017a740' ccb command time out!
arcmsr0: scsi id 14 lun 0 cmd=0x12 srb='0xffffff800017a9a0' ccb command time out!
arcmsr0: scsi id 15 lun 0 cmd=0x12 srb='0xffffff800017ac00' ccb command time out!
arcmsr0: scsi id 17 lun 0 cmd=0x12 srb='0xffffff800017ae60' ccb command time out!
run_interrupt_driven_hooks: still waiting after 300 seconds for xpt_config
arcmsr0: scsi id 0 lun 0 cmd=0x12 srb='0xffffff800017b0c0' ccb command time out!
arcmsr0: scsi id 1 lun 0 cmd=0x12 srb='0xffffff800017b320' ccb command time out!
arcmsr0: scsi id 2 lun 0 cmd=0x12 srb='0xffffff800017b580' ccb command time out!
arcmsr0: scsi id 3 lun 0 cmd=0x12 srb='0xffffff800017b7e0' ccb command time out!
arcmsr0: scsi id 4 lun 0 cmd=0x12 srb='0xffffff800017ba40' ccb command time out!
arcmsr0: scsi id 5 lun 0 cmd=0x12 srb='0xffffff800017bca0' ccb command time out!
arcmsr0: scsi id 6 lun 0 cmd=0x12 srb='0xffffff800017bf00' ccb command time out!
arcmsr0: scsi id 7 lun 0 cmd=0x12 srb='0xffffff800017c160' ccb command time out!
arcmsr0: scsi id 8 lun 0 cmd=0x12 srb='0xffffff800017c3c0' ccb command time out!
arcmsr0: scsi id 9 lun 0 cmd=0x12 srb='0xffffff800017c620' ccb command time out!
arcmsr0: scsi id 10 lun 0 cmd=0x12 srb='0xffffff800017c880' ccb command time out!
arcmsr0: scsi id 11 lun 0 cmd=0x12 srb='0xffffff800017cae0' ccb command time out!
arcmsr0: scsi id 12 lun 0 cmd=0x12 srb='0xffffff800017cd40' ccb command time out!
arcmsr0: scsi id 13 lun 0 cmd=0x12 srb='0xffffff800017cfa0' ccb command time out!
arcmsr0: scsi id 14 lun 0 cmd=0x12 srb='0xffffff800017d200' ccb command time out!
arcmsr0: scsi id 15 lun 0 cmd=0x12 srb='0xffffff800017d460' ccb command time out!
arcmsr0: scsi id 17 lun 0 cmd=0x12 srb='0xffffff800017d6c0' ccb command time out!
arcmsr0: scsi id 0 lun 0 cmd=0x12 srb='0xffffff800017d920' ccb command time out!
arcmsr0: scsi id 1 lun 0 cmd=0x12 srb='0xffffff800017db80' ccb command time out!
arcmsr0: scsi id 2 lun 0 cmd=0x12 srb='0xffffff800017dde0' ccb command time out!
arcmsr0: scsi id 3 lun 0 cmd=0x12 srb='0xffffff800017e040' ccb command time out!
arcmsr0: scsi id 4 lun 0 cmd=0x12 srb='0xffffff800017e2a0' ccb command time out!
arcmsr0: scsi id 5 lun 0 cmd=0x12 srb='0xffffff800017e500' ccb command time out!
arcmsr0: scsi id 6 lun 0 cmd=0x12 srb='0xffffff800017e760' ccb command time out!
arcmsr0: scsi id 7 lun 0 cmd=0x12 srb='0xffffff800017e9c0' ccb command time out!
arcmsr0: scsi id 8 lun 0 cmd=0x12 srb='0xffffff800017ec20' ccb command time out!
arcmsr0: scsi id 9 lun 0 cmd=0x12 srb='0xffffff800017ee80' ccb command time out!
arcmsr0: scsi id 10 lun 0 cmd=0x12 srb='0xffffff800017f0e0' ccb command time out!
arcmsr0: scsi id 11 lun 0 cmd=0x12 srb='0xffffff800017f340' ccb command time out!
arcmsr0: scsi id 12 lun 0 cmd=0x12 srb='0xffffff800017f5a0' ccb command time out!
arcmsr0: scsi id 13 lun 0 cmd=0x12 srb='0xffffff800017f800' ccb command time out!
arcmsr0: scsi id 14 lun 0 cmd=0x12 srb='0xffffff800017fa60' ccb command time out!
arcmsr0: scsi id 15 lun 0 cmd=0x12 srb='0xffffff800017fcc0' ccb command time out!
arcmsr0: scsi id 17 lun 0 cmd=0x12 srb='0xffffff800017ff20' ccb command time out!
arcmsr0: scsi id 0 lun 0 cmd=0x12 srb='0xffffff8000180180' ccb command time out!
arcmsr0: scsi id 1 lun 0 cmd=0x12 srb='0xffffff80001803e0' ccb command time out!
arcmsr0: scsi id 2 lun 0 cmd=0x12 srb='0xffffff8000180640' ccb command time out!
arcmsr0: scsi id 3 lun 0 cmd=0x12 srb='0xffffff80001808a0' ccb command time out!
arcmsr0: scsi id 4 lun 0 cmd=0x12 srb='0xffffff8000180b00' ccb command time out!
arcmsr0: scsi id 5 lun 0 cmd=0x12 srb='0xffffff8000180d60' ccb command time out!
arcmsr0: scsi id 6 lun 0 cmd=0x12 srb='0xffffff8000180fc0' ccb command time out!
arcmsr0: scsi id 7 lun 0 cmd=0x12 srb='0xffffff8000181220' ccb command time out!
arcmsr0: scsi id 8 lun 0 cmd=0x12 srb='0xffffff8000181480' ccb command time out!
arcmsr0: scsi id 9 lun 0 cmd=0x12 srb='0xffffff80001816e0' ccb command time out!
arcmsr0: scsi id 10 lun 0 cmd=0x12 srb='0xffffff8000181940' ccb command time out!
arcmsr0: scsi id 11 lun 0 cmd=0x12 srb='0xffffff8000181ba0' ccb command time out!
arcmsr0: scsi id 12 lun 0 cmd=0x12 srb='0xffffff8000181e00' ccb command time out!
arcmsr0: scsi id 13 lun 0 cmd=0x12 srb='0xffffff8000182060' ccb command time out!
arcmsr0: scsi id 14 lun 0 cmd=0x12 srb='0xffffff80001822c0' ccb command time out!
arcmsr0: scsi id 15 lun 0 cmd=0x12 srb='0xffffff8000182520' ccb command time out!
arcmsr0: scsi id 17 lun 0 cmd=0x12 srb='0xffffff8000182780' ccb command time out!

It takes a very long time to load since I have to wait the 300 seconds to timeout.

After the bootup completes I get:

Code:

interrupt storm detected on "irq19:"; throttling interrupt source

This happens every second. Google searching lead me to add hw.intr_storm_threshold=200000 to sysctls section of FreeNAS GUI. No change in anything. I still get the error every second.

So I then ran camcontrol dev list

Code:

<VMware Virtual disk 1.0>          at scbus2 target 0 lun 0 (pass0,da0)
<Areca RAID controller R001>       at scbus3 target 16 lun 0 (pass1)

Here's the output of vmstat -i:

Code:

interrupt                          total       rate
irq1: atkbd0                          20          0
irq6: fdc0                            85          0
irq18: em0                          2897          3
irq19: arcmsr0                  40563545      42122
cpu0: timer                       384270        399
irq256: mpt0                        4736          4
cpu2: timer                       142241        147
cpu3: timer                       142214        147
cpu1: timer                       142238        147
Total                           41382246      42972

If I run the areca CLI it does detect the controller and work fine, so I'm fairly certain that the driver is loading and is fully functional. But if I go look at disk no disks are detected nor can I import the ZFS pool I created previously when i booted from the USB key.

I've search and searched and I'm coming up with nothing. A few other people have made posts on the internet with no replies back or any fixes that seemed to work with similar problems. So far I've tried:

1. Changing FreeNAS to use 1 CPU physical socket and 1 core per socket(default was was 1 physical socket and 4 cores)
2. Changing the RAID controller to JBOD mode. This seems to disable passthrough in the controller BIOS.
3. Disabled the USB Mouse function, onboard NICs, onboard Gigabyte SATA controller and onboard Firewire in the system BIOS. I won't be using any of those anyway.
4. Switched the VM SATA controller from the default of "LSI Logic SAS" to "LSI Logic Parallel".
5. Tried disabling acceleration for the VM and Hide the NX/XD flag from guest.


Any other ideas of what I should try? I'm pretty much out of good ideas...

 

[cyberjock]

Ok.. I got it to work.. once. While playing around with settings. I successfully imported my ZFS and did a few things. I shutdown the VM to change a couple of settings to narrow down what I did to get it to work, but now I can't get it to work again.. sigh.

 

[jgreco]

Very sorry to hear it. As I mentioned privately, PCI passthrough - awesome when it works - is highly dependent on everything being engineered to support it. Gigabyte may have been able to give you a beta BIOS that "supports" it, but in my experience, so much stuff in the PC world is "stick an engineer on it until it works ... hey it works! must be ready to ship it!" which usually means that something will just barely work with the most common hardware, until someone comes up with something interesting, where it breaks.

It would be very nice to see if we could isolate if it was the mainboard or the RAID controller. Nice as that Areca may be, it is kind of an off-brand for purposes of VT-d, just as the Gigabyte board is also kind of an off-brand for purposes of VT-d. I've got equally uneasy feelings about the both.

1) was a good idea for purposes of working around potential FreeBSD issues, 2-5) were unlikely to have impact, and I'm not coming up with any immediate ideas for you to try.

While it won't help make it work, you could probably get a better feel for the problem by trying a version of Linux known to work with the controller. If it craps out under that when in an ESXi VM, then you know not to concentrate on FreeBSD based resolutions. Also, if you have other PCIe cards that you could stick in the same slot and pass through, particularly a different controller or network card, you could start narrowing the possibilities for where the problem is.

Also, I don't think I need to tell you, but I'll say it for everyone else's benefit: if you've got tetchy hardware for this sort of application, get it stable, then TEST, TEST, TEST, and then test some more, before you trust any data to it!

 

[cyberjock]

#1 was the only good idea I could find on the internet. #2-5 was out of desperation.. you probably noticed as you go down the list it was far less likely to work. I'm still baffled that it worked that one time and then never again. I thought that the big change that mattered was changing the CPU/MMU virtualization from "Automatic" to "Use software for instruction set and MMU virtualization". But it doesn't work now. It might have been a fluke. I do have another 24 port RAID controller(Highpoint RocketRAID 3560) but I didn't really want to try it because there isn't SMART support for it yet. I opened a ticket for it a few days ago, but I'm not sure if it will ever work. I was unable to get the Highpoint CLI to work at all, but William was. He said it was a very screwed up program and due to the complexity of trying to implement it there is a chance it will never be added to FreeNAS.

So I guess the real question with the Highpoint, if I was to use it as a long term option, would be "How important is SMART info?"

I'm going to give the controller a try though. I have nothing to lose but if the Highpoint controller doesn't work then at least I can somewhat confirm that I should give up my ambitions of trying to get ESXi to work.

Edit: I wish there was a way to change the 300 second delay to something like 30 seconds. Trying different sysctls really makes the 5 minute wait time suck.

Another thing I noticed is that when the system is POSTing the RAID controller says it is on IRQ 11, but in the VM its IRQ 19. I have no idea if that is a symptom, cause, or normal.

 

[cyberjock]

so... I plugged in the Highpoint controller and booted up ESXi. I was able to use the passthrough of a device "Unknown RAID Controller".

Rebooted ESXi and set up FreeNAS. Hard drive worked perfectly the first time. Already tested unplugging a drive and several other small things. Everything seems to work perfectly...excepy SMART. In order to use SMART you HAVE to use the Highpoint CLI. But the Highpoint CLI doesn't work if you just extract the files from the zip and use them. You get some really bizarre error message. William somehow was able to figure it out and get the CLI to work. No clue how he did it though. I'm not a master at FreeBSD, so I don't even know where I'd start to make the CLI work. I googled the heck out of the error 2 months ago and I couldn't find anything anywhere where someone even knew what the error meant. It was funny that William had the CLI working in about 30 minutes(he's a badass).

Maybe I can PM him with some clues as to how he got it working.

So now i'm going to do some reliability tests and see how things go. I have a hunch that at the end of all of this I'm going to be left with a choice. Don't use ESXi or use ESXi but accept the fact that I don't have SMART info and may never have SMART info through FreeNAS. I guess that I could always make a second VM and change the VM settings every 2 weeks to let another OS like Windows gather SMART info and see if everything is okay. If I do use ESXi I plan to have a Windows VM anyway.

At this point I'd really like to see ESXi in action, especially after all of the time I've put into this.

 

[jgreco]

Another thing I noticed is that when the system is POSTing the RAID controller says it is on IRQ 11, but in the VM its IRQ 19. I have no idea if that is a symptom, cause, or normal.

Normal, I'd say. The host system is still having to do some interaction for you, and I'm pretty sure(*) that under PCI passthru it is still ESXi shouldering and routing the interrupt. PCIe involves some rather complex $#!+ for interrupts to maintain legacy illusions while working with several newer technologies... but even in the case of a shared interrupt, these days it is unusual for that to go far afield.

(*) as in a vague fuzzy memory of something I read years ago and am too lazy to go research for this discussion, sorry.

 

[jgreco]

http://www.intel.com/technology/itj/2006/v10i3/2-io/5-platform-hardware-support.htm

Found that, which might give you some insight into the complexity of the parlor tricks used to handle PCI passthrough. If you get through it, you'll probably appreciate that it's nice that Gigabyte made the effort, and also appreciate what a minor miracle it is that anything works in modern computing systems... I sometimes fondly remember the days back when a processor was a few thousand transistors (MOS 6502 -> ~3500!) and it was possible to crack a microcomputer open and actually make changes to the core circuitry...

 

[cyberjock]

That's an amazing link. I've decided I'll try installing FreeBSD 8.3 and see what happens. I figure if I get the same thing in FreeBSD then I'm giving up on the Areca for good.

I really wish I knew why the card worked the one time it did. :(

 

[jgreco]

I hear LSI makes a really nice 24 port controller...

Now you have GOT to know you had that comin', right?

And this is in no way an endorsement of LSI or their controllers...

 

[cyberjock]

I hear LSI makes a really nice 24 port controller...

Now you have GOT to know you had that comin', right?

And this is in no way an endorsement of LSI or their controllers...

You know, I looked at that exact post earlier when I was searching the forums. I was wondering if you were going to comment about it... Glad to see it came full circle!

I got a very good price on my Areca controller. I really wanted this exact controller because for quite a while this was their flagship and was renowned for reliability even with hard drives that are customarily not compatible with RAID controllers. I bought it and it did fix my problems at the time. I'm really torn at the moment as to how I want to proceed. The Highpoint seems to be perfect for what I want, except no SMART support(but may come someday). I really only see 3 options:

1. Use ESXi with the Highpoint controller(downside.. no SMART).
2. Use FreeNAS from a USB stick(everything works but I'll have to run a separate machine for Plex)
3. Use FreeNAS from a USB stick for now, but look at purchasing a new RAID controller in the future that I know WILL work how I want. Of course, this assumes I can even find a 24 port controller that does exactly what I want.

 

[jgreco]

I would suggest that it might be worth contacting Areca to see if they have any comments or suggestions. They appear to have written their own driver for FreeBSD, and it looks professionally written, and modified within the last year, so it is quite possible that a tech support request with an explanation of what you are trying to do could cause them to see if the problem appears to be with their side, and I suspect that if it is, they'd fix it if possible.

You have to remember, though, it might NOT be the pricey RAID card that's at fault here, it could be the cheap and easily replaced Gigabyte motherboard. However, before spending money, it'd be best to see if you could determine which thing(s) are likely to be at fault.

 

[cyberjock]

Well, I've run some more tests. In installed Windows 7 in a VM and was able to install the Areca driver without any problems. No errors in the error log and I seem to be able to use the card as normal.

I installed FreeBSD 8.3 as well as 9.0 and while both bootup, detect, and load the Areca drive they both have the same issues with the CCB command timeout. I didn't let them bootup completely to find out if the IRQ storm will occur, but I don't think it really matters since the CCB timeout seems to be a giveaway that something is broken.

I've emailed Areca to see if they have any recommendations.

Edit: I did try changing the VM from "FreeBSD 64 bit" to "Other 64 bit" just to see if it would matter. It didn't.

 

[jgreco]

Ooo, then there's a middlin' good chance the hardware's compatible. For VM funness, trying a well-supported Linux variant in a VM would be informative as well.

 

[cyberjock]

Which Linux variant would you like to see tested? :D

 

[jgreco]

Whatever's convenient and considered to be "well supported" by Areca, of course. The point here is to see if we can narrow this down to "some FreeBSD driver bug" or something like that.

 

[cyberjock]

Well, Areca only specifies drivers for Red Hand Enterprise Linux(http://www.areca.com.tw/support/s_linux/linux.htm). So I'm going to try Ubuntu and Linux Mint first. Either they'll be built-in or I'm going to have a heck of a fun time figuring out how to compile the drivers from source code.

 

[cyberjock]

So I installed Ubuntu 64bit 12.10. It installed without a hitch. But it didn't see the hard drive attached to the controller. After a few minutes of uptime a message would popup saying that a system error occurred and wanted to upload the report info. I clicked no since I had never seen that message before and I hadn't run any programs. About 2-3 minutes later the system froze. Rebooted and then tried installing updates. System froze again after a few minutes. Did another reboot which then paniced within 2 seconds of POST. So I apparently killed the system when I rebooted the system while updates were in progress. I don't know where to look to see what drivers were loaded so I can't verify anything. I'm not a Linux guy. In fact, I'm more familiar with FreeBSD than Linux.

I also installed Linux Mint 14.1 64 bit Cinnamon. I chose the "Other (64bit)" virtual machine since Mint isn't a choice. VM boots up fine and installed fine. Just like Ubuntu there was no disk found. I didn't know where to look to see if an Areca driver was loading or not. But looking at the "Disks" app my 250GB drive was not seen.

So I take it that either Linux doesn't have driver support unless I add it(yuk!) or the controller just doesn't work completely with the passthrough. It's possible it was a fluke that it worked in Windows, but I already deleted the Windows VM. I may recreate the VM depending on what Areca says when they reply.

If you know of any commands you'd like me to run in Mint or Ubuntu feel free to post them. I'll reply with the appropriate responses from the OSes.

 

[jgreco]

No, really, because I don't know anything about the controller or the driver, the point I'm trying to clarify is whether this is a FreeBSD-specific problem or not. It sounds like this probably affects the Areca under Linux too, which is not what I was hoping for... it would have been great for it to be something specific to FreeBSD, meaning either driver or system handling of interrupts or something like that - those sorts of bugs are generally repairable if you find the right people.

I don't really have any great suggestions for things to try in Linux, I was hoping for either a straight "it works" or "it doesn't". The only real hope would be that maybe the driver wasn't installed or wasn't installed quite right, which you might be able to work through by following Areca's docs (if any). But my guess is that it isn't worth much screwing around with. My gut feeling is that either the Areca or the mainboard isn't suitable for VT-d as-is, quite possibly a little bit of both.

My suggested course of action:

Wait for Areca to respond, hope for good news

If that fails - contact Gigabyte and see if you can get them to touch bases with whoever over there is doing beta VT-d work.

 

[cyberjock]

I've already exchanged a few email with Areca. They asked if I had the latest firmware and driver. I am using the latest firmware and the driver FreeNAS uses is newer than what is listed on their website.

Its definitely not looking good. It's just odd that the Highpoint works perfectly and the Areca doesn't. I really wish I knew how to troubleshoot the issue more. :(

 

[jgreco]

The latest driver song and dance means you're probably still talking with first tier tech support. See if you can persevere through a few more rounds.

Basically the way this works out is this.

1) It could be a setting in the BIOS that you've missed. Walking through and experimenting a bit might not be bad.

2) It could be that the BIOS itself isn't really up to the task and was only tested for doing trite stuff like passing a video card to a VM. This would be firmly in Gigabyte's realm.

3) It could be that the Gigabyte hardware is fundamentally incapable of correct VT-d, like maybe "it wasn't quite wired right for it". This is unlikely but possible.

4) It could be that the Areca hardware is fundamentally incapable of VT-d. Working fine in Windows is one thing, but sometimes manufacturers go to extreme lengths to make it "work" (by working around detectable brokenness of a hardware platform, but not doing the same in the UNIX drivers).

5) It could be that the Areca UNIX drivers needs tweaking to make them more resilient on a VT-d platform.

6) It is possible that the hypervisor - the glue holding this all together - is in some way screwed up or contributing to brokenness.

So, you're really doing the right things to try to resolve this. Since just running out and buying new hardware is an expensive "fix", and it isn't clear which expensive bit of hardware is at fault, ... sigh