GPU Isolation on i5-13500

[bbradley]

Hi,

I've tried getting help in Discord but didn't get much response. I am having issues isolating my GPU for use in Apps and VMs, but I thought it was supported on the 6.x kernel from what I read.

It is the integrated Intel Graphics on an i5-13500. Interestingly TrueNAS is reporting it as Alderlake even though it should be Raptor Lake, but I don't know if that's related. The error I get trying to isolate it is:

GPU pci slot(s) consists of devices which cannot be isolated from host.

I think this is the reason that I am unable to use my GPU in any apps either. For instance, transcoding doesn't work on Plex. I was however to create a VM and add the PCI device directly. This worked and transcoding is working in the VM. But, I cannot properly pass through the GPU because of the above error.

Kernel version from uname -r is 6.1.63-production+truenas which I believe should support raptor lake. And I am on the latest Cobia release TrueNAS-SCALE-23.10.1.3

Am I missing something? Is this hardware supported?

 

[HoneyBadger]

The GPU generation didn't change between Alder and Raptor (to my recollection) so this might just be a cosmetic challenge in the UI for the GPU revision.

You won't be able to isolate the entire GPU as it's being used by the host system for console output; but you should also be able to leverage it with Apps without isolation.

When setting up Plex, did you assign "1" GPU resource from intel/i915?

 

[bbradley]

The GPU generation didn't change between Alder and Raptor (to my recollection) so this might just be a cosmetic challenge in the UI for the GPU revision.

You won't be able to isolate the entire GPU as it's being used by the host system for console output; but you should also be able to leverage it with Apps without isolation.

When setting up Plex, did you assign "1" GPU resource from intel/i915?

I did. It appears to pass through just fine, and seemingly without error. I can even see it in my list of available transcoding devices in plex and select that for use. It appears to initialize fine if I am reading the logs correctly.

However, it errors out when trying to actually transcode. The logs suggest it doesn't seem to actually have access to it. This is true across the official scale app, the truecharts apps, and even Jellyfin which I setup just to test this. So I have to believe something isn't right with how it is being passed through, I just don't know what.

Feb 21, 2024 21:14:16.883 [140301768887096] DEBUG - [Req#13fc1/Transcode] Codecs: hardware transcoding: testing API vaapi for device '/dev/dri/renderD128' (AlderLake-S GT1)
Feb 21, 2024 21:14:16.883 [140301768887096] ERROR - [Req#13fc1/Transcode] [FFMPEG] - Failed to initialise VAAPI connection: -1 (unknown libva error).
Feb 21, 2024 21:14:16.883 [140301768887096] DEBUG - [Req#13fc1/Transcode] Codecs: hardware transcoding: opening hw device failed - probably not supported by this system, error: I/O error

Also, you mention the GPU being used for console output -- Is that true in my case? More info, I have an IPMI card which I believe is set to be the primary display, so that thei GPU isn't actually in use. Everything should be piping through the IPMI VGA output. I'll need to verify that in the BIOS to be sure but that is how I intended to set it up.

 

[HoneyBadger]

@bbradley What do you get from output of ls -lah /dev/dri? You should see devices named "render" inside.

I'm not 100% certain but I recall some mention of needing to connect a physical display device or EDID "dummy" dongle in order to cause the kernel to register the iGPU as being able to provide the render devices.

Grepping the output of lspci -v and dmesg to look for references to the i915 driver may also be helpful.

 

[bbradley]

@bbradley What do you get from output of ls -lah /dev/dri? You should see devices named "render" inside.

I'm not 100% certain but I recall some mention of needing to connect a physical display device or EDID "dummy" dongle in order to cause the kernel to register the iGPU as being able to provide the render devices.

Grepping the output of lspci -v and dmesg to look for references to the i915 driver may also be helpful.

Yes this is something I checked before getting started because I was are of the same issues you're referring to. I think everything is all good there but let me know if I am mistaken.

bbradley@truenas:~$ ls -lah /dev/dri
total 0
drwxr-xr-x 3 root root 120 Feb 23 09:51 .
drwxr-xr-x 20 root root 4.7K Feb 23 09:51 ..
drwxr-xr-x 2 root root 100 Feb 23 09:51 by-path
crw-rw---- 1 root video 226, 0 Feb 22 19:18 card0
crw-rw---- 1 root video 226, 1 Feb 23 09:51 card1
crw-rw---- 1 root render 226, 128 Feb 23 09:51 renderD128

Note that card1 and renderD128 go away when I start the VM that passes through the PCI. Probably expected.

bbradley@truenas:~$ lspci -v | grep i915
Kernel driver in use: i915
Kernel modules: i915

bbradley@truenas:~$ sudo dmesg | grep i915
[ 15.287714] i915 0000:00:02.0: [drm] VT-d active for gfx access
[ 15.287756] i915 0000:00:02.0: [drm] Using Transparent Hugepages
[ 15.288356] i915 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=io+mem
[ 15.288794] mei_hdcp 0000:00:16.0-b638ab7e-94e2-4ea2-a552-d1c54b627f04: bound 0000:00:02.0 (ops i915_hdcp_component_ops [i915])
[ 15.289638] i915 0000:00:02.0: [drm] Finished loading DMC firmware i915/adls_dmc_ver2_01.bin (v2.1)
[ 15.410314] i915 0000:00:02.0: [drm] GuC firmware i915/tgl_guc_70.bin version 70.5.1
[ 15.410337] i915 0000:00:02.0: [drm] HuC firmware i915/tgl_huc.bin version 7.9.3
[ 15.425690] i915 0000:00:02.0: [drm] HuC authenticated
[ 15.425861] i915 0000:00:02.0: [drm] GuC submission disabled
[ 15.425879] i915 0000:00:02.0: [drm] GuC SLPC disabled
[ 15.449224] [drm] Initialized i915 1.6.0 20201103 for 0000:00:02.0 on minor 1
[ 15.451571] snd_hda_intel 0000:00:1f.3: bound 0000:00:02.0 (ops i915_audio_component_bind_ops [i915])
[ 15.452262] i915 0000:00:02.0: [drm] Cannot find any crtc or sizes
[ 15.452412] i915 0000:00:02.0: [drm] Cannot find any crtc or sizes
[44196.258041] i915 0000:00:02.0: [drm] VT-d active for gfx access
[44196.259048] i915 0000:00:02.0: [drm] Using Transparent Hugepages
[44196.265202] mei_hdcp 0000:00:16.0-b638ab7e-94e2-4ea2-a552-d1c54b627f04: bound 0000:00:02.0 (ops i915_hdcp_component_ops [i915])
[44196.279956] i915 0000:00:02.0: [drm] Finished loading DMC firmware i915/adls_dmc_ver2_01.bin (v2.1)
[44196.373772] i915 0000:00:02.0: [drm] GuC firmware i915/tgl_guc_70.bin version 70.5.1
[44196.373851] i915 0000:00:02.0: [drm] HuC firmware i915/tgl_huc.bin version 7.9.3
[44196.389247] i915 0000:00:02.0: [drm] HuC authenticated
[44196.390723] i915 0000:00:02.0: [drm] GuC submission disabled
[44196.390914] i915 0000:00:02.0: [drm] GuC SLPC disabled
[44196.398957] [drm] Initialized i915 1.6.0 20201103 for 0000:00:02.0 on minor 1
[44196.400852] snd_hda_intel 0000:00:1f.3: bound 0000:00:02.0 (ops i915_audio_component_bind_ops [i915])
[44196.402544] i915 0000:00:02.0: [drm] Cannot find any crtc or sizes
[44196.403039] i915 0000:00:02.0: [drm] Cannot find any crtc or sizes
[44196.403562] i915 0000:00:02.0: [drm] Cannot find any crtc or sizes
[52370.245159] i915 0000:00:02.0: [drm] VT-d active for gfx access
[52370.247019] i915 0000:00:02.0: [drm] Using Transparent Hugepages
[52370.255720] i915 0000:00:02.0: [drm] Finished loading DMC firmware i915/adls_dmc_ver2_01.bin (v2.1)
[52370.256658] mei_hdcp 0000:00:16.0-b638ab7e-94e2-4ea2-a552-d1c54b627f04: bound 0000:00:02.0 (ops i915_hdcp_component_ops [i915])
[52370.358255] i915 0000:00:02.0: [drm] GuC firmware i915/tgl_guc_70.bin version 70.5.1
[52370.358338] i915 0000:00:02.0: [drm] HuC firmware i915/tgl_huc.bin version 7.9.3
[52370.372929] i915 0000:00:02.0: [drm] HuC authenticated
[52370.373720] i915 0000:00:02.0: [drm] GuC submission disabled
[52370.373957] i915 0000:00:02.0: [drm] GuC SLPC disabled
[52370.382389] [drm] Initialized i915 1.6.0 20201103 for 0000:00:02.0 on minor 1
[52370.384163] snd_hda_intel 0000:00:1f.3: bound 0000:00:02.0 (ops i915_audio_component_bind_ops [i915])
[52370.385878] i915 0000:00:02.0: [drm] Cannot find any crtc or sizes
[52370.386331] i915 0000:00:02.0: [drm] Cannot find any crtc or sizes

With regards to dmesg, when I previously tested this before passing into VM I got nothing. I can reboot and check again if needed.

Also regarding the hdmi dummy plug -- I had the same thought. I tested by plugging in a TV at one point. Do you think this is sufficient to rule that out?

 

[HoneyBadger]

I'm afraid I haven't got anything newer than a 10th-gen Intel iGPU system to test with, so I'm not able to really follow along much further personally, sorry.

Using the TV as a monitor should have been enough to act as a display device, if that was the issue; but it looks like that may not be the case here.