Here’s my notes and steps I took on my MS-01 to get fully-functional Hardware Acceleration on the iGPU on my i9-13900H, including Tone Mapping.
I don’t claim to be perfectly competent in all of these things, I’m just hoping to share the things I’ve learned and cobbled together from youtube videos, forum posts, and other how-to’s spattered all over the internet, and thought it’d be valuable to share the things I learned in a way that hopefully is search-friendly enough for others to find and maybe contribute to.
Notes
Problem: X710 NIC dislikes bridge VLAN awareness
Flood of “Error I40E_AQ_RC_ENOSPC, forcing overflow promiscuous on PF” in dmesg.
Solution:
Reduce the number of VLANs set up for the Intel X710 NIC
- If you’re using a VLAN aware bridge note that by default it sets the filters for vlans 2-4096 - please adapt the needed VLANs in /etc/network/interfaces
https://bugzilla.proxmox.com/show_bug.cgi?id=256
I set the vlan range in etc/network/interfaces to 2-180, since I only need vlans locally in the first hundred or so. Some users in the linked thread seem to imply that there’s a hardware limitation on the number of VLANs supported, but I couldn’t find an actual answer in the datasheets for what that limitation is on the X710 NICs that are in the MS-01.
Problem: MeshCommander can’t stay connected to vPro
This is an annoyance that seems to stem from the NIC (i226-LM) that’s vPro enabled not maintaining a DHCP lease independently of the OS. In my environment, I’m using that NIC on a separate management VLAN and don’t want Proxmox and things running on that one specifically.
Solution:
Set the IP address statically in the vPro configuration
Network Settings → IPv4 state → Static IP address.
Setup of Proxmox on the MS-01 - Enabling VFIO for the iGPU
- Install Proxmox VE 8.1.4 - Kernel Linux 6.5.11-8-pve
- Using EFI with ZFS mirrored SSDs as MS-01 Storage, with Secure Boot enabled
- EFI + Secure Boot + ZFS → Bootloader is GRUB
- BIOS Changes
- Ensure all virtualization things (SR-IOV, etc.) are enabled
- Set onboard devices > video Hybrid
- Follow these instructions https://www.derekseaman.com/2023/11/proxmox-ve-8-1-windows-11-vgpu-vt-d-passthrough-with-intel-alder-lake.html to setup the i915-sriov-dkms driver, set up the GRUB configuration, and PCI configuration
- NOTE: I wasn’t actually seeing the correct output from
dmesg | grep i915until I cleaned up the flood ofError I40E_AQ_RC_ENOSPCin my dmesg output from boot. But I did successfully see the virtual functions when I ranlspci | grep VGA - My GRUB config:
Make sure to update GRUB and initramfs.GRUB_DEFAULT=0 GRUB_TIMEOUT=5 GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian` GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt i915.enable_guc=3 i915.max_vfs=7" GRUB_CMDLINE_LINUX=""
- NOTE: I wasn’t actually seeing the correct output from
- Back to Wendel’s post here Intel i915 sr-iov mode for Flex 170 + Proxmox PVE Kernel 6.5
- Set up the PCIe devices as Resources in the Datacenter section. Not sure if that mattered or not for getting this working, but it seems like a good practice.
Jellyfin on Proxmox via LXC
So getting Jellyfin running in docker inside a VM inside Proxmox didn’t work for me, but I suspect there was some driver funkiness happening that caused me issues, and by the time I figured all that out, I had set up Jellyfin as an LXC and got that working WITH hardware transcoding and Tone Mapping!
Initial Good State
After installing the LXC container using the script from the proxmox helper scripts, and using all the defaults, I bumped the settings for the container to 50GB disk, 4GB of RAM, and left it with 2 cores, since it’s plenty fast on a i9-13900H. The larger disk is necessary because of the space needed for the transcode cache - While transcoding a very large and long film, I accidentally filled my container’s storage (it’s default was 8GB) - at which point it became completely unresponsive.
It’s also running in a privileged container, which I hope to maybe restrict somewhat.
I modified the LXC configuration as well, to change the passed through render device (though it seems like, since it’s privileged, I’m not sure if this matters). On my machine renderD128 should be the root iGPU device, and 129-134 are the 7 Virtual Functions (VFs). The idea here that I’m trying to achieve is that the LXC only has access to one of the GPU VirtIO functions, rather than the whole card.
lxc.mount.entry: /dev/dri dev/dri none bind,optional,create=dir
lxc.mount.entry: /dev/dri/renderD129 dev/dri/renderD129 none bind,optional,create=file
While transcoding a 2K HDR10 video, I get about 100FPS. A 4K HDR10 video transcodes at around 83FPS. Non-HDR and/or standard 1080p goes at 300+FPS, so it’s pretty good from that perspective.
Fixing the OpenCL Failure - Update intel-opencl-icd!
Initially, once normal transcoding was working, I still couldn’t get anything that needed tone mapping (HDR) to transcode without an error in Jellyfin. In the logs, the last few lines of the FFmpeg log showed:
Transcode log:
[AVHWDeviceContext @ 0x55f4ad1088c0] Failed to get number of OpenCL platforms: -1001. Device creation failed: -19. Failed to set value 'opencl=ocl@va' for option 'init_hw_device': No such device Error parsing global options: No such device
This was a hard one to crack, since it seems that we’re on the leading edge of software drivers and support for using this kind of hardware in this way. But on the Jellyfin documentation here: Intel GPU | Jellyfin I more closely read point #6:
“For the latest products like N95/N100 and Arc A380, support is provided in
23.xx.xxxxxand newer. Otherwise install from Intel compute-runtime repository.”
I was using version 22.xx - which seems like it’s insufficient for the Raptor Lake i9-13900H, and the LXC repositories (Ubuntu 22.04.4 LTS) didn’t have an update. I followed the instructions for the installation on the linked github release page to install version 22.09.28717.12. After installing that, running the command below, I no longer had any red text around OpenCL!
/usr/lib/jellyfin-ffmpeg/ffmpeg -v verbose -init_hw_device vaapi=va:/dev/dri/renderD129 -init_hw_device opencl@va
note to specify the correct render device
This should print:
ffmpeg version 5.1.4-Jellyfin Copyright (c) 2000-2023 the FFmpeg developers
built with gcc 11 (Ubuntu 11.4.0-1ubuntu1~22.04)
configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-libs=-lfftw3f --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-libdrm --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libsvtav1 --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=amd64 --enable-libshaderc --enable-libplacebo --enable-vulkan --enable-opencl --enable-vaapi --enable-amf --enable-libmfx --enable-ffnvcodec --enable-cuda --enable-cuda-llvm --enable-cuvid --enable-nvdec --enable-nvenc
libavutil 57. 28.100 / 57. 28.100
libavcodec 59. 37.100 / 59. 37.100
libavformat 59. 27.100 / 59. 27.100
libavdevice 59. 7.100 / 59. 7.100
libavfilter 8. 44.100 / 8. 44.100
libswscale 6. 7.100 / 6. 7.100
libswresample 4. 7.100 / 4. 7.100
libpostproc 56. 6.100 / 56. 6.100
[AVHWDeviceContext @ 0x55ecaa3d14c0] libva: VA-API version 1.20.0
[AVHWDeviceContext @ 0x55ecaa3d14c0] libva: Trying to open /usr/lib/jellyfin-ffmpeg/lib/dri/iHD_drv_video.so
[AVHWDeviceContext @ 0x55ecaa3d14c0] libva: Found init function __vaDriverInit_1_20
[AVHWDeviceContext @ 0x55ecaa3d14c0] libva: va_openDriver() returns 0
[AVHWDeviceContext @ 0x55ecaa3d14c0] Initialised VAAPI connection: version 1.20
[AVHWDeviceContext @ 0x55ecaa3d14c0] VAAPI driver: Intel iHD driver for Intel(R) Gen Graphics - 24.1.1 (f5f09c4).
[AVHWDeviceContext @ 0x55ecaa3d14c0] Driver not found in known nonstandard list, using standard behaviour.
[AVHWDeviceContext @ 0x55ecaa3ff280] 0.0: Intel(R) OpenCL Graphics / Intel(R) Iris(R) Xe Graphics
[AVHWDeviceContext @ 0x55ecaa3ff280] Intel QSV to OpenCL mapping function found (clCreateFromVA_APIMediaSurfaceINTEL).
[AVHWDeviceContext @ 0x55ecaa3ff280] Intel QSV in OpenCL acquire function found (clEnqueueAcquireVA_APIMediaSurfacesINTEL).
[AVHWDeviceContext @ 0x55ecaa3ff280] Intel QSV in OpenCL release function found (clEnqueueReleaseVA_APIMediaSurfacesINTEL).
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...
Use -h to get full help or, even better, run 'man ffmpeg'
Success!
Monitoring iGPU from Proxmox Host
Running intel_gpu_top without any flags leads to a segmentation fault. Make sure to put in the -d sriov flag.
root@proxmox1:~# intel_gpu_top -d sriov
That seems to do the trick.
Jellyfin Playback Settings
Hardware Acceleration: Intel QuickSync (QSV)
Enabled decoding: Everything except VP8
Prefer OS native DXVA or VA-API hardware decoders
Enable hardware encoding
Enable VPP Tone mapping
Enable Tone Mapping
Everything else at the defaults.
Hardware Setup
- RAM: Crucial CT48G56C46S5.M16B1 48 GB x2 Kit @ 5200MT/s - total 96GB
- Disk: ZFS Mirrored Crucial CT2000P2SSD8 2TB nvme
- HDMI Dummy Plug installed - this was like $3 on amazon and I figured it couldn’t hurt, since I wanted to make sure I can use the remote vPro console since I’m running this machine headless.
Everything else is basically stock.
I hope that this proves useful to folks. I’ll answer questions and provide updates if / as I learn more. My overall plan is to migrate all my containers off my Synology and onto this, as well as my LibreNMS VM from there, too - hopefully that all goes more smoothly than moving Jellyfin.
