VFIO/Passthrough in 2023 - Call to Arms

I am still using a dated i9 9900K with a Geforce 3080ti and a Radeon RX 6600 with Proxmox. I realise, that I am still pretty much a Linux newbie and don’t know much about the inner workings and most of the various linux CLIs commands - but I am very happy with my daily driver I build.

What didn’t work:

  • Doing vfio in a Desktop distro (Arch / Manjaro), to much latency, to many workarounds (hugepages setup, core seperation / pinning, the need to recomple kernels after a change added 2min boot time with VMs which have more than 16GB RAM) and a lot of other issues which required constant mantainance
  • passthrough of legacy hardware (Windows XP, Win7 with older non-UEFI GPU cards), seabios passthrough - my goal was to build a retro PC based on (more) modern hardware (Intel i7 8700K CPU and mainboard during that time), no HW / software or KVM solution could produce a successfull GPU passthrough > I am better off with PCem nowadays for these old games
  • HyperV nested virtualisation tanks performance in KVM

What worked:

  • 2 gamers, one PC setup using cheapish consumer hardware
  • using link files to get rid of unpredictable network interface names while moving PCie devices around (for example replacing a m.2 would isolate my Proxmox host because the network interface name wouldn’t match in the network config): NetworkInterfaceNames - Debian Wiki
  • ZFS - creating, maintaining and expanding pools is a breeze
  • passing though SATA controllers to use Windows storage spaces in a VM (workaround for a short time after importing my bare metal system back in 2021)
  • Proxmos NAS with various hardware passed through
  • no need to isolate HW with vfio-pci in Proxmox
  • Mac OS VM with GPU passthrough
  • the janky ass PCIe - USB (like) extenders which I also use to connect a 10Gbit network card to a 1x PCIe slot (hey, still faster than a vanilla 1Gbit connetion) and my USB 3.0 PCIe cards

Edit:

  • UEFI flashing a passed through LSI card (MINI-SAS) with OFW in a VM
  • Building a second storage PC with Proxmox which uses the flashed LSI controller

Here are older pics of my system