GPU Pass-Through; Pop!_OS - nVidia & AMD GPU's

[Querzion]

Hardware/System;

CPU; RyZen 1700X 8x2 Cores
GPU; RX580 dual (8GB)
GPU; nVidia Quadro 2000 (1GB)
RAM; 32 GB

Distro;
Now - Arch Linux - ArchTitus script Fork
Later Today or Tomorrow - Pop!_OS

So what is the question…?

Should I only have one GPU in the computer at the installation and then add the other after the install? Because I am going to use the RX580 for VM pass-through. While the other for Pop!

I don’t want drivers for both of them to be installed, if that is a problem. I was going to use Arch from the beginning, but I have a bit of a quirks to fix with my installation scripts so I’m going to use a VM in order to fix them, because I don’t want to have to install my computer over and over, since there are some bugs, that blocked me from using the VM Hypervisor & VB, but now I’m going over to Pop! after question post.

If there are a thread here that I have missed to read. Please point me to it. Thanks!

Also, how much memory and cores should Pop! have in order to be stable?

[robbbot]

There is no reason to not have both GPU’s in at install time since you can just change which GPU you want to bind with vfio-pci in your initramdisk anytime after installation.

As for how much memory / cores you “should” use is up to you, and is easily configurable even after your VM is setup. So depending on the workload you can just adjust it. Just don’t give 100% of your cores / memory.

[Querzion]

@robbbot

There is no reason to not have both GPU’s in at install time since you can just change which GPU you want to bind with vfio-pci in your initramdisk anytime after installation.

That’s good, but I was thinking of the drivers. Isn’t Pop! going to install both drivers?

@robbbot

As for how much memory / cores you “should” use is up to you, and is easily configurable even after your VM is setup. So depending on the workload you can just adjust it. Just don’t give 100% of your cores / memory.

That I know, but I’m going to lock the core usage to the VM’s and when they are on, limit the OS to not being able to use more then what it is told to use, I don’t know if it needs 8GB and one core with one thread, or two cores with two threads and then give the VM 6 or 7 with threads, it optimizes the VM’s funtions and gameplay.

[robbbot]

The drivers being “installed” or not doesn’t really make any difference, as they are just kernel modules. Before loading them you can just bind vfio-pci to whichever your going to be passing in during the boot process via your initramdisk / mkinitcpio.

For example I have the following script being referenced in my /etc/mkinitcpio.conf
/etc/mkinitcpio.conf

MODULES=(vfio_pci vfio vfio_iommu_type1 vfio_virqfd)
FILES=(/usr/local/bin/vfio-pci-override.sh)

/usr/local/bin/vfio-pci-override.sh

#!/bin/sh

for i in /sys/bus/pci/devices/*/boot_vga; do
    if [ $(cat "$i") -eq 0 ]; then # 0 vega, 1 navi
        GPU="${i%/boot_vga}"
        AUDIO="$(echo "$GPU" | sed -e "s/0$/1/")"
        echo "vfio-pci" > "$GPU/driver_override"
        if [ -d "$AUDIO" ]; then
            echo "vfio-pci" > "$AUDIO/driver_override"
        fi
    fi
done

modprobe -i vfio-pci

This allows me to switch which card I have bound to vfio-pci so I can pass in either my Vega or 6800xt by changing a flag.

I usually passed in half my cores / ram to the VM if I’m gaming, but really haven’t had to boot up my VM since last January since Proton’s been so good.