Return to Level1Techs.com

Fedora 28 > PCI-E/GPU Passthrough > Linux Guest

Software & Operating Systems Linux
#1

Problem/Purpose:
I have Fedora 28 passing my GPU through to my Windows 10 VM fine - via QEMU/KVM.
I’m now trying to have GPU passthrough work on a Linux guest VM. I’m currently trying Ubuntu, but any distro is fine. The purpose is for cryptomining - to use my GTX 1070 for mining; but integegrated graphics for just regular daily use within Fedora at the same time.

What I’ve tried:
I vaguely remember there being issues discussed in one of Level1’s videos. Something regarding the boot process regarding drivers, but I can’t seem to find any guides or the video pointing me the right direction…
I’ve gotten as far as running Linux VMs in Spice and installing the Nvidia drivers. When I switch over from Spice/QXL to PCI/VMVGA, I can only get as far as a black or purple (Ubuntu) screen.
I’ve tried these Windows10 virsh xml tags, in various combinations, on the Linux guest as well:

<hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='1234567890ab'/>
    </hyperv>
    <kvm>
      <hidden state='on'/>
    </kvm>

^ These haven’t helped.

I’ve seen a few vague posts regarding VNC and SSH into the VM to do driver installs?
Seems to me that drivers might be claimed by noveau and failing? Video fails after boot screen.

Specs
OS: Fedora 28 TwentyEight
Kernel: x86_64 Linux 4.16.5-300.fc28.x86_64
CPU: Intel Core i7-4790K
iGPU: Mesa DRI Intel® Haswell Desktop
dGPU: EVGA GTX 1070 FTW (confirmed passthrough with Windows10 VM)
RAM: 32GB

TL;DR
Guidance for Fedora Nvidia passthrough to Linux guest in QEMU/KVM.
Thank you for any help. First post.

1 Like
#2

Hi

Did You end up solving this?

also . would you have link for the step by step or anything you used for Pcie passthrough for Fedora 28 to Windows VM (KVM/Qemu)?

im looking for a good one to follow myself

Thanks

Viktor

#3

Are you using i440FX or Q35 for the chipset? I’m rather certain that you cannot passthrough to a linux guest with i440FX as it (as the VM sees it) plugs the card directly into the root PCI bus, which linux doesn’t like. I haven’t heard anybody getting this working. But from my own experimentation with Q35 I was able to get the card up with a driver on it, I never shifted to running a display server on it however.

#4

Can you post your XML or other pertinent info? Digging through mail threads got me to what looks like the kernel at least recognizing the hardware properly, but it still crashes the entire VM the instant I try to run nvidia-settings or just force the drivers to run by not having a VirtIO display.

My current XML config is attached as: https://pastebin.com/9cjatwkW

#5

I’ve gotten this working on F27 passing a 10 series nvidia card to ubuntu VM.

In addition to hidden state, you need to set a vendor id (just make one up that isn’t nvidia). Can get you more details when I’m by my rig if you still need help.

I use my vm for mining and ML. However I couldn’t get X11 to work, but terminal works fine for those things for me. If you want a slightly more friendly environment you can SSH to the VM from a graphical one, that way you can at least scroll and not have to pipe every long command into less. :slight_smile:

I’ve gotten it working with i440FX @SesameStreetThug. I’m a kvm noob and I only use i440FX as I’m not too good at successfully backing up EFI VMs yet :sweat_smile:

#6

Awesome, do you have a working X11?

1 Like
#7

I can get X11 working if I treat the Linux vm like a Windows one. That is, plugging nvidia card into separate monitor, removing spice display, and passing keyboard and mouse in directly.

I vaguely remember having more luck getting a spice display working with nvidia drivers when using a vm with ubuntu 18.04, X11, and no-install-recommends ubuntu-desktop. I never fully finished setting it up tho, so can’t confirm 100%.

#8

I never tried too hard as the VM is really just for mining and neural net training. I either ssh or write the code in other environments and just move it there via git. @SesameStreetThug

#9

Your mining hashes are within what you would expect from the GPU and everything?

#10

A couple things I forgot
: I’ve always had the smoothest experience using drivers direct from nvidia, not using the ones in the ubuntu repos. I used the run files,
But I’ve heard good stuff about adding their repos. In any scenario there’s nothing FOSS about it, it’s just a matter of what works. That said repos (ubuntu’s or nvidia’s) will allow for more automatic compatibility with kernel updates.

Ubuntu repos for nvidia place everything in a different place and things like tensorflow or mining programs won’t look for them there unless you make a symlink or compile them right.

Also I think I actually had ubuntu 18.04 on wayland before installing nvidia drivers, which then switched it to X. Dunno if that matters or helps with VirtIO/spice/etc.

#11

Yes 1050ti gets 12-14 mH with preconpiled ethminer.

Edit: tweaking parallel cuda hash command line parameter can help optimize the hash for cards 1060+