AMD vs nVidia for GPU Passthrough gaming

Hardware Hub GPU
1

Hi all,

So, i’ve built my new PC, but there’s still one more thing to buy - GPU.

Currently I have:

  • MOBO Asus strix z370-i
  • CPU i5 8600k
  • RAM 32 Gigs 3200 Corsair something
  • m.2 drives and all that
  • PSU CoolerMaster V650 [RS650-AFBAG1-EU]

Planning to use Ubuntu 18.04 KDE, which is downloading as I type this. Initially a dual boot system which will likely transition to full ubuntu use over time.

So I wonder if there’s any difference in support\ease of setup of GPU Passthrough VM with AMD or nVidia card?

Personally I always favored nVidia cards, mainly because of PhysX and CUDA as well as whole set of GameWorks goodies available only on nVidia. For example can’t wait for the nVidia Flex to finally appear in some game.

So if it’s the same, I’d go with nVidia. However if AMD is much better supported on Linux and specifically easier to use in a passthrough setup, AND they are generally cheaper… That would be enough pros for me to go for the red team.

Now that I think of it… I never even used an AMD card before, huh.

( Discussion is also available on Reddit: https://www.reddit.com/r/linux_gaming/comments/8krv90/amd_vs_nvidia_for_gpu_passthrough_gaming/ )

Thanks for your attention.

2

It doesn’t matter which one you use, pass-thru is pass-thru. The quirks come depending on how you get the pass-thru working.

I haven’t actually done this myself because I’m too poor, but I’ve read tons of threads about people setting this stuff up and the problems have never been about the card, just using outdated QEMU, kernel, etc.

3

I think to vaguely remember that one Level1 video stated it’s easier with AMD due to better driver support. (Afaik,some Nvidia cards decide to stop running if the figure out that they are used in a virtual environment. After all, that’s what the Quadro cards are for :wink: ).
That being said, there are other people on this forum who might be able to offer more help than I.

4

K900 on Reddit says though, that nVidia is generally hostile towards GPU Passthrough on Consumer cards, actively working to detect and prevent it, because they only want this feature to be available on Quadro cards.

And now that I look at the prices I see that, at least here, AMD alternatives for nVidia cards (1070 and vega 56 for example) are generally 100-150$ more expensive, which kind of makes the decision much more complicated. Since if AMD cards were cheaper, as I initially thought, and they were easier to passthrough then it was an easy decision - two pros. But now… I’ll have to pay more for ‘ease of setup’.

So, what I’m saying is that now I need to visualize and understand the amount of work required to passthrough nvidia vs amd, and what kind of problems it brings. I don’t think that I would want to look for a fix after every nVidia driver update, so that would make paying more for AMD card at least worth it. However if it’s not that bad, then I’d go with nVidia.

Unless, of course, I’m mistaken and vega 56 is not an alternative of a 1070.

5

for gaming, V56 is on about par with the Fury. No reason to get it unless you need the extra VRAM or want to do Machine learning stuff.

6

Vega56 is technically the better buy, but with inflated pricing its a hard sale IMO.

https://youtu.be/uhTnTJymytQ?t=41 (latest)

The AMD drivers are pretty close to NVIDIA under Linux as for performance if your aware of amdgpu-dc

7

Everything you said in the first 2 paragraphs is true.
I would recommend looking up the solution for the nvidia driver error, and see if its something that you feel comfortable dealing with. I’ve never passed through an Nvidia card myself, but a friend of mine has, so I know it’s possible (and not too difficult according to him).

8

Passing through an NV card requires 2 more lines in your .xml and is well documented. It’s easy and trouble free. I think NV is a shitty company, but they make superior video cards, so which one to pass through is really up to you and what performance/features you expect in your gaming.

9

As I am aware there are some downsides to those 2 lines of XML in your VM definition. The hypervisor essentially has to lie to the guest about its status as a VM, to make it think it is a real machine. This causes some VM friendly optimizations in Windows to be disabled, causing some CPU related performance issues. These issues aren’t super obvious, but may become noticeable in things like audio and in some other areas potentially.

I do agree with you about Nvidia however. While being anti-consumer, their cards are very price competitive with AMD currently. And the linux performance is currently very good (especially in Vulkan). Because of this I actually use an Nvidia card for my HOST and an AMD card for my GUEST.

1 Like
10

@hondaman As someone who already has a GTX980, but just can’t cram it into a new build, I would expect a good performance :smiley:

I usually tend to make my build future proof. For example my old gaming rig (and it hurts me to call it ‘old’, it is still standing next to me, waiting to be transported for storage, until it can live on as a NAS-media server) server me well for over 9 years. It was i7 2600, 16 gigs of ram, and the only thing I upgraded a few years back was GPU, from 580 to 980. It was still giving 40-60 fps in most games on medium-high and even some ultra settings. But just recently I noticed that it’s CPU usage started to stall games, even PUBG, CPU was not very often but regularly hitting 100% mark, stalling the game, and as usual it happened in the worst moments possible :slight_smile: And also I just grew tired of it’s size (ATX) and noise, and well. It was time to finally build something brand new. It just could not support a lot of new technologies like m.2 or newer GPUs.

@SesameStreetThug Well, that’s the kind of information I’m looking for. But as someone relatively new to linux, I am not very familiar with searching information on it, and so far it proved much faster to just ask people who know it :slight_smile:

I’m not worried about spending a bit more time on initial setup. I am worried if I need to spend a few hours or more every time nVidia releases new driver. I don’t have the luxury of such time wasted on pointless task. And with no guarantee that nVidia won’t come up with a solution to block it for good.

But if it’s possible to make it so Windows and GPU won’t know that they are running in VM, that sounds like the only viable solution to me.

I also can’t use 2 GPUs, miniITX card, only 1 slot :slight_smile:

And I do plan to stick to 1080p gaming for the most part.

So yeah. So far, comparing AMD and nVidia for the last few days, I’m coming to this conclusion:

  • AMD can compete with nVidia in raw performance. There’s even information that AMD runs faster on Linux.
  • AMD’s control panel is something I could just pay for, if I had to use it more than once or twice of course
  • AMD has ReLive as alternative to ShadowPlay, which is good enough for me
  • AMD has better OpenCL performance, but no CUDA support (obvious, of course, but still). Might be important in case I want to use this machine for things like video editing, machine learning (<3 Python), or GPU rendering. Which I do, even though I have a separate rig for such work.
  • AMD is easier to passthrough, and they are not shitty about it, making it future-proof and easy to do, as well as, potentially, giving better performance in VM due to the fact that there are no hacks involved and Window can be optimized for VM performance
  • AMD’s price and especially price\performance ratio sucks
  • AMD, even though we’re all tired of this joke, still has heating problems. Well, not OVERheating, but still, it’s gonna get warm and noisy
  • nVidia has a large pro in terms of it’s GameWorks, both for devs and gamers. I am a sucker for things like volume lighting, physics, dynamic hair, flex, all that. If a game has it it’s like 40% of time spent for me just playing with those things. After spending hours simulating those things for VFX work, it’s always such a relieve to see them working in realtime.
  • Once again, price! 1080ti is cheaper than RX Vega 64, it’s close to Vega 56… which, in truth, makes the only real reason to go for AMD - ease of passing it through.

Did I miss anything?

11

I don’t know if it would be blocked for good, but it is absolutely possible for them to make it much more difficult. This should be a concern to anybody passing through an Nvidia GPU, might might come to a point where you have to stay on an old version of the driver etc.

Well, it depends. AMD is getting more competitive on linux, especially recently. For Vulkan however, Nvidia is much faster on linux. The RADV driver just isn’t there yet. Take a look at some benchmarks on phoronix.com and you’ll see what I mean.

One benefit that you get with Nvidia is that the technology that accelerates shadowplay (NVENC) can be used in other software such as OBS and FFMPEG in both Windows and Linux. I would double check to see if you can do this with AMD’s ReLive, which may require special software from AMD and may not be available in the Linux driver.

I may be wrong about this, but I was under the impression that the RX[4,5]80 was a huge improvement over the R9 [2,3]90 in this regard.

One other thing to keep in mind:
While using your intel iGPU for graphics on the host, it is possible to switch to your graphics card without booting. You have to restart your display server but it is possible, and easy to script. The problem here is that it is only possible on the free driver (AMD), due to the way pass-through works. Basically; to pass through a graphics card you have to detach the standard graphics driver from the device, and attach the vfio-pci driver to it so that the hypervisor can use it. The problem here is that once the proprietary Nvidia driver is attached to the card, it cannot be detached without rebooting. The AMD driver can be detached from the radeon driver and attached to the vfio-pci driver (or vice versa) without rebooting, as long as the display server isn’t running (in order to detach radeon) and the VM isn’t running (in order to detach vfio-pci).

12

I was originally planning to use my RX 580 for GPU passthrough because AMD doesn’t try to prevent the use of consumer-grade Radeons in VMs. However, I got bitten by the “Radeon reset bug” and couldn’t get my VM to a stable state with AMD.

Basically, this bug prevents you from rebooting your VM. Instead, you have to reboot the VM and the host OS. It’s very annoying.

So now my setup consists of an RX 580 running on a Fedora Core 28 host, and a GTX 1050 Ti attached to a Windows 10 VM. The problem with Nvidia in a VM is that it takes extra trickery to fool the Nvidia driver to load in a VM. Forum member GrayBoltWolf has an excellent tutorial here The guide pertains to Debian Stretch, but the Nvidia specific configuration should be applicable to any KVM install.

13

You mean - in Linux host? Or in VM as well? I can stay on nouveau for Linux, it works well enough for my tasks so far.

@imrazor Oh yeah, I’ve seen that tutorial.

14

In the linux host. I’m curious what card you have for the nouveau driver to be working. On newer cards the nouveau driver doesn’t work at all. The reason I bring up the driver attach/detach is that it allows you to play games on linux or in the VM with only one GPU, all it takes is a display server restart (and some other stuff).

I thought they fixed that, but I guess not. I remember having the issue on my 290x but seemed it was fixed pretty quickly. Perhaps the much newer cards are still affected.

15

Which one has the performance you want.

16

It seems to be partially fixed. While I could sometimes reboot a VM with my RX 580 in it, sometimes it would boot up with graphical problems. I’d also see errors in the system logs that were indicative of the Radeon reset bug. I couldn’t find a whole lot of information, but it seems like some reference RX 480/580 GPUs have the problem as well as older generations. And, of course, I have a reference RX 580.

17

Maybe it regressed. I probably wouldn’t notice, to be honest. I boot the VM once every other month maybe. I’d much rather just play games natively at this point. I don’t have a ton of time to spend playing games, and there’s enough games on Linux where I can’t play them all anyway. Not to mention console emulators and all of that.

1 Like
18

Sorry for falling out of this thread.

After a lot of consideration, with huge thanks to this community as well as reddit /r/linux_gaming and /r/vfio I decided to go with nVidia card, EVGA GTX 1080 (not Ti), which fit perfectly in my case, has very low temps even at high load, and performs just perfectly with my 2K monitor. It might be a bit overkill for 1080p and 2K gaming, but I thought that more powerful card will be able to work at lower load and result in lower noise and temps, as it did. Also - a bit more future proof, though it’s hard to say since nVidia is launching 11XX (or 20XX) soon, and from what I gather this new generation sounds like a huge jump in performance to me, with their aim for raytracing.

For now I’m just booting into windows whenever I want to play a game, and do everything else in Linux.

I’d love to play natively on Linux, but I mostly play with friends, and a lot of games are not available on Linux. Even PUBG.

I did not yet have time to try virtualization after my first failed attempt, but I still hope to get there one day. Not that I’m bothered much by rebooting, it’s actually quite fast and allows me to focus more on certain tasks, keeping things separate. So when I’m in Windows I know I should focus on gaming and not try to work at the same time. When I’m in Linux I’m less tempted to launch a game, and can focus on work and stuff.

19

It all down to pricing for me… I like AMD because there embracing open source. But when I put down dirty australian plastic money. I want performance.

Thanks to mining AMD and Nvidia are chilling and only releasing professional GPU’s because even old cards make sales at stupid prices.

20

The reset bug can force you to restart the whole host, or you can set windows up so that it disables the gpu on shutdown, as explained in this reinitialization fix
The conclusion of this post was :

you shouldn’t have any issues with your VM failing to start whenever you explicitly reset your VM or stability issues over subsequent resets. I have noticed however, that restarts due to Windows Update still seem to cause issues, perhaps because these scripts aren’t run.

2 Likes