OK, first the good: I got GPU passthrough to work on my Threadripper system. Had to cannibalize my old system to get a GPU for the host OS because the GPU I initially ordered for this purpose was DOA (well, at least it was probably the cheapest part in the whole computer…). But aside from that the whole process was remarkably painless.
I’m doing this under Debian BTW, though it’s the testing branch. Debian stable is just too out of date. But this was all pretty easy to setup if you use testing.
Because I know people will be curious, the motherboard I’m running is the ASRock X399 Taichi and I haven’t seen anyone talk about its IOMMU groups before. They seem basically perfect to me. Not only did I not have any conflicts I don’t really see any potential conflicts. Both GPUs are in their own groups (and they’re in the 16x slots). I have two M.2 slots occupied, one of which is in group 0 along with some random stuff like USB and SATA controllers and ethernet. But the other is completely on its own so unless for some reason you want to pass through all your NVMe storage there shouldn’t be a problem. So it’s definitely a good board to consider for this kind of stuff.
The bad: I have the same problem everyone does with Vega where you have to actually power down the whole system to do anything with your Vega card after shutting down a VM that’s using it. That seems a little ridiculous to me. It practically defeats the purpose of running a VM instead of just dual booting, unless you just leave the VM running all the time…
Also my performance under the VM is awful, though I suspect that might be related to the NPT bug I’ve heard people talk about. I haven’t actually done anything to bypass that yet and I’ve read conflicting claims about what you should do. I feel like this point really needs some clarification.
And interesting aside: Since no driver is loaded for Vega initially the card seems to default to running its fan at full speed all the time. Even though the card is not even being used for anything. This is really noisy (it’s a reference blower card after all)…and seems like a waste of power and a good way to wear out the fan too. However as soon as you start up a VM that uses ithuts up.
Anyway, that’s the current state of things. Since this is a hot topic right now I thought people might be interested.