The “correct” GPU to use for AI is nvidia. But, I already have an AMD GPU, and a pretty good one too. The ROCm SDK is presently missing from my distro (Debian), and it uses a kernel module. A Ubuntu VM with either half or all of my AMD GPU passed through would accelerate AI significantly over pure CPU work, and would hopefully fix a few other problems I’ve been having while I’m at it.
As I mentioned before, I also do gaming and windows-based CAD on this same system. So now I have two use cases, which means I have to balance the GPU between them. Being able to wholly allocate the GPU for one VM, then turn that VM off and boot up the other VM would be ideal. Or, to reach for the stars, some kind of dynamic allocation scheme, whereby whatever VMs try to use the GPU get an even share of what’s available.
Oversubscription with virtual GPUs would be amazing. Just assign 100% of your GPU to each VM and have them dynamically adjust actual capacity after load.
On your W11 instance, is Hyperv hypervisor enabled?
bcdedit /set hypervisorlaunchtype off with admin privilieged CMD.
I run with x2avic forced, host passthrough, and all Hyperv enlightenments on.
With the hyperv hypervisor on, I get these in dmesg: kvm_amd: kvm [2961]: vcpu0, guest rIP: 0xfffff8178329cc19 Unhandled WRMSR(0xc0010115) = 0x0
I’m not sure if it is the AVIC hardware, some CPU bit falsely passing through, or whatever screwing me over; I just haven’t had the time to further isolate what’s up. Otherwise, it’s very stable and I always try to use AVIC to lower DPC latency. I do notice the improvement.
Also I’m on Zen3 Ryzen 5950x. Just something to note because I remember reading up anything prior had some sorta hardware bug about AVIC and vmexits… I think…?
Please do share or create something! All Ik about AMD AVIC is through reading random but related patch series on the linux kernel and some r/VFIO threads.
I need to learn how to interact with the Linux Kernel Mailing List grr evil outdated email
@CodeDragon57 Not sure this fits your use case but there’s a software solution for sharing K&M (but not V) that does support Linux:
I used it a little years ago, it worked well and I think I only stopped using it because I dumped Windoze entirely. (Yes it works across different OSes too.) Well, also I didn’t want to keep running two machines all the time - power use and all.
I tried that once, but I ended up requesting a refund because it didn’t seem to work for me. Perhaps I configured it wrong or something. At any rate, what I do now works. The beauty of VFIO is that I run them both on one machine. Once Handbrake finally supports Intel Arc cards in Linux, I won’t have to run Windows anymore. I’ll never understand why OSS projects like that treat Linux as a second-class citizen.
Just asked out of curiosity, I don’t have a problem but my system always gives this error message when starting a VM, does anyone else have this on AM5?
Just wanted to mention that the “static ReBAR” patch for qemu I talked about in this post was merged into master in commit b2896c1b on May 10th, and while I’m not sure which exact release it made it into, it’s definitely in as of v8.1.0. That means that anyone using a recent qemu who boots their 7900XTX-based VM while the graphics card has the increased BAR size should have SAM recognized and available in the Radeon software (it can also be confirmed in Device Manager).
I have passthrough working on the Asrock B650m PG Riptide, Fedora/Nobara host, Windows guest. Setup was quick, I can post more details on what options were needed.
My 5700g/6900xt ITX build is still going strong. Finally enabled resizable bar, in November, and things went well.
Only complaint is I wish AM4 had more memory bandwidth. Looking Glass has been super reliable for me, and the greatest surprise to me was when audio support landed. <3
I’m a bit late to the game (just watched the call to arms video yesterday). I’ve been doing VFIO in one manner or another at work for years now. I’ve decided to make Linux my main OS at home and run Windows on a VM for any games that warrant it given the anti-consumer / anti-privacy direction that Microsoft has been headings towards.
I’m mostly settled on:
AMD Ryzen 7 7800X3D 4.2 GHz 8-Core Processor
MSI MAG X670E TOMAHAWK WIFI ATX AM5 Motherboard
G.Skill Trident Z5 Neo 64 GB (2 x 32 GB) DDR5-6000 CL30 Memory
and reusing the rest of what I have in my old machine
I’ve been strongly looking at the 7950X3D because it’d be nice to have 8 of the lower performing cores for the host while allocating 8 of the higher performing cores to the VM. However, there is a lot of conflicting information about the host using core0 which sits on the 3D cache CCD. I feel like if I’m going to be stuck with 7 cores on the VM, then I might as well just roll with a 7800X3D and save the $200.
I don’t know if AMD is still talking to level1tech about VFIO, but this sort of thing has definitely muddied the waters on figuring out what part to purchase. The reports that I’ve read on the IOMMU grouping (network and misc. I/O) for the x670 is also a bit aggravating.
Been using 7950X3D for months, with zero issues.
Although I have set my UEFI setting to “Prefer Frequency cores”, Linux uses all of them, no matter what (Kernel 6.6.3).
I was afraid of the Core0 issue, but with or without core isolation* I don’t have any issues with either OS while using both.
My “Individual Core Usage” widget, shows even 100% usage on all 3D cores, while playing, and host doesn’t seem to be bothered.
*I am using vfio-isolate with “move” parameter, but even without it, I didn’t see any difference.
(Yes it works across different OSes too.) Well, also I didn’t want to keep running two machines all the time - power use and all.
Looking Glass has been super reliable for me, and the greatest surprise to me was when audio support landed. <3