So an update on this that I just discovered and I will update the guide with it:
What you found is normal. In kernel 4.8 there are changes which help with IOMMU separation, and that random PCI controller you see in that group is normal. I have the same thing. I have it working without the ACS patch on kernel 4.8!
Yeah, I was going to mention that. Mine works that way as well.
I do need some help with CPU pinning, because for some reason it's fighting me. I have never had a problem with pinning before, so I don't understand what the problem is.
Also, I think I may have found the issue that was causing the VM to not boot, and it was in the config file. It caused the new VM to not boot as well.
I'll elaborate tomorrow sometime. Been a long couple days at work and I am tired.
Updated the guide with the audio section. Perfect audio in the guest now through Pulseaudio.
I'm still testing my virt manger config but for my audio issues I tried that config and still had problems and then used then the recommendations on this page and worked for me.
I used to have to do that but if you change the "nographics_allow_host_audio" then you don't have to put the env variable in to the config.
After you said that, I did some more tests and it worked the first time, after shutting down the vm a starting the vm up I would always cause audio jitter even if I restarted the vm or rebooted the pc.
Using all the suggested audio parameters mentioned on the link works every time.
I am also using archlinux and not debian which may be cause, but it can also be difference in hardware.
OK, turkey coma is over.
I just fiddled with the CPU pinning for a bit and finally have it working.
For some reason I couldn't get two physical cores and their corresponding threads to be assigned to the VM properly.
This is the current working config for the pinning:
<vcpu placement='static'>4</vcpu>
<cputune>
<vcpupin vcpu='0' cpuset='2'/>
<vcpupin vcpu='1' cpuset='3'/>
<vcpupin vcpu='2' cpuset='6'/>
<vcpupin vcpu='3' cpuset='7'/>
</cputune>I don't know why that works now, because I am almost certain that I tried that exact setup a few times and it didn't work. Maybe it has something to do with the spacing. I don't know. I was trying to get this setup the entire time, making variations in the first line and it kept taking two or three cores out of the cputune section.
All I know is that my VM boots and Windows shows that it has 2 cores and 2 threads available.
Thanks Gray!
Everything worked as expected up to "the fun part" but then when I try to start virt-manager I get this error.
Unable to connect to libvirt.
Verify that the 'libvirtd' daemon is running. (It is.)
Libvirt URI is: qemu:///system
Traceback (most recent call last):
File "/usr/share/virt-manager/virtManager/connection.py", line 904, in doopen
self.backend.open(self.do_creds_password)
File "/usr/share/virt-manager/virtinst/connection.py", line 148, in open
open_flags)
File "/usr/lib/python2.7/dist-packages/libvirt.py", line 105, in openAuth
if ret is None:raise libvirtError('virConnectOpenAuth() failed')
libvirtError: Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
I'm using Debian Stretch with all updates. I have two questions about the quoted lines in bold.
I have uri_default = "qemu:///session" in /etc/libvirt/libvirt.conf and not commented, so why does virt-manager report that the URI is qemu:///system in this error message?
The last quoted line, about /var/run/libvirt/libvirt-sock, shows up in a lot of Google results dating back several years, and there seem to be multiple underlying causes, possibly varying with versions of libvirt & virt-manager. So far, I have continued to see the /var/run/libvirt/libvirt-sock error even after creating the missing file then giving it ownership & permissions that I expected to work based on what I've read so far:
touch /var/run/libvirt/libvirt-sock
chown user:libvirt /var/run/libvirt/libvirt-sock
chmod 0777 /var/run/libvirt/libvirt-sock
/etc/init.d/libvirtd restart
/etc/init.d/libvirt-guests restart
Emphatic thanks in advance for any assistance!
You should be running libvirt as root. To connect to it you also have to run virtmanager as root. When you open it it should ask you for your password (or the root account). If not it won't be able to connect to it.
So make sure you have libvirt running as root and you have escalated the GUI to the proper level as well.
Thanks for creating this Gray! I've been doing passthrough for about a year now and spent a lot of time tuning certain things, such as adding IO threads, tuning storage parameters (I ended up settling on a raw image, cache none IO mode native as it was fastest), but the one thing I couldn't find a lot of information on was audio. I have a wireless USB headset and at first I was passing that through to use for audio but I knew there had to be a way to have it be able to play on my speakers as well which is how I found out about doing it with spice. I was using a spice channel with the spice guest tools for audio for quite some time, and that worked really well, but I was also forced to have basically a second screen due to it , if I removed it from virt-manager it would just add it back, and sometimes it would change the resolution on my monitor before/after logging in, or it wouldn't show up on my monitor at all but rather inside the QEMU window that you could open from virt-manager. I did have that display "turned off", I'd disable the Microsoft Basic Adapter that would show up and it would eventually be enabled again. The pulse audio solution is much more elegant and I stressed the VM and it was still playing fine, only when I ran stress-ng on the host did it have some issues.
I don't use the extra command line arguments that you've listed right at the end of the tutorial, I just removed the hyper V sections and added the other format to hide KVM and set CPU to host. I've also enabled suspend to memory and suspend to disk.
<kvm>
<hidden state='on'/>
</kvm>
<cpu mode='host-passthrough'>
<topology sockets='1' cores='3' threads='2'/>
</cpu>
<pm>
<suspend-to-mem enabled='yes'/>
<suspend-to-disk enabled='yes'/>
</pm>Is your suspend-to-mem and suspend-to-disk working with a GPU?
Yes it is. :) As long as I suspend/hibernate the VM before I put the host to sleep (I don't shut down often) I can just start it back up. I even pass through my USB 3.1 controller so I can use my KVM display port switch and when I hibernate the VM since it "shuts off" the USB controller gets passed back to the host. When I sleep/suspend to mem the VM goes into suspend mode in virt-manager, but I can't recall if it breaks after I put the host to sleep or not, I think it still turns on fine. To wake it back up from sleep/suspend to mem I have to use dompmwakeup from a virsh terminal.
@lessaj Awesome, thanks for the info. I've been trying to figure that out for a while.
Now, I'm having some issues with a recently acquired fury that I'm trying to pass through in place of my 970. libvirt keeps giving me device in use errors when I try to start the VM and when it passes through, I'm able to use it as long as I don't have the AMD drivers installed.
I went through the whole vfio-pci attachment process again, and I'm sitting here scratching my head. I even created a new domain in libvirt.
For the record, I'm using the arch-vfio kernel and I had more luck with an nvidia card than an amd one. My host card is a r9 380, I'm wondering if amdgpu is causing issues trying to bind to the fury, even after we attached vfio-pci to it.
I'm considering swapping in my 970 as the host device for now, but I hate nvidia cards in Linux with a passion. Any advice would be awesome. I've been scratching my head on this one for a while, and I'd rather not nuke and pave.
You can't have the Nvidia or AMD drivers loaded on the host. The drivers will interfere with vfio. Look up blacklisting modules for specific pci devices, as well as the XEN pci stub module. One of those might help.
yeah. damn. Up until today, I always had one of each vendor, so handling driver fencing was cake.
Guess I'll be learnin' me some new stuffs today.
I was not able to get audio properly working no matter what I tried and just decided to pass the onboard sound to the VM using virt manager (black listing not required). This worked without issues to my surprise and I was able to install my onboard sound drivers to the VM and get the full 5.1 audio. Of course when the VM is running there is no sound to the host but as soon as the VM is shutdown the host regains audio with no problems. I would have liked to have audio working on both at the same time but I can live with it, as the VM's only usage is for gaming.
The other option would have been to get a second sound card and pass that through to the VM but this would require the use of some sort of audio mixer as I would now have to connect 2 sound cards to one set of speakers. This was just further complicating the setup and adding cost so I gave up on this option for now.
I noticed that on my "passed through" GPU the fans stay on permanently until the windows VM is first started. The fans then stay off after the windows VM is shut down. Is there a way to remedy this or is this just how it works at the moment?
Also AMD 8350 users who have this setup working properly, please share your CPU topology:-). I have been testing out different socket and thread combinations on virt manager but I have gotten horrible performance on my VM with all the combinations I tried so far. I haven't looked into CPU pinning yet, if that is what is required to get this working properly.
Since the GPU doesn't have a driver controlling it until the VM boots it will do weird things.
And AMD doesn't have very good virtualization acceleration, so it's probably that.
I would pin your cpu, set up 1 socket, 4 (or 2) cores, 1 thread. Pin it to CPU 4,5,6,7. (assuming 0 indexed)
Also, I'm not sure if you're using windows 10, but I've heard that 8350's don't do so hot with windows 10. #justamdthings
Fury cards have a peculiar "reset" bug that makes them even worse for passthrough than nVidia cards, which just need to be tricked in Windows.
I would try Fury on the host & passthrough the 380. It's not the best for games but I bet it works easily.