I am trying to configure two VMs, that run at the same time for optimal CPU usage.
In short:
Host: Manjaro
Gaming VM: Windows 10 with VGA, SSD, NVME, LAN passthrough running on SSD
Coding VM: Windows 10 with webcam passthrough running on qcow2 file
So, my daily routine is that I am working on my “Coding VM” and when I have breaks, or long periods without work (waiting for answers, approvals), I am playing games on my “Gaming VM”.
The problem I am trying to solve is that on my “Gaming VM”, I have big framedrops (140 → 70), with no apparent reason, while the GPU usage is <70%. I have seen differences while changing settings in CPU pinning, therefore I assume it is CPU related, but haven’t been able to solve it. Most probably I am doing something wrong in CPU configuration, but I don’t know what…
When I understand this correctly you use all of the available 12 Cores for either the emulation or computation of the gaming VM. Simultaneously your other VM as well as all the threads of you host operating system will have to be scheduled on some of these cores as well. That means both the host OS as well as any other VMs may interfere with your games while they compete for resources. Pinning only means that your gaming VM will run on the cores specified, but it is no isolation. Other threads can be scheduled on these cores as well.
What I would do is limit the gaming VM to 6 cores, plus an additional 1 or 2 cores for emulation and IO. Then install vfio-isolate to shield the gaming VM from the rest of the system. vfio-isolate can reserve the specified cores for the gaming VM and keep the host as well as any other VMs from accessing them.
You can initiate vfio-isolate via a qemu hook like the following example for a 16 Core processor with a VM called gaming on the last 8 cores ,in /etc/libvirt/hooks/qemu:
Thank you for your reply. Actually, I totally forgot that I have done some sort of isolation already…
Don’t remember installing vfio-isolate, but I am using below scripts:
As far as I know vfio-isolate is doing a lot more things. When you run something like stress -c 24 on the host OS, is it affecting your gaming VMs performance? Maybe you can try if vfio-isolate helps anyways. Install vfio-isolate, comment out your scripts, edit the one I posted. I certainly know I can stress my host as much as I want the VM has no issues. Your issue could originate from a entirely different problem still.
HNODE specifies which cores should be used by the host. MNODE specifies the cores used by the guest for computation. Core 7 and Core 23 is what I have used for my gaming VMs emulator and IO set. It should be left out of both.
Let it use the cores from HNODE. As soon as you stop the gaming VM the coding VM will have access to all cores again. The script restores to original state when the gaming VM ist stopped and since you did not pin any cores in the coding VM it will be automatically scheduled to free cores.
if [ "$1" != "gaming" ]; then
exit 0
fi
This part basically means ignore the rest of the script when it is not the gaming VM. The hook is invoked every time any VM is started or stopped.
vfio-isolate generates the UNDOFILE when it is run. This options is just there in case you have a special use case or different distribution and want to specify another place where it is temporarely stored.
Please test again is everything works. You now should be able to run stress -c 24 and it should use all your cores. You can check this in any system monitor you have. When you start your gaming VM core utilization should be restricted to the cores you defined in HNODE and what you specified in MNODE should only be utilized if your gaming VM is used.
If you use the stress -c 24 it will always tell you 24 cpu because that is the setting you just made. You need to run this command in a terminal and check in a separate system monitor which cores are getting used.
Did you miss to paste part of the script because half of it is missing. In my first post you can’t see the entire script you have to scroll down. The small window with the script is scrollable!
Ooops…
But still it is not working…
As soon as I run the “stress -c 24” FPS drops to 40-50 and it is almost unresponsive.
This is the corrected script:
Also you need to make the script executable with sudo chmod +x /etc/libvirt/hooks/qemu
and then restart at least libvirt with sudo systemctl restart libvirtd.service
Ok… Maybe that was the problem on the first place…
I had the scripts under the wrong folder all along…
There is a folder ‘/etc/libvirt/qemu/’, I created ‘hooks’ in it and put there my prepare, release, started scripts, and obviously they didn’t work…
Now I moved the qemu file you suggested under ‘/etc/libvirt/’ folder, made it executable (it should have been, as I edited my old one) and tried again, but still the same…
I will try a reboot a bit later and comeback.
edit: found this in my logs: libvirtd.service internal error: End of file from qemu monitor
Can it be related?
File "/usr/lib/python3.9/site-packages/vfio_isolate/cpuset.py", line 57, in get_cpus
with open(self.__path("cpuset.cpus"), "r") as f:
FileNotFoundError: [Errno 2] No such file or directory: '/sys/fs/cgroup/cpuset/host.slice/cpuset.cpus'
.........
File "/usr/lib/python3.9/site-packages/vfio_isolate/cpuset.py", line 43, in create
os.mkdir(self.__path())
FileNotFoundError: [Errno 2] No such file or directory: '/sys/fs/cgroup/cpuset/host.slice'