CPU optimization between two VMs

Operating Systems & Open Source Virtualization
21

This should not happen anymore. vfio-isolate shields the VM from the rest. Your script also looks right to me, I double checked. My guess is that the script is not being invoked correctly.

case "$2" in
"prepare")
	enable_isolation
        echo "prepare" >> /home/<yourusername>/qemu_hook.log
	;;
"started")
	;;
"release")
	disable_isolation
       echo "release" >> /home/<yourusername>/qemu_hook.log
	;;
esac

Can you maybe edit the script like the code I pasted above. I am not entirely sure if qemu can only invoke scripts or if it can also run shell commands in this hook. This should, hopefully, output a file called qemu_hook.log in your users home directory. This way you could at least see if the functions to enable and disable the isolation are even getting run!

22

Yes, it wrote to the log, in both ā€œprepareā€ and ā€œreleaseā€ events. But still, it doesnā€™t work. As soon as I fire the stress, the VM becomes almost unusable.

edit: and yes, since qemu is a bash script, it should be run from the terminal, and isolate the terminal PID. But these errors should appear there.

23

Found it!

For some reason, which I cannot recall, I had below setting in my Kernel parameters:

systemd.unified_cgroup_hierarchy=1

In order for the isolation to work, it needs to be 0

Again thank you for your efforts, really appreciate it! :metal: :+1:

24

Youā€™re welcome. Does your gaming VM behave correctly now or it still has framedrops?

25

Looks ok! Couple of games have framedrops, but they are badly designed/implemented.
FurMark for example, stays stable with stress running and the feeling remains excellent.

1 Like
26

Well no, but yesā€¦
So, When I run the ā€œstressā€, I have zero framedropsā€¦
However, when I have both VMs on, it doesā€¦
And the 2nd VM doesnā€™t have any PIN, script, etc, and the only passthrough devices are a webcam and a NVME diskā€¦
Can this part cause the issue (I have the same on both VMs) ?

  <iothreads>2</iothreads>
  <cputune>
    <emulatorpin cpuset="3,15"/>
    <iothreadpin iothread="1" cpuset="3"/>
    <iothreadpin iothread="2" cpuset="15"/>
  </cputune>
27

@anon86748826 Is the above setting correct for my Gaming VM?
If yes, what should I use for my Coding VM? The same or different cores? Currently I have nothing in the section nor , but my Gaming VM suffers CPU performanceā€¦ :pensive:

28

I would not use the same cores for the coding VM. Try not pinning anything in the coding VM and see if the framedrops in the gaming VM vanish. If this kind of isolation does not help I am a bit lost what to try next.

1 Like
29

I did remove all settings from Coding VM, but this didnā€™t help.
However while looking around, I found two possible solutions.

  1. I enabled Huge Pages. It was something I forgot to do, as I had issues before. I did the steps again and now looks ok. Didnā€™t see any obvious performance change, but didnā€™t get worse, so I kept it
  2. After reading at this post, I disabled SVM, and that made the trick. However I need to search a bit deeper, as this blocks the Hyper-V from running in the guest machine, something that helps few games (with certain anti-cheats) to run. Currently I only have one in that category, so no big deal, but in the future I may have to find another way
30

@lI_Simo_Hayha_Il Beware that the newest version of vfio-isolate, depending on your setup, might need changes to the start script! Depending on which cgroup version is used by your OS. You can read up on the changes on the developers Github page. The hook script has to look something like this for cgroups v2:

#!/bin/bash

if [ "$1" != "gaming" ]; then
	exit 0
fi

HCPUS=0-6,16-22

MCPUS=8-15,24-31

UNDOFILE=/var/run/libvirt/qemu/vfio-isolate-undo.bin

disable_isolation () {
	vfio-isolate \
		restore $UNDOFILE

	taskset -pc 0-31 2 # kthreadd reset
}

enable_isolation () {
	vfio-isolate \
		-u $UNDOFILE \
		drop-caches \
                cpuset-modify --cpus C$HCPUS /system.slice \
		cpuset-modify --cpus C$HCPUS /user.slice \
		compact-memory \
		irq-affinity mask C$MCPUS

	taskset -pc $HCPUS 2 # ktreadd only on host cores
}



case "$2" in
"prepare")
	enable_isolation
	;;
"started")
	;;
"release")
	disable_isolation
	;;
esac
2 Likes
31

Is C%HCPUS /user.slice \ a typo?
Should it be C$HCPUS ?

32

You are correct, I missed that.

1 Like
33

Any idea why both ways (individual scripts & vfio-isolate) stopped working recently?
I am not sure if it was since the last Kernel update or before thatā€¦