[GUIDE] Package ryzen npt/acs patches into Fedora kernel package

As of kernel 4.14.3, The NPT patch is part of the default kernel shipped in Fedora 27 and so this guide is no longer relevant. This post will be left as guide to adding custom patches to packaged Fedora kernel.

If you’ve followed this guide to install the NPT or ACS patches, you’ll notice that it requires you to install the patched kernel manually. This isn’t usually an issue unless you also use things like the Nvidia driver, VirtualBox or other things that require a kernel module to be built in which case, the builds may fail with logs citing kernel-devel-uname-r package not found or similar.

This guide will show you how to build proper packages for your patched kernel. At least in Fedora.


STEP 1: Dependencies

In order to build the kernel, you need the right tools. Run these commands to install them:

sudo dnf groupinstall "Development Tools"
sudo dnf install rpmdevtools
sudo dnf build-dep kernel

This installed the tools needed to compile code, build RPM packages and other stuff needed to build the kernel.

Next, run:
rpmdev-setuptree
to create the environment to build RPM packages.

STEP 2: Get the sources

Can’t build a kernel without source code!

cd ~/rpmbuild/SOURCES
sudo dnf download --source kernel
rpm2cpio kernel-* | cpio -i --make-directories
mv kernel-*.src.rpm ../SRPMS

What these commands do is move you to the source directory for the kernel, download the source for the latest release kernel, extract the source and move the original source RPM into the source RPM folder.

Next, get the patches: [ACS patch] [NPT patch] Skip the ACS patch if you don’t need it.
For the ACS patch, create a file called linux-vfio.patch in your downloads folder and copy/paste the text into it.

Use these commands to move the patches into the sources directory if you’re lazy.

mv ~/Downloads/KVM-SVM-obey-guest-PAT.patch ~/rpmbiuld/SOURCES
mv ~/Downloads/linux-vfio.patch ~/rpmbuild/SOURCES

Now move the kernel.spec file into the SPECS folder and move to that folder yourself.

mv kernel.spec ../SPECS
cd ../SPECS
STEP 3: Editing stuff

Now. open the kernel.spec file in your favourite text editor and change the line:

# define buildid .local

to

%define buildid .patched

This should be at about line 26.

Now add out patches to the list. Scroll down and look for lines that look like: Patch000: This-is-a.patch and keep scrolling until you find # END OF PATCH DEFINITIONS.
Just above that line add:

Patch998: linux-vfio.patch
Patch999: KVM-SVM-obey-guest-PAT.patch

Skip the linux-vfio.patch line if you didn’t download the ACS patch.

Then save the file. Now onto the “fun” part.

STEP 4: Building

Depending on how fast you CPU is, this may take anywhere between 15 minutes to hours.
Run this command, sit back, and relax. Maybe watch the latest Level1News or something.

rpmbuild -ba kernel.spec

This is compiling the kernel and building the RPM packages for you to install.

STEP 5: Installing

Once that’s done, install the freshly built packages:

cd ../RPMS/x86_64
sudo dnf install kernel-core*.rpm kernel-devel*.rpm kernel-headers*.rpm kernel-modules*.rpm

Then reboot, select the patched kernel in the boot menu and enjoy.


Unfortunately, this will have to be re-done every time you want to update the kernel. Perhaps I should set up a copr repo or something to make this as pain-free as possible.

This guide can also be followed to add other patches to the kernel if you substitute the NPT and ACS patches with your own patches.

Good luck, and happy patching!

2 Likes

After patched,I tested PUBG and discovered one thing on the Ryzen platform.
Ryzen [email protected]
Giga Aorus AX370 Gaming 5
GTX1070x2
VM1 has 0~7 pinned
VM2 has 8~15 pinned

quality: all ultra 1080p

when VM1 play PUBG alone, FPS at 77 avg.(looks like normal)
when VM1 and VM2 play PUBG at the same time ,both FPS at 42~46 (very bad)

but when test cpu bench mark at the same time,all vm can reach 1/2 ryzen 1700.

The power suply is 1000Watt

Two problems here:

  1. Qemu doesn’t yet support Ryzen well, it doesn’t set the hyperthreading flag and as such the guest scheduler treats all threads as complete cores.
  2. PUBG is terribly optimized and is very CPU hungry.
1 Like

But the CPU benchmark is very high ,close to i7.
I watched a PUBG test video , he uses ryzen 1700 ,only 8 threads (1/2 CPU) were very high usage,and the averange FPS was 77 ,even better than i7 7700k with same GPU.

Intel CPUs have similar problem,Ryzen CPUs are more obvious.

during my test, the CPU usage were always below 50%.

Are you benchmarking them both at once? It’s a useless comparison if you are not.

i had.
and the result was normal.
both can reach 1/2 ryzen 1700.

Intel CPUs like 7700k may reach 60~64 FPS,avg, but also below Bare metal(70~78)

But, Half of 7700k was very weak ,maybe reached i3-6100 level.

I’d say then you are hitting the limits of what the platform can do, remember that VMs introduce overheads, such as additional task switching, additional IRQ latency and IO bandwidth.

CPU benchmarking only tests the CPU
GPU benchmarking only tests the GPU
Games, tax both at once…

My suggestion is to run 3dMark on each, at once, and see what happens as it also performs a mixed CPU/GPU test. If that doesn’t yield any performance issues I suggest you give each VM it’s own HDD too as you might be HDD I/O bound.

I would say though there is little you can do, my bet is on system memory bandwidth, games are memory intensive and you are now sharing your DDR with two VMs, causing memory access contention.

Edit: 1GB huge page tables might help if you are not using them already.

Edit2: I know for a fact that PUBG is terrible with it’s memory access running a 16GB native windows system out of ram after a few hours of play, reporting it is out of video ram (it lies). I have a 1080Ti, 11GB of video ram should be more then enough for textures, and windows reports PUBG is consuming all system memory available leading up to the crash.

Thank you for reply.
The GPU performace is always close to Bare metal.

3Dmark scores were very close to Bare metal,maybe 95%.
GPUZ sensors never reports any issue,maybe vRel cap(1.05000V),But Bare metal had the same issue.

my hosts always use SSD ,No HDD,so the I/O bound never exists.

I had tested Hugepages (1GB), even worth than transparent hugepages sometime.

About 5month ago,I heard that PUBG was very popular, so my hosts added DRAM,sothat every VM can have 16GiB DRAM

Every host alwasy have 2 channel memory.
I tested 1 channel very long ago,and the result was very very terrible.

Yeah I had tested 3200MHZ DRAM(G.skill).with 4.8Ghz i7 7700k.

the FPS(PUBG) may reach 70~74,very close to Bare metal.

But I guess its reason was overclocking, overclocking will always bring performace boost

Because 2400MHZ DRAM with 4.8Ghz i7 7700k can reach 70 FPS.DRAM bandwith maybe not the real reason.

Games like Overwatch performanced vergy good,very close to Bare metal.

Not 8GiB per VM , 16GiB per VM, small mistake.

Please edit your posts instead of posting one liners continually.

In that case I would say you’re hitting memory contention issues, as I stated PUBG is terribly optimized, and comparing the Intel’s MMU to Ryzen is like comparing apples to oranges. It is up to the MMU to decide how to physically address the memory.

agree with you.

New to this forum,I dont know it can edit posted messages,sorry.

My conclusion is that ,both Ryzen and Intel CPUs had this issue, Intel performences better

League of Legends had the same issue.
140FPS when 1 VM
90~100 FPS when 2 VM.
tested 2 month ago,Uhhhh, maybe .

You’re not going to get an exact 50/50 split on performance, it is that simple. Different workloads will perform for better or worse, it depends on what is in use.

Think about it…

  1. VM1 wants memory that is physically located in DIMM1
  2. VM2 wants memory that is also physicially located in DIMM1
  3. VM2 has to wait until the memory controller can perform a second read to get the contents at that physical location.
  4. VM1 issues a read that is again in DIMM1, and now has to wait on VM2

For a program that is using a ton of memory, this is going to happen and performance will suffer. For games like Overwatch that have literally millions of dollars poured into optimization, don’t waste RAM, everything that can be is pre-loaded into video ram. Their system memory footprint and access patterns is small enough to not see much contention with the other VMs

Not using NPT is always going to suffer. 1GB HPT will actually help quite a bit in many workloads, and shouldn’t ever cause a performance degradation. Again, I think you’re hitting memory contention issues which is why you see a performance loss when using this.

1 Like

agree with you again.

Just an afterthought, see if you can turn off dual gannged memory mode in the bios, this may help. This will allow the CPU to read from each DIMM independently rather then trying to perform very wide reads from two at once.

maybe good ,i will do some research.

I had seen some developers were doing many work on KVM tweak ,like CPU Idle halting(schedule), exitless timer…

virtualization performanced very very terrible before the hardware accelerators came out(VTx SVM …)

Maybe we can see some good news in the future.

I want to try to solve this problem,do you have some advice on Intel platform .

I have used the patch on the 4.13 kernel with succes but for some reason I get build errors when building the kernel for any newer version. Any clues?

Using F27

Build Output

Applying: KVM: SVM: obey guest PAT
error: patch failed: arch/x86/kvm/svm.c:3626
error: arch/x86/kvm/svm.c: patch does not apply
Patch failed at 0097 KVM: SVM: obey guest PAT
The copy of the patch that failed is found in: .git/rebase-apply/patch
When you have resolved this problem, run “git am --continue”.
If you prefer to skip this patch, run “git am --skip” instead.
To restore the original branch and stop patching, run “git am --abort”.
error: Bad exit status from /var/tmp/rpm-tmp.fakVG1 (%prep)

RPM build errors:
line 963: Possible unexpanded macro in: %package %{pae}-core
line 963: Possible unexpanded macro in: Provides: kernel-%{pae}-core-uname-r = 4.14.3-300.patched.fc27.x86_64+%{pae}
line 963: Possible unexpanded macro in: Provides: kernel-x86_64 = 4.14.3-300.patched.fc27+%{pae}
line 963: Possible unexpanded macro in: Provides: kernel-uname-r = 4.14.3-300.patched.fc27.x86_64+%{pae}
line 963: Possible unexpanded macro in: %package %{pae}
line 963: Possible unexpanded macro in: Requires: kernel-%{pae}-core-uname-r = 4.14.3-300.patched.fc27.x86_64+%{pae}
line 963: Possible unexpanded macro in: Requires: kernel-%{pae}-modules-uname-r = 4.14.3-300.patched.fc27.x86_64+%{pae}
line 963: Possible unexpanded macro in: %package %{pae}-devel
line 963: Possible unexpanded macro in: Provides: kernel-devel-x86_64 = 4.14.3-300.patched.fc27+%{pae}
line 963: Possible unexpanded macro in: Provides: kernel-devel-uname-r = 4.14.3-300.patched.fc27.x86_64+%{pae}
line 963: Possible unexpanded macro in: %package %{pae}-modules
line 963: Possible unexpanded macro in: Provides: kernel-modules-x86_64 = 4.14.3-300.patched.fc27+%{pae}
line 963: Possible unexpanded macro in: Provides: kernel-modules = 4.14.3-300.patched.fc27+%{pae}
line 963: Possible unexpanded macro in: Provides: kernel-%{pae}-modules-uname-r = 4.14.3-300.patched.fc27.x86_64+%{pae}
line 963: Possible unexpanded macro in: Requires: kernel-uname-r = 4.14.3-300.patched.fc27.x86_64+%{pae}
line 963: Possible unexpanded macro in: %package %{pae}-modules-extra
line 963: Possible unexpanded macro in: Provides: kernel-%{pae}-modules-extra-x86_64 = 4.14.3-300.patched.fc27+%{pae}
line 963: Possible unexpanded macro in: Provides: kernel-%{pae}-modules-extra = 4.14.3-300.patched.fc27+%{pae}
line 963: Possible unexpanded macro in: Provides: kernel-%{pae}-modules-extra-uname-r = 4.14.3-300.patched.fc27.x86_64+%{pae}
line 963: Possible unexpanded macro in: Requires: kernel-uname-r = 4.14.3-300.patched.fc27.x86_64+%{pae}
line 963: Possible unexpanded macro in: Requires: kernel-%{pae}-modules-uname-r = 4.14.3-300.patched.fc27.x86_64+%{pae}
line 963: Possible unexpanded macro in: %package %{pae}-debuginfo
line 979: Possible unexpanded macro in: %package %{pae}debug-core
line 979: Possible unexpanded macro in: Provides: kernel-%{pae}debug-core-uname-r = 4.14.3-300.patched.fc27.x86_64+%{pae}debug
line 979: Possible unexpanded macro in: Provides: kernel-x86_64 = 4.14.3-300.patched.fc27+%{pae}debug
line 979: Possible unexpanded macro in: Provides: kernel-uname-r = 4.14.3-300.patched.fc27.x86_64+%{pae}debug
line 979: Possible unexpanded macro in: %package %{pae}debug
line 979: Possible unexpanded macro in: Requires: kernel-%{pae}debug-core-uname-r = 4.14.3-300.patched.fc27.x86_64+%{pae}debug
line 979: Possible unexpanded macro in: Requires: kernel-%{pae}debug-modules-uname-r = 4.14.3-300.patched.fc27.x86_64+%{pae}debug
line 979: Possible unexpanded macro in: %package %{pae}debug-devel
line 979: Possible unexpanded macro in: Provides: kernel-devel-x86_64 = 4.14.3-300.patched.fc27+%{pae}debug
line 979: Possible unexpanded macro in: Provides: kernel-devel-uname-r = 4.14.3-300.patched.fc27.x86_64+%{pae}debug
line 979: Possible unexpanded macro in: %package %{pae}debug-modules
line 979: Possible unexpanded macro in: Provides: kernel-modules-x86_64 = 4.14.3-300.patched.fc27+%{pae}debug
line 979: Possible unexpanded macro in: Provides: kernel-modules = 4.14.3-300.patched.fc27+%{pae}debug
line 979: Possible unexpanded macro in: Provides: kernel-%{pae}debug-modules-uname-r = 4.14.3-300.patched.fc27.x86_64+%{pae}debug
line 979: Possible unexpanded macro in: Requires: kernel-uname-r = 4.14.3-300.patched.fc27.x86_64+%{pae}debug
line 979: Possible unexpanded macro in: %package %{pae}debug-modules-extra
line 979: Possible unexpanded macro in: Provides: kernel-%{pae}debug-modules-extra-x86_64 = 4.14.3-300.patched.fc27+%{pae}debug
line 979: Possible unexpanded macro in: Provides: kernel-%{pae}debug-modules-extra = 4.14.3-300.patched.fc27+%{pae}debug
line 979: Possible unexpanded macro in: Provides: kernel-%{pae}debug-modules-extra-uname-r = 4.14.3-300.patched.fc27.x86_64+%{pae}debug
line 979: Possible unexpanded macro in: Requires: kernel-uname-r = 4.14.3-300.patched.fc27.x86_64+%{pae}debug
line 979: Possible unexpanded macro in: Requires: kernel-%{pae}debug-modules-uname-r = 4.14.3-300.patched.fc27.x86_64+%{pae}debug
line 979: Possible unexpanded macro in: %package %{pae}debug-debuginfo
Bad exit status from /var/tmp/rpm-tmp.fakVG1 (%prep)

The NPT patch is now included in kernel 4.14.3 which is why you’re getting the message:

I’ll edit the original post to reflect this. Thanks for letting me know!

Ow, I thought it would be in 4.15 and higher. Good to know