Hi there,
I’m trying to change my setup to use the kernel module instead.
I’ve been following the docs but I’ve encountered a problem.
I’ve made the necessary changes but now Virt-Manager fails to connect to libvirt.
Output of ls -l /dev/kvmfr0:
crw-rw---- 1 ian kvm 510, 0 Oct 25 08:44 /dev/kvmfr0
My cgroup_device_acl block in /etc/libvirt/qemu.conf:
Contents of /etc/apparmor.d/local/abstractions/libvirt-qemu:
Honestly don’t know what went wrong. I’d be happy for any advice! 
Update: still haven’t found a way to make it work.
I’ll try to post as much information as possible, hopefully someone spots the error in my configuration. I’d love to make this work!
Unable to connect to libvirt qemu:///system.
Cannot recv data: Connection reset by peer
Libvirt URI is: qemu:///system
Traceback (most recent call last):
File "/usr/share/virt-manager/virtManager/connection.py", line 923, in _do_open
self._backend.open(cb, data)
File "/usr/share/virt-manager/virtinst/connection.py", line 171, in open
conn = libvirt.openAuth(self._open_uri,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/site-packages/libvirt.py", line 147, in openAuth
raise libvirtError('virConnectOpenAuth() failed')
libvirt.libvirtError: Cannot recv data: Connection reset by peer
<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
<name>Gaming</name>
<uuid>b7f5a6fa-a759-47a9-8512-21550245c11a</uuid>
<metadata>
<libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0">
<libosinfo:os id="http://microsoft.com/win/10"/>
</libosinfo:libosinfo>
</metadata>
<memory unit='KiB'>16777216</memory>
<currentMemory unit='KiB'>16777216</currentMemory>
<memoryBacking>
<hugepages/>
</memoryBacking>
<vcpu placement='static'>8</vcpu>
<iothreads>1</iothreads>
<cputune>
<vcpupin vcpu='0' cpuset='1'/>
<vcpupin vcpu='1' cpuset='2'/>
<vcpupin vcpu='2' cpuset='3'/>
<vcpupin vcpu='3' cpuset='4'/>
<vcpupin vcpu='4' cpuset='6'/>
<vcpupin vcpu='5' cpuset='7'/>
<vcpupin vcpu='6' cpuset='8'/>
<vcpupin vcpu='7' cpuset='9'/>
<emulatorpin cpuset='0,5'/>
<iothreadpin iothread='1' cpuset='10-11'/>
</cputune>
<os firmware='efi'>
<type arch='x86_64' machine='pc-q35-9.1'>hvm</type>
<firmware>
<feature enabled='no' name='enrolled-keys'/>
<feature enabled='no' name='secure-boot'/>
</firmware>
<loader readonly='yes' type='pflash'>/usr/share/edk2/x64/OVMF_CODE.fd</loader>
<nvram template='/usr/share/edk2/x64/OVMF_VARS.fd'>/var/lib/libvirt/qemu/nvram/Gaming_VARS.fd</nvram>
</os>
<features>
<acpi/>
<apic/>
<hyperv mode='custom'>
<relaxed state='on'/>
<vapic state='on'/>
<spinlocks state='on' retries='8191'/>
<vpindex state='on'/>
<synic state='on'/>
<stimer state='on'/>
<reset state='on'/>
<frequencies state='on'/>
</hyperv>
<vmport state='off'/>
</features>
<cpu mode='host-passthrough' check='none' migratable='off'>
<topology sockets='1' dies='1' clusters='1' cores='4' threads='2'/>
<cache mode='passthrough'/>
<maxphysaddr mode='passthrough' limit='40'/>
</cpu>
<clock offset='localtime'>
<timer name='rtc' tickpolicy='catchup'/>
<timer name='pit' tickpolicy='delay'/>
<timer name='hpet' present='no'/>
<timer name='hypervclock' present='yes'/>
</clock>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>destroy</on_crash>
<pm>
<suspend-to-mem enabled='no'/>
<suspend-to-disk enabled='no'/>
</pm>
<devices>
<emulator>/usr/bin/qemu-system-x86_64</emulator>
<disk type='file' device='disk'>
<driver name='qemu' type='raw' cache='none' io='threads' discard='unmap'/>
<source file='/var/lib/libvirt/images/win10.img'/>
<target dev='sda' bus='scsi'/>
<boot order='1'/>
<address type='drive' controller='0' bus='0' target='0' unit='0'/>
</disk>
<controller type='usb' index='0' model='qemu-xhci' ports='15'>
<address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
</controller>
<controller type='pci' index='0' model='pcie-root'/>
<controller type='pci' index='1' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='1' port='0x10'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/>
</controller>
<controller type='pci' index='2' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='2' port='0x11'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
</controller>
<controller type='pci' index='3' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='3' port='0x12'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/>
</controller>
<controller type='pci' index='4' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='4' port='0x13'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
</controller>
<controller type='pci' index='5' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='5' port='0x14'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/>
</controller>
<controller type='pci' index='6' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='6' port='0x15'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x5'/>
</controller>
<controller type='pci' index='7' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='7' port='0x16'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x6'/>
</controller>
<controller type='pci' index='8' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='8' port='0x17'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x7'/>
</controller>
<controller type='pci' index='9' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='9' port='0x18'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0' multifunction='on'/>
</controller>
<controller type='pci' index='10' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='10' port='0x19'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x1'/>
</controller>
<controller type='pci' index='11' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='11' port='0x1a'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x2'/>
</controller>
<controller type='pci' index='12' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='12' port='0x1b'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x3'/>
</controller>
<controller type='pci' index='13' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='13' port='0x1c'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x4'/>
</controller>
<controller type='pci' index='14' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='14' port='0x1d'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x5'/>
</controller>
<controller type='pci' index='15' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='15' port='0x8'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
</controller>
<controller type='pci' index='16' model='pcie-to-pci-bridge'>
<model name='pcie-pci-bridge'/>
<address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
</controller>
<controller type='scsi' index='0' model='virtio-scsi'>
<driver queues='8' iothread='1'/>
<address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
</controller>
<controller type='sata' index='0'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
</controller>
<interface type='network'>
<mac address='52:54:00:1d:35:91'/>
<source network='default'/>
<model type='virtio'/>
<address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
</interface>
<input type='mouse' bus='virtio'>
<address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
</input>
<input type='keyboard' bus='virtio'>
<address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
</input>
<input type='mouse' bus='ps2'/>
<input type='keyboard' bus='ps2'/>
<graphics type='spice' port='-1' autoport='no'>
<listen type='address'/>
<image compression='off'/>
<gl enable='no'/>
</graphics>
<sound model='ich9'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x1b' function='0x0'/>
</sound>
<audio id='1' type='pipewire' runtimeDir='/run/user/1000'>
<input name='qemuinput'/>
<output name='qemuoutput'/>
</audio>
<video>
<model type='none'/>
</video>
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
</source>
<address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
</hostdev>
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
</source>
<address type='pci' domain='0x0000' bus='0x08' slot='0x00' function='0x0'/>
</hostdev>
<watchdog model='itco' action='reset'/>
<memballoon model='none'/>
</devices>
<qemu:commandline>
<qemu:arg value='-device'/>
<qemu:arg value='{"driver":"ivshmem-plain","id":"shmem0","memdev":"looking-glass"}'/>
<qemu:arg value='-object'/>
<qemu:arg value='{"qom-type":"memory-backend-file","id":"looking-glass","mem-path":"/dev/kvmfr0","size":33554432,"share":true}'/>
</qemu:commandline>
</domain>
I’m pretty sure it has something to do with the cgroup_device_acl block. As soon as I uncomment that block, I can no longer start any VMs.
# Master configuration file for the QEMU driver.
# All settings described here are optional - if omitted, sensible
# defaults are used.
# Use of TLS requires that x509 certificates be issued. The default is
# to keep them in /etc/pki/qemu. This directory must contain
#
# ca-cert.pem - the CA master certificate
# server-cert.pem - the server certificate signed with ca-cert.pem
# server-key.pem - the server private key
#
# and optionally may contain
#
# dh-params.pem - the DH params configuration file
#
# If the directory does not exist, libvirtd will fail to start. If the
# directory doesn't contain the necessary files, QEMU domains will fail
# to start if they are configured to use TLS.
#
# In order to overwrite the default path alter the following. This path
# definition will be used as the default path for other *_tls_x509_cert_dir
# configuration settings if their default path does not exist or is not
# specifically set.
#
#default_tls_x509_cert_dir = "/etc/pki/qemu"
# The default TLS configuration only uses certificates for the server
# allowing the client to verify the server's identity and establish
# an encrypted channel.
#
# It is possible to use x509 certificates for authentication too, by
# issuing an x509 certificate to every client who needs to connect.
#
# Enabling this option will reject any client who does not have a
# certificate signed by the CA in /etc/pki/qemu/ca-cert.pem
#
# The default_tls_x509_cert_dir directory must also contain
#
# client-cert.pem - the client certificate signed with the ca-cert.pem
# client-key.pem - the client private key
#
# If this option is supplied it provides the default for the "_verify" option
# of specific TLS users such as vnc, backups, migration, etc. The specific
# users of TLS may override this by setting the specific "_verify" option.
#
# When not supplied the specific TLS users provide their own defaults.
#
#default_tls_x509_verify = 1
#
# Libvirt assumes the server-key.pem file is unencrypted by default.
# To use an encrypted server-key.pem file, the password to decrypt
# the PEM file is required. This can be provided by creating a secret
# object in libvirt and then to uncomment this setting to set the UUID
# of the secret.
#
# NB This default all-zeros UUID will not work. Replace it with the
# output from the UUID for the TLS secret from a 'virsh secret-list'
# command and then uncomment the entry
#
#default_tls_x509_secret_uuid = "00000000-0000-0000-0000-000000000000"
# VNC is configured to listen on 127.0.0.1 by default.
# To make it listen on all public interfaces, uncomment
# this next option.
#
# NB, strong recommendation to enable TLS + x509 certificate
# verification when allowing public access
#
#vnc_listen = "0.0.0.0"
# Enable this option to have VNC served over an automatically created
# unix socket. This prevents unprivileged access from users on the
# host machine, though most VNC clients do not support it.
#
# This will only be enabled for VNC configurations that have listen
# type=address but without any address specified. This setting takes
# preference over vnc_listen.
#
#vnc_auto_unix_socket = 1
# Enable use of TLS encryption on the VNC server. This requires
# a VNC client which supports the VeNCrypt protocol extension.
# Examples include vinagre, virt-viewer, virt-manager and vencrypt
# itself. UltraVNC, RealVNC, TightVNC do not support this
#
# It is necessary to setup CA and issue a server certificate
# before enabling this.
#
#vnc_tls = 1
# In order to override the default TLS certificate location for
# vnc certificates, supply a valid path to the certificate directory.
# If the provided path does not exist, libvirtd will fail to start.
# If the path is not provided, but vnc_tls = 1, then the
# default_tls_x509_cert_dir path will be used.
#
#vnc_tls_x509_cert_dir = "/etc/pki/libvirt-vnc"
# Uncomment and use the following option to override the default secret
# UUID provided in the default_tls_x509_secret_uuid parameter.
#
#vnc_tls_x509_secret_uuid = "00000000-0000-0000-0000-000000000000"
# The default TLS configuration only uses certificates for the server
# allowing the client to verify the server's identity and establish
# an encrypted channel.
#
# It is possible to use x509 certificates for authentication too, by
# issuing an x509 certificate to every client who needs to connect.
#
# Enabling this option will reject any client that does not have a
# certificate (as described in default_tls_x509_verify) signed by the
# CA in the vnc_tls_x509_cert_dir (or default_tls_x509_cert_dir).
#
# If this option is not supplied, it will be set to the value of
# "default_tls_x509_verify". If "default_tls_x509_verify" is not supplied either,
# the default is "0".
#
#vnc_tls_x509_verify = 1
# The default VNC password. Only 8 bytes are significant for
# VNC passwords. This parameter is only used if the per-domain
# XML config does not already provide a password. To allow
# access without passwords, leave this commented out. An empty
# string will still enable passwords, but be rejected by QEMU,
# effectively preventing any use of VNC. Obviously change this
# example here before you set this.
#
#vnc_password = "XYZ12345"
# Enable use of SASL encryption on the VNC server. This requires
# a VNC client which supports the SASL protocol extension.
# Examples include vinagre, virt-viewer and virt-manager
# itself. UltraVNC, RealVNC, TightVNC do not support this
#
# It is necessary to configure /etc/sasl2/qemu.conf to choose
# the desired SASL plugin (eg, GSSPI for Kerberos)
#
#vnc_sasl = 1
# The default SASL configuration file is located in /etc/sasl2/
# When running libvirtd unprivileged, it may be desirable to
# override the configs in this location. Set this parameter to
# point to the directory, and create a qemu.conf in that location
#
#vnc_sasl_dir = "/some/directory/sasl2"
# QEMU implements an extension for providing audio over a VNC connection,
# though if your VNC client does not support it, your only chance for getting
# sound output is through regular audio backends. By default, libvirt will
# disable all QEMU sound backends if using VNC, since they can cause
# permissions issues. Enabling this option will make libvirtd honor the
# QEMU_AUDIO_DRV environment variable when using VNC.
#
#vnc_allow_host_audio = 0
# SPICE is configured to listen on 127.0.0.1 by default.
# To make it listen on all public interfaces, uncomment
# this next option.
#
# NB, strong recommendation to enable TLS + x509 certificate
# verification when allowing public access
#
#spice_listen = "0.0.0.0"
# Enable use of TLS encryption on the SPICE server.
#
# It is necessary to setup CA and issue a server certificate
# before enabling this.
#
#spice_tls = 1
# In order to override the default TLS certificate location for
# spice certificates, supply a valid path to the certificate directory.
# If the provided path does not exist, libvirtd will fail to start.
# If the path is not provided, but spice_tls = 1, then the
# default_tls_x509_cert_dir path will be used.
#
#spice_tls_x509_cert_dir = "/etc/pki/libvirt-spice"
# Enable this option to have SPICE served over an automatically created
# unix socket. This prevents unprivileged access from users on the
# host machine.
#
# This will only be enabled for SPICE configurations that have listen
# type=address but without any address specified. This setting takes
# preference over spice_listen.
#
#spice_auto_unix_socket = 1
# The default SPICE password. This parameter is only used if the
# per-domain XML config does not already provide a password. To
# allow access without passwords, leave this commented out. An
# empty string will still enable passwords, but be rejected by
# QEMU, effectively preventing any use of SPICE. Obviously change
# this example here before you set this.
#
#spice_password = "XYZ12345"
# Enable use of SASL encryption on the SPICE server. This requires
# a SPICE client which supports the SASL protocol extension.
#
# It is necessary to configure /etc/sasl2/qemu.conf to choose
# the desired SASL plugin (eg, GSSPI for Kerberos)
#
#spice_sasl = 1
# The default SASL configuration file is located in /etc/sasl2/
# When running libvirtd unprivileged, it may be desirable to
# override the configs in this location. Set this parameter to
# point to the directory, and create a qemu.conf in that location
#
#spice_sasl_dir = "/some/directory/sasl2"
# Enable use of TLS encryption on the chardev TCP transports.
#
# It is necessary to setup CA and issue a server certificate
# before enabling this.
#
#chardev_tls = 1
# In order to override the default TLS certificate location for character
# device TCP certificates, supply a valid path to the certificate directory.
# If the provided path does not exist, libvirtd will fail to start.
# If the path is not provided, but chardev_tls = 1, then the
# default_tls_x509_cert_dir path will be used.
#
#chardev_tls_x509_cert_dir = "/etc/pki/libvirt-chardev"
# The default TLS configuration only uses certificates for the server
# allowing the client to verify the server's identity and establish
# an encrypted channel.
#
# It is possible to use x509 certificates for authentication too, by
# issuing an x509 certificate to every client who needs to connect.
#
# Enabling this option will reject any client that does not have a
# certificate (as described in default_tls_x509_verify) signed by the
# CA in the chardev_tls_x509_cert_dir (or default_tls_x509_cert_dir).
#
# If this option is not supplied, it will be set to the value of
# "default_tls_x509_verify". If "default_tls_x509_verify" is not supplied either,
# the default is "1".
#
#chardev_tls_x509_verify = 1
# Uncomment and use the following option to override the default secret
# UUID provided in the default_tls_x509_secret_uuid parameter.
#
# NB This default all-zeros UUID will not work. Replace it with the
# output from the UUID for the TLS secret from a 'virsh secret-list'
# command and then uncomment the entry
#
#chardev_tls_x509_secret_uuid = "00000000-0000-0000-0000-000000000000"
# Enable use of TLS encryption for all VxHS network block devices that
# don't specifically disable.
#
# When the VxHS network block device server is set up appropriately,
# x509 certificates are required for authentication between the clients
# (qemu processes) and the remote VxHS server.
#
# It is necessary to setup CA and issue the client certificate before
# enabling this.
#
#vxhs_tls = 1
# In order to override the default TLS certificate location for VxHS
# backed storage, supply a valid path to the certificate directory.
# This is used to authenticate the VxHS block device clients to the VxHS
# server.
#
# If the provided path does not exist, libvirtd will fail to start.
# If the path is not provided, but vxhs_tls = 1, then the
# default_tls_x509_cert_dir path will be used.
#
# VxHS block device clients expect the client certificate and key to be
# present in the certificate directory along with the CA master certificate.
# If using the default environment, default_tls_x509_verify must be configured.
# Since this is only a client the server-key.pem certificate is not needed.
# Thus a VxHS directory must contain the following:
#
# ca-cert.pem - the CA master certificate
# client-cert.pem - the client certificate signed with the ca-cert.pem
# client-key.pem - the client private key
#
#vxhs_tls_x509_cert_dir = "/etc/pki/libvirt-vxhs"
# Uncomment and use the following option to override the default secret
# UUID provided in the default_tls_x509_secret_uuid parameter.
#
# NB This default all-zeros UUID will not work. Replace it with the
# output from the UUID for the TLS secret from a 'virsh secret-list'
# command and then uncomment the entry
#
#vxhs_tls_x509_secret_uuid = "00000000-0000-0000-0000-000000000000"
# Enable use of TLS encryption for all NBD disk devices that don't
# specifically disable it.
#
# When the NBD server is set up appropriately, x509 certificates are required
# for authentication between the client and the remote NBD server.
#
# It is necessary to setup CA and issue the client certificate before
# enabling this.
#
#nbd_tls = 1
# In order to override the default TLS certificate location for NBD
# backed storage, supply a valid path to the certificate directory.
# This is used to authenticate the NBD block device clients to the NBD
# server.
#
# If the provided path does not exist, libvirtd will fail to start.
# If the path is not provided, but nbd_tls = 1, then the
# default_tls_x509_cert_dir path will be used.
#
# NBD block device clients expect the client certificate and key to be
# present in the certificate directory along with the CA certificate.
# Since this is only a client the server-key.pem certificate is not needed.
# Thus a NBD directory must contain the following:
#
# ca-cert.pem - the CA master certificate
# client-cert.pem - the client certificate signed with the ca-cert.pem
# client-key.pem - the client private key
#
#nbd_tls_x509_cert_dir = "/etc/pki/libvirt-nbd"
# Uncomment and use the following option to override the default secret
# UUID provided in the default_tls_x509_secret_uuid parameter.
#
# NB This default all-zeros UUID will not work. Replace it with the
# output from the UUID for the TLS secret from a 'virsh secret-list'
# command and then uncomment the entry
#
#nbd_tls_x509_secret_uuid = "00000000-0000-0000-0000-000000000000"
# In order to override the default TLS certificate location for migration
# certificates, supply a valid path to the certificate directory. If the
# provided path does not exist, libvirtd will fail to start. If the path is
# not provided, but TLS-encrypted migration is requested, then the
# default_tls_x509_cert_dir path will be used. Once/if a default certificate is
# enabled/defined, migration will then be able to use the certificate via
# migration API flags.
#
#migrate_tls_x509_cert_dir = "/etc/pki/libvirt-migrate"
# The default TLS configuration only uses certificates for the server
# allowing the client to verify the server's identity and establish
# an encrypted channel.
#
# It is possible to use x509 certificates for authentication too, by
# issuing an x509 certificate to every client who needs to connect.
#
# Enabling this option will reject any client that does not have a
# certificate (as described in default_tls_x509_verify) signed by the
# CA in the migrate_tls_x509_cert_dir (or default_tls_x509_cert_dir).
#
# If this option is not supplied, it will be set to the value of
# "default_tls_x509_verify". If "default_tls_x509_verify" is not supplied
# either, the default is "1".
#
#migrate_tls_x509_verify = 1
# Uncomment and use the following option to override the default secret
# UUID provided in the default_tls_x509_secret_uuid parameter.
#
# NB This default all-zeros UUID will not work. Replace it with the
# output from the UUID for the TLS secret from a 'virsh secret-list'
# command and then uncomment the entry
#
#migrate_tls_x509_secret_uuid = "00000000-0000-0000-0000-000000000000"
# By default TLS is requested using the VIR_MIGRATE_TLS flag, thus not requested
# automatically. Setting 'migate_tls_force' to "1" will prevent any migration
# which is not using VIR_MIGRATE_TLS to ensure higher level of security in
# deployments with TLS.
#
#migrate_tls_force = 0
# In order to override the default TLS certificate location for backup NBD
# server certificates, supply a valid path to the certificate directory. If the
# provided path does not exist, libvirtd will fail to start. If the path is
# not provided, but TLS-encrypted backup is requested, then the
# default_tls_x509_cert_dir path will be used.
#
#backup_tls_x509_cert_dir = "/etc/pki/libvirt-backup"
# The default TLS configuration only uses certificates for the server
# allowing the client to verify the server's identity and establish
# an encrypted channel.
#
# It is possible to use x509 certificates for authentication too, by
# issuing an x509 certificate to every client who needs to connect.
#
# Enabling this option will reject any client that does not have a
# certificate (as described in default_tls_x509_verify) signed by the
# CA in the backup_tls_x509_cert_dir (or default_tls_x509_cert_dir).
#
# If this option is not supplied, it will be set to the value of
# "default_tls_x509_verify". If "default_tls_x509_verify" is not supplied either,
# the default is "1".
#
#backup_tls_x509_verify = 1
# Uncomment and use the following option to override the default secret
# UUID provided in the default_tls_x509_secret_uuid parameter.
#
# NB This default all-zeros UUID will not work. Replace it with the
# output from the UUID for the TLS secret from a 'virsh secret-list'
# command and then uncomment the entry
#
#backup_tls_x509_secret_uuid = "00000000-0000-0000-0000-000000000000"
# By default, if no graphical front end is configured, libvirt will disable
# QEMU audio output since directly talking to alsa/pulseaudio may not work
# with various security settings. If you know what you're doing, enable
# the setting below and libvirt will passthrough the QEMU_AUDIO_DRV
# environment variable when using nographics.
#
#nographics_allow_host_audio = 1
# Override the port for creating both VNC and SPICE sessions (min).
# This defaults to 5900 and increases for consecutive sessions
# or when ports are occupied, until it hits the maximum.
#
# Minimum must be greater than or equal to 5900 as lower number would
# result into negative vnc display number.
#
# Maximum must be less than 65536, because higher numbers do not make
# sense as a port number.
#
#remote_display_port_min = 5900
#remote_display_port_max = 65535
# VNC WebSocket port policies, same rules apply as with remote display
# ports. VNC WebSockets use similar display <-> port mappings, with
# the exception being that ports start from 5700 instead of 5900.
#
#remote_websocket_port_min = 5700
#remote_websocket_port_max = 65535
# The default security driver is SELinux. If SELinux is disabled
# on the host, then the security driver will automatically disable
# itself. If you wish to disable QEMU SELinux security driver while
# leaving SELinux enabled for the host in general, then set this
# to 'none' instead. It's also possible to use more than one security
# driver at the same time, for this use a list of names separated by
# comma and delimited by square brackets. For example:
#
# security_driver = [ "selinux", "apparmor" ]
#
# Notes: The DAC security driver is always enabled; as a result, the
# value of security_driver cannot contain "dac". The value "none" is
# a special value; security_driver can be set to that value in
# isolation, but it cannot appear in a list of drivers.
#
#security_driver = "selinux"
# If set to non-zero, then the default security labeling
# will make guests confined. If set to zero, then guests
# will be unconfined by default. Defaults to 1.
#security_default_confined = 1
# If set to non-zero, then attempts to create unconfined
# guests will be blocked. Defaults to 0.
#security_require_confined = 1
# The user for QEMU processes run by the system instance. It can be
# specified as a user name or as a user id. The qemu driver will try to
# parse this value first as a name and then, if the name doesn't exist,
# as a user id.
#
# Since a sequence of digits is a valid user name, a leading plus sign
# can be used to ensure that a user id will not be interpreted as a user
# name.
#
# Some examples of valid values are:
#
# user = "qemu" # A user named "qemu"
# user = "+0" # Super user (uid=0)
# user = "100" # A user named "100" or a user with uid=100
#
user = "ian"
# The group for QEMU processes run by the system instance. It can be
# specified in a similar way to user.
#group = "libvirt-qemu"
# Whether libvirt should dynamically change file ownership
# to match the configured user/group above. Defaults to 1.
# Set to 0 to disable file ownership changes.
#dynamic_ownership = 1
# Whether libvirt should remember and restore the original
# ownership over files it is relabeling. Defaults to 1, set
# to 0 to disable the feature.
#remember_owner = 1
# What cgroup controllers to make use of with QEMU guests
#
# - 'cpu' - use for scheduler tunables
# - 'devices' - use for device access control
# - 'memory' - use for memory tunables
# - 'blkio' - use for block devices I/O tunables
# - 'cpuset' - use for CPUs and memory nodes
# - 'cpuacct' - use for CPUs statistics.
#
# NB, even if configured here, they won't be used unless
# the administrator has mounted cgroups, e.g.:
#
# mkdir /dev/cgroup
# mount -t cgroup -o devices,cpu,memory,blkio,cpuset none /dev/cgroup
#
# They can be mounted anywhere, and different controllers
# can be mounted in different locations. libvirt will detect
# where they are located.
#
#cgroup_controllers = [ "cpu", "devices", "memory", "blkio", "cpuset", "cpuacct" ]
# This is the basic set of devices allowed / required by
# all virtual machines.
#
# As well as this, any configured block backed disks,
# all sound device, and all PTY devices are allowed.
#
# This will only need setting if newer QEMU suddenly
# wants some device we don't already know about.
#
cgroup_device_acl = [
"/dev/null", "/dev/full", "/dev/zero",
"/dev/random", "/dev/urandom",
"/dev/ptmx", "/dev/kvm",
"/dev/userfaultfd"
"/dev/kvmfr0"
]
#
# RDMA migration requires the following extra files to be added to the list:
# "/dev/infiniband/rdma_cm",
# "/dev/infiniband/issm0",
# "/dev/infiniband/issm1",
# "/dev/infiniband/umad0",
# "/dev/infiniband/umad1",
# "/dev/infiniband/uverbs0"
# The default format for QEMU/KVM guest save images is raw; that is, the
# memory from the domain is dumped out directly to a file. If you have
# guests with a large amount of memory, however, this can take up quite
# a bit of space. If you would like to compress the images while they
# are being saved to disk, you can also set "zstd", "lzop", "gzip", "bzip2",
# or "xz" for save_image_format. Note that this means you slow down the process
# of saving a domain in order to save disk space.
#
# save_image_format is used when you use 'virsh save' or 'virsh managedsave'
# at scheduled saving, and it is an error if the specified save_image_format
# is not valid, or the requested compression program can't be found.
#
# dump_image_format is used when you use 'virsh dump' at emergency
# crashdump, and if the specified dump_image_format is not valid, or
# the requested compression program can't be found, this falls
# back to "raw" compression.
#
# snapshot_image_format specifies the compression algorithm of the memory save
# image when an external snapshot of a domain is taken. This does not apply
# on disk image format. It is an error if the specified format isn't valid,
# or the requested compression program can't be found.
#
#save_image_format = "raw"
#dump_image_format = "raw"
#snapshot_image_format = "raw"
# When a domain is configured to be auto-dumped when libvirtd receives a
# watchdog event from qemu guest, libvirtd will save dump files in directory
# specified by auto_dump_path. Default value is /var/lib/libvirt/qemu/dump
#
#auto_dump_path = "/var/lib/libvirt/qemu/dump"
# When a domain is configured to be auto-dumped, enabling this flag
# has the same effect as using the VIR_DUMP_BYPASS_CACHE flag with the
# virDomainCoreDump API. That is, the system will avoid using the
# file system cache while writing the dump file, but may cause
# slower operation.
#
#auto_dump_bypass_cache = 0
# When a domain is configured to be auto-started, enabling this flag
# has the same effect as using the VIR_DOMAIN_START_BYPASS_CACHE flag
# with the virDomainCreateWithFlags API. That is, the system will
# avoid using the file system cache when restoring any managed state
# file, but may cause slower operation.
#
#auto_start_bypass_cache = 0
# If provided by the host and a hugetlbfs mount point is configured,
# a guest may request huge page backing. When this mount point is
# unspecified here, determination of a host mount point in /proc/mounts
# will be attempted. Specifying an explicit mount overrides detection
# of the same in /proc/mounts. Setting the mount point to "" will
# disable guest hugepage backing. If desired, multiple mount points can
# be specified at once, separated by comma and enclosed in square
# brackets, for example:
#
# hugetlbfs_mount = ["/dev/hugepages2M", "/dev/hugepages1G"]
#
# The size of huge page served by specific mount point is determined by
# libvirt at the daemon startup.
#
# NB, within these mount points, guests will create memory backing
# files in a location of $MOUNTPOINT/libvirt/qemu
#
#hugetlbfs_mount = "/dev/hugepages"
# Path to the setuid helper for creating tap devices. This executable
# is used to create <source type='bridge'> interfaces when libvirtd is
# running unprivileged. libvirt invokes the helper directly, instead
# of using "-netdev bridge", for security reasons.
# If this is not an absolute path, the program will be searched for
# in $PATH as well as a few additional directories.
#bridge_helper = "qemu-bridge-helper"
# If enabled, libvirt will have QEMU set its process name to
# "qemu:VM_NAME", where VM_NAME is the name of the VM. The QEMU
# process will appear as "qemu:VM_NAME" in process listings and
# other system monitoring tools. By default, QEMU does not set
# its process title, so the complete QEMU command (emulator and
# its arguments) appear in process listings.
#
#set_process_name = 1
# If max_processes is set to a positive integer, libvirt will use
# it to set the maximum number of processes that can be run by qemu
# user. This can be used to override default value set by host OS.
# The same applies to max_files which sets the limit on the maximum
# number of opened files.
#
#max_processes = 0
#max_files = 0
# If max_threads_per_process is set to a positive integer, libvirt
# will use it to set the maximum number of threads that can be
# created by a qemu process. Some VM configurations can result in
# qemu processes with tens of thousands of threads. systemd-based
# systems typically limit the number of threads per process to
# 16k. max_threads_per_process can be used to override default
# limits in the host OS.
#
#max_threads_per_process = 0
# If max_core is set to a non-zero integer, then QEMU will be
# permitted to create core dumps when it crashes, provided its
# RAM size is smaller than the limit set.
#
# Be warned that the core dump will include a full copy of the
# guest RAM, if the 'dump_guest_core' setting has been enabled,
# or if the guest XML contains
#
# <memory dumpCore="on">...guest ram...</memory>
#
# If guest RAM is to be included, ensure the max_core limit
# is set to at least the size of the largest expected guest
# plus another 1GB for any QEMU host side memory mappings.
#
# As a special case it can be set to the string "unlimited" to
# to allow arbitrarily sized core dumps.
#
# By default the core dump size is set to 0 disabling all dumps
#
# Size is a positive integer specifying bytes or the
# string "unlimited"
#
#max_core = "unlimited"
# Determine if guest RAM is included in QEMU core dumps. By
# default guest RAM will be excluded if a new enough QEMU is
# present and host kernel supports it. Setting this to '1' will
# force guest RAM to always be included in QEMU core dumps.
#
# This setting will be ignored if the guest XML has set the
# dumpCore attribute on the <memory> element.
#
#dump_guest_core = 1
# mac_filter enables MAC addressed based filtering on bridge ports.
# This currently requires ebtables to be installed.
#
#mac_filter = 1
# By default, PCI devices below non-ACS switch are not allowed to be assigned
# to guests. By setting relaxed_acs_check to 1 such devices will be allowed to
# be assigned to guests.
#
#relaxed_acs_check = 1
# In order to prevent accidentally starting two domains that
# share one writable disk, libvirt offers two approaches for
# locking files. The first one is sanlock, the other one,
# virtlockd, is then our own implementation. Accepted values
# are "sanlock" and "lockd".
#
#lock_manager = "lockd"
# Set limit of maximum APIs queued on one domain. All other APIs
# over this threshold will fail on acquiring job lock. Specially,
# setting to zero turns this feature off.
# Note, that job lock is per domain.
#
#max_queued = 0
###################################################################
# Keepalive protocol:
# This allows qemu driver to detect broken connections to remote
# libvirtd during peer-to-peer migration. A keepalive message is
# sent to the daemon after keepalive_interval seconds of inactivity
# to check if the daemon is still responding; keepalive_count is a
# maximum number of keepalive messages that are allowed to be sent
# to the daemon without getting any response before the connection
# is considered broken. In other words, the connection is
# automatically closed approximately after
# keepalive_interval * (keepalive_count + 1) seconds since the last
# message received from the daemon. If keepalive_interval is set to
# -1, qemu driver will not send keepalive requests during
# peer-to-peer migration; however, the remote libvirtd can still
# send them and source libvirtd will send responses. When
# keepalive_count is set to 0, connections will be automatically
# closed after keepalive_interval seconds of inactivity without
# sending any keepalive messages.
#
#keepalive_interval = 5
#keepalive_count = 5
# Use seccomp syscall filtering sandbox in QEMU.
# 1 == filter enabled, 0 == filter disabled
#
# Unless this option is disabled, QEMU will be run with
# a seccomp filter that stops it from executing certain
# syscalls.
#
#seccomp_sandbox = 1
# Override the listen address for all incoming migrations. Defaults to
# 0.0.0.0, or :: if both host and qemu are capable of IPv6.
#migration_address = "0.0.0.0"
# The default hostname or IP address which will be used by a migration
# source for transferring migration data to this host. The migration
# source has to be able to resolve this hostname and connect to it so
# setting "localhost" will not work. By default, the host's configured
# hostname is used.
#migration_host = "host.example.com"
# Override the port range used for incoming migrations.
#
# Minimum must be greater than 0, however when QEMU is not running as root,
# setting the minimum to be lower than 1024 will not work.
#
# Maximum must not be greater than 65535.
#
#migration_port_min = 49152
#migration_port_max = 49215
# Timestamp QEMU's log messages (if QEMU supports it)
#
# Defaults to 1.
#
#log_timestamp = 0
# Location of master nvram file
#
# This configuration option is obsolete. Libvirt will follow the
# QEMU firmware metadata specification to automatically locate
# firmware images. See docs/interop/firmware.json in the QEMU
# source tree. These metadata files are distributed alongside any
# firmware images intended for use with QEMU.
#
# NOTE: if ANY firmware metadata files are detected, this setting
# will be COMPLETELY IGNORED.
#
# ------------------------------------------
#
# When a domain is configured to use UEFI instead of standard
# BIOS it may use a separate storage for UEFI variables. If
# that's the case libvirt creates the variable store per domain
# using this master file as image. Each UEFI firmware can,
# however, have different variables store. Therefore the nvram is
# a list of strings when a single item is in form of:
# ${PATH_TO_UEFI_FW}:${PATH_TO_UEFI_VARS}.
# Later, when libvirt creates per domain variable store, this list is
# searched for the master image. The UEFI firmware can be called
# differently for different guest architectures. For instance, it's OVMF
# for x86_64 and i686, but it's AAVMF for aarch64. The libvirt default
# follows this scheme.
#nvram = [
# "/usr/share/OVMF/OVMF_CODE.fd:/usr/share/OVMF/OVMF_VARS.fd",
# "/usr/share/OVMF/OVMF_CODE.secboot.fd:/usr/share/OVMF/OVMF_VARS.fd",
# "/usr/share/AAVMF/AAVMF_CODE.fd:/usr/share/AAVMF/AAVMF_VARS.fd",
# "/usr/share/AAVMF/AAVMF32_CODE.fd:/usr/share/AAVMF/AAVMF32_VARS.fd"
#]
# The backend to use for handling stdout/stderr output from
# QEMU processes.
#
# 'file': QEMU writes directly to a plain file. This is the
# historical default, but allows QEMU to inflict a
# denial of service attack on the host by exhausting
# filesystem space
#
# 'logd': QEMU writes to a pipe provided by virtlogd daemon.
# This is the current default, providing protection
# against denial of service by performing log file
# rollover when a size limit is hit.
#
#stdio_handler = "logd"
# QEMU gluster libgfapi log level, debug levels are 0-9, with 9 being the
# most verbose, and 0 representing no debugging output.
#
# The current logging levels defined in the gluster GFAPI are:
#
# 0 - None
# 1 - Emergency
# 2 - Alert
# 3 - Critical
# 4 - Error
# 5 - Warning
# 6 - Notice
# 7 - Info
# 8 - Debug
# 9 - Trace
#
# Defaults to 4
#
#gluster_debug_level = 9
# virtiofsd debug
#
# Whether to enable the debugging output of the virtiofsd daemon.
# Possible values are 0 or 1. Disabled by default.
#
#virtiofsd_debug = 1
# To enhance security, QEMU driver is capable of creating private namespaces
# for each domain started. Well, so far only "mount" namespace is supported. If
# enabled it means qemu process is unable to see all the devices on the system,
# only those configured for the domain in question. Libvirt then manages
# devices entries throughout the domain lifetime. This namespace is turned on
# by default.
#namespaces = [ "mount" ]
# This directory is used for memoryBacking source if configured as file.
# NOTE: big files will be stored here
#memory_backing_dir = "/var/lib/libvirt/qemu/ram"
# Path to the SCSI persistent reservations helper. This helper is
# used whenever <reservations/> are enabled for SCSI LUN devices.
# If this is not an absolute path, the program will be searched for
# in $PATH as well as a few additional directories.
#pr_helper = "qemu-pr-helper"
# Path to the SLIRP networking helper.
#slirp_helper = "/usr/bin/slirp-helper"
# Path to the dbus-daemon
# If this is not an absolute path, the program will be searched for
# in $PATH.
#dbus_daemon = "dbus-daemon"
# User for the swtpm TPM Emulator
#
# Default is 'tss'; this is the same user that tcsd (TrouSerS) installs
# and uses; alternative is 'root'
#
#swtpm_user = "tss"
#swtpm_group = "tss"
# For debugging and testing purposes it's sometimes useful to be able to disable
# libvirt behaviour based on the capabilities of the qemu process. This option
# allows to do so. DO _NOT_ use in production and beaware that the behaviour
# may change across versions.
#
#capability_filters = [ "capname" ]
# 'deprecation_behavior' setting controls how the qemu process behaves towards
# deprecated commands and arguments used by libvirt.
#
# This setting is meant for developers and CI efforts to make it obvious when
# libvirt relies on fields which are deprecated so that it can be fixes as soon
# as possible.
#
# Possible options are:
# "none" - (default) qemu is supposed to accept and output deprecated fields
# and commands
# "omit" - qemu is instructed to omit deprecated fields on output, behaviour
# towards fields and commands from qemu is not changed
# "reject" - qemu is instructed to report an error if a deprecated command or
# field is used by libvirtd
# "crash" - qemu crashes when an deprecated command or field is used by libvirtd
#
# For both "reject" and "crash" qemu is instructed to omit any deprecated fields
# on output.
#
# The "reject" option is less harsh towards the VMs but some code paths ignore
# errors reported by qemu and thus it may not be obvious that a deprecated
# command/field was used, thus it's suggested to use the "crash" option instead.
#
# In cases when qemu doesn't support configuring the behaviour this setting is
# silently ignored to allow testing older qemu versions without having to
# reconfigure libvirtd.
#
# DO NOT use in production.
#
#deprecation_behavior = "none"
# If this is set then QEMU and its threads will run in a separate scheduling
# group meaning no other process will share Hyper Threads of a single core with
# QEMU. Each QEMU has its own group.
#
# Possible options are:
# "none" - (default) neither QEMU or any of its helper processes are placed
# into separate scheduling group
# "vcpus" - only QEMU vCPU threads are placed into a separate scheduling group,
# emulator threads and helper processes remain outside of the group
# "emulator" - only QEMU and its threads (emulator + vCPUs) are placed into
# separate scheduling group, helper processes remain outside of
# the group
# "full" - both QEMU and its helper processes are placed into separate
# scheduling group
#sched_core = "none"
# Using nbdkit to access remote disk sources
#
# If this is set then libvirt will use nbdkit to access remote disk sources
# when available. nbdkit will export an NBD share to QEMU rather than having
# QEMU attempt to access the remote server directly.
#
# Possible values are 0 or 1. Default value is 1. Please
# note that the default might change in future releases.
#
#storage_use_nbdkit = 1
Commenting out this block and running systemctl restart libvirtd.service allows for VMs to be started again.
Any ideas what might be the culprit?
You’re missing a comma after /dev/userfaultfd. Change:
cgroup_device_acl = [
"/dev/null", "/dev/full", "/dev/zero",
"/dev/random", "/dev/urandom",
"/dev/ptmx", "/dev/kvm",
"/dev/userfaultfd"
"/dev/kvmfr0"
]
To:
cgroup_device_acl = [
"/dev/null", "/dev/full", "/dev/zero",
"/dev/random", "/dev/urandom",
"/dev/ptmx", "/dev/kvm",
"/dev/userfaultfd",
"/dev/kvmfr0"
]
You gotta be kidding me. 
I knew it would be something trivial, but damn, a comma. This is embarrassing.
Just tried it and while the framerate didn’t improve, it does feel less sluggish now.
Thank you for your help!!