Dmesg is spammed with "nvidia-nvlink: Nvlink Core is being initialized"

I’ve upgraded from Lubuntu 18.10 to Kubuntu 19.10 for my work-gaming combo PC and now the VM with PCI passthrough is very unstable, compared to how it was in 18.10.

The kernel driver in use is vfio-pci,

user@user-MS-7B44:/mnt/raid/QEMU$ lspci -nnv
02:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP104 [GeForce GTX 1070 Ti] [10de:1b82] (rev a1) (prog-if 00 [VGA controller])
        Subsystem: CardExpert Technology GP104 [GeForce GTX 1070 Ti] [10b0:1b82]
        Flags: fast devsel, IRQ 10
        Memory at de000000 (32-bit, non-prefetchable) [disabled] [size=16M]
        Memory at c0000000 (64-bit, prefetchable) [disabled] [size=256M]
        Memory at d0000000 (64-bit, prefetchable) [disabled] [size=32M]
        I/O ports at e000 [disabled] [size=128]
        Expansion ROM at df000000 [disabled] [size=512K]
        Capabilities: <access denied>
        Kernel driver in use: vfio-pci
        Kernel modules: nvidiafb, nouveau, nvidia_drm, nvidia

but every 300 ms this is spammed in dmesg logs

user@user-MS-7B44:/mnt/raid/QEMU$ dnesg
[  327.609906] nvidia-nvlink: Nvlink Core is being initialized, major device number 511
[  327.610231] NVRM: The NVIDIA probe routine was not called for 1 device(s).
[  327.610231] NVRM: This can occur when a driver such as: 
               NVRM: nouveau, rivafb, nvidiafb or rivatv 
               NVRM: was loaded and obtained ownership of the NVIDIA device(s).
[  327.610231] NVRM: Try unloading the conflicting kernel module (and/or
               NVRM: reconfigure your kernel without the conflicting
               NVRM: driver(s)), then try loading the NVIDIA kernel module
               NVRM: again.
[  327.610232] NVRM: No NVIDIA devices probed.
[  327.610424] nvidia-nvlink: Unregistered the Nvlink Core, major device number 511

which makes me think this might be why is unstable.

How do I stop it from probing the device?
I’ve tried to add

blacklist nouveau
options nouveau modeset=0

in /etc/modprobe.d/blacklist-nouveau.conf, but it has no effect.

It stopped when I uninstalled nvidia driver.
Instability still persists though, so it was unrelated.