How are the drives attached to the VM? I experienced the same issues when passsing through physical storage devices with virtio, same goes for QCOW and raw disk images as there is a clear bottleneck in the way the emulation of storage devices are handled. If this is the case you may also want to give this a look as it suggests some optimisations you can do to improve disk performance in QEMU/Libvirt that may also help with this issue.
Another option which is what solved it for me was installing Windows onto physical disks and passing through a seperate SATA controller to the virtual machine. If your motherboard has more than one SATA controller to spare you can pass it through to the VM or use a PCIe SATA card and attach to the disks to that.
How much ram are you giving the guest? By far the biggest improvement I saw was when I passed through a sata controller with win10 installed on it already. In virtmanager, pass through the sata card (that you will have passed through with vfio-pci) and tell virtmanager to boot to that sata card. Not sure if you 've tried exactly that, but that’s what I’d first suggest.
I had the same problems with the networkdrivers as well. Changed to Virtio and installed drivers from the iso and everything went smooth.
Take a look at the resources used by the guest when this is happening, this could give you an hint.