The thing to understand with hid ports is they are grouped together internally for soft device mapping cutover.
If you have one misbehaving hid device it can bring down the whole stack. Can you rotate devices off of the hid ports to the usb3 ports one at a time to find the culprit? Reboot the kvm after switching from hid to USB3 also.