Shortcut key not registering when used in a short time frame

I’m using the keyboard shortcut a lot to change between computer 1 and 2 (ctrl + ctrl + 1 ~ 4).

However, I’m having a issue where I have to press the windows key before changing back to the initial computer. It appears that the shortcut does not register. So I end up doing the shortcut, not hearing the confirmation beep, pressing the windows key twice, redoing the shortcut sequence and then it works.

Is this behavior expected?

EDIT: I got a 4port KM Switch

Try it with scroll lock instead of ctrl in case something is debouncing ctrl