Was planning the same thing, but the only thing I wish the pi-kvm had, for such a setup, is the ability to control 4 ATX power and reset buttons at the same time, through an expansion hat or something. You can only control one PC (and see the activity LEDs).
I guess you could mod the pi itself with the GPIO to control random relay switches (the 3v ones I think, for the gpio) and do momentary presses or N seconds holds to close the circuit via a cli script through SSH. But it’s jank.