There is a built-in throttle & deny system in Nginx, but I’d like to not target one single server if possible. Nginx is quite speedy at processing the 407 urls I am already catching, and I know Apache & others (thttpd, Lighttpd, etc) can also do that, while maintaining reletively low overheads.
But not every webserver has a throttle & deny system, and ideally the final “binary” needs to be able to sample then flag at least 4 invalid attempts in less that a second per IP request (just like a router or firewall does).
The principle being the more useless traffic blocked upstream, the cleaner (and hence faster) the network in between. The result being vaugley similar to what users experience after installing (and using) PiHole for bogus DNS traffic.
And in reality this actually means 4 denied attempts, so after a reponse has been made, the idea is to send small requests based on that info to an internal endpoint on a router/firewall/switch that will then instigate an IP block/ban/deny/blackhole for a short period of time on the internet side of an internal lan, ideally pushing the same sort of data upstream to the next hop device, so (in an ideal world) blocking would eventually be done at the first hop where the offending IP address enters the internet, not the last hop before entering an endpoint gateway (and definitely not the targeted server which may already be under load serving legitimate content or services).
There is nothing wrong with Perl or Python dealing with this sort of information at the targeted host, but ideally you want something that can handle 10K hits per second, or 100K hits/s, or maybe even 1M hits/s even.
So far what I have (which isn’t sampling live) appears to already have reduce things to a more than bareable outbound traffic rate (AVG <10Kb/s and falling), and yet it only consists of some 400+ lines of code in 10 scripts, of which only 4 are run every hour in less than 1 sec. while consuming less that 1% of a single core CPU.