[Devember 2021] brutally simple firewall

Dev Zone Code
, ,
1

Only a couple of days left in December, and I previously said I would both upload the project to GitHub and post it for Devember2021, so here it is.

Brutality Bonus (updated)

Brutal? How brutal? Well of 3882 haxor urls served to indiviuals @ 1k/s, these over 100Mb:

101244928 = 28 hours
102637568
112189768
112914432
118325248
123443296
129685528
133234272
135237632
136433664
136470528
136470528
159956992
160206848
178102272
188473344
195564912
209518592
299053056 = 83 Hours

What is it? SS:FragWhare

SSFW is a super simple firewall (for search purposes), Whare (farry) is Maori for House, so FragHaus. The brutality of SSFW comes from the setting of how many IPv4 occurances it will scan for in sshd log entries and webserver logs, combined with a haxor list of known web urls that try to commandeer your device, appliance or VM, and an (optional) “here have this 4.5Gb file at 1K/s for your efforts” brutality bonus.

Because I have been testing this on my Linode sevrer (signed up via the link in the news videos) for just over 6 months now, I have a pretty solid IPv4 block list, and known urls capture list, which I am also making available as part of this project (a Devember Project Present if you like).

It comes with some simple php “viewer” scripts that give you an idea and overview of whats going on, and simplifies IPv4 verification queries, with the filesystem being used to supply the extracted information (so they too can be web accessible if desired), including the total blocked IPv4 addresses along with how many were blocked in the last hour and last 24 hours (all of which is easier, quicker and simpler to do via the filesystem).

SSFW is meant for systems where the bloat of regular alternative firewall systems is otherwise not wanted, nor warrented. To that end, there is also a standalone version of the sshd (gerka) service script, if you dont need webserver protection.

SSFW is designed to be managed from within the webserver tree, as the webserver user, but does NOT use or assign sudo privileges.

The Plan

For #devember2021 my plan is to write an installer, which means I can finally upload the code, and obviscate any personal hardcoded names and locations, which would make it easy to detect and abuse if they were not randomised or unique at install time.

I figure I can do this in 4-ish days if I try really hard (maybe). The extra bits that I will add if there is time (or after the deadline) are the extended documentation and notes. In the meantime, all the scripts are documented in the code, and all bar one have full commandline help. Again they have all been thoroughly tested for more than nearly 7 months now.

NOTE: the webserver testing and scripts have only been done with Nginx so far, others to come as I only have one server atm. The nginx scripts can easily be adapted if others want to use them for other webservers. This is the other reason for supplying a “capture list”, which can be easily manipulated into the required format.

The Extras

I will be adding this text to the project, but an upfront notice is definitely require if you are NOT using this firewall system on a Linode server instance.

  • the blocked IPv4 ranges include ALL DigitalOcean ranges bar one, and the new Microsoft Azure cloud ranges as they get used and abused by attackers.
  • there are alot of blocked ranges and IP address that originate in the Netherlands and China, with some minor ranges from other parts of South Eastern Asia and their surroundings, and some from the Baulkins.
  • if you try and fail any sshd connection or key exchange (kex) that IPv4 address will be blocked by default based on the settings you use. 1 failed sshd attempt records the IP address 2-3 times in the log output. (Linode has remote + weblish access which allows this to be fixed if it causes a problem).

Yes, there are individually blocked IPv4 addresses from every single DigitalOcean range allocatrion, bar one, and they have some 30-50 range allocations covering millions of IPv4 addresses. I have verified 2 attempts from assigned Linode ranges, both were deallocated before or after I verified them (kudos to Linode for not allowing abuse via their networks).

By my calculations, both this firewall system and the block lists, can be used in part or in conjunction with other “software”, like iptables or pihole, without conflicts.

Anyways

Wish me luck (I have regular+random power issues here).

Cheers

Paul

12 Likes
2

Good luck and Merry Christmas!

2 Likes
3

Well the README.md is about as complete as its ever going to be, so at least you will know what to expect from the code and installer when it goes up. The mentioned “regex fix last night” under SSFW Structure was in the range lookup, it was saying “58.x.” was the same as “8.x”, now it knows better.

I forgot ro mention that SS:FragWhare was originally just the name of the gerka service , but I think it makes sense to maintain the “SSFW” acronym and “super simple firewall” for search purposes (thats the text I used to try and find something to do this originally).

EDIT: I just have to patch in some new urls (about 20 or so from the last 4-5 months) for the url block list before I will upload those (before the rest of the code), but I have forgotten how to use it (its been 4 months and I only used it 1x per month before that to update the url block list), good thing it is apparently additive (not replacitive) by default :slight_smile:

4

Bah, I stuck up a Design Decission document for the hell of it.

ED: does anyone know of any coders more prolific at documentation than coding?

:slight_smile:

5

Well I guess it was just a matter of time, I just added that last DigitalOcean range due to 2 seperate url attemps on the 29th UTC (today, or yesterday for me), both looking for wordpress, both within 5 seconds of each other. Yes they were from seperate IPv4 addresses.

Edit: It seem the range might be in Australia, a traceroute from Texas goes through one of the Singapore servers on the Telstra global network, but there are 5 hops after that without name resolution, so I am making an educated guess.

6

added install.sh that can test your input settings. Should have enough variables from that to patch the scripts within 3 hours. We will see …

7

Well I got the basis on the install script done, and upload the first of the shell scripts with the substitutions in. Unfortunately is a new archive script I have been testing for about 4 months now and ATM it is both Nginx and Linode specific - they are quirks and I need to figure out how to abstact them.

But its the New Year now, and I need a break, so I am gonna watch the latest episode of Expanse then get back to it.

8

done, 6 hours instead of 3 hours, not bad. Actually 3 hours if you dont include the “new” backup script. Standalone is there. Forgot to upload the block lists first, duh.

EDIT: everything should work, but I nginx’d a couple of the process script, so they wont work with the cronjob scripts out-of-the-box. This is fine because it really needs a webserver script re-linker anyway, just to make it easier to setup the defaults for a specific webserver

NOTE: remember that the standalone script does not need anything else to be useful - instructions are in the source code of the ss-fragwhare.sh script (or the link in the OP)

9

I shall quote this “officially” 1st New Year commit message

rename: I woke up just now, looked at the last entry and …

just realised now that what I originally wanted is what the gerkas
are, a super simple firewall, the rest is just fluff from me wanting
to automate various stuff, generate logs and block list, and adding
other forms of block analysis & feedback, to simplify what you need to type on
(a sometimes dodgey) remote server link thats SSH but not.

But Hey, the “fluff” has an installer now :slight_smile:

I will try and get the blocklists up today (now that I emptied the logs - duwh)

10

“oh I do love to be beside the seaside” - lalala

A day at the beach, and I have automated the block list GH uploads (one of the benefits of a camper), including a single static named “latest” archive (which contains the other archives), and a “latest.txt” so it can be referenced and easily automated.

There are 5 individual archive files available, 2x as filesystem data, 2x as text file data, and seperate (not hidden) range text file data.

Cheers & Happy New Year

Paul

2 Likes
11

I decided to put up a statically named ssfw-blocklists/known_urls.txt as well, for those interested in what has been tried over a 6 month period. By changing /blob/ to /raw/ in the url you can downbload it or otherwise look at in its raw text formatting.

I am thinking of doing something similar for sshd “Failed password for invalid user” names, too.

Hopefully both of these will help people, projects, developers and administrators better secure webserver and sshd access.

EDIT: again a reminder that the IPv4 addresses and ranges need to be screened before using them, as (unless you are on Linode) there is a high chance some of your users, connecting api servers, or hosting services will be in there.

1 Like
12

just a note on the known_urls.txt:

There are some urls missing, like a lot of the comon PHPmyAdmin urls. This is because I wrote a custom header include for when the nginx config is generated, so those urls need to be culled from the data 1st. The same is true for some common Wordpress urls. Here is part of that custom include:

		## fake locations (`/.env` is linked to a 4Gb ISO)
		#  search will stop on `location = ` match
		#  `aio threads` is non-blocking, uses less ram
		#  but the log entries only on connection close

		location ~ .txt$ {
			try_files $uri		/.env;
		}

		location ~ .env$ {
			try_files $uri		/.env;
		}

		location ~ .xml$ {
			try_files $uri		/.xml;
		}
		
		# These are custom multi-match uri's and so
		# need to be removed from the pre-processor

		location ~ print.css$ {
			try_files $uri		/.css;
		}

		location ~ /phpunit {
			try_files $uri 		/.php-html;
		}

		location ~* /pma {
			try_files $uri		/.php-html;
		}

		location ~* /phpmyadmin {
			try_files $uri		/.php-html;
		}

yes those are hidden files in the root of the webserver, and they are all soft-linked to the brutality bonus (random 4.5Gb file, which may or may not be a Win11 ISO file)

this is part of the mime include related to the above

		location = /.env {
			access_log      /some-svr_log-dir/nginx-haxors.log;
			default_type    text/plain;
			index			.env;
			limit_rate		1k;
			sendfile		on;
			aio			threads;
		}

		location = /.css {
			access_log      /some-svr_log-dir/nginx-haxors.log;
			default_type	text/css;
			index			.css;
			limit_rate		1k;
			sendfile		on;
			aio			threads;
		}

		location = /.xml {
			access_log      /some-svr_log-dir/nginx-haxors.log;
			default_type	text/xml;
			index			.xml;
			limit_rate		1k;
			sendfile		on;
			aio			threads;
		}

		location = /.php-html {
			access_log      /some-svr_log-dir/nginx-haxors.log;
			default_type	text/html;
			index			.php-html;
			limit_rate		1k;
			sendfile		on;
			aio			threads;
		}

Cheers

Paul

13

I added a user script to generate the blocklist archives, which outputs a download url, along with a download extracter that only require the diff in the url (ie the unique date part).

I also added an extra way to delete ranges. If you just take a quick glance at the tools scripts, you might see ~_remove-blocked-ipv4.sh (at bottom of listing) before you see ~_block-ipv4-range.sh (at top of listing), and not immediately realize the latter has a del option, so I added that ability (delete range) too, not just plain IPv4 addresses.

I am trying to come up with a way to perform a range block in some intelligent way. Toward that end, I added a script that will give you an ASN when supplied with a IPv4, as a bonus, it also outputs the CIDR range for that IP address as well. This only works for public IP addresses, and relies on an external api service (which is documented in the source code too).

You can then take that ASN value and use it to query the (public) BGP, and it will show you all the IP ranges assigned to that entity. This was how I was able to realise just how many DigitalOcean servers are used for hacking attempts against both sshd and the webserver.

As a result of this, I may also look at upstreaming certain items into some of the dynamic protection systems that use AI and monitoring solutions that are designed to protect, and that are currently available, some of which provide web frontend, and may be intergrated into a server management solutions (like PiHole is).

14

I just replace 2 ranges with a higher blocked range (3 days ago), because there were 30+ extra IPv4 addresses outside those 2 ranges, and the result (may not be an actual correlation) has been a halving in the per day block rate.

A stats update:

3 days ago: 6-7 blocked IPv4 per day 
currently:  3 blocked IPv4 per day
total blocked IPv4 ranges: 273
total blocked IPv4: 6513

It took 2 months to get the 1st 4000, and 2 months to get the next 2000, while it has taken 3 months to get that last 500. After I blocked a couple of the new Microsoft Azure Cloud Service ranges (there were no MS ranges blocked before that), there were near 0 blocks per day for almost 2 weeks.

2 Likes
15

hmm… 3 per day lasted 3 days, back up to 10 per day atm, looks like even hackers take christmas and new years holidays (at least for a bit).

just doing some range checking, had no range entries for DigitalOcean’s 178.128 IP address block, out of 41 blocked IPv4 addresses (since late August when I block most DigitalOcean ranges), 16 unique ranges are present (just a couple of mid and high are missing), so I range banned 178.128.0.0/16 instead of the 19 individual ranges (most are /20 address blocks).

While looking up some other ip addresses, I also saw a server address of: mongo.blahblah.mongodb.ondigitalocean.com

Not sure if that domain is bound to the same ASN though, guess there are ways to check, just thought it was interesting (thats either an SSH or a webserver attack coming from a MongoDB server at DigitalOcean).

EDIT: FYI

  • DigitalOcean have over 700 IPv4 ranges assigned to their (main) ASN, mostly in the US & UK, and a large number in NL, but there ranges cover a good part of the planet (country assigned IP addresses).
  • You would think (via media) that most IPv4 address traceroutes would end up in Russia, China, (and the US if you read between the lines) but they dont, most go back to the Netherlands.
  • If attacks come from at least one of every IPv4 address range a specific ASN entity owns (supprisingly not DigitalOcean this time), and those ranges are used by various other (somethimes seemingly unrelated) hosting and ISP services (in 10 different countries), does that show collusion, a government agency, a private agency, or a criminal organisation of some sort?
1 Like
16

blocked : 13 in last hour : 46 in last 24 hours
blocked : 11 in last hour : 39 in last 24 hours
blocked : 16 in last hour : 55 in last 24 hours

nah, business as usual. (yeah thats 3 days in a row now :slight_smile: )

I must have just caught a break for some reason when I only got 3/24hrs for 3 days, who knows, but its still way higher than the averages I saw last year.

With that many new IPv4 per day being added, chances are there are some more ranges that can be blocked too.

EDIT:
yeah, BIG patterns. the system log files have rotated since yesterday, and all 177 sshd entries are unique, but there are about 12 new ranges that can be blocked.

17

6898 blocked : 2 in last hour : 64 in last 24 hours

Got my first automated sub domain attack a couple of days ago, 420 subdomains in less than 1 minute.

Because they were all to root of the web server (ie. legit), they are not factored into any IPv4 block analysis, so a manual block was used.

However, it gave me a chance to write an ASN prefix allocation (IPv4 range allocations) scraper, which I can pipe to the range blocker script. This in itself brought me up against something that has frustrated me on the odd occasion, CloudFlare.

This time I defeated CloudFlares JS browser probe with PhantomJS, but I dont want to need that as a prerequisite for a firewall (python, qtwekit, nodejs - all of which are fat), plus the online ASN tool I use streams a single line of content, so its only really useful from my desktop machine. I got the data from somewhere much cleaner, but I do see a difference is their output.

On top of that I realised the extra data I wanted to see (what and why a block occured) is needed to create valid DShield logs (for SANS Internet Storm Centre submissions), so “cool”, gives me something else to think about too (v2?).

And that brought me to the idea of the standalone Gerka’s having the option of placing something funny in the message log, eg:

Jan 25 17:58:00 localhost cron.info crond[2078]: USER root pid 881 cmd /work/www/tools/unique_block-ipv4-redo.sh
Jan 25 17:59:13 localhost ssfw.info ssfw-ipv4-update[32192]: 17.11.0.14 and 4 others shot in the back, buried deep [added](5) 
Jan 25 18:06:25 localhost auth.info sshd[1329]: Failed password for root from 81.150.9.251 port 52661 ssh2
Jan 25 18:06:25 localhost auth.info sshd[1329]: Received disconnect from 81.150.9.251 port 52661:11: Bye Bye [preauth]
Jan 25 18:06:25 localhost auth.info sshd[1329]: Disconnected from authenticating user root 81.150.9.251 port 52661 [preauth]
Jan 25 18:06:35 localhost ssfw.info ssfw-sshd-gerka[11961]: 81.150.9.251 was shot down, Bye Bye urself, pesky haxor [root](1)
Jan 25 18:15:00 localhost cron.info crond[2078]: USER root pid 1807 cmd run-parts /etc/periodic/15min
Jan 25 18:15:00 localhost cron.info crond[2078]: USER www pid 1808 cmd /work/www/tools/unique-svr_cron-jobs.sh 15min
Jan 25 18:16:13 localhost ssfw.info ssfw-cron-nginx[32291]: 179.11.11.142 wiped the floor clean, guts everywhere [sshd](33)
Jan 25 18:16:33 localhost ssfw.info ssfw-cron-nginx[32291]: 6.114.0.141 decimated after barrage, 5kR1p7 kiddie [subdomain](420) 
Jan 25 18:33:05 localhost ssfw.info ssfw-sshd-gerka[11961]: 8.15.19.25 faught hard, died lonely, janator mode [root](16)
Jan 25 18:56:35 localhost ssfw.info ssfw-sshd-gerka[11961]: 1.150.9.51 sniped in 4 seconds, trophy hunter [user](8)
Jan 25 19:15:00 localhost cron.info crond[2078]: USER root pid 1809 cmd run-parts /etc/periodic/15min
Jan 25 19:15:00 localhost cron.info crond[2078]: USER www pid 1810 cmd /work/www/tools/unique-svr_cron-jobs.sh 15min
Jan 25 19:16:13 localhost ssfw.info ssfw-cron-sshd[37290]: 17.1.11.14 sniper on sniper action, keys recovered [kex](1)
Jan 25 19:16:33 localhost ssfw.info ssfw-cron-nginx[37290]: 176.114.0.141 posted death certificate, \0x that [haxor](3)

well its just an idea, those log files are so dry, it would make sandpaper thirsty. :slight_smile:

Anyhoo

Cheers

Paul

1 Like
18

I have come across what looks like (maybe) a professional hacker, or someone looking to make money of a results of what they find (I cant see it as just collecting the target results as an end game).

I noticed a couple of /.git/config probes in last moths logs (2 to be exact), and the only thing they had in common was the HTTP_AGENT (python-requests/2.9.1) and the fact they only pulled the exact same amount of data (remember I have all haxor pulls linked to a 4Gb ISO file) being 8192 bytes.

I went back over the previous logs, and the same sort of probes go back to August 2021, when I started the project.

What raised my eyebrows was of all 25 probes only 1 ipv4 came up 2x, and that was the 1st two probes, at the same time (Alpine does not record anything less than seconds).

Why I think there is something worse going on here than just a “script kiddie” being nosey or annoying, is that every single DNS lookup (that I checked) of those 24 unique IPv4 addresses resulted in a _something_.linodeusercontent.com url.

When I did a geolocation lookup, every single one was from a different Linode service location around the world.

My thinking is that they are looking for private origin repos (ie cloned private repos) as opposed to public or cloned public repos.

I am not sure what to do about this. I know the support staff at Linode will look at it, but I dont know weather they are interested in tracking this person down.

I would say they have already been successful at it, which is why they keep trying. Since either interlectual property or money are involved I dont think “a slap on the wrist” is the right thing to do here.

Then there is the problem of “from which country did they sign up”.

Hmm… (just going over the log entries again - I greped them into a seperate file)

It seems like 2 seperate people, one pulls 8192 bytes with an HTTP_AGENT of Mozilla/5.0 (but always from a different platform, with a different Chrome version number) about once a month.

The other person using python agent always pulls 4.5 - 5.5 Mb before disconnect. Actually they are often in groups of 2-3 probes (from different IPv4), and when they are in groups, they just about always start at 1:30am GMT/UTC (1 starts a 10am) and the last one is always between 5pm & 10pm GMT/UTC (when there are 3 probes, the middle one is always bewteen 4pm and 5pm)

FWIW my server is in Texas, GMT is default for Alpine Linux, and its exactly 12hrs behind me (which is why I left it set that way - my clock on my desk is set to GMT/UTC).

FYI there is one Go-http-client/1.1 in there too, just once near 1:30am, but not from a Linode server (but yes, the Netherlands again)

Anyways, something to think about …

1 Like
19

This is what a coordinated, distributed and controlled, but sustained attack looks like. You can’t call it a probe, because the blocked IPv4 Addresses are coming from failed sshd username attempts. You might call it a soft attack as each SSH attempt comes from a different IP Address, and there is (usually) a descent time difference before the next attempt.

Having checked, I had already range blocked 43.128.0.0/15 and 43.154.0.0/16 , which is why none of those ranges are present. I am not 100% sure if this coordination is limited to this network, but ALL of these 43.* addresses list [email protected] as their “Abuse contact”, and they ALL list location as either US or SG (2 of Tencent international Cloud locations). There are 122 unique IPv4 here. For every 10 addresses 2 were for root user attempts, the rest are a random collection of known possible Services or Users username attempts.

( date -I seconds | PID | sleep # | sshd Gerka | ip route add blackhole * )

2022-02-01T03:25:29+0000#28607#10#monitor#added: blackhole 43.155.106.140 
2022-02-01T05:30:53+0000#28607#10#monitor#added: blackhole 43.155.74.70 
2022-02-01T05:56:24+0000#28607#10#monitor#added: blackhole 43.254.156.42 
2022-02-01T09:17:01+0000#28607#2#monitor#added: blackhole 43.155.68.119 
2022-02-01T10:20:03+0000#28607#10#monitor#added: blackhole 43.134.201.169 
2022-02-01T15:45:00+0000#28607#10#monitor#added: blackhole 43.155.74.252 
2022-02-02T00:35:19+0000#28607#10#monitor#added: blackhole 43.134.187.246 
2022-02-02T05:19:31+0000#28607#10#monitor#added: blackhole 43.153.6.100 
2022-02-02T07:30:29+0000#28607#10#monitor#added: blackhole 43.155.60.53 
2022-02-02T22:26:39+0000#28607#10#monitor#added: blackhole 43.130.45.123 
2022-02-02T23:19:30+0000#28607#10#monitor#added: blackhole 43.155.68.111 
2022-02-03T00:45:10+0000#28607#1#monitor#added: blackhole 43.134.202.54 
2022-02-03T02:13:24+0000#28607#10#monitor#added: blackhole 43.156.42.138 
2022-02-03T08:22:08+0000#28607#5#monitor#added: blackhole 43.155.100.182 
2022-02-03T08:38:09+0000#28607#10#monitor#added: blackhole 43.155.92.208 
2022-02-03T16:03:26+0000#28607#10#monitor#added: blackhole 43.155.101.118 
2022-02-03T16:04:21+0000#28607#10#monitor#added: blackhole 43.130.44.186 
2022-02-03T18:26:40+0000#28607#10#monitor#added: blackhole 43.156.54.220 
2022-02-03T21:40:02+0000#28607#10#monitor#added: blackhole 43.155.68.187 
2022-02-04T05:17:23+0000#28607#10#monitor#added: blackhole 43.156.46.96 
2022-02-04T21:49:12+0000#28607#10#monitor#added: blackhole 43.155.115.152 
2022-02-04T22:38:49+0000#28607#10#monitor#added: blackhole 43.155.64.136 
2022-02-04T23:24:51+0000#28607#10#monitor#added: blackhole 43.134.202.87 
2022-02-05T00:39:33+0000#28607#10#monitor#added: blackhole 43.156.46.55 
2022-02-05T03:02:00+0000#28607#9#monitor#added: blackhole 43.155.93.82 
2022-02-05T03:05:05+0000#28607#10#monitor#added: blackhole 43.135.158.214 
2022-02-05T21:01:31+0000#28607#10#monitor#added: blackhole 43.155.60.117 
2022-02-05T21:48:07+0000#28607#10#monitor#added: blackhole 43.155.60.206 
2022-02-05T21:51:32+0000#28607#10#monitor#added: blackhole 43.156.48.174 
2022-02-06T05:03:12+0000#28607#10#monitor#added: blackhole 43.155.78.35 
2022-02-06T05:22:14+0000#28607#10#monitor#added: blackhole 43.155.72.149 
2022-02-06T08:02:39+0000#28607#10#monitor#added: blackhole 43.134.205.14 
2022-02-06T13:55:19+0000#28607#10#monitor#added: blackhole 43.155.97.128 
2022-02-06T14:34:55+0000#28607#10#monitor#added: blackhole 43.156.46.178 
2022-02-06T14:55:01+0000#28607#10#monitor#added: blackhole 43.155.118.244 
2022-02-06T15:27:57+0000#28607#10#monitor#added: blackhole 43.155.65.167 
2022-02-06T16:30:21+0000#28607#6#monitor#added: blackhole 43.134.207.150 
2022-02-07T14:46:46+0000#28607#10#monitor#added: blackhole 43.156.45.112 
2022-02-07T16:46:54+0000#28607#10#monitor#added: blackhole 43.153.33.120 
2022-02-07T21:24:42+0000#28607#10#monitor#added: blackhole 43.153.5.122 
2022-02-08T01:08:18+0000#28607#10#monitor#added: blackhole 43.134.237.89 
2022-02-08T05:20:49+0000#28607#10#monitor#added: blackhole 43.155.63.36 
2022-02-08T10:08:18+0000#28607#10#monitor#added: blackhole 43.155.114.143 
2022-02-08T15:33:46+0000#28607#10#monitor#added: blackhole 43.132.135.222 
2022-02-08T18:50:17+0000#28607#10#monitor#added: blackhole 43.153.32.58 
2022-02-09T06:08:47+0000#28607#10#monitor#added: blackhole 43.132.251.88 
2022-02-09T19:53:35+0000#28607#10#monitor#added: blackhole 43.132.251.145 
2022-02-09T20:30:41+0000#28607#10#monitor#added: blackhole 43.155.64.249 
2022-02-09T23:10:56+0000#28607#10#monitor#added: blackhole 43.156.47.247 
2022-02-09T23:12:31+0000#28607#10#monitor#added: blackhole 43.132.180.108 
2022-02-10T14:18:37+0000#28607#10#monitor#added: blackhole 43.155.71.41 
2022-02-10T18:03:23+0000#28607#10#monitor#added: blackhole 43.155.107.219 
2022-02-10T22:27:32+0000#28607#10#monitor#added: blackhole 43.153.15.29 
2022-02-11T03:36:08+0000#28607#10#monitor#added: blackhole 43.134.202.107 
2022-02-11T04:42:21+0000#28607#10#monitor#added: blackhole 43.155.67.153 
2022-02-11T07:40:35+0000#28607#10#monitor#added: blackhole 43.130.61.158 
2022-02-11T10:46:10+0000#28607#10#monitor#added: blackhole 43.155.94.254 
2022-02-11T11:19:56+0000#28607#10#monitor#added: blackhole 43.134.199.32 
2022-02-11T15:59:48+0000#28607#10#monitor#added: blackhole 43.132.246.223 
2022-02-11T16:20:44+0000#28607#10#monitor#added: blackhole 43.134.194.179 
2022-02-11T16:27:24+0000#28607#10#monitor#added: blackhole 43.153.21.119 
2022-02-11T18:31:58+0000#28607#10#monitor#added: blackhole 43.155.67.205 
2022-02-11T23:39:30+0000#28607#10#monitor#added: blackhole 43.131.91.178 
2022-02-11T23:40:06+0000#28607#8#monitor#added: blackhole 43.135.165.22 
2022-02-12T00:13:02+0000#28607#10#monitor#added: blackhole 43.155.60.208 
2022-02-12T02:37:13+0000#28607#10#monitor#added: blackhole 43.155.72.11 
2022-02-12T10:22:39+0000#28607#10#monitor#added: blackhole 43.134.224.138 
2022-02-12T16:48:40+0000#28607#10#monitor#added: blackhole 43.135.160.246 
2022-02-12T19:54:41+0000#28607#10#monitor#added: blackhole 43.255.30.192 
2022-02-13T00:01:43+0000#28607#10#monitor#added: blackhole 43.133.201.165 
2022-02-13T00:04:39+0000#28607#10#monitor#added: blackhole 43.130.227.235 
2022-02-13T00:44:30+0000#28607#10#monitor#added: blackhole 43.155.111.188 
2022-02-13T06:44:47+0000#28607#10#monitor#added: blackhole 43.153.30.8 
2022-02-13T07:02:02+0000#28607#10#monitor#added: blackhole 43.133.188.254 
2022-02-13T12:01:22+0000#28607#10#monitor#added: blackhole 43.155.102.117 
2022-02-13T12:14:03+0000#28607#10#monitor#added: blackhole 43.135.155.97 
2022-02-13T12:47:18+0000#28607#10#monitor#added: blackhole 43.135.166.27 
2022-02-14T06:33:23+0000#28607#10#monitor#added: blackhole 43.156.42.20 
2022-02-14T08:06:00+0000#28607#10#monitor#added: blackhole 43.135.166.170 
2022-02-14T15:30:30+0000#28607#6#monitor#added: blackhole 43.155.95.49 
2022-02-14T20:47:00+0000#28607#3#monitor#added: blackhole 43.134.195.243 
2022-02-14T23:17:31+0000#28607#10#monitor#added: blackhole 43.156.47.186 
2022-02-14T23:55:52+0000#28607#3#monitor#added: blackhole 43.156.42.69 
2022-02-15T01:44:05+0000#28607#3#monitor#added: blackhole 43.135.160.150 
2022-02-15T02:11:46+0000#28607#10#monitor#added: blackhole 43.134.29.71 
2022-02-15T03:33:29+0000#28607#10#monitor#added: blackhole 43.254.158.205 
2022-02-15T04:41:58+0000#28607#10#monitor#added: blackhole 43.130.40.251 
2022-02-15T04:51:19+0000#28607#3#monitor#added: blackhole 43.152.201.119 
2022-02-15T05:44:47+0000#28607#10#monitor#added: blackhole 43.134.201.159 
2022-02-15T06:44:34+0000#28607#10#monitor#added: blackhole 43.155.74.159 
2022-02-15T06:46:28+0000#28607#10#monitor#added: blackhole 43.155.105.151 
2022-02-15T08:43:06+0000#28607#10#monitor#added: blackhole 43.155.63.228 
2022-02-15T09:09:32+0000#28607#10#monitor#added: blackhole 43.155.65.87 
2022-02-15T09:58:32+0000#28607#10#monitor#added: blackhole 43.155.89.111 
2022-02-15T12:01:44+0000#28607#10#monitor#added: blackhole 43.152.197.68 
2022-02-15T13:56:03+0000#28607#10#monitor#added: blackhole 43.134.189.8 
2022-02-15T14:08:38+0000#28607#10#monitor#added: blackhole 43.156.46.58 
2022-02-15T20:30:07+0000#28607#8#monitor#added: blackhole 43.155.69.61 
2022-02-15T22:45:19+0000#28607#10#monitor#added: blackhole 43.254.158.247 
2022-02-15T23:19:56+0000#28607#10#monitor#added: blackhole 43.135.155.141 
2022-02-15T23:44:58+0000#28607#10#monitor#added: blackhole 43.156.46.175 
2022-02-16T01:01:08+0000#28607#10#monitor#added: blackhole 43.132.247.122 
2022-02-16T01:03:34+0000#28607#10#monitor#added: blackhole 43.153.33.202 
2022-02-16T01:07:39+0000#28607#10#monitor#added: blackhole 43.153.1.208 
2022-02-16T02:19:23+0000#28607#10#monitor#added: blackhole 43.130.45.216 
2022-02-16T02:39:19+0000#28607#10#monitor#added: blackhole 43.132.253.248 
2022-02-16T03:41:07+0000#28607#10#monitor#added: blackhole 43.155.115.206 
2022-02-16T03:48:23+0000#28607#5#monitor#added: blackhole 43.155.86.60 
2022-02-16T06:44:27+0000#28607#9#monitor#added: blackhole 43.134.201.127 
2022-02-16T07:24:34+0000#28607#10#monitor#added: blackhole 43.135.160.40 
2022-02-16T08:48:07+0000#28607#10#monitor#added: blackhole 43.155.74.56 
2022-02-16T17:50:14+0000#28607#10#monitor#added: blackhole 43.155.118.222 
2022-02-17T00:04:16+0000#28607#10#monitor#added: blackhole 43.153.31.162 
2022-02-17T07:17:57+0000#28607#10#monitor#added: blackhole 43.135.155.245 
2022-02-17T09:05:59+0000#28607#10#monitor#added: blackhole 43.255.29.95 
2022-02-17T10:45:12+0000#28607#10#monitor#added: blackhole 43.131.196.209 
2022-02-17T17:49:47+0000#28607#10#monitor#added: blackhole 43.155.84.158 
2022-02-17T19:28:04+0000#28607#10#monitor#added: blackhole 43.132.147.235 
2022-02-18T03:55:32+0000#28607#10#monitor#added: blackhole 43.155.92.145 
2022-02-18T06:22:16+0000#28607#10#monitor#added: blackhole 43.153.5.123 
2022-02-18T07:39:09+0000#28607#10#monitor#added: blackhole 43.153.9.186 
2022-02-18T10:35:26+0000#28607#10#monitor#added: blackhole 43.135.157.13

( date -I seconds | PID | sleep # | sshd Gerka | ip route add blackhole * )

This is only for the current month. Even at 2/10 thats still 24 seperate attempts to crack root user (easy to tick off from a password list), and we are only halfway through the month. These sorts of corodinated, but distributed attacks are NOT seen by Automated Analytics or Firewall Protection Algorythims.

Its these sorts of attempts that is the reason why I posted the original thread Automated Network Threat Response with an upstream component, because I can guarentee that my server is not the only one getting hit inside the network where this server is physically (location) and logically (network) located. This is the current failure of End Point Service Providers Security Protection, whereas most other attempts can be blocked or protected against based on Maths, Time, Repitition, Patterns, or known block-lists.

These attempts are only a pattern AFTER a “Network Whois” has been done on each individual IP Address (which is quite time consuming and resource intensive).

The other (more common) pattern of this type is to not even have the IP Address either look the similar, or come from the same AS-ORIGIN’s. Then only the timing and username attempts can join together those data points, as they are often much more condensed in attack form.

As a Service provider at their source, both of these “attack forms” are easier to see, because there is a master controller they are coordinating with, so there are bursts of traffic coming and going to the same external IP Address, something that analytics could easily flag as “potential abuse” at least (and often happens already, to a certain degree) - but obviously not in this case.

Anyway, after adding the new range block , I’ll also add an over-range block, probably an /8 or /9 (depending on who else gets allocated a 43.* range - I already had another 105 blocked previously before analysing these ones).

1 Like
20

UPDATE:
after further analysis I would say that, currently, everything (98%) coming into sshd is from the same controller based on the logs (13th - 18th). Normally you see multiple entries for the same username when the attempts are not from the same controllers, OR different usernames and multiple root attempts from the same IP Address.

That would put the last 60-odd entries (listed above) mingled across 491 sshd entries for the same period. Here is the analysis of the usernames, so you can see what I mean, most are 1, and there are 272 unique usernames there (after root is removed):

      1 123456
      1 3
      1 ADMIN
      1 Guest
      1 abc
      1 adm
     11 admin
      1 ahmed
      1 alvaro
      1 amg
      1 anaconda
      1 anup
      2 api
      1 appadmin
      1 apple
      1 appltest
      1 apps
      1 arjun
      2 ark
      1 asu
      1 auditoria
      1 backend
      1 backup
      1 backupuser
      1 ball
      1 beatriz
      1 bianca
      1 bitrix
      2 bot
      1 bounce
      1 bran
      1 bruno
      1 business
      1 caja01
      1 carlos
      1 ccs
      1 christian
      1 client1
      1 cod4server
      1 coder
      1 csb
      1 culture
      1 daniel
      1 danilo
      1 danny
      1 data
      1 debian
      1 deploy
      1 deploy2
      1 deployer
      1 deployment
      1 deployop
      1 dev
      1 developer
      1 devteam
      1 devuser
      1 dieter
      1 dk
      1 dms
      2 docker
      1 docs
      1 dp
      1 ds
      1 ec2-user
      1 elasticsearch
      1 element
      1 elias
      1 els
      1 emil
      1 emily
      2 erika
      1 es
      1 felix
      1 felomina
      1 finance
      1 frederick
      1 ftpadmin
      2 ftpuser
      1 g
      1 gaurav
      1 gay
      1 gerencia
      1 getmail
      1 github
      1 gnats
      1 gp
      2 gpadmin
      1 guest
      1 guest2
      1 hadi
      1 hana
      1 haoyu
      1 hh
      1 house
      1 icinga
      1 infowarelab
      1 invent
      1 invoices
      1 irina
      1 itadmin
      1 ivan
      1 jasmin
      1 java
      1 jefferson
      1 jenkins
      1 jesse
      3 john
      1 johnny
      1 join
      1 jordi
      1 jt
      1 julia
      1 julian
      1 kfserver
      1 krishna
      1 ldm
      1 lehrer
      1 leo
      1 leticia
      1 lg
      1 libuuid
      1 linaro
      1 lisa
      1 local
      1 lxy
      1 magento
      1 mailserver
      1 mami
      1 manu
      1 manuel
      1 mario
      1 matlab
      3 mc
      1 mh
      1 michelle
      1 minecraft
      1 mingyang
      3 mob
      1 mobil
      1 mongod
      1 mos
      1 musicbot
      2 musikbot
      1 myftp
      1 mysql
      1 nagios
      1 ncs
      1 netflow
      1 noc
      1 nq
      2 odoo
      1 ok
      1 online
      1 opc
      1 open
      1 opu
      9 oracle
      1 orauat
      1 osm
      1 ov
      1 patrick
      1 payroll
      1 phpmyadmin
      1 portal
      4 postgres
      1 prakash
      1 prof
      1 project
      1 q3server
      1 qce
      1 qq
      1 quentin
      1 rachid
      1 ramesh
      1 rd
      1 reboot
      1 rex
      1 rik
      1 rob
      1 rolland
      1 romeo
      1 ronald
      1 samba1
      1 samir
      1 sdtdserver
      3 server
      2 sg
      1 shijie
      1 shiva
      1 silvia
      1 simon
      1 sk
      1 slave
      1 sms
      2 solr
      1 sonar
      2 sonarqube
      1 soporte
      1 splunk
      3 student
      1 student1
      1 student3
      1 suo
      1 super
      1 suporte
      3 support
      1 teacher
      2 teamspeak3
      1 temp
      1 template
      1 tes
      8 test
      3 test1
      1 test10
      1 test123
      1 testbed
      1 testdev
      1 tester
      1 testftp
      1 testmail
      3 testuser
      1 tiago
      1 todds
      1 tomas
      1 tommy
      1 toro
      1 trial
      1 ts
      1 ts3
      1 ttf
      1 tuser
      1 tv
      1 twang
      1 ty
      8 ubuntu
      1 uftp
      1 uno50
      1 urbackup
      4 user
      1 user02
      1 user1
      1 user13
      1 user15
      1 user2
      2 user5
      1 username
      2 usuario
      1 util
      3 vbox
      1 vboxuser
      1 viewer
      2 vision
      1 vsftpd
      1 webconfig
      1 weblogic
      1 webs
      1 whmcs
      1 wordpress
      1 www-data
      1 wy
      1 x
      1 xbmc
      1 xm
      1 yan
      1 yoko
      1 ys
      1 yx
      2 zabbix
      1 zhang
      1 zhangyt
      1 zhu
      1 zoom

FWIW I got that information from this command line, where messages.sshd contains pre-culled sshd log entries, and 146 root attempts (including 16 in previous post) are “grepped out” of the incoming results:

grep -v root ../logs/messages.sshd | cut -d \: -f 4 | cut -d \  -f 7 | sort | uniq -c > ../43.usernames.txt

messages.sshd is generated every 3 hours from the messages & messages.0 log files (the shortest time I have recorded for the log rotation on this server - every 200Kb), before a cron job uses it to check for anything that the Gerka might have missed, as that only scans for duplicate IPv4’s in the last 10-20 lines of the actual log file (which has a bunch of cruft in it as well).

result based on previous posts log output:

  • 1st automated range block adding 21 new 43.* ranges (sweet)
  • and over-range block is 43.128.0.0/9 (bye bye Tencent’s qcloud.net)