As stable as all our Linux servers are, when problems come up they certainly are something…
I have a Debian Webserver that’s been chugging along nicely. Yesterday, problems started to appear. First our Nagios Server wasn’t able to monitor it anymore. The NRPE couldn’t find it’s plugins allthough they worked locally.
Now, i’m unable to connect through SSH from two machines (on internal, one external). I don’t care for NRPE, but ssh is a problem. I’m able to work on the VM through Console.
The Problem is rather non-descriptive. My Client just tells me:
Connection to 10.101.0.100 closed by remote host.
Connection to 10.101.0.100 closed.
On the Server, in auth.log, i’m getting this:
So far, i’ve looked at hosts.allow and hosts.deny, both are empty. Allowing sshd for all doesn’t change anything.
sshd is configured to not allow root Login, accept Public Keys (which i’m using) and i temporarily enabled Password Auth to try it, without success. There is no fail2ban, no firewall (firewall is sophos hardware and didn’t change) on the server, no selinux (debian machine) etc.
I’m completely lost. All i could google up where some Problems with openssl and openssh a few months back, but those should be fixed and i have newer versions than where those fixes where added.
Additional Information:
OS: Debian bullseye (don’t ask me why this thing is running testing. No clue. Maybe downgrade to stable?)
OpenSSL Version: 1.1.1d
OpenSSH Version: 7.9p1
Feel free to ask for any further information. I’m willing to test anything to get this working. I can snapshot that thing and try stuff out. The website is still up and working, so there’s no influence on produciton so far.
Worst case, i can restore from 2 days ago, but i’m not sure if this will help long term. I’d rather find the Problem and fix it.
Edit 1:
SInce i’m working on this live, i’ll add information as it comes up.
I’m now trying to downgrade to stable through pinning to that release.
Not sure if this applies to me, but here is an issue with openssl 1.1.1d that seems to be related. No clue if debian backported a fix or if this is specific to certain openssh versions. I think downgrading should roll this back to an earlier version, so we’ll see.
Makes me wonder what happens, when Debian bullseye becomes stable and if this Problem would persist.
