Automating SSH with RSA Keys: The authentication keeps prompting for a password after offering keys. Why?

The systems are both CentOS 7 installations. I’m trying to configure key based SSH Authentication. This is what I get:

# ssh [email protected] -v
OpenSSH_7.4p1, OpenSSL 1.0.2k-fips  26 Jan 2017
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 58: Applying options for *
debug1: Connecting to target.domain.com [10.0.5.40] port 22.
debug1: Connection established.
debug1: identity file /home/user/.ssh/id_rsa type 1
debug1: key_load_public: No such file or directory
debug1: identity file /home/user/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/user/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/user/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/user/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/user/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/user/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/user/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.4
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.4
debug1: match: OpenSSH_7.4 pat OpenSSH* compat 0x04000000
debug1: Authenticating to target.domain.com:22 as 'user'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: [email protected] MAC: <implicit> compression: none
debug1: kex: client->server cipher: [email protected] MAC: <implicit> compression: none
debug1: kex: curve25519-sha256 need=64 dh_need=64
debug1: kex: curve25519-sha256 need=64 dh_need=64
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:6h5hwXDIaE4rHPqZESx+bsoLFFMfowZB8vQ4pAaVG6o
debug1: Host 'target.domain.com' is known and matches the ECDSA host key.
debug1: Found key in /home/user/.ssh/known_hosts:1
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /home/user/.ssh/id_rsa
debug1: Authentications that can continue: publickey,password
debug1: Trying private key: /home/user/.ssh/id_dsa
debug1: Trying private key: /home/user/.ssh/id_ecdsa
debug1: Trying private key: /home/user/.ssh/id_ed25519
debug1: Next authentication method: password
[email protected]'s password:

Checking target.domain.com:

# cat /home/user/.ssh/authorized_keys
ssh-rsa <correct-rsa-key> [email protected]

This was placed by doing:

ssh-copy-id -i ~/.ssh/id_rsa.pub [email protected]

Which succeeded. The target’s logs:

# cat /var/log/messages | grep ssh
# journalctl _COMM=sshd > sshd_logs
# cat sshd_logs
Oct 01 08:39:06 target.domain.com sshd[32108]: Connection closed by 10.0.5.88 port 48274 [preauth]
Oct 01 08:47:33 target.domain.com sshd[32361]: Connection closed by 10.0.5.88 port 48424 [preauth]
Oct 01 08:52:21 target.domain.com sshd[40466]: Received signal 15; terminating.
Oct 01 08:52:21 target.domain.com sshd[32515]: Server listening on 10.0.5.73 port 22.
Oct 01 08:54:38 target.domain.com sshd[32575]: Connection closed by 10.0.5.88 port 48518 [preauth]

Note: Nothing is returned by cat /var/log/messages | grep ssh.

I’m at a roadblock because nothing says why the key isn’t accepted or used. It’s offered, but authentication doesn’t happen? wat

check permissions on the client side / server side

.ssh folder should be 700
auth_keys file in folder .ssh should be 600
known hosts and public key can be 644

2 Likes
# ls -lha /home/user/.ssh
total 16K
drwx------. 2 user user   80 Oct  1 09:14 .
drwxr-xr-x. 3 user user   39 Sep 27 13:48 ..
-rw-------. 1 user user  409 Oct  1 09:14 authorized_keys
-rw-------. 1 user user 1.8K Oct  1 09:13 id_rsa
-rw-r--r--. 1 user user  408 Oct  1 09:13 id_rsa.pub
-rw-r--r--. 1 user user  195 Sep 27 13:49 known_hosts

Turns out it was the SELinux type being off. It wasn’t ssh_home_t on the files inside the directory. This fixed the issue:

semanage fcontext -a -t ssh_home_t /home/user/.ssh
semanage fcontext -a -t ssh_home_t /home/user/.ssh/id_rsa
semanage fcontext -a -t ssh_home_t /home/user/.ssh/id_rsa.pub
semanage fcontext -a -t ssh_home_t /home/user/.ssh/authorized_keys
semanage fcontext -a -t ssh_home_t /home/user/.ssh/known_hosts
restorecon -v /home/user/.ssh
2 Likes

Restorecon usually fixes that. The context night be lost depending on how you put the file there. Same for files in the pki folder (vpn configs etc.)

In my experience, SELinux causes all* the problems. Does it actually do anything other than break things?

“*” Ok, maybe not literally all of them, but IIRC actually literally all of the enterprise hardware/software I’ve used had “turn off SELinux” somewhere in their “getting started” doc.

Very bad practice IMO
SELinux can save you if you have it working.

What does it actually do, though, that the network firewall won’t protect you from anyway?

I’m any case, I’d tried leaving it on when I first started working at my old job (I quit it a couple months ago), but nothing worked and my boss wanted me to follow the directions for everything. Next time I go back there (he’s still got a couple power strips and a USB hub of mine), I’ll mention it to him, though.

read the link I posted above.

1 Like

Ah, ok, I’d somehow missed the link. Thanks!

Yeah, on systems that use selinux, I almost always have to fix contexts after messing with the ssh keys.


And yeah, SELinux is a requirement for any reasonably hardened system in the enterprise.

Anyone who thinks disabling SELinux is acceptable would never get past the first interview with me.

2 Likes

blacklisted. if such a thing existed…

depends on level of experience, but basically that.

SElinux is a tremendous pain in the ass. We turn it off everywhere we’re allowed to. Elsewhere it’s just a long-ass slog fixing our apps to run properly with it enforcing.

It’s not that much of a pain once you summit the cliff of a learning curve that it has.

Spend a week working with it and you’ll be able to solve almost any issue within 20 minutes.

1 Like

Custom app with dozens of various microservices, running on a client’s custom component/container management system that nobody else ever heard of. Yeah, it ain’t 20 minutes.

Hire someone who knows what they are doing then is my suggestion. It isnt hard if you know what your doing with it.

1 Like

Sounds like you’ve got dozens of issues then. 20 minutes times dozens.

More importantly, you’re sacrificing security for convenience. Learn to properly deploy your applications and you won’t have these problems.

Yeah, I’ll pass along your criticism to R&D. That’ll help.

1 Like

Any time hon.

1 Like