Packer fails to connect to a machine image when public key algorithm "ssh-rsa" is deprecated (OpenSSH 8.2)

[dbilling]

Overview of the Issue

OpenSSH 8.2 announced they are deprecating the ssh key algorithm "ssh-rsa" because it uses a sha-1 hash. sha-1 is has been proven weak. Soon folks will be using such a version of OpenSSH and linux distros will be incorporating said version, and packer will be not be compatible with those linux distributions.

Please note that OpenSSH is not deprecating RSA keys. those will still work. You just can't use the "ssh-rsa" sha-1 based key algorithm to verify possession of the private key.

See OpenSSH 8.2 posting here: https://www.openssh.com/txt/release-8.2

I attempted to have packer connect to such a machine where I intentionally removed the "ssh-rsa" public key algorithm by configuing a line such as the following in /etc/ssh/ssh_config on my CentOS machine (note the line does not include "ssh-rsa":

PubkeyAcceptedKeyTypes=rsa-sha2-256,[email protected],ecdsa-sha2-nistp256,[email protected],ecdsa-sha2-nistp384,[email protected],rsa-sha2-512,[email protected],ecdsa-sha2-nistp521,[email protected]

Note that ssh-rsa is not included in the PubkeyAcceptedKeyTypes, so I am simulating what the world will look like when OpenSSH decommissions this algorithm

Reproduction Steps

I recreated in AWS with builder amazon-ebs builder by creating a source AMI that has the sshd config line shown above.

Packer version

From packer version
1.5.5

Simplified Packer Buildfile

Any build file that connects via ssh should exhibit this issue when the destination starting image has "ssh-rsa" disabled

Operating system and Environment details

I ran packer 1.5.5 on macOS 10.13.6 high sierra to recreate the failure. Note that if I ssh directly to the a VM launched using the very same AMI from the very SAME mac, the SSH connection works fine. This shows the problem must be packer's internal use of Golang SSH and their supported SSH algorithms that is the issue here.

Log Fragments and crash.log files

You have recreated the issue if you see errors similar to this when PACKER_LOG=1 is on:
2020/04/03 12:34:47 packer-builder-amazon-ebs plugin: Using host value: 52.204.31.86
2020/04/03 12:34:47 packer-builder-amazon-ebs plugin: [INFO] Attempting SSH connection to 52.204.31.86:22...
2020/04/03 12:34:47 packer-builder-amazon-ebs plugin: [DEBUG] reconnecting to TCP connection for SSH
2020/04/03 12:34:47 packer-builder-amazon-ebs plugin: [DEBUG] handshaking with SSH
2020/04/03 12:34:50 packer-builder-amazon-ebs plugin: [DEBUG] SSH handshake err: ssh: handshake failed: ssh: unable to authenticate, attempted methods [publickey none], no supported methods remain
2020/04/03 12:34:50 packer-builder-amazon-ebs plugin: [DEBUG] Detected authentication error. Increasing handshake attempts.

I also believe this is the same issue that is causing: #8609 but I wrote this one up for better description and clarity of the underlying issue and its importance.

[SwampDragons]

Just to comment on this explicitly -- I agree with this diagnosis and have confirmed it via code; I'm communicating largely on the other issue just for coherence, but this writeup was very helpful to me. So thanks :)

[SwampDragons]

Already posted this update on #8609 but sharing here:

I can't fix this inside of Packer without re-implementing a huge portion of the golang crypto/ssh library, which I don't want to do. However, I've opened a patch to try to fix this upstream: https://go-review.googlesource.com/c/crypto/+/240717

I'm not sure how long it will take to get merged. In the meantime, I'll talk with our security team about the implications of maintaining our own fork of the crypto library in order to un-block the Packer users who are affected by this.

I'm going to close this issue since it is a duplicate of #8609 and I want to track all the work in one place. Thanks for opening though -- your writeup made it much easier for me to conceptualize the issue.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.