Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[v8.x backport] Support both OpenSSL 1.1.0 and 1.0.2 #18622

Closed
wants to merge 25 commits into from

Conversation

MylesBorins
Copy link
Contributor

replacement for #18326

This is a backport of #16130 which includes a patch from the fedora backport.

Refs: https://src.fedoraproject.org/fork/zvetlik/rpms/nodejs/tree/openssl-110

@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. v8.x labels Feb 7, 2018
@MylesBorins MylesBorins changed the title Openssl 1.1.0 8.x [v8.x backport] Support both OpenSSL 1.1.0 and 1.0.2 Feb 7, 2018
@MylesBorins
Copy link
Contributor Author

CI: https://ci.nodejs.org/job/node-test-pull-request/13013/

TLDR; everything looks good, just need explicit sign off from someone, preferably a member of @nodejs/crypto as e4e0cf6 is a new commit

In OpenSSL 1.1.0, X509_STORE_CTX is opaque and thus cannot be
stack-allocated. This works in OpenSSL 1.1.0 and 1.0.2. Adapted from PR

PR-URL: nodejs#16130
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
This is cherry-picked from PR nodejs#8491 and then tidied up. The original had
an unnecessarily large diff and messed up some public/private bits.

PR-URL: nodejs#16130
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
Based on a build of OpenSSL 1.1.0f.

The exact sizes are not particularly important (the original value was
missing all the objects hanging off anyway), only that V8 garbage
collector be aware that there is some memory usage beyond the sockets
themselves.

PR-URL: nodejs#16130
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
These are OpenSSL-internal APIs that are no longer accessible in 1.1.0
and weren't necessary. OpenSSL will push its own errors and, if it
doesn't, the calling code would handle it anyway.

PR-URL: nodejs#16130
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
This is cherry-picked from PR nodejs#8491 and tidied up. This change does
*not* account for the larger ticket key in OpenSSL 1.1.0. That will be
done in a follow-up commit as the 48-byte ticket key is part of Node's
public API.

rvagg: removed BORINGSSL defines before landing

PR-URL: nodejs#16130
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
Add a regression test for openssl/openssl#4384.

PR-URL: nodejs#16130
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
Parts of this were cherry-picked from PR nodejs#8491. Note that this only
works with OpenSSL 1.0.2 or 1.1.0g or later. 1.1.0g is, as of writing,
not yet released, but the fix is on the branch. See
openssl/openssl#4384.

PR-URL: nodejs#16130
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
The callbacks are all no-ops in OpenSSL 1.1.0. This isn't necessary (the
functions still exist for compatibility), but silences some warnings and
avoids allocating a few unused mutexes.

PR-URL: nodejs#16130
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
In OpenSSL 1.1.0, EVP_CIPHER_CTX must be heap-allocated. Once we're
heap-allocating them, there's no need in a separate initialised_ bit.
The presence of ctx_ is sufficient.

PR-URL: nodejs#16130
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
OpenSSL 1.1.0 requires EVP_MD_CTX be heap-allocated.

PR-URL: nodejs#16130
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
1.1.0 requires EVP_MD_CTX be heap-allocated. In doing so, move the Init
and Update hooks to shared code because they are the same between Verify
and Sign.

PR-URL: nodejs#16130
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
OpenSSL 1.1.0 requries HMAC_CTX be heap-allocated.

PR-URL: nodejs#16130
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
In OpenSSL 1.1.0, EVP_dss1() is removed. These hash names were exposed
in Node's public API, so add compatibility hooks for them.

PR-URL: nodejs#16130
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
This aligns the documentation with reality. This API never did what Node
claims it did.

The SSL_CIPHER_get_version function just isn't useful. In OpenSSL 1.0.2,
it always returned the string "TLSv1/SSLv3" for anything but SSLv2
ciphers, which Node does not support. Note how test-tls-multi-pfx.js
claims that ECDHE-ECDSA-AES256-GCM-SHA384 was added in TLSv1/SSLv3 which
is not true. That cipher is new as of TLS 1.2. The OpenSSL 1.0.2
implementation is:

char *SSL_CIPHER_get_version(const SSL_CIPHER *c)
{
    int i;

    if (c == NULL)
        return ("(NONE)");
    i = (int)(c->id >> 24L);
    if (i == 3)
        return ("TLSv1/SSLv3");
    else if (i == 2)
        return ("SSLv2");
    else
        return ("unknown");
}

In OpenSSL 1.1.0, SSL_CIPHER_get_version changed to actually behave as
Node documented it, but this changes the semantics of the function and
breaks tests. The cipher's minimum protocol version is not a useful
notion to return to the caller here, so just hardcode the string at
"TLSv1/SSLv3" and document it as legacy.

PR-URL: nodejs#16130
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
Some errors in the two versions are different. The test-tls-no-sslv3 one
because OpenSSL 1.1.x finally does version negotiation properly. 1.0.x's
logic was somewhat weird and resulted in very inconsistent errors for
SSLv3 in particular.

Also the function codes are capitalized differently, but function codes
leak implementation details, so don't assert on them to begin with.

PR-URL: nodejs#16130
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
"sha" in OpenSSL refers to SHA-0 which was removed from OpenSSL 1.1.0
and is insecure. Replace it with SHA-256 which is present in both 1.0.2
and 1.1.0. Short of shipping a reimplementation in Node, this is an
unavoidable behavior change with 1.1.0.

PR-URL: nodejs#16130
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
OpenSSL 1.0.x used a 48-byte ticket key, but OpenSSL 1.1.x made it
larger by using a larger HMAC-SHA256 key and using AES-256-CBC to
encrypt. However, Node's public API exposes the 48-byte key. Implement
the ticket key callback to restore the OpenSSL 1.0.x behavior.

PR-URL: nodejs#16130
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
OpenSSL 1.1.0 rejects RSA keys smaller than 1024 bits by default. Fix
the tests to use larger ones. This test only cares that the PEM blob be
missing a trailing newline. Certificate adapted from
test/fixtures/cert.pem.

PR-URL: nodejs#16130
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
This test is testing what happens to the server if the client shuts off
the connection (so the server sees ECONNRESET), but the way it does it
is convoluted. It uses a static RSA key exchange with a tiny (384-bit)
RSA key. The server doesn't notice (since it is static RSA, the client
acts on the key first), so the client tries to encrypt a premaster and
fails:

  rsa routines:RSA_padding_add_PKCS1_type_2:data too large for key size
  SSL routines:ssl3_send_client_key_exchange:bad rsa encrypt

OpenSSL happens not to send an alert in this case, so we get ECONNRESET
with no alert. This is quite fragile and, notably, breaks in OpenSSL
1.1.0 now that small RSA keys are rejected by libssl. Instead, test by
just connecting a TCP socket and immediately closing it.

PR-URL: nodejs#16130
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
These are both no-ops in OpenSSL 1.1.0.

PR-URL: nodejs#16130
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
OpenSSL 1.1.0 disables anonymous ciphers unless building with
enable-weak-crypto. Avoid unnecessary dependencies on these ciphers in
tests.

PR-URL: nodejs#16130
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
This test is testing the workaround for an OpenSSL 1.0.x bug, which was
fixed in 1.1.0. With the bug fixed, the test expectations need to change
slightly.

PR-URL: nodejs#16130
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
This is kind of hairy. OpenSSL 1.0.2 ignored the return value and always
treated everything as SSL_TLSEXT_ERR_NOACK (so the comment was wrong and
Node was never sending a warning alert). OpenSSL 1.1.0 honors
SSL_TLSEXT_ERR_NOACK vs SSL_TLSEXT_ERR_FATAL_ALERT and treats everything
unknown as SSL_TLSEXT_ERR_FATAL_ALERT.

Since this is a behavior change (tests break too), start by aligning
everything on SSL_TLSEXT_ERR_NOACK. If sending no_application_protocol
is desirable in the future, this can by changed to
SSL_TLSEXT_ERR_FATAL_ALERT with whatever deprecation process is
appropriate.

However, note that, contrary to
https://rt.openssl.org/Ticket/Display.html?id=3463#txn-54498,
SSL_TLSEXT_ERR_FATAL_ALERT is *not* useful to a server with no fallback
protocol. Even if such mismatches were rejected, such a server must
*still* account for the fallback protocol case when the client does not
advertise ALPN at all. Thus this may not be worth bothering.

PR-URL: nodejs#16130
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
Fixing the rest will be rather involved. I think the cleanest option is
to deprecate the method string APIs which are weird to begin with.

PR-URL: nodejs#16130
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
@MylesBorins
Copy link
Contributor Author

@gibfahn I think this is good to land. I'll do so Tuesday if there are no objections

gibfahn pushed a commit that referenced this pull request Feb 18, 2018
In OpenSSL 1.1.0, X509_STORE_CTX is opaque and thus cannot be
stack-allocated. This works in OpenSSL 1.1.0 and 1.0.2. Adapted from PR

PR-URL: #16130
Backport-PR-URL: #18622
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
gibfahn pushed a commit that referenced this pull request Feb 18, 2018
This is cherry-picked from PR #8491 and then tidied up. The original had
an unnecessarily large diff and messed up some public/private bits.

PR-URL: #16130
Backport-PR-URL: #18622
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
gibfahn pushed a commit that referenced this pull request Feb 18, 2018
Based on a build of OpenSSL 1.1.0f.

The exact sizes are not particularly important (the original value was
missing all the objects hanging off anyway), only that V8 garbage
collector be aware that there is some memory usage beyond the sockets
themselves.

PR-URL: #16130
Backport-PR-URL: #18622
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
gibfahn pushed a commit that referenced this pull request Feb 18, 2018
These are OpenSSL-internal APIs that are no longer accessible in 1.1.0
and weren't necessary. OpenSSL will push its own errors and, if it
doesn't, the calling code would handle it anyway.

PR-URL: #16130
Backport-PR-URL: #18622
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
gibfahn pushed a commit that referenced this pull request Feb 18, 2018
This is cherry-picked from PR #8491 and tidied up. This change does
*not* account for the larger ticket key in OpenSSL 1.1.0. That will be
done in a follow-up commit as the 48-byte ticket key is part of Node's
public API.

rvagg: removed BORINGSSL defines before landing

PR-URL: #16130
Backport-PR-URL: #18622
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
gibfahn pushed a commit that referenced this pull request Feb 18, 2018
Add a regression test for openssl/openssl#4384.

PR-URL: #16130
Backport-PR-URL: #18622
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
gibfahn pushed a commit that referenced this pull request Feb 18, 2018
Parts of this were cherry-picked from PR #8491. Note that this only
works with OpenSSL 1.0.2 or 1.1.0g or later. 1.1.0g is, as of writing,
not yet released, but the fix is on the branch. See
openssl/openssl#4384.

PR-URL: #16130
Backport-PR-URL: #18622
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
gibfahn pushed a commit that referenced this pull request Feb 18, 2018
The callbacks are all no-ops in OpenSSL 1.1.0. This isn't necessary (the
functions still exist for compatibility), but silences some warnings and
avoids allocating a few unused mutexes.

PR-URL: #16130
Backport-PR-URL: #18622
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
gibfahn pushed a commit that referenced this pull request Feb 18, 2018
In OpenSSL 1.1.0, EVP_CIPHER_CTX must be heap-allocated. Once we're
heap-allocating them, there's no need in a separate initialised_ bit.
The presence of ctx_ is sufficient.

PR-URL: #16130
Backport-PR-URL: #18622
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
gibfahn pushed a commit that referenced this pull request Feb 18, 2018
OpenSSL 1.1.0 requires EVP_MD_CTX be heap-allocated.

PR-URL: #16130
Backport-PR-URL: #18622
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
gibfahn pushed a commit that referenced this pull request Feb 18, 2018
1.1.0 requires EVP_MD_CTX be heap-allocated. In doing so, move the Init
and Update hooks to shared code because they are the same between Verify
and Sign.

PR-URL: #16130
Backport-PR-URL: #18622
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
gibfahn pushed a commit that referenced this pull request Feb 18, 2018
OpenSSL 1.1.0 requries HMAC_CTX be heap-allocated.

PR-URL: #16130
Backport-PR-URL: #18622
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
gibfahn pushed a commit that referenced this pull request Feb 18, 2018
In OpenSSL 1.1.0, EVP_dss1() is removed. These hash names were exposed
in Node's public API, so add compatibility hooks for them.

PR-URL: #16130
Backport-PR-URL: #18622
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
gibfahn pushed a commit that referenced this pull request Feb 18, 2018
This aligns the documentation with reality. This API never did what Node
claims it did.

The SSL_CIPHER_get_version function just isn't useful. In OpenSSL 1.0.2,
it always returned the string "TLSv1/SSLv3" for anything but SSLv2
ciphers, which Node does not support. Note how test-tls-multi-pfx.js
claims that ECDHE-ECDSA-AES256-GCM-SHA384 was added in TLSv1/SSLv3 which
is not true. That cipher is new as of TLS 1.2. The OpenSSL 1.0.2
implementation is:

char *SSL_CIPHER_get_version(const SSL_CIPHER *c)
{
    int i;

    if (c == NULL)
        return ("(NONE)");
    i = (int)(c->id >> 24L);
    if (i == 3)
        return ("TLSv1/SSLv3");
    else if (i == 2)
        return ("SSLv2");
    else
        return ("unknown");
}

In OpenSSL 1.1.0, SSL_CIPHER_get_version changed to actually behave as
Node documented it, but this changes the semantics of the function and
breaks tests. The cipher's minimum protocol version is not a useful
notion to return to the caller here, so just hardcode the string at
"TLSv1/SSLv3" and document it as legacy.

PR-URL: #16130
Backport-PR-URL: #18622
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
gibfahn pushed a commit that referenced this pull request Feb 18, 2018
Some errors in the two versions are different. The test-tls-no-sslv3 one
because OpenSSL 1.1.x finally does version negotiation properly. 1.0.x's
logic was somewhat weird and resulted in very inconsistent errors for
SSLv3 in particular.

Also the function codes are capitalized differently, but function codes
leak implementation details, so don't assert on them to begin with.

PR-URL: #16130
Backport-PR-URL: #18622
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
gibfahn pushed a commit that referenced this pull request Feb 18, 2018
"sha" in OpenSSL refers to SHA-0 which was removed from OpenSSL 1.1.0
and is insecure. Replace it with SHA-256 which is present in both 1.0.2
and 1.1.0. Short of shipping a reimplementation in Node, this is an
unavoidable behavior change with 1.1.0.

PR-URL: #16130
Backport-PR-URL: #18622
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
gibfahn pushed a commit that referenced this pull request Feb 18, 2018
OpenSSL 1.0.x used a 48-byte ticket key, but OpenSSL 1.1.x made it
larger by using a larger HMAC-SHA256 key and using AES-256-CBC to
encrypt. However, Node's public API exposes the 48-byte key. Implement
the ticket key callback to restore the OpenSSL 1.0.x behavior.

PR-URL: #16130
Backport-PR-URL: #18622
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
gibfahn pushed a commit that referenced this pull request Feb 18, 2018
OpenSSL 1.1.0 rejects RSA keys smaller than 1024 bits by default. Fix
the tests to use larger ones. This test only cares that the PEM blob be
missing a trailing newline. Certificate adapted from
test/fixtures/cert.pem.

PR-URL: #16130
Backport-PR-URL: #18622
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
gibfahn pushed a commit that referenced this pull request Feb 18, 2018
This test is testing what happens to the server if the client shuts off
the connection (so the server sees ECONNRESET), but the way it does it
is convoluted. It uses a static RSA key exchange with a tiny (384-bit)
RSA key. The server doesn't notice (since it is static RSA, the client
acts on the key first), so the client tries to encrypt a premaster and
fails:

  rsa routines:RSA_padding_add_PKCS1_type_2:data too large for key size
  SSL routines:ssl3_send_client_key_exchange:bad rsa encrypt

OpenSSL happens not to send an alert in this case, so we get ECONNRESET
with no alert. This is quite fragile and, notably, breaks in OpenSSL
1.1.0 now that small RSA keys are rejected by libssl. Instead, test by
just connecting a TCP socket and immediately closing it.

PR-URL: #16130
Backport-PR-URL: #18622
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
gibfahn pushed a commit that referenced this pull request Feb 18, 2018
These are both no-ops in OpenSSL 1.1.0.

PR-URL: #16130
Backport-PR-URL: #18622
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
gibfahn pushed a commit that referenced this pull request Feb 18, 2018
OpenSSL 1.1.0 disables anonymous ciphers unless building with
enable-weak-crypto. Avoid unnecessary dependencies on these ciphers in
tests.

PR-URL: #16130
Backport-PR-URL: #18622
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
gibfahn pushed a commit that referenced this pull request Feb 18, 2018
This test is testing the workaround for an OpenSSL 1.0.x bug, which was
fixed in 1.1.0. With the bug fixed, the test expectations need to change
slightly.

PR-URL: #16130
Backport-PR-URL: #18622
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
gibfahn pushed a commit that referenced this pull request Feb 18, 2018
This is kind of hairy. OpenSSL 1.0.2 ignored the return value and always
treated everything as SSL_TLSEXT_ERR_NOACK (so the comment was wrong and
Node was never sending a warning alert). OpenSSL 1.1.0 honors
SSL_TLSEXT_ERR_NOACK vs SSL_TLSEXT_ERR_FATAL_ALERT and treats everything
unknown as SSL_TLSEXT_ERR_FATAL_ALERT.

Since this is a behavior change (tests break too), start by aligning
everything on SSL_TLSEXT_ERR_NOACK. If sending no_application_protocol
is desirable in the future, this can by changed to
SSL_TLSEXT_ERR_FATAL_ALERT with whatever deprecation process is
appropriate.

However, note that, contrary to
https://rt.openssl.org/Ticket/Display.html?id=3463#txn-54498,
SSL_TLSEXT_ERR_FATAL_ALERT is *not* useful to a server with no fallback
protocol. Even if such mismatches were rejected, such a server must
*still* account for the fallback protocol case when the client does not
advertise ALPN at all. Thus this may not be worth bothering.

PR-URL: #16130
Backport-PR-URL: #18622
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
gibfahn pushed a commit that referenced this pull request Feb 18, 2018
Fixing the rest will be rather involved. I think the cleanest option is
to deprecate the method string APIs which are weird to begin with.

PR-URL: #16130
Backport-PR-URL: #18622
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
@gibfahn
Copy link
Member

gibfahn commented Feb 18, 2018

Landed in 56401a4...072902a

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants