TLS tests fail on newer versions of macOS

[kraih]

• Mojolicious version: 8.26
• Perl version: v5.30.0
• Operating system: macOS 10.15.1

Steps to reproduce the behavior

Run tests with TEST_TLS=1 and OpenSSL 1.1.1d from Homebrew.

Expected behavior

t/mojo/tls.t .......................... ok

Actual behavior

#   Failed test 'AES256-SHA has been negotiatied'
#   at t/mojo/tls.t line 55.
#          got: 'TLS_AES_256_GCM_SHA384'
#     expected: 'AES256-SHA'

#   Failed test 'AES256-SHA has been negotiatied'
#   at t/mojo/tls.t line 57.
#          got: 'TLS_AES_256_GCM_SHA384'
#     expected: 'AES256-SHA'
# Looks like you failed 2 tests of 6.
t/mojo/tls.t ............................... Dubious, test returned 2 (wstat 512, 0x200)

[rage311]

I don't know whether this is helpful or not, but the tls.t tests pass on OpenBSD 6.6 (-current) with LibreSSL 3.0.2.

% uname -rsvp
OpenBSD 6.6 GENERIC.MP#424 amd64

% openssl version
LibreSSL 3.0.2

~/dev/mojo % TEST_TLS=1 prove -l t/mojo/tls.t
t/mojo/tls.t .. ok   
All tests successful.
Files=1, Tests=6,  0 wallclock secs ( 0.04 usr  0.01 sys +  0.56 cusr  0.06 csys =  0.67 CPU)
Result: PASS

[tianon]

It seems odd that it's affecting OpenSSL from Homebrew, but could it be similar to what's described in FiloSottile/mkcert#174 (comment) ? (Changes to maximum certificate validity in newer macOS)

[kraih]

I just installed LibreSSL 2.9.2 with Homebrew, and all tests still pass with it. Looks like it's only OpenSSL.

[marked]

• Ubuntu 19.10
• OpenSSL 1.1.1c 28 May 2019
• Perl (v5.28.1, linux)
• Mojolicious (8.32, Supervillain)

$ export TEST_TLS=1
$ prove tls.t
tls.t .. 1/?
#   Failed test 'AES256-SHA has been negotiatied'
#   at tls.t line 55.
#          got: 'TLS_AES_256_GCM_SHA384'
#     expected: 'AES256-SHA'

#   Failed test 'AES256-SHA has been negotiatied'
#   at tls.t line 57.
#          got: 'TLS_AES_256_GCM_SHA384'
#     expected: 'AES256-SHA'
# Looks like you failed 2 tests of 6.
tls.t .. Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/6 subtests

Test Summary Report
-------------------
tls.t (Wstat: 512 Tests: 6 Failed: 2)
  Failed tests:  4, 6
  Non-zero exit status: 2
Files=1, Tests=6,  0 wallclock secs ( 0.02 usr  0.01 sys +  0.25 cusr  0.01 csys =  0.29 CPU)
Result: FAIL

[marked]

What you're seeing is OpenSSL 1.1.1 added support for TLSv1.3 while I'm guessing LibreSSL only goes up to TLSv1.2. The new protocol version has a different/disjoint set of ciphers.

The docs mentions v1.2 a bunch.

$ perldoc Mojo::IOLoop::TLS
...
        tls_version => 'TLSv1_2'

Nonetheless, no particular version is specified in mojo/lib/Mojo/IOLoop/TLS.pm
Setting this somewhere will enable the test to pass. Opening a PR demonstration.

[kraih]

There is another test that fails.

===(    6593;3  15/?  25/?  27/?  1/?  15/?  14/?  148/?  0/? )=========
#   Failed test 'AES256-SHA has been negotiatied'
#   at t/mojo/ioloop_tls.t line 341.
#          got: 'TLS_AES_256_GCM_SHA384'
#     expected: 'AES256-SHA'

#   Failed test 'right result'
#   at t/mojo/ioloop_tls.t line 363.
#          got: undef
#     expected: 'accepted'
# Looks like you failed 2 tests of 37.
t/mojo/ioloop_tls.t ........................ Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/37 subtests

[kraih]

This should be resolved now. 42c3e07...9353f52