-
-
Notifications
You must be signed in to change notification settings - Fork 90
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cherrypy SSL stops responding "ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:2471)" (after 5 intermediate tracebacks TLSV1_ALERT_UNKNOWN_CA or SSLV3_ALERT_CERTIFICATE_UNKNOWN) #346
Comments
Also from the same Radarr 3.0.0 (but now with different client approach): different intermediate traceback C:\mini-cherrypy>python mini-cherrypy-server.py
python version: 3.9.0 (tags/v3.9.0:9cf6752, Oct 5 2020, 15:34:40) [MSC v.1927 64 bit (AMD64)]
cherrpy version: 18.6.0
[08/Dec/2020:12:28:29] ENGINE Listening for SIGTERM.
[08/Dec/2020:12:28:29] ENGINE Bus STARTING
CherryPy Checker:
The Application mounted at '' has an empty config.
[08/Dec/2020:12:28:29] ENGINE Set handler for console events.
[08/Dec/2020:12:28:29] ENGINE Started monitor thread 'Autoreloader'.
[08/Dec/2020:12:28:29] ENGINE Serving on https://127.0.0.1:4400
[08/Dec/2020:12:28:29] ENGINE Bus STARTED
[08/Dec/2020:12:28:35] ENGINE socket.error 1
Traceback (most recent call last):
File "C:\Users\myname\AppData\Local\Programs\Python\Python39\lib\site-packages\cheroot\server.py", line 1273, in communicate
req.parse_request()
File "C:\Users\myname\AppData\Local\Programs\Python\Python39\lib\site-packages\cheroot\server.py", line 702, in parse_request
success = self.read_request_line()
File "C:\Users\myname\AppData\Local\Programs\Python\Python39\lib\site-packages\cheroot\server.py", line 743, in read_request_line
request_line = self.rfile.readline()
File "C:\Users\myname\AppData\Local\Programs\Python\Python39\lib\site-packages\cheroot\server.py", line 300, in readline
data = self.rfile.readline(256)
File "C:\Users\myname\AppData\Local\Programs\Python\Python39\lib\_pyio.py", line 559, in readline
b = self.read(nreadahead())
File "C:\Users\myname\AppData\Local\Programs\Python\Python39\lib\_pyio.py", line 538, in nreadahead
readahead = self.peek(1)
File "C:\Users\myname\AppData\Local\Programs\Python\Python39\lib\_pyio.py", line 1134, in peek
return self._peek_unlocked(size)
File "C:\Users\myname\AppData\Local\Programs\Python\Python39\lib\_pyio.py", line 1141, in _peek_unlocked
current = self.raw.read(to_read)
File "C:\Users\myname\AppData\Local\Programs\Python\Python39\lib\socket.py", line 704, in readinto
return self._sock.recv_into(b)
File "C:\Users\myname\AppData\Local\Programs\Python\Python39\lib\ssl.py", line 1241, in recv_into
return self.read(nbytes, buffer)
File "C:\Users\myname\AppData\Local\Programs\Python\Python39\lib\ssl.py", line 1099, in read
return self._sslobj.read(len, buffer)
ssl.SSLError: [SSL: SSLV3_ALERT_CERTIFICATE_UNKNOWN] sslv3 alert certificate unknown (_ssl.c:2621)
...
ssl.SSLError: [SSL: SSLV3_ALERT_CERTIFICATE_UNKNOWN] sslv3 alert certificate unknown (_ssl.c:2621)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\myname\AppData\Local\Programs\Python\Python39\lib\threading.py", line 950, in _bootstrap_inner
self.run()
File "C:\Users\myname\AppData\Local\Programs\Python\Python39\lib\site-packages\cheroot\workers\threadpool.py", line 120, in run
keep_conn_open = conn.communicate()
File "C:\Users\myname\AppData\Local\Programs\Python\Python39\lib\site-packages\cheroot\server.py", line 1302, in communicate
self._conditional_error(req, '500 Internal Server Error')
File "C:\Users\myname\AppData\Local\Programs\Python\Python39\lib\site-packages\cheroot\server.py", line 1345, in _conditional_error
req.simple_response(response)
File "C:\Users\myname\AppData\Local\Programs\Python\Python39\lib\site-packages\cheroot\server.py", line 1111, in simple_response
self.conn.wfile.write(EMPTY.join(buf))
File "C:\Users\myname\AppData\Local\Programs\Python\Python39\lib\site-packages\cheroot\makefile.py", line 438, in write
res = super().write(val, *args, **kwargs)
File "C:\Users\myname\AppData\Local\Programs\Python\Python39\lib\site-packages\cheroot\makefile.py", line 36, in write
self._flush_unlocked()
File "C:\Users\myname\AppData\Local\Programs\Python\Python39\lib\site-packages\cheroot\makefile.py", line 45, in _flush_unlocked
n = self.raw.write(bytes(self._write_buf))
File "C:\Users\myname\AppData\Local\Programs\Python\Python39\lib\socket.py", line 722, in write
return self._sock.send(b)
File "C:\Users\myname\AppData\Local\Programs\Python\Python39\lib\ssl.py", line 1173, in send
return self._sslobj.write(data)
ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:2471) |
It sounds like your CherryPy app uses a self-signed certificate that is not added to Radarr's chain of trust. And so when Radarr doesn't confirm the received certificate as trusted, it issues a TLS alert that CherryPy-side (stdlib SSL module really) receives and raises as an exception, then it attempts to process that exception at a higher level, attempts responding with HTTP 500 and gets another exception while attempting to write on a closed transport. @sanderjo @Safihre can you confirm that this is also happening under GNU/Linux? Can you confirm that it's a regression that doesn't happen before 8.4.5? I'm trying to understand if it's in any way related to the refactoring @liamstask has been working on lately. Also, side note: in our TLS tests, we use trusted certificates. It sounds like we need to add tests with the untrusted certs too. |
Sounds like an explanation!
So far: all on Linux, Radarr in a docker (
I'll try to install that version via pip (on Windows)
Yes, please! |
Cross-testing between Linux and Windows: Radarr (in docker) on Linux ... connecting to cheroot-mini-server on Windows ... No SSL errors! That is weird. That means Radarr-on-Linux behaves differently than Radarr-on-Windows? The other way around: Radarr on Windows ... connecting to cheroot-mini-server on Linux ... errors on Linux!
|
On Linux, in a env, I'm able to install cheroot 8.4.4. From Windows I connect to that Linux instance, and ... SSL errors. So 8.4.4 does not solve it. EDIT
|
Since upgrading from cheroot 8.4.6 to 8.5.1 we're also experiencing the same (or at least a very similar) issue both under Linux (SLES15 64bit) and Windows (10 64bit). We've encounterd both
and
We also noticed that Firefox (84.0.2 (64-Bit)) was more likely to display the page anyway, while accessing the page with Chrome ( 81.0.4044.92) would request a bunch of resources before the server would stop responding alltogether (w/o error) and just "hang" - no other requests (no matter from which browser) would be served after the sever hangs. Downgrading to 8.4.8 resolved the issue for now. We're using Python 3.7.5 64 bit and CherryPy 18.6.0 |
The hotfix released as 8.5.2 seems to fix the issue, at least AFAICT (using the code provided by sanderjo in this issue's first post as well as using the new fix in our application). While with version 8.5.1 FF would sometimes raise an error and Chrome would hang after 5 to10 requests, with 8.5.2 I can see neither errors nor does the application hang after over 100 requests. Cheers =) |
A DoS would happen in many situations, including TLS errors and attempts to close the underlying sockets erroring out. This patch aims to prevent a situation when the worker threads are killed by arbitrary exceptions that bubble up to their entry point layers that aren't handled properly or at all. PR #649 Fixes #358 Fixes #354 Ref #310 Ref #346 Ref #375 Ref #599 Ref #641 Resolves #365
Hi,
SABnzbd sub-author here, CC to @Safihre
I'm not sure if this is cheroot or cherrypy. The PoC program does a "import cherrypy", and the traceback is in cheroot, so I'm reporting this under cheroot.
❓ I'm submitting a ...
🐞 Describe the bug. What is the current behavior?
After 5 combined (failed) SSL/TLS connections from a certain SSL client (Radarr 3.0.0.4204, based on .NET Core), cherrypy stops responding completely with
ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:2471)
The program is still 'running'
Reproducable on Windows 10.
FWIW self-signed-certificate
❓ What is the motivation / use case for changing the behavior?
Cherrypy stops responding.
💡 To Reproduce
Start minimal cherrypy HTTPS server:
Let Radarr 3.0.0.4204 connect 5 times. Cherrypy spits out tracebacks, and after 5th attempt stops responding completely.
💡 Expected behavior
No tracebacks, keep responding.
Or, maybe as a workaround, cherrypy/cheroot can respawn itself (just like it does if it detects a new config)?
📋 Details
See below
📋 Environment
📋 Additional context
Full program output, including tracebacks
The text was updated successfully, but these errors were encountered: