Threads started in exit handler are still running after their thread states are destroyed #104690
Comments
chgnrdv
added
the
type-crash
|
I am not sure if this is a bug. The thread cleanup happens before Lines 1761 to 1777 in d1732fe So you need to clean up those import atexit
import sys
import threading
import gc
def t0():
pass
def t1():
t = threading.Thread(target=t0)
t.start()
t.join()
def f():
t = threading.Thread(target=t1)
t.start()
t.join()
atexit.register(f)
exit() |
|
CC @vstinner, who might be an expert here |
|
Surely, it's an edge case, and in real Python code these threads (whose creation at interpreter exit also looks strange to me) should be manually joined. But I found this worth reporting anyway :) |
|
Looks like the same issue was spotted earlier in #103824 as reproducible with |
|
See Python/pylifecycle.c: // Block some operations.
tstate->interp->finalizing = 1;
// Wrap up existing "threading"-module-created, non-daemon threads.
wait_for_thread_shutdown(tstate);
// Make any remaining pending calls.
_Py_FinishPendingCalls(tstate);
/* The interpreter is still entirely intact at this point, and the
* exit funcs may be relying on that. In particular, if some thread
* or exit func is still waiting to do an import, the import machinery
* expects Py_IsInitialized() to return true. So don't say the
* runtime is uninitialized until after the exit funcs have run.
* Note that Threading.py uses an exit func to do a join on all the
* threads created thru it, so this also protects pending imports in
* the threads created via Threading.
*/
_PyAtExit_Call(tstate->interp);It's too late to spawn new threads. Currently, there is a race condition is there is a pending pthread_create() call while |
|
IMO we should also disallow spawning new processes at this point: os.fork(), subprocess.Popen, etc. |
I'd exclude subprocess.Popen from that - its child process isn't using the Python interpreter anymore. (barring |
gpshead
changed the title
gpshead
changed the title
…on (#104826) Disallow thread creation and fork at interpreter finalization. in the following functions, check if interpreter is finalizing and raise `RuntimeError` with appropriate message: * `_thread.start_new_thread` and thus `threading` * `posix.fork` * `posix.fork1` * `posix.forkpty` * `_posixsubprocess.fork_exec` when a `preexec_fn=` is supplied. --------- Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com> Co-authored-by: Gregory P. Smith <[email protected]>
…lization (pythonGH-104826) Disallow thread creation and fork at interpreter finalization. in the following functions, check if interpreter is finalizing and raise `RuntimeError` with appropriate message: * `_thread.start_new_thread` and thus `threading` * `posix.fork` * `posix.fork1` * `posix.forkpty` * `_posixsubprocess.fork_exec` when a `preexec_fn=` is supplied. --------- (cherry picked from commit ce558e6) Co-authored-by: chgnrdv <[email protected]> Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com> Co-authored-by: Gregory P. Smith <[email protected]>
…alization (GH-104826) (#105277) gh-104690 Disallow thread creation and fork at interpreter finalization (GH-104826) Disallow thread creation and fork at interpreter finalization. in the following functions, check if interpreter is finalizing and raise `RuntimeError` with appropriate message: * `_thread.start_new_thread` and thus `threading` * `posix.fork` * `posix.fork1` * `posix.forkpty` * `_posixsubprocess.fork_exec` when a `preexec_fn=` is supplied. --------- (cherry picked from commit ce558e6) Co-authored-by: chgnrdv <[email protected]> Co-authored-by: Gregory P. Smith <[email protected]>
|
Thanks. IMO this change will help to keep internals consistent. Spawning process and thread while Python is being finalized is fragile and can lead to crashes. |
thread_run() of _threadmodule.c now calls _PyThreadState_CheckConsistency() to check if tstate is a dangling pointer. Rename ceval_gil.c is_tstate_valid() to _PyThreadState_CheckConsistency() to reuse it in _threadmodule.c.
thread_run() of _threadmodule.c now calls _PyThreadState_CheckConsistency() to check if tstate is a dangling pointer when Python is built in debug mode. Rename ceval_gil.c is_tstate_valid() to _PyThreadState_CheckConsistency() to reuse it in _threadmodule.c.
…hon#109056) thread_run() of _threadmodule.c now calls _PyThreadState_CheckConsistency() to check if tstate is a dangling pointer when Python is built in debug mode. Rename ceval_gil.c is_tstate_valid() to _PyThreadState_CheckConsistency() to reuse it in _threadmodule.c. (cherry picked from commit f63d378)
…hon#109056) thread_run() of _threadmodule.c now calls _PyThreadState_CheckConsistency() to check if tstate is a dangling pointer when Python is built in debug mode. Rename ceval_gil.c is_tstate_valid() to _PyThreadState_CheckConsistency() to reuse it in _threadmodule.c. (cherry picked from commit f63d378)
…09056) (#109134) gh-104690: thread_run() checks for tstate dangling pointer (#109056) thread_run() of _threadmodule.c now calls _PyThreadState_CheckConsistency() to check if tstate is a dangling pointer when Python is built in debug mode. Rename ceval_gil.c is_tstate_valid() to _PyThreadState_CheckConsistency() to reuse it in _threadmodule.c. (cherry picked from commit f63d378)
…09056) (#109133) gh-104690: thread_run() checks for tstate dangling pointer (#109056) thread_run() of _threadmodule.c now calls _PyThreadState_CheckConsistency() to check if tstate is a dangling pointer when Python is built in debug mode. Rename ceval_gil.c is_tstate_valid() to _PyThreadState_CheckConsistency() to reuse it in _threadmodule.c. (cherry picked from commit f63d378)
Reproduced on main (663c049), bisected to 283ab0e.
Output (can also pass without crash or segv depending on at which point
t0tries to access itststate):Stack traces:
Looks like some race condition between main thread and threads started and still running at finalization, whose
tstatemain thread is trying to delete, although I'm not an expert and didn't make further debugging.Platform:
Linux 5.10.0-21-amd64 #1 SMP Debian 5.10.162-1 (2023-01-21) x86_64 GNU/LinuxLinked PRs
The text was updated successfully, but these errors were encountered: