bpo-39360: Ensure all workers exit when finalizing a multiprocessing Pool #19009
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pablogsal
reviewed
Mar 15, 2020
Misc/NEWS.d/next/Library/2020-03-15-05-41-05.bpo-39360.cmcU5p.rst
Outdated
Show resolved
Hide resolved
pablogsal
reviewed
Mar 15, 2020
pablogsal
reviewed
Mar 15, 2020
Co-Authored-By: Pablo Galindo <[email protected]>
pablogsal
approved these changes
Mar 15, 2020
isidentical
force-pushed
the
bpo-39360
branch
5 times, most recently
from
b1a94ba to
13aa61f
Compare
|
Thanks @isidentical for the PR, and @pablogsal for merging it 🌮🎉.. I'm working now to backport this PR to: 3.8. |
miss-islington
pushed a commit
to miss-islington/cpython
that referenced
this pull request
Mar 15, 2020
…Pool (pythonGH-19009) When the pull is not used via the context manager or terminate() is called, there is a system in multiprocessing.util that handles finalization of all pools via an atexit handler (the Finalize) class. This class registers the _terminate_pool handler in the registry of finalizers of the module, and that registry is called on interpreter exit via _exit_function. The problem is that the "happy" path with the context manager or manual call to finalize() does some extra steps that _terminate_pool does not. The step that is not executed when the atexit() handler calls _terminate_pool is pinging the _change_notifier queue to unblock the maintenance threads. This commit moves the notification to the _terminate_pool function so is called from both code paths. Co-authored-by: Pablo Galindo <[email protected]> (cherry picked from commit ac10e0c) Co-authored-by: Batuhan Taşkaya <[email protected]>
|
GH-19023 is a backport of this pull request to the 3.8 branch. |
miss-islington
added a commit
that referenced
this pull request
Mar 15, 2020
…Pool (GH-19009) When the pull is not used via the context manager or terminate() is called, there is a system in multiprocessing.util that handles finalization of all pools via an atexit handler (the Finalize) class. This class registers the _terminate_pool handler in the registry of finalizers of the module, and that registry is called on interpreter exit via _exit_function. The problem is that the "happy" path with the context manager or manual call to finalize() does some extra steps that _terminate_pool does not. The step that is not executed when the atexit() handler calls _terminate_pool is pinging the _change_notifier queue to unblock the maintenance threads. This commit moves the notification to the _terminate_pool function so is called from both code paths. Co-authored-by: Pablo Galindo <[email protected]> (cherry picked from commit ac10e0c) Co-authored-by: Batuhan Taşkaya <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
https://bugs.python.org/issue39360