Use available_parallelism instead of num_cpus
#10427
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
`std::thread::available_parallelism` has been stabilized since 1.59.0. Also, we don't want to block timing data output, so if parallelism data is not available the table will display `ncpu=n/a` instead.
|
r? @ehuss (rust-highfive has picked a reviewer for you, use r? to override) |
rust-highfive
added
the
S-waiting-on-review
alexcrichton
approved these changes
Feb 28, 2022
| let jobs = jobs.or(cfg.jobs).unwrap_or(::num_cpus::get() as u32); | ||
| let jobs = match jobs.or(cfg.jobs) { | ||
| Some(j) => j, | ||
| None => available_parallelism()?.get() as u32, |
There was a problem hiding this comment.
Could the error-propagation here have some context indicating what the error is? (e.g. something like "failed to determine the amount of parallelism available")
|
I get the impression that |
|
Thanks for the review. I think @ehuss is right. Ref: |
|
@ehuss @weihanglo |
|
@bors r+ |
|
📌 Commit 4f706ae has been approved by |
bors
added
S-waiting-on-bors
|
☀️ Test successful - checks-actions |
Dylan-DPC
added a commit
to Dylan-DPC/rust
that referenced
this pull request
Mar 9, 2022
Update cargo 11 commits in 3d6970d50e30e797b8e26b2b9b1bdf92dc381f34..65c82664263feddc5fe2d424be0993c28d46377a 2022-02-28 19:29:07 +0000 to 2022-03-09 02:32:56 +0000 - Remove remaining 2 warn(clippy::*) instances (rust-lang/cargo#10438) - Use `available_parallelism` instead of `num_cpus` (rust-lang/cargo#10427) - Wait up to one second while waiting for curl (rust-lang/cargo#10456) - Improve code coverage (rust-lang/cargo#10460) - Don't recommend leaking tokens into the console history (rust-lang/cargo#10458) - fix some typos (rust-lang/cargo#10454) - Use `extend` instead of `push`ing in a loop (rust-lang/cargo#10453) - Use locked_version more (rust-lang/cargo#10449) - Disable dependabot (rust-lang/cargo#10443) - Update git2 dependencies (rust-lang/cargo#10442) - Stop gating stable features (rust-lang/cargo#10434)
Dylan-DPC
added a commit
to Dylan-DPC/rust
that referenced
this pull request
Mar 9, 2022
Update cargo 11 commits in 3d6970d50e30e797b8e26b2b9b1bdf92dc381f34..65c82664263feddc5fe2d424be0993c28d46377a 2022-02-28 19:29:07 +0000 to 2022-03-09 02:32:56 +0000 - Remove remaining 2 warn(clippy::*) instances (rust-lang/cargo#10438) - Use `available_parallelism` instead of `num_cpus` (rust-lang/cargo#10427) - Wait up to one second while waiting for curl (rust-lang/cargo#10456) - Improve code coverage (rust-lang/cargo#10460) - Don't recommend leaking tokens into the console history (rust-lang/cargo#10458) - fix some typos (rust-lang/cargo#10454) - Use `extend` instead of `push`ing in a loop (rust-lang/cargo#10453) - Use locked_version more (rust-lang/cargo#10449) - Disable dependabot (rust-lang/cargo#10443) - Update git2 dependencies (rust-lang/cargo#10442) - Stop gating stable features (rust-lang/cargo#10434)
bors
added a commit
that referenced
this pull request
Jun 8, 2022
[beta] Revert #10427: switch from num_cpus This temporarily reverts #10427 (Use available_parallelism instead of num_cpus) per the discussion at rust-lang/rust#97549. `available_parallelism` does not handle cgroups v1 on Linux unlike num_cpus. I am concerned that this potentially affects a significant percentage of users. For example, Docker just added cgroups v2 support last year. Various Linux distributions have only recently switched to it as the default. The following is what I can find on the web: * Fedora (since 31) * Arch Linux (since April 2021) * openSUSE Tumbleweed (since c. 2021) * Debian GNU/Linux (since 11) * Ubuntu (since 21.10) * RHEL and RHEL-like distributions (since 9) This also appears to affect CircleCI. The consequence is that Cargo ends up using too much parallelism and can run out of memory. I'm not sure what to do about 1.63. If std adds support for cgroups v1, then I don't think there is anything to do there. Otherwise I think we should revert similarly if that doesn't happen.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Status: blocked on rust-lang/rust#92697
Just realized that we can drop one dependency 🎉
Also, we don't want to block timing data output, so if parallelism data is not available, the table will display
ncpu=n/ainstead.