std: available_parallelism using native netbsd api first #112226
Conversation
|
r? @cuviper (rustbot has picked a reviewer for you, use r? to override) |
rustbot
added
S-waiting-on-review
library/std/src/sys/unix/thread.rs
Outdated
| #[cfg(target_os = "netbsd")] | ||
| { | ||
| unsafe { | ||
| let set = libc::_cpuset_create(); | ||
| if !set.is_null() { | ||
| let mut count: usize = 0; | ||
| if libc::pthread_getaffinity_np(libc::pthread_self(), libc::_cpuset_size(set), set) == 0 { | ||
| #[cfg(target_pointer_width = "32")] | ||
| const MAXCPUS: u64 = 32; | ||
| #[cfg(target_pointer_width = "64")] | ||
| const MAXCPUS: u64 = 256; | ||
| for i in 0..MAXCPUS { | ||
| match libc::_cpuset_isset(i, set) { | ||
| -1 => break, | ||
| 0 => continue, | ||
| _ => count = count + 1, | ||
| } | ||
| } | ||
| } | ||
| libc::_cpuset_destroy(set); | ||
| if count > 0 { | ||
| return Ok(NonZeroUsize::new_unchecked(count)); | ||
| } | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
Since this doesn't need to affect the rest of the code until it hits a final return, can the majority of this block be moved out-of-line into a Rust function so the in-line code is something like if let Some(x) = netbsd_available_parallelism() { return Ok(x) };?
There was a problem hiding this comment.
I don't know, maybe that's actually harder to read/follow? The platform grouping just feels odd with two mutually-exclusive cfg blocks inside what is already a cfg-if branch. What do you think?
There was a problem hiding this comment.
since these two blocks are not gonna be reusable pieces, would not make much sense to make them as separate calls.
There was a problem hiding this comment.
Well, there's plenty of other called-once fn in std, they're just usually used as an abstraction boundary (so you don't have to look at both if only one is relevant). I'm not married to that idea, though, I'm mostly just wondering because it is getting... well, has been for some time a bit hard to read std::sys in general, so I'm wondering what directions might be good for improvement.
There was a problem hiding this comment.
The linux cgroup stuff has been extracted into its own module for example.
library/std/src/sys/unix/thread.rs
Outdated
| const MAXCPUS: u64 = 32; | ||
| #[cfg(target_pointer_width = "64")] | ||
| const MAXCPUS: u64 = 256; | ||
| for i in 0..MAXCPUS { |
There was a problem hiding this comment.
I'm wary of hard-coding MAXCPUS. Would it be sufficient to just count unbounded (or to u64::MAX) and let -1 => break do its job?
library/std/src/sys/unix/thread.rs
Outdated
| if count > 0 { | ||
| return Ok(NonZeroUsize::new_unchecked(count)); |
There was a problem hiding this comment.
| if count > 0 { | |
| return Ok(NonZeroUsize::new_unchecked(count)); | |
| if let Some(count) = NonZeroUsize::new(count) { | |
| return Ok(count); |
before falling back to existing code paths like FreeBSD does.
|
Thanks! @bors r+ |
bors
added
S-waiting-on-bors
…mpiler-errors Rollup of 7 pull requests Successful merges: - rust-lang#111074 (Relax implicit `T: Sized` bounds on `BufReader<T>`, `BufWriter<T>` and `LineWriter<T>`) - rust-lang#112226 (std: available_parallelism using native netbsd api first) - rust-lang#112474 (Support 128-bit enum variant in debuginfo codegen) - rust-lang#112662 (`#[lang_item]` for `core::ptr::Unique`) - rust-lang#112665 (Make assumption functions in new solver take `Binder<'tcx, Clause<'tcx>>`) - rust-lang#112684 (Disable alignment checks on i686-pc-windows-msvc) - rust-lang#112706 (Add `SyntaxContext::is_root`) r? `@ghost` `@rustbot` modify labels: rollup
…ism". First off, I have it on good authority this code is Just Wrong: by default a given thread does not have affinity to any specific set of CPUs. This particular change came with this pull request: rust-lang#112226 However, even worse, this code causes a segmentation fault for certain NetBSD target architectures in the "bootstrap" program when building rust natively on those platforms. So far armv7/9.0, powerpc/10.0_BETA, and i386/9.3 all crash with a segmentation fault. However, for some strange reason, this isn't consistent across the board: riscv64/current, amd64/10.0_BETA, aarch64/9.0 and sparc64/10.0_BETA all pass this hurdle. A trivial C reimplementation also doesn't crash on any of these systems, ref. the thread which starts at https://mail-index.netbsd.org/current-users/2023/10/10/msg044510.html but also always prints 0. However, if we get a SEGV running this code, the entire build fails, of course. So ... while I do not have a full explanation for the SEGVs, this undoes the addition from pull request 112226, and restores the ability to build rust natively on the above flagged-as-problematical platforms.
before falling back to existing code paths like FreeBSD does.