Tracking Issue for unix_sigpipe

[Enselic]

---

---

---

↓↓↓↓ Important ↓↓↓↓

The ui changed from an attribute to a compiler flag: #124480, so the below description is out of date. Someone (maybe me) should update the description.

↑↑↑↑ Important ↑↑↑↑

---

---

---

The feature gate for the issue is #![feature(unix_sigpipe)].
It enables a new fn main() attribute #[unix_sigpipe = "..."].

Usage

Any simple Rust program that writes a sizeable amount of data to stdout will panic if its output is limited via pipes.

fn main() {
    loop {
        println!("hello world");
    }
}

% ./main | head -n 1
hello world
thread 'main' panicked at 'failed printing to stdout: Broken pipe (os error 32)', library/std/src/io/stdio.rs:1016:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrac

To prevent panicking we can use the new attribute:

#![feature(unix_sigpipe)]

#[unix_sigpipe = "sig_dfl"]
fn main() {
    loop {
        println!("hello world");
    }
}

% ./main | head -n 1
hello world

More Info

Please refer to the unstable book section for more details. In short:

#[unix_sigpipe = "..."]	Behaviour
sig_ign	Set SIGPIPE handler to SIG_IGN before invoking fn main(). Default behaviour since 2014.
sig_dfl	Set SIGPIPE handler to SIG_DFL before invoking fn main().
inherit	Leave SIGPIPE handler untounched before entering fn main().

The problem with the current SIGPIPE code in libstd as well as several other aspects of this problem is discussed extensively at these places:

• Should Rust still ignore SIGPIPE by default? #62569
• Spurious "broken pipe" error messages, when used in typical UNIX shell pipelines #46016
• https://rust-lang.zulipchat.com/#narrow/stream/219381-t-libs/topic/Proposal.3A.20First.20step.20towards.20solving.20the.20SIGPIPE.20problem

Naming convention

The naming follows the convention used by #![windows_subsystem = "windows|console"] where the values "windows" and "console" have the same names as the actual linker flags: /SUBSYSTEM:WINDOWS and /SUBSYSTEM:CONSOLE.

The names sig_ign and sig_dfl comes from the signal handler names SIG_IGN and SIG_DFL.

Steps

• Implement the feature:
  • Support #[unix_sigpipe = "inherit|sig_dfl"] on fn main() to prevent ignoring SIGPIPE #97802
  • Fix build with #[unix_sigpipe = "..."] support in rustc miri#2532
  • Change process spawning to inherit the parent's signal mask by default #101077 (made sigpipe::DEFAULT distinct)
  • Migrate rustc_passes diagnostics #102110 (improved diagnostics)
  • Add checks for the signature of the start lang item #106092
  • Regression test println!() panic message on ErrorKind::BrokenPipe #108980
• Use the attribute in a broad set of non-test-case places to learn how it works in practice.
  • #[unix_sigpipe = "sig_dfl"]
    • rustc: Use unix_sigpipe instead of rustc_driver::set_sigpipe_handler #102587
    • rustdoc: Use unix_sigpipe instead of rustc_driver::set_sigpipe_handler #103495
    • https://github.com/moonrepo/espresso/blob/e3f429b01bfd9a0a8956f11b1bc9120084c42d3c/crates/cli/src/main.rs#L18
    • https://github.com/trinitronx/intro-to-rust-kvstore/blob/2c26260a837c33f193cf26cecf49279675c3a6a3/src/main.rs#L8
  • #[unix_sigpipe = "sig_ign"]
  • #[unix_sigpipe = "inherit"]
• Remove rustc_driver::set_sigpipe_handler()
  • Remove rustc_driver::set_sigpipe_handler() #103536
• Add a test for #[unix_sigpipe = "inherit"] that test that the disposition is actually inherited, rather than assuming SIG_DFL shall always be inherited.
• Final comment period (FCP)
  • Stabilize #[unix_sigpipe = "sig_dfl"] on fn main() #120832
• Stabilization PR
  • Stabilize #[unix_sigpipe = "sig_dfl"] on fn main() #120832

Unresolved Questions That Blocks Stabilisation

• How can we make it easy to put fn lang_start() in an external crate that can be compiled with stable?
• We should rename the attribute and attribute values to things that reflect what they do rather than how they do it.

• #[unix_sigpipe = "sig_dfl"]
  • None
• #[unix_sigpipe = "sig_ign"]
  • currently child processes get SIG_IGN, but arguably they should get SIG_DFL since that is what most programs assume, and we explicitly made it that way before.
    • Note that we can implement this without running code right before exec, see DRAFT: Use a noop SIGPIPE handler instead of SIG_IGN #121578 for the trick.
  • Should the current default be implemented with a noop signal handler?
• #[unix_sigpipe = "inherit"]
  • Is the name clear enough? Maybe rename to unchanged?

Unresolved Questions That Does Not Block Stabilisation

Because these questions can be resolved incrementally after stabilization.

• What is the long-term plan with regards to changing the default behaviour with regards to ignoring SIGPIPE, if we want to do it at all?
  • Should Rust still ignore SIGPIPE by default? #62569

Resolved Questions

• We don't want to change to a -Z unix_sigpipe flag instead, see https://rust-lang.zulipchat.com/#narrow/stream/219381-t-libs/topic/Proposal.3A.20First.20step.20towards.20solving.20the.20SIGPIPE.20problem/near/285499895, at least not initially.
• Should we only have the third sigpipe: u8 argument to fn lang_start() on Unix platform via cfg?
  Answer: No, this is not allowed, see top level comment in https://github.com/rust-lang/rust/blob/master/src/tools/tidy/src/pal.rs
• Should we stabilize sig_dfl or is inherit and sig_ign sufficient?
  Answer: There are noteworthy examples of real projects that has opted to use SIG_DFL to solve the BrokenPipe problem. Notably rustc itself. So if we don't stabilize sig_dfl, such projects can't make use of our new attribute. Therefore, we also need to stabilize sig_dfl.
• Should the attribute go on fn main() or on the top-level module (#![unix_sigpipe="..."])?
  Answer: It makes a lot of semantic sense to have the attribute on fn main(), because it is a way to configure what the Rust runtime should do before fn main() is invoked. For libraries, no entry point code that modifies SIGPIPE is generated, so allowing the attribute in these situations does not make much sense. See Change process spawning to inherit the parent's signal mask by default #101077 (comment) for small-scale discussion.
• Can we write the code in a way that allows lto to remove the _signal stub code completely? With a bool it works (see Support #[unix_sigpipe = "inherit|sig_dfl"] on fn main() to prevent ignoring SIGPIPE #97802 (comment)), but with the current u8 we might need to do some tweaks. Answer: There are currently 4 values and I see no feasible way to reduce it to 2.
• Can and should we alter the BrokenPipe error message and make it suggest to use the new attribute? Answer: No, because that would mean we would end up giving developer advice to users that can't act on the advice.
• Does this have any impact on defining a stable ABI? Answer: No, because ABI discussion are about enabling things such as calling Rust functions in binary artifacts produced by an older Rust compiler than the current one. That we changed the ABI of fn lang_start() is not relevant. And a stable Rust ABI is not even close (see Define a Rust ABI rfcs#600).
• Can we use MSG_NOSIGNAL with send() etc instead of setting SIGPIPE globally? Answer: No, because there is no equivalent for write(), and it would incur an extra syscall for each write-operation, which is likely to have significant performance drawbacks.

Disclaimer: I have taken the liberty to mark some questions resolved that I find unlikely to be controversial. If you would like me to create a proper discussion ticket for any of the resolved or unresolved questions, please let me know!

About tracking issues

Tracking issues are used to record the overall progress of implementation. They are also used as hubs connecting to other relevant issues, e.g., bugs or open design questions. A tracking issue is however not meant for large scale discussion, questions, or bug reports about a feature. Instead, open a dedicated issue for the specific matter and add the relevant feature gate label.

@rustbot label +T-libs

[jpmckinney]

For anyone else seeing this on nightly:

error: cannot find attribute `unix_sigpipe` in this scope
   --> src/main.rs:137:3
    |
137 | #[unix_sigpipe = "sig_dfl"]
    |   ^^^^^^^^^^^^

error[E0635]: unknown feature `unix_sigpipe`
 --> src/main.rs:1:12
  |
1 | #![feature(unix_sigpipe)]
  |            ^^^^^^^^^^^^

It seems like this feature is gone and is replaced with a compiler flag: #124480

Note that when doing a web search for these language features or compiler flags, it's not obvious which version of the Unstable Book you're looking at.

Nightly: https://doc.rust-lang.org/nightly/unstable-book/compiler-flags/on-broken-pipe.html
Beta: https://doc.rust-lang.org/beta/unstable-book/language-features/unix-sigpipe.html

[RalfJung]

Turns out making this a compiler flag is also not great, at least in the context of build systems like rustc bootstrap itself that build a ton of stuff. It's non-trivial to figure out when to set and when to not set this compiler flag, and a lot of time has been spend tweaking this logic in bootstrap and it's still wrong -- leading to a whole sequence of issues of the form "when I do A and then B, bootstrap rebuilds more than it should", often caused by some difference in RUSTFLAGS, such as this flag. It makes a lot more sense to annotate this in the program that wants the behavior (e.g. an attribute of the main function, or a std API invoked early in main) than in the build system that has to build tons of different programs that may want different values for this flag.

Also conceptually, this is a library thing, not a compiler thing, so having it be a compiler flag seems odd.

See here for more.

[Enselic]

I agree this is a library problem and not a compiler or language problem. The current solution I envision involves adding libraries named something like

libsigpipe_error-91d1108bf6191a61.rlib
libsigpipe_kill-4b832a03827ff95e.rlib

to the sysroot. Similar to how we already have

libpanic_abort-91d1108bf6191a61.rlib
libpanic_unwind-4b832a03827ff95e.rlib

So choosing how SIGPIPE will behave means choosing what library to use.

What I can't currently envision is how we could allow choosing what SIGPIPE library to use without introducing a new compiler flag like -Con-broken-pipe=..., similar to -Cpanic=... that already exist. But there might be a solution.

[RalfJung]

Why does it have to be a compile-time decision at all? Why can't we have std::os::unix::set_sigpipe_behavior(...) to be invoked early in main?

[Enselic]

Because SIGPIPE is setup before main is invoked. So there'd be no way to change SIGPIPE disposition 0 times. It would be changed two times. Which

• matters for seccomp filters
• does not live up up to Rust's standards as a systems programming langauge IMHO

Note that that workaround can already be used (and is already quite widely used), except the helper function is not provided by libstd.

[RalfJung]

Well at some point you pick a tradeoff. Personally, I don't think the complexity of a compiler switch is justified here. Maybe one day we will have a lang/compiler feature that can carry this, rather than needing a bunch of special support just for this feature. For instance, it seems to me this could be done with this "overrideable defauly fn" mechanism that is intended to generalize panic handlers and the global allocator. But the status quo is actively painful for rustc development, as noted above.

[jstarks]

Note that that workaround can already be used (and is already quite widely used), except the helper function is not provided by libstd.

No. There is no stable workaround today. The correct behavior for Unix tools that want to behave like Unix tools is to inherit the signal disposition from the parent process. Rust throws away the original disposition, and it cannot be recovered.

I still think a reasonable tradeoff is to save the original disposition and provide some way of restoring it via a function in early in main(). It's not perfect (there's a race window where an externally-injected SIGPIPE will be ignored), but it's a hell of a lot simpler than the perfect solutions. I don't think the seccomp issue is a real one; it seems perfectly reasonable to say that Rust programs must be able to call sigaction.

(I also think we should do a better job of steering users toward "inherit" in the docs. Right now, the docs recommend "kill" for programs producing textual output. This is not the default behavior for C (which is "inherit"), it is not a standard convention/recommendation for C programs (programs typically either explicitly ignore SIGPIPE or do nothing and so inherit the disposition), and it is not aligned with POSIX requirements for standard inbox tools (which requires that tools inherit the SIGPIPE disposition). I don't think Rust has any good reason to recommend against existing convention here.)

[bjorn3]

What about having libstd mask off SIGPIPE by default rather than disabling is entirely and then provide a function to restore the SIGPIPE mask to the original value? If there is a SIGPIPE injected before the restore function is called, it will still trigger once SIGPIPE is unmasked by the restore function. All tools that want to be compatible with the unix behavior have to do then is call this restore function at the top of main.

[sendittothenewts]

What about having libstd mask off SIGPIPE by default rather than disabling is entirely and then provide a function to restore the SIGPIPE mask to the original value? If there is a SIGPIPE injected before the restore function is called, it will still trigger once SIGPIPE is unmasked by the restore function. All tools that want to be compatible with the unix behavior have to do then is call this restore function at the top of main.

If the restorer is not called, then SIGPIPE would remain masked indefinitely, which would be strange. The mask would be inherited across fork and exec, thus exporting this strangeness to child processes, even non-Rust ones.

Also, any Rust program currently working around this issue by calling signal(SIGPIPE, SIG_DFL) would be re-broken by this.

[bjorn3]

The mask would be inherited across fork and exec, thus exporting this strangeness to child processes, even non-Rust ones.

That also happens with disabling the signal rather than masking it, right?

[sendittothenewts]

The mask would be inherited across fork and exec, thus exporting this strangeness to child processes, even non-Rust ones.

That also happens with disabling the signal rather than masking it, right?

Yes, I didn't mean to imply that signal dispositions are not inherited. Rather, by switching to masking SIGPIPE instead of setting its disposition to SIG_IGN, the child will still inherit the block, but now in an observably different and more unusual way, which may require the (potentially non-Rust) child to handle the situation differently. For example, using sigaction to install a custom signal handler or reset it to SIG_DFL would no longer work properly. And even if the child does go the extra mile and unmask the signal, it now risks getting killed by a SIGPIPE that was triggered by its parent and has already been duly handled as EPIPE.

I'd say that @jstarks has already put forward the correct, idiomatically POSIX solution in #97889 (comment)