From 7e4955e5027f5cc333884690070e57cbb2618378 Mon Sep 17 00:00:00 2001
From: Miguel Ojeda <ojeda@kernel.org>
Date: Sat, 24 Feb 2024 20:55:23 +0100
Subject: [PATCH] rustc: document the jobserver

Explicitly document that the jobserver may be used by `rustc` and show
the warning to increase the chances that this document is found when
searching for solutions online.

In particular, add a section about the interaction with build systems,
which is intended to contain recommendations on how to integrate `rustc`
with different built systems.

For GNU Make, recommend using the `+` indicator. In addition, add a
note about the issue with GNU Make 4.3 since it is important that users
realize they should do this even if they do not expect parallelism from
`rustc`.  Finally, show how to workaround the issue of `$(shell ...)`
calls in recursive Make (which e.g. was needed for the Linux kernel).

The GNU Make 4.4 case under `--jobserver-style=pipe` is not added since
it got fixed after Rust 1.76.0 already (i.e. `rustc` will not warn if
it finds the negative file descriptors).

For CMake, recommend using `JOB_SERVER_AWARE` and show a workaround using
`$(MAKE)` for earlier versions (when using the Makefile generator).

From: https://github.com/rust-lang/rust/issues/120515
Cc: @petrochenkov @belovdv @weihanglo @bjorn3
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
---
 src/doc/rustc/src/SUMMARY.md   |  1 +
 src/doc/rustc/src/jobserver.md | 86 ++++++++++++++++++++++++++++++++++
 2 files changed, 87 insertions(+)
 create mode 100644 src/doc/rustc/src/jobserver.md

diff --git a/src/doc/rustc/src/SUMMARY.md b/src/doc/rustc/src/SUMMARY.md
index 1f9307203bc3c..345cf61469135 100644
--- a/src/doc/rustc/src/SUMMARY.md
+++ b/src/doc/rustc/src/SUMMARY.md
@@ -3,6 +3,7 @@
 - [What is rustc?](what-is-rustc.md)
 - [Command-line Arguments](command-line-arguments.md)
     - [Codegen Options](codegen-options/index.md)
+- [Jobserver](jobserver.md)
 - [Lints](lints/index.md)
     - [Lint Levels](lints/levels.md)
     - [Lint Groups](lints/groups.md)
diff --git a/src/doc/rustc/src/jobserver.md b/src/doc/rustc/src/jobserver.md
new file mode 100644
index 0000000000000..da82b0df7c629
--- /dev/null
+++ b/src/doc/rustc/src/jobserver.md
@@ -0,0 +1,86 @@
+# Jobserver
+
+Internally, `rustc` may take advantage of parallelism. `rustc` will coordinate
+with the build system calling it if a [GNU Make jobserver] is passed in the
+`MAKEFLAGS` environment variable. Other flags may have an effect as well, such
+as [`CARGO_MAKEFLAGS`]. If a jobserver is not passed, then `rustc` will choose
+the number of jobs to use.
+
+Starting with Rust 1.76.0, `rustc` will warn if a jobserver appears to be
+available but is not accessible, e.g.:
+
+```console
+$ echo 'fn main() {}' | MAKEFLAGS=--jobserver-auth=3,4 rustc -
+warning: failed to connect to jobserver from environment variable `MAKEFLAGS="--jobserver-auth=3,4"`: cannot open file descriptor 3 from the jobserver environment variable value: Bad file descriptor (os error 9)
+  |
+  = note: the build environment is likely misconfigured
+```
+
+## Integration with build systems
+
+The following subsections contain recommendations on how to integrate `rustc`
+with build systems so that the jobserver is handled appropriately.
+
+### GNU Make
+
+When calling `rustc` from GNU Make, it is recommended that all `rustc`
+invocations are marked as recursive in the `Makefile` (by prefixing the command
+line with the `+` indicator), so that GNU Make enables the jobserver for them.
+For instance:
+
+<!-- ignore-tidy-tab -->
+
+```make
+x:
+	+@echo 'fn main() {}' | rustc -
+```
+
+In particular, GNU Make 4.3 (a widely used version as of 2024) passes a simple
+pipe jobserver in `MAKEFLAGS` even when it was not made available for the child
+process, which in turn means `rustc` will warn about it. For instance, if the
+`+` indicator is removed from the example above and GNU Make is called with e.g.
+`make -j2`, then the aforementioned warning will trigger.
+
+For calls to `rustc` inside `$(shell ...)` inside a recursive Make, one can
+disable the jobserver manually by clearing the `MAKEFLAGS` variable, e.g.:
+
+```make
+S := $(shell MAKEFLAGS= rustc --print sysroot)
+
+x:
+	@$(MAKE) y
+
+y:
+	@echo $(S)
+```
+
+### CMake
+
+CMake 3.28 supports the `JOB_SERVER_AWARE` option in its [`add_custom_target`]
+command, e.g.:
+
+```cmake
+cmake_minimum_required(VERSION 3.28)
+project(x)
+add_custom_target(x
+    JOB_SERVER_AWARE TRUE
+    COMMAND echo 'fn main() {}' | rustc -
+)
+```
+
+For earlier versions, when using CMake with the Makefile generator, one
+workaround is to have [`$(MAKE)`] somewhere in the command so that GNU Make
+treats it as a recursive Make call, e.g.:
+
+```cmake
+cmake_minimum_required(VERSION 3.22)
+project(x)
+add_custom_target(x
+    COMMAND DUMMY_VARIABLE=$(MAKE) echo 'fn main() {}' | rustc -
+)
+```
+
+[GNU Make jobserver]: https://www.gnu.org/software/make/manual/html_node/POSIX-Jobserver.html
+[`CARGO_MAKEFLAGS`]: https://doc.rust-lang.org/cargo/reference/environment-variables.html
+[`add_custom_target`]: https://cmake.org/cmake/help/latest/command/add_custom_target.html
+[`$(MAKE)`]: https://www.gnu.org/software/make/manual/html_node/MAKE-Variable.html