./x.py build --on-fail=/bin/bash has no effect

[devurandom]

Trying to look into #43982, I was running ./x.py build --jobs=4 --on-fail=/bin/bash. Once I hit the ICE, the build exited without dropping me to a shell as I had expected. @nagisa wanted to try and repro this.

re-uploaded build log as a gist:
https://gist.github.com/Mark-Simulacrum/f660624e0195d733da752649a6cf7538

[Mark-Simulacrum]

I can reproduce, but so far have not been able to determine the cause.

[nagisa]

It appears as if cargo(?) has been changed recently to:

1. Redirect stdin to /dev/null;
2. Pipe stdout to some internal buffer which is never printed

thus breaking the hack that this feature was relying on to work.

[nagisa]

Not sure if we want to fix this feature or just reduce it to a plain --on-fail=print-env which would print you the environment variables to use when invoking the failed rustc invocation manually.

[Mark-Simulacrum]

I would prefer that we fix this feature... but that seems somewhat complicated, based on your conclusions. I'd be okay with --on-fail=print-cmd. We could just always do that, though, since it never seems wrong...

[devurandom]

If I may wish for something, that would be for an easy and convenient way to debug ICE's like #43982. I would not necessarily need to be dropped to a shell when something fails -- capturing the debug output of rustc would be a much better option for me.

[mati865]

Triage: the issue still persists.

[jyn514]

triage: the behavior here has changed recently:

#!/bin/sh
// stdin_empty.sh

use_real_rustc=0

for flag in "$@"; do
	case $flag in
		-v*|--print=file-names)
			use_real_rustc=1;
			break
			;;
		*) ;;
	esac
done

if [ $use_real_rustc = 1 ]; then
	exec rustc +$RUSTUP_TOOLCHAIN "$@"
fi

timeout .1 bash -c 'read -r x'
status=$?
if [ $status = 124 ]; then
	# stdin not closed (we timed out)
	exit 0
else
	# stdin closed
	exit 1
fi

$ RUSTC=./stdin_empty.sh cargo +1.22 check
warning: unused manifest key: package.edition
   Compiling inner v0.1.0 (file:///home/jnelson/rust-lang/test-rust)
    Finished dev [unoptimized + debuginfo] target(s) in 0.10 secs
$ RUSTC=./stdin_empty.sh cargo +stable check
    Checking inner v0.1.0 (/home/jnelson/rust-lang/test-rust)
error: could not compile `inner`
$ RUSTC=./stdin_empty.sh cargo +beta check
    Checking inner v0.1.0 (/home/jnelson/rust-lang/test-rust)
    Finished dev [unoptimized + debuginfo] target(s) in 0.16s

I tried --on-fail=bash with #98673, but it behaves very strangely:

Did not run successfully: exit status: 1oc                              
"/home/jnelson/rust-lang/rust2/build/x86_64-unknown-linux-gnu/stage0/bin/rustc" "--crate-name" "alloc" "--edition=2021" "library/alloc/src/lib.rs" "--error-format=json" "--json=diagnostic-rendered-ansi,artifacts,future-incompat" "--crate-type" "lib" "--emit=dep-info,metadata" "-C" "opt-level=3" "-C" "embed-bitcode=no" "-C" "debuginfo=0" "-C" "metadata=96c2fa262ac0ad06" "-C" "extra-filename=-96c2fa262ac0ad06" "--out-dir" "/home/jnelson/rust-lang/rust2/build/x86_64-unknown-linux-gnu/stage0-std/x86_64-unknown-linux-gnu/release/deps" "--target" "x86_64-unknown-linux-gnu" "-C" "incremental=/home/jnelson/rust-lang/rust2/build/x86_64-unknown-linux-gnu/stage0-std/x86_64-unknown-linux-gnu/release/incremental" "-L" "dependency=/home/jnelson/rust-lang/rust2/build/x86_64-unknown-linux-gnu/stage0-std/x86_64-unknown-linux-gnu/release/deps" "-L" "dependency=/home/jnelson/rust-lang/rust2/build/x86_64-unknown-linux-gnu/stage0-std/release/deps" "--extern" "compiler_builtins=/home/jnelson/rust-lang/rust2/build/x86_64-unknown-linux-gnu/stage0-std/x86_64-unknown-linux-gnu/release/deps/libcompiler_builtins-97dc3bf1f5390237.rmeta" "--extern" "core=/home/jnelson/rust-lang/rust2/build/x86_64-unknown-linux-gnu/stage0-std/x86_64-unknown-linux-gnu/release/deps/libcore-5670a3f7cd4c9169.rmeta" "--cfg=bootstrap" "-Csymbol-mangling-version=legacy" "-Zunstable-options" "-Zmacro-backtrace" "-Clink-args=-Wl,-z,origin" "-Clink-args=-Wl,-rpath,$ORIGIN/../lib" "-Zunstable-options" "-Csplit-debuginfo=off" "-Cprefer-dynamic" "-Cllvm-args=-import-instr-limit=10" "-Zcrate-attr=doc(html_root_url=\"https://doc.rust-lang.org/nightly/\")" "-Z" "binary-dep-depinfo" "-Wrust_2018_idioms" "-Wunused_lifetimes" "-Wsemicolon_in_expressions_from_macros" "-Dwarnings" "--sysroot" "/home/jnelson/rust-lang/rust2/build/x86_64-unknown-linux-gnu/stage0-sysroot" "-Z" "force-unstable-if-unmarked"
-------------
whoami


^Cbash-5.0$ 
    Building [==============>            ] 20/36: alloc  

I think cargo has done something to the terminal that breaks stdout from bash (maybe buffered output or something like that?). But this is definitely closer than in the previous release :)

[jyn514]

ok, this does work on beta:

$ cat interactive.sh
#!/bin/sh
/usr/bin/env bash --norc --noprofile -i "$@" </dev/tty >/dev/tty
$ x check --on-fail=$(realpath ./interactive.sh)

I'll see if I can find some way to make this less painful for people who don't want to mess with /dev/tty, etc.