Auto merge of rust-lang#114305 - lqd:bootstrap-strip, r=ozkanonur

Strip unexpected debuginfo from `libLLVM.so` and `librustc_driver.so` when not requesting any debuginfo

As seen in rust-lang#114175 and in [this zulip discussion](https://rust-lang.zulipchat.com/#narrow/stream/247081-t-compiler.2Fperformance/topic/Artifact.20sizes/near/379302655), there's still some small amount of debuginfo in LLVM's shared library on linux, even when not requesting it (nightly CI), coming from `libstdc++`.

```
$ readelf --debug-dump=info ~/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/libLLVM-16-rust-1.73.0-nightly.so | grep DW_TAG_compile_unit -A5 | grep DW_AT_comp_dir | cut -d ":" -f 2- | counts
101 counts
(  1)       39 (38.6%, 38.6%):  (indirect string, offset: 0x7): /tmp/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/libsupc++
(  2)       38 (37.6%, 76.2%):  (indirect string, offset: 0x43fb2): /tmp/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/src/c++11
(  3)       23 (22.8%, 99.0%):  (indirect string, offset: 0x18ed8): /tmp/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/src/c++98
(  4)        1 ( 1.0%,100.0%):  (indirect string, offset: 0x53f04): /tmp/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/src
```

Similarly, here's `librustc_driver.so` when not requesting debuginfo from either rustc or the tools (nightly CI), coming e.g. from our LLVM wrapper:
```
$ readelf --debug-dump=info ~/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-e534b3a316089f5f.so | grep DW_TAG_compile_unit -A5 | grep DW_AT_comp_dir | cut -d ":" -f 2- | counts
116 counts
(  1)       34 (29.3%, 29.3%):  (indirect string, offset: 0x3c11): /tmp/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/libsupc++
(  2)       32 (27.6%, 56.9%):  (indirect string, offset: 0x9753c): /tmp/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/src/c++11
(  3)       25 (21.6%, 78.4%):  (indirect string, offset: 0x393bd): /tmp/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/src/c++98
(  4)       23 (19.8%, 98.3%):  (indirect string, offset: 0x33ed3): /cargo/registry/src/index.crates.io-6f17d22bba15001f/compiler_builtins-0.1.98
(  5)        1 ( 0.9%, 99.1%):  (indirect string, offset: 0xaffff): /rustc/0d95f9132909ae7c5f2456748d0ffd1c3ba4a8e8
(  6)        1 ( 0.9%,100.0%):  (indirect string, offset: 0xb604a): /tmp/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/src
```

To reduce the size of distributed artifacts, this PR strips debuginfo from the LLVM and `rustc_driver` shared libraries, when:
- no debuginfo is requested when building LLVM: `link-shared` is true, `optimize` is true and `release-debuginfo` is false
- no debuginfo is requested when building the rustc driver:
  - `debuginfo-level-rustc` and `debuginfo-level-tools` are off.
  - when building with a stage != 0 compiler: since this is about the distributed artifacts, there's no need to do this at other stages.
- for both: on a x64 linux host and target where `strip -g` is available and fixes the issue (I don't know how to strip debuginfo from a `.dylib` on mac). The LLVM BOLTed .so, and `librustc_driver.so` are big there, and this will help a little. Other targets/hosts can be added in the future if we want to.

rust-lang#114175 did the same thing unconditionally in `opt-dist`, prior to BOLTing LLVM. But this should only be used in conjunction with the other config options mentioned above, and which `opt-dist` doesn't know about. Therefore, it makes more sense as in bootstrap when building LLVM and rustc when applicable and no debuginfo is requested.

This shouldn't interact badly with CI caching builds and artifacts, right?

---

From the other PR, `libLLVM-16-rust-1.73.0-nightly.so` prior to rust-lang#114141:
- master: 173.13 MiB
- stripped debuginfo: 165.12 MiB (-8 MiB, -4.6%)

`libLLVM-16-rust-1.73.0-nightly.so` after rust-lang#114141:
- master: 121.13 MiB
- stripped debuginfo: 113.12 MiB (still -8 MiB, -6.6%)

`librustc_driver.so`:
- master: 118.58 MiB
- stripped debuginfo: 106.46 MiB (-12 MiB, -10.2%)

(Results are also available in this most recent [perf run's artifact sizes](https://perf.rust-lang.org/compare.html?start=b321edd1b2d4bd00c7b4611e8f20a03ee7b77023&end=810ab570d5d27facb91806e5d9847815d9dac22a&stat=instructions%3Au&tab=artifact-size))

src/bootstrap/compile.rs

@@ -23,7 +23,7 @@ use crate::builder::crate_description;
 use crate::builder::Cargo;
 use crate::builder::{Builder, Kind, PathSet, RunConfig, ShouldRun, Step, TaskPath};
 use crate::cache::{Interned, INTERNER};
-use crate::config::{LlvmLibunwind, RustcLto, TargetSelection};
+use crate::config::{DebuginfoLevel, LlvmLibunwind, RustcLto, TargetSelection};
 use crate::dist;
 use crate::llvm;
 use crate::tool::SourceType;
@@ -888,16 +888,37 @@ impl Step for Rustc {
             compiler.host,
             target,
         );
+        let stamp = librustc_stamp(builder, compiler, target);
         run_cargo(
             builder,
             cargo,
             vec![],
-            &librustc_stamp(builder, compiler, target),
+            &stamp,
             vec![],
             false,
             true, // Only ship rustc_driver.so and .rmeta files, not all intermediate .rlib files.
         );
 
+        // When building `librustc_driver.so` (like `libLLVM.so`) on linux, it can contain
+        // unexpected debuginfo from dependencies, for example from the C++ standard library used in
+        // our LLVM wrapper. Unless we're explicitly requesting `librustc_driver` to be built with
+        // debuginfo (via the debuginfo level of the executables using it): strip this debuginfo
+        // away after the fact.
+        // FIXME: to make things simpler for now, limit this to the host and target where we know
+        // `strip -g` is both available and will fix the issue, i.e. on a x64 linux host that is not
+        // cross-compiling. Expand this to other appropriate targets in the future.
+        if builder.config.rust_debuginfo_level_rustc == DebuginfoLevel::None
+            && builder.config.rust_debuginfo_level_tools == DebuginfoLevel::None
+            && target == "x86_64-unknown-linux-gnu"
+            && target == builder.config.build
+        {
+            let target_root_dir = stamp.parent().unwrap();
+            let rustc_driver = target_root_dir.join("librustc_driver.so");
+            if rustc_driver.exists() {
+                output(Command::new("strip").arg("--strip-debug").arg(rustc_driver));
+            }
+        }
+
         builder.ensure(RustcLink::from_rustc(
             self,
             builder.compiler(compiler.stage, builder.config.build),

src/bootstrap/config.rs

@@ -50,7 +50,7 @@ pub enum DryRun {
     UserSelected,
 }
 
-#[derive(Copy, Clone, Default)]
+#[derive(Copy, Clone, Default, PartialEq, Eq)]
 pub enum DebuginfoLevel {
     #[default]
     None,

src/bootstrap/llvm.rs

@@ -485,25 +485,56 @@ impl Step for Llvm {
 
         cfg.build();
 
-        // When building LLVM with LLVM_LINK_LLVM_DYLIB for macOS, an unversioned
-        // libLLVM.dylib will be built. However, llvm-config will still look
-        // for a versioned path like libLLVM-14.dylib. Manually create a symbolic
-        // link to make llvm-config happy.
-        if builder.llvm_link_shared() && target.contains("apple-darwin") {
+        // Helper to find the name of LLVM's shared library on darwin and linux.
+        let find_llvm_lib_name = |extension| {
             let mut cmd = Command::new(&res.llvm_config);
             let version = output(cmd.arg("--version"));
             let major = version.split('.').next().unwrap();
-            let lib_name = match llvm_version_suffix {
-                Some(s) => format!("libLLVM-{major}{s}.dylib"),
-                None => format!("libLLVM-{major}.dylib"),
+            let lib_name = match &llvm_version_suffix {
+                Some(version_suffix) => format!("libLLVM-{major}{version_suffix}.{extension}"),
+                None => format!("libLLVM-{major}.{extension}"),
             };
+            lib_name
+        };
 
+        // When building LLVM with LLVM_LINK_LLVM_DYLIB for macOS, an unversioned
+        // libLLVM.dylib will be built. However, llvm-config will still look
+        // for a versioned path like libLLVM-14.dylib. Manually create a symbolic
+        // link to make llvm-config happy.
+        if builder.llvm_link_shared() && target.contains("apple-darwin") {
+            let lib_name = find_llvm_lib_name("dylib");
             let lib_llvm = out_dir.join("build").join("lib").join(lib_name);
             if !lib_llvm.exists() {
                 t!(builder.symlink_file("libLLVM.dylib", &lib_llvm));
             }
         }
 
+        // When building LLVM as a shared library on linux, it can contain unexpected debuginfo:
+        // some can come from the C++ standard library. Unless we're explicitly requesting LLVM to
+        // be built with debuginfo, strip it away after the fact, to make dist artifacts smaller.
+        // FIXME: to make things simpler for now, limit this to the host and target where we know
+        // `strip -g` is both available and will fix the issue, i.e. on a x64 linux host that is not
+        // cross-compiling. Expand this to other appropriate targets in the future.
+        if builder.llvm_link_shared()
+            && builder.config.llvm_optimize
+            && !builder.config.llvm_release_debuginfo
+            && target == "x86_64-unknown-linux-gnu"
+            && target == builder.config.build
+        {
+            // Find the name of the LLVM shared library that we just built.
+            let lib_name = find_llvm_lib_name("so");
+
+            // If the shared library exists in LLVM's `/build/lib/` or `/lib/` folders, strip its
+            // debuginfo. Note: `output` will propagate any errors here.
+            let strip_if_possible = |path: PathBuf| {
+                if path.exists() {
+                    output(Command::new("strip").arg("--strip-debug").arg(path));
+                }
+            };
+            strip_if_possible(out_dir.join("lib").join(&lib_name));
+            strip_if_possible(out_dir.join("build").join("lib").join(&lib_name));
+        }
+
         t!(stamp.write());
 
         res