Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rust-lld: add rpath entry to the correct lib folder #112247

Merged
merged 2 commits into from
Jun 4, 2023

Conversation

lqd
Copy link
Member

@lqd lqd commented Jun 3, 2023

An explanation, for our linux rustup toolchain:

  • lld / rust-lld is built as a regular LLVM tool, but is not distributed via the llvm-tools component. It's distributed by default, like a regular rust binary, like cargo and rustc. The general expected setup is: binaries in bin and libraries in lib, so the rpath we use for a bin/$executable is $ORIGIN/../lib.
  • However, rust-lld is not in the same location as our other executables ($root/bin), it's in $root/lib/rustlib/$host/bin/. The current rpath thus expects the LLVM's shared library to be in $root/lib/rustlib/$host/lib/.
  • That .so is only present in $root/lib, causing rust-lld doesn't work on Linux unless executed with rustup wrapper #80703. (LLVM's shared library is also copied to $root/lib/rustlib/$host/lib/ with the llvm-tools component, so it also was a workaround for the issue)

rustup's LD_LIBRARY_PATH overrides made this discrepancy invisible when we switched to llvm.link-shared = true, and this only showed up when running rustc or rust-lld's executables directly.

To fix this we could:

  • copy the .so to this expected location all the time, but that seems wasteful.
  • or, add an rpath entry when building LLD, which seems preferable to me (but I don't know if it could cause issues).

This PR does the latter, tweaking how bootstrap builds LLD to point to the expected directory, and fixes #80703.

(Since this is related to P-high issues about switching to lld by default, I'll cc @petrochenkov to keep them updated.)

rust-lld is not located in the same directory as the other binaries that
point to ../lib, but in a deeper directory in lib. So we need to point
a few layers up for rust-lld to find the LLVM shared library without
rustup's LD_LIBRARY_PATH overrides.
@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Jun 3, 2023
@lqd
Copy link
Member Author

lqd commented Jun 3, 2023

@bors try

@bors
Copy link
Contributor

bors commented Jun 3, 2023

⌛ Trying commit 17d321c with merge 6d05280785ea61c31f635fbfb61e8f988212a58b...

@bors
Copy link
Contributor

bors commented Jun 3, 2023

☀️ Try build successful - checks-actions
Build commit: 6d05280785ea61c31f635fbfb61e8f988212a58b (6d05280785ea61c31f635fbfb61e8f988212a58b)

@lqd lqd marked this pull request as ready for review June 3, 2023 18:38
@lqd
Copy link
Member Author

lqd commented Jun 3, 2023

The issue is fixed on my local builds, and linking the wasm example works fine, but I was hoping we could get a wasm32-unknown-unknown target on try builds. It appears not to be the case, so we can't reproduce the issue exactly, but we can simulate it.

On the try parent (1e17cef9e29961636db63730784caea4b7d47ff5), the wasm linking issue is the same as the following:

$ readelf -d ~/.rustup/toolchains/1e17cef9e29961636db63730784caea4b7d47ff5/lib/rustlib/x86_64-unknown-linux-gnu/bin/rust-lld | grep PATH
 0x000000000000001d (RUNPATH)            Library runpath: [$ORIGIN/../lib:/checkout/obj/build/x86_64-unknown-linux-gnu/llvm/lib]

$ ldd ~/.rustup/toolchains/1e17cef9e29961636db63730784caea4b7d47ff5/lib/rustlib/x86_64-unknown-linux-gnu/bin/rust-lld | grep LLVM
 libLLVM-16-rust-1.72.0-nightly.so => not found

$ ~/.rustup/toolchains/1e17cef9e29961636db63730784caea4b7d47ff5/lib/rustlib/x86_64-unknown-linux-gnu/bin/rust-lld
.../rust-lld: error while loading shared libraries: libLLVM-16-rust-1.72.0-nightly.so: cannot open shared object file: No such file or directory

while on this PR's 6d05280785ea61c31f635fbfb61e8f988212a58b try:

$ readelf -d ~/.rustup/toolchains/6d05280785ea61c31f635fbfb61e8f988212a58b/lib/rustlib/x86_64-unknown-linux-gnu/bin/rust-lld | grep PATH
 0x000000000000001d (RUNPATH)            Library runpath: [$ORIGIN/../../../:$ORIGIN/../lib:/checkout/obj/build/x86_64-unknown-linux-gnu/llvm/lib]

$ ldd ~/.rustup/toolchains/6d05280785ea61c31f635fbfb61e8f988212a58b/lib/rustlib/x86_64-unknown-linux-gnu/bin/rust-lld | grep LLVM
 libLLVM-16-rust-1.72.0-nightly.so => /home/lqd/.rustup/toolchains/6d05280785ea61c31f635fbfb61e8f988212a58b/lib/rustlib/x86_64-unknown-linux-gnu/bin/../../../libLLVM-16-rust-1.72.0-nightly.so (0x00007fffeea00000)

$ ~/.rustup/toolchains/6d05280785ea61c31f635fbfb61e8f988212a58b/lib/rustlib/x86_64-unknown-linux-gnu/bin/rust-lld
lld is a generic driver.
Invoke ld.lld (Unix), ld64.lld (macOS), lld-link (Windows), wasm-ld (WebAssembly) instead

so we should be good to go here.

r? bootstrap

@Mark-Simulacrum
Copy link
Member

@bors r+

@bors
Copy link
Contributor

bors commented Jun 3, 2023

📌 Commit 4117b5e has been approved by Mark-Simulacrum

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 3, 2023
bors added a commit to rust-lang-ci/rust that referenced this pull request Jun 4, 2023
…iaskrgr

Rollup of 3 pull requests

Successful merges:

 - rust-lang#112247 (rust-lld: add rpath entry to the correct `lib` folder)
 - rust-lang#112274 (Migrate GUI colors test to original CSS color format)
 - rust-lang#112277 (Don't require the output from libtest to be valid UTF-8)

Failed merges:

 - rust-lang#112251 (rustdoc: convert `if let Some()` that always matches to variable)

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit ef2c64b into rust-lang:master Jun 4, 2023
@bors
Copy link
Contributor

bors commented Jun 4, 2023

⌛ Testing commit 4117b5e with merge 101fa90...

@rustbot rustbot added this to the 1.72.0 milestone Jun 4, 2023
@lqd lqd deleted the lld-rpath branch June 4, 2023 22:22
@bors
Copy link
Contributor

bors commented Jun 5, 2023

💥 Test timed out

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jun 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

rust-lld doesn't work on Linux unless executed with rustup wrapper
4 participants