Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Linking modifier whole-archive and link time cfg are not supported on bundled static libraries #99429

Closed
petrochenkov opened this issue Jul 18, 2022 · 3 comments
Labels
A-linkage Area: linking into static, shared libraries and binaries T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@petrochenkov
Copy link
Contributor

petrochenkov commented Jul 18, 2022

This is more or less a part of #81490.

If you try to combine +whole-archive with (possibly implicit) +bundle you will encounter this error: "linking modifiers +bundle and +whole-archive are not compatible with each other when generating rlibs".
The whole-archive modifier is significantly limited due to this issue and cannot be used in foo-sys rlibs bundling their native libfoo.a dependencies, which is one of the main use cases.

We also have to enable the +whole-archive modifier implicitly in one ad hoc case (https://github.com/rust-lang/rust/blob/master/compiler/rustc_codegen_ssa/src/back/link.rs#L2209-L2213) for backward compatibility, because it cannot be enabled manually.

cfg for bundled static libraries was implemented just enough to make the case of musl libc bundled into libc crate to work, but it doesn't work in general case.
(Note that musl libc no longer uses bundling, see #72274.)

Also see #88085 and #88161 for some discussion related to this issue.

@petrochenkov
Copy link
Contributor Author

We need to make bundled libraries behave more like regular static libraries libfoo.a.

The simplest way, which is also most compatible with traditional tooling is to literally move them out of rlibs to separate .a files, like this

.
├── libfoo-sys.rlib
└── libfoo-sys.rlib.bundle
    └── libfoo.a

It solves all the issues above:

  • libfoo.a can be passed to linker as a separate static library, with preserving all the static library semantics, thus it can be
    • wrapped into -whole-archive ... -no-whole-archive
    • not passed at all if some relevant link time cfg is false
    • moved to some other place on the command line if necessary, not necessarily next to libfoo-sys.rlib
  • it's also fast, rustc doesn't have to extract object files to some temporary .a file on the fly or something like that, when static library semantics are necessary.

The problem is that we'll have to teach cargo to track these new extra files.
Third party build systems will also need to support this (although it's possible that they never use bundling and always link their native libraries in their own way).

If the separate file solution doesn't work out we'll need to figure out how to bundle object files into rlibs in some smarter way, with preserving their old structure, and then how to repack them on the fly if they need to be represented as a static library.

@petrochenkov petrochenkov added A-linkage Area: linking into static, shared libraries and binaries T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Jul 18, 2022
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this issue Aug 2, 2022
rustc-docs: Be less specific about the representation of `+bundle`

For rust-lang#99429.
r? `@bjorn3`
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this issue Aug 14, 2022
change rlib format to distinguish native dependencies

Another one method to solve problem mentioned in rust-lang#99429.

Changed .rlib format, it contains all bundled native libraries as archieves.
At link time rlib is unpacked and native dependencies linked separately.
New behavior hidden under separate_native_rlib_dependencies flag.
TaKO8Ki added a commit to TaKO8Ki/rust that referenced this issue Aug 16, 2022
change rlib format to distinguish native dependencies

Another one method to solve problem mentioned in rust-lang#99429.

Changed .rlib format, it contains all bundled native libraries as archieves.
At link time rlib is unpacked and native dependencies linked separately.
New behavior hidden under separate_native_rlib_dependencies flag.
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this issue Sep 2, 2022
change rlib format to distinguish native dependencies

Another one method to solve problem mentioned in rust-lang#99429.

Changed .rlib format, it contains all bundled native libraries as archieves.
At link time rlib is unpacked and native dependencies linked separately.
New behavior hidden under separate_native_rlib_dependencies flag.
bors added a commit to rust-lang-ci/rust that referenced this issue Sep 13, 2022
change rlib format to distinguish native dependencies

Another one method to solve problem mentioned in rust-lang#99429.

Changed .rlib format, it contains all bundled native libraries as archieves.
At link time rlib is unpacked and native dependencies linked separately.
New behavior hidden under separate_native_rlib_dependencies flag.
@petrochenkov
Copy link
Contributor Author

Support for this was implemented in #105601 under a feature gate #![feature(packed_bundled_libs)].
I'm closing this issue, but we need to open a tracking issue for that feature.

@petrochenkov
Copy link
Contributor Author

The tracking issue - #108081.

RalfJung pushed a commit to RalfJung/rust-analyzer that referenced this issue Apr 20, 2024
rustc-docs: Be less specific about the representation of `+bundle`

For rust-lang/rust#99429.
r? `@bjorn3`
RalfJung pushed a commit to RalfJung/rust-analyzer that referenced this issue Apr 27, 2024
rustc-docs: Be less specific about the representation of `+bundle`

For rust-lang/rust#99429.
r? `@bjorn3`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-linkage Area: linking into static, shared libraries and binaries T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests

1 participant