Linking to private symbols from frameworks between compilation units #91372
Comments
|
Sorry if the issue is not very well explained, I'm very inexperienced with how linking works |
See also rust-windowing#1626. The `cocoa` crate links to AppKit, which made the symbol `CGDisplayCreateUUIDFromDisplayID` from ColorSync (which AppKit uses internally) available to us (on macOS 10.13 or above). Lately something changed in rustc so that this no longer works, see rust-lang/rust#91372. So instead, we link to AppKit directly within the `winit` crate, which, for now, allows us to use the private symbol.
|
This is affecting |
ehuss
added
O-macos
|
With #90499 Rust sets I will look into this more. |
|
I can confirm that using
You need to build a binary, so that the linker is actually invoked. Try building an example (e.g. |
|
I can clarify this a little more: If you were building a Rust program for an Apple platform that linked to APIs with availability somewhere between 10.7+ and 12.0+, and you weren't setting your own deployment target, #90499 has revealed a bug in your build process. Your binaries were only ever going to run on recent OS releases, but you would have been unaware without testing. For coreaudio, if it really requires APIs from 11.0+, that seems a bit recent/limiting for a library that's existed since forever on macOS. If the authors were unaware of this, #90499 has revealed that they need to do more work to attain backwards compatibility. There will be more breakage like it, but it is the good kind, because anyone using newer Apple APIs should be aware of their minimum platform requirement, just as Xcode forces you to be. |
|
Hmm, this change had more impact than I thought. (Open-source) clang does a dance to determine which minimum Macos version to pass as part of the target triple to LLVM and to ld via I think ideally we should probably try to mimic this behavior somewhat so we don't confront users with having to set |
See also rust-windowing#1626. The `cocoa` crate links to AppKit, which made the symbol `CGDisplayCreateUUIDFromDisplayID` from ColorSync (which AppKit uses internally) available to us (on macOS 10.13 or above). Lately something changed in rustc so that this no longer works, see rust-lang/rust#91372. So instead, we link to AppKit directly within the `winit` crate, which, for now, allows us to use the private symbol.
Actually, that makes a lot of sense, thanks! I fully agree that it's better to catch these issues when developing (instead of later, when users complain that the application doesn't work on their older system)! I've opened rust-windowing/winit#2078 to fix the linking in |
|
Note that this is a breaking change for macOS users, before, they didn't have to specify Additionally, the affected crates (that we know about) are quite widely used (at least half of all Rust games depend on I don't really know the procedure for this, but I would like to propose reverting #90499 (at least from the current beta) to give the affected crates more time to update. Again, I really do agree with the change, I just also want to minimize the breakage macOS users will experience from upgrading Rust. |
Mark-Simulacrum
added
the
regression-from-stable-to-beta
rustbot
added
the
I-prioritize
|
Like I wrote above I think ideally we should match clang's behavior and thus fall back to the default of the currently configured Macos SDK. That would mean looking at the |
@hkratz I think we should revert the change until you find a way to do this - it seems to be causing a lot of breakage and my understanding is the goal was just to turn a runtime error into a link time error. |
…rt, r=Mark-Simulacrum Revert setting a default for the MACOSX_DEPLOYMENT_TARGET env var for linking This reverts commit b376f56, which is the main part of rust-lang#90499, because it turns out that this causes a good amount of breakage in crates relying on the old behavior. In particular `winit`, `coreaudio` and crates that depend on them are affected. Fixes rust-lang#91372. Background: Before rust-lang#90499 the behavior was the following: If MACOSX_DEPLOYMENT_TARGET is not set, we pass the minimum supported OS version to LLVM but not to the linker. The linker default depends on the Xcode version and the version of the OS it is running on. That caused one known problem in libcurl with the most recent Xcode versions. rust-lang#90499 passed the minumum supported version (10.7 for Macos x86-64) to the linker instead. This has shown to be problematic because some crates such as winit, coreaudio implicitly expect a newer minimum OS version. The libcurl issue has been fixed independently (see alexcrichton/curl-rust#417), so a revert should not really be problematic. Eventually we should probably mimic clang's behavior and fall back to the default of the currently configured Macos SDK for both the LLVM min os target version and MACOSX_DEPLOYMENT_TARGET for linking. That would entail looking at the `Version` property of the `SDKSettings.json` in the currently configured SDK.
…rt, r=Mark-Simulacrum Revert setting a default for the MACOSX_DEPLOYMENT_TARGET env var for linking This reverts commit b376f56, which is the main part of rust-lang#90499, because it turns out that this causes a good amount of breakage in crates relying on the old behavior. In particular `winit`, `coreaudio` and crates that depend on them are affected. Fixes rust-lang#91372. Background: Before rust-lang#90499 the behavior was the following: If MACOSX_DEPLOYMENT_TARGET is not set, we pass the minimum supported OS version to LLVM but not to the linker. The linker default depends on the Xcode version and the version of the OS it is running on. That caused one known problem in libcurl with the most recent Xcode versions. rust-lang#90499 passed the minumum supported version (10.7 for Macos x86-64) to the linker instead. This has shown to be problematic because some crates such as winit, coreaudio implicitly expect a newer minimum OS version. The libcurl issue has been fixed independently (see alexcrichton/curl-rust#417), so a revert should not really be problematic. Eventually we should probably mimic clang's behavior and fall back to the default of the currently configured Macos SDK for both the LLVM min os target version and MACOSX_DEPLOYMENT_TARGET for linking. That would entail looking at the `Version` property of the `SDKSettings.json` in the currently configured SDK.
…rt, r=Mark-Simulacrum Revert setting a default for the MACOSX_DEPLOYMENT_TARGET env var for linking This reverts commit b376f56, which is the main part of rust-lang#90499, because it turns out that this causes a good amount of breakage in crates relying on the old behavior. In particular `winit`, `coreaudio` and crates that depend on them are affected. Fixes rust-lang#91372. Background: Before rust-lang#90499 the behavior was the following: If MACOSX_DEPLOYMENT_TARGET is not set, we pass the minimum supported OS version to LLVM but not to the linker. The linker default depends on the Xcode version and the version of the OS it is running on. That caused one known problem in libcurl with the most recent Xcode versions. rust-lang#90499 passed the minumum supported version (10.7 for Macos x86-64) to the linker instead. This has shown to be problematic because some crates such as winit, coreaudio implicitly expect a newer minimum OS version. The libcurl issue has been fixed independently (see alexcrichton/curl-rust#417), so a revert should not really be problematic. Eventually we should probably mimic clang's behavior and fall back to the default of the currently configured Macos SDK for both the LLVM min os target version and MACOSX_DEPLOYMENT_TARGET for linking. That would entail looking at the `Version` property of the `SDKSettings.json` in the currently configured SDK.
…rt, r=Mark-Simulacrum Revert setting a default for the MACOSX_DEPLOYMENT_TARGET env var for linking This reverts commit b376f56, which is the main part of rust-lang#90499, because it turns out that this causes a good amount of breakage in crates relying on the old behavior. In particular `winit`, `coreaudio` and crates that depend on them are affected. Fixes rust-lang#91372. Background: Before rust-lang#90499 the behavior was the following: If MACOSX_DEPLOYMENT_TARGET is not set, we pass the minimum supported OS version to LLVM but not to the linker. The linker default depends on the Xcode version and the version of the OS it is running on. That caused one known problem in libcurl with the most recent Xcode versions. rust-lang#90499 passed the minumum supported version (10.7 for Macos x86-64) to the linker instead. This has shown to be problematic because some crates such as winit, coreaudio implicitly expect a newer minimum OS version. The libcurl issue has been fixed independently (see alexcrichton/curl-rust#417), so a revert should not really be problematic. Eventually we should probably mimic clang's behavior and fall back to the default of the currently configured Macos SDK for both the LLVM min os target version and MACOSX_DEPLOYMENT_TARGET for linking. That would entail looking at the `Version` property of the `SDKSettings.json` in the currently configured SDK.
rustbot
removed
the
I-prioritize
|
reopening as this is a beta-regression, and we keep those open until beta fix lands (or we explicitly decline to fix). |
…f not set." This reverts commit b376f56, which is the main part of rust-lang#90499, because it turns out that this causes a good amount of breakage in crates relying on the old behavior. Fixes rust-lang#91372.
|
Backport merged. |
|
Just for reference, the linking in the noted crates have since been fixed:
|
This seems related to rust-lang/rust#91372 . Without `resolver = "2"`: ``` [nix-shell:~/code/clones/roc]$ cargo run examples/hello-rust/Hello.roc Finished dev [unoptimized + debuginfo] target(s) in 0.28s Running `target/debug/roc examples/hello-rust/Hello.roc` thread '<unnamed>' panicked at 'Failed to rebuild src/lib.rs - stderr of the `cargo build` command was: Updating crates.io index Compiling libc v0.2.116 Compiling cfg-if v1.0.0 Compiling proc-macro2 v1.0.36 ... Compiling gilrs v0.8.2 Compiling gltf-json v0.16.0 Compiling wgpu-hal v0.12.4 error: DX12 API enabled on non-Windows OS. If your project is not using resolver="2" in Cargo.toml, it should. --> /Users/jan/.cargo/registry/src/github.com-1ecc6299db9ec823/wgpu-hal-0.12.4/src/lib.rs:53:1 | 53 | compile_error!("DX12 API enabled on non-Windows OS. If your project is not using resolver=\"2\" in Cargo.toml, it should."); | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ error[E0432]: unresolved import `super::dx12` --> /Users/jan/.cargo/registry/src/github.com-1ecc6299db9ec823/wgpu-hal-0.12.4/src/lib.rs:68:20 | 68 | pub use super::dx12::Api as Dx12; | ^^^^ could not find `dx12` in the crate root Compiling hexasphere v6.0.0 Compiling bevy_macro_utils v0.6.0 For more information about this error, try `rustc --explain E0432`. error: could not compile `wgpu-hal` due to 2 previous errors warning: build failed, waiting for other jobs to finish... error: build failed ', compiler/build/src/link.rs:984:27 note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Any { .. }', cli/src/build.rs:235:50 🔨 Rebuilding host... [nix-shell:~/code/clones/roc]$ ``` With `resolver = "2"`: ``` [nix-shell:~/code/clones/roc]$ cargo run examples/hello-rust/Hello.roc Finished dev [unoptimized + debuginfo] target(s) in 0.86s Running `target/debug/roc examples/hello-rust/Hello.roc` 🔨 Rebuilding host... Done! Undefined symbols for architecture x86_64: "_AudioComponentFindNext", referenced from: __ZN9coreaudio10audio_unit9AudioUnit14new_with_flags17heb6ec65f112e010bE in host.o "_AudioComponentInstanceDispose", referenced from: __ZN74_$LT$coreaudio..audio_unit..AudioUnit$u20$as$u20$core..ops..drop..Drop$GT$4drop17h75c103dc6ced4686E in host.o "_AudioComponentInstanceNew", referenced from: __ZN9coreaudio10audio_unit9AudioUnit14new_with_flags17heb6ec65f112e010bE in host.o "_AudioObjectGetPropertyData", referenced from: __ZN4cpal4host9coreaudio5macos9enumerate13audio_devices17haf753430364e01ffE in host.o __ZN4cpal4host9coreaudio5macos9enumerate20default_input_device17h2d1534837211f2f8E in host.o __ZN4cpal4host9coreaudio5macos9enumerate21default_output_device17h083911c236280accE in host.o __ZN4cpal4host9coreaudio5macos6Device4name17h68e2f994f5671fa5E in host.o __ZN4cpal4host9coreaudio5macos6Device17supported_configs17h11dc942236696a63E in host.o __ZN4cpal4host9coreaudio5macos6Device14default_config17hdbd3efef31ad18f9E in host.o __ZN4cpal4host9coreaudio5macos6Device22build_input_stream_raw13rate_listener17h0d393e1cb81ceb75E in host.o ... "_AudioObjectGetPropertyDataSize", referenced from: __ZN4cpal4host9coreaudio5macos9enumerate13audio_devices17haf753430364e01ffE in host.o __ZN4cpal4host9coreaudio5macos6Device17supported_configs17h11dc942236696a63E in host.o "_AudioOutputUnitStart", referenced from: __ZN9coreaudio10audio_unit9AudioUnit5start17h3f1dd3512794e2c6E in host.o "_AudioOutputUnitStop", referenced from: __ZN9coreaudio10audio_unit9AudioUnit4stop17h8195cacdfa008254E in host.o "_AudioUnitGetProperty", referenced from: __ZN9coreaudio10audio_unit12get_property17hcf8e3ea3616b46c8E in host.o __ZN9coreaudio10audio_unit12get_property17h521e60fcc0df83d1E in host.o __ZN9coreaudio10audio_unit12get_property17hbd4c129c859bc308E in host.o "_AudioUnitInitialize", referenced from: __ZN9coreaudio10audio_unit9AudioUnit14new_with_flags17heb6ec65f112e010bE in host.o __ZN9coreaudio10audio_unit9AudioUnit10initialize17hb6a2c15df5611a8bE in host.o "_AudioUnitSetProperty", referenced from: __ZN9coreaudio10audio_unit12set_property17h7ac04f580fde2d6bE in host.o __ZN9coreaudio10audio_unit12set_property17h7fe7a189aa012be8E in host.o __ZN9coreaudio10audio_unit12set_property17h4c55c6b1a7e57f73E in host.o __ZN9coreaudio10audio_unit12set_property17he956119f0b29327cE in host.o "_AudioUnitUninitialize", referenced from: __ZN9coreaudio10audio_unit9AudioUnit12uninitialize17h3386111a26093dcbE in host.o __ZN74_$LT$coreaudio..audio_unit..AudioUnit$u20$as$u20$core..ops..drop..Drop$GT$4drop17h75c103dc6ced4686E in host.o "_CFStringGetCString", referenced from: __ZN4cpal4host9coreaudio5macos6Device4name17h68e2f994f5671fa5E in host.o "_CFStringGetCStringPtr", referenced from: __ZN4cpal4host9coreaudio5macos6Device4name17h68e2f994f5671fa5E in host.o ld: symbol(s) not found for architecture x86_64 [nix-shell:~/code/clones/roc]$ ```
This seems related to rust-lang/rust#91372 . ``` [nix-shell:~/code/clones/roc]$ cargo run examples/hello-rust/Hello.roc Finished dev [unoptimized + debuginfo] target(s) in 0.40s Running `target/debug/roc examples/hello-rust/Hello.roc` 🔨 Rebuilding host... Done! Undefined symbols for architecture x86_64: "_AudioComponentFindNext", referenced from: __ZN9coreaudio10audio_unit9AudioUnit14new_with_flags17heb6ec65f112e010bE in host.o "_AudioComponentInstanceDispose", referenced from: __ZN74_$LT$coreaudio..audio_unit..AudioUnit$u20$as$u20$core..ops..drop..Drop$GT$4drop17h75c103dc6ced4686E in host.o "_AudioComponentInstanceNew", referenced from: __ZN9coreaudio10audio_unit9AudioUnit14new_with_flags17heb6ec65f112e010bE in host.o "_AudioObjectGetPropertyData", referenced from: __ZN4cpal4host9coreaudio5macos9enumerate13audio_devices17haf753430364e01ffE in host.o __ZN4cpal4host9coreaudio5macos9enumerate20default_input_device17h2d1534837211f2f8E in host.o __ZN4cpal4host9coreaudio5macos9enumerate21default_output_device17h083911c236280accE in host.o __ZN4cpal4host9coreaudio5macos6Device4name17h68e2f994f5671fa5E in host.o __ZN4cpal4host9coreaudio5macos6Device17supported_configs17h11dc942236696a63E in host.o __ZN4cpal4host9coreaudio5macos6Device14default_config17hdbd3efef31ad18f9E in host.o __ZN4cpal4host9coreaudio5macos6Device22build_input_stream_raw13rate_listener17h0d393e1cb81ceb75E in host.o ... "_AudioObjectGetPropertyDataSize", referenced from: __ZN4cpal4host9coreaudio5macos9enumerate13audio_devices17haf753430364e01ffE in host.o __ZN4cpal4host9coreaudio5macos6Device17supported_configs17h11dc942236696a63E in host.o "_AudioOutputUnitStart", referenced from: __ZN9coreaudio10audio_unit9AudioUnit5start17h3f1dd3512794e2c6E in host.o "_AudioOutputUnitStop", referenced from: __ZN9coreaudio10audio_unit9AudioUnit4stop17h8195cacdfa008254E in host.o "_AudioUnitGetProperty", referenced from: __ZN9coreaudio10audio_unit12get_property17hcf8e3ea3616b46c8E in host.o __ZN9coreaudio10audio_unit12get_property17h521e60fcc0df83d1E in host.o __ZN9coreaudio10audio_unit12get_property17hbd4c129c859bc308E in host.o "_AudioUnitInitialize", referenced from: __ZN9coreaudio10audio_unit9AudioUnit14new_with_flags17heb6ec65f112e010bE in host.o __ZN9coreaudio10audio_unit9AudioUnit10initialize17hb6a2c15df5611a8bE in host.o "_AudioUnitSetProperty", referenced from: __ZN9coreaudio10audio_unit12set_property17h7ac04f580fde2d6bE in host.o __ZN9coreaudio10audio_unit12set_property17h7fe7a189aa012be8E in host.o __ZN9coreaudio10audio_unit12set_property17h4c55c6b1a7e57f73E in host.o __ZN9coreaudio10audio_unit12set_property17he956119f0b29327cE in host.o "_AudioUnitUninitialize", referenced from: __ZN9coreaudio10audio_unit9AudioUnit12uninitialize17h3386111a26093dcbE in host.o __ZN74_$LT$coreaudio..audio_unit..AudioUnit$u20$as$u20$core..ops..drop..Drop$GT$4drop17h75c103dc6ced4686E in host.o ld: symbol(s) not found for architecture x86_64 [nix-shell:~/code/clones/roc]$ ```
This seems related to rust-lang/rust#91372 . ``` [nix-shell:~/code/clones/roc]$ cargo run examples/hello-rust/Hello.roc Finished dev [unoptimized + debuginfo] target(s) in 0.40s Running `target/debug/roc examples/hello-rust/Hello.roc` 🔨 Rebuilding host... Done! Undefined symbols for architecture x86_64: "_AudioComponentFindNext", referenced from: __ZN9coreaudio10audio_unit9AudioUnit14new_with_flags17heb6ec65f112e010bE in host.o "_AudioComponentInstanceDispose", referenced from: __ZN74_$LT$coreaudio..audio_unit..AudioUnit$u20$as$u20$core..ops..drop..Drop$GT$4drop17h75c103dc6ced4686E in host.o "_AudioComponentInstanceNew", referenced from: __ZN9coreaudio10audio_unit9AudioUnit14new_with_flags17heb6ec65f112e010bE in host.o "_AudioObjectGetPropertyData", referenced from: __ZN4cpal4host9coreaudio5macos9enumerate13audio_devices17haf753430364e01ffE in host.o __ZN4cpal4host9coreaudio5macos9enumerate20default_input_device17h2d1534837211f2f8E in host.o __ZN4cpal4host9coreaudio5macos9enumerate21default_output_device17h083911c236280accE in host.o __ZN4cpal4host9coreaudio5macos6Device4name17h68e2f994f5671fa5E in host.o __ZN4cpal4host9coreaudio5macos6Device17supported_configs17h11dc942236696a63E in host.o __ZN4cpal4host9coreaudio5macos6Device14default_config17hdbd3efef31ad18f9E in host.o __ZN4cpal4host9coreaudio5macos6Device22build_input_stream_raw13rate_listener17h0d393e1cb81ceb75E in host.o ... "_AudioObjectGetPropertyDataSize", referenced from: __ZN4cpal4host9coreaudio5macos9enumerate13audio_devices17haf753430364e01ffE in host.o __ZN4cpal4host9coreaudio5macos6Device17supported_configs17h11dc942236696a63E in host.o "_AudioOutputUnitStart", referenced from: __ZN9coreaudio10audio_unit9AudioUnit5start17h3f1dd3512794e2c6E in host.o "_AudioOutputUnitStop", referenced from: __ZN9coreaudio10audio_unit9AudioUnit4stop17h8195cacdfa008254E in host.o "_AudioUnitGetProperty", referenced from: __ZN9coreaudio10audio_unit12get_property17hcf8e3ea3616b46c8E in host.o __ZN9coreaudio10audio_unit12get_property17h521e60fcc0df83d1E in host.o __ZN9coreaudio10audio_unit12get_property17hbd4c129c859bc308E in host.o "_AudioUnitInitialize", referenced from: __ZN9coreaudio10audio_unit9AudioUnit14new_with_flags17heb6ec65f112e010bE in host.o __ZN9coreaudio10audio_unit9AudioUnit10initialize17hb6a2c15df5611a8bE in host.o "_AudioUnitSetProperty", referenced from: __ZN9coreaudio10audio_unit12set_property17h7ac04f580fde2d6bE in host.o __ZN9coreaudio10audio_unit12set_property17h7fe7a189aa012be8E in host.o __ZN9coreaudio10audio_unit12set_property17h4c55c6b1a7e57f73E in host.o __ZN9coreaudio10audio_unit12set_property17he956119f0b29327cE in host.o "_AudioUnitUninitialize", referenced from: __ZN9coreaudio10audio_unit9AudioUnit12uninitialize17h3386111a26093dcbE in host.o __ZN74_$LT$coreaudio..audio_unit..AudioUnit$u20$as$u20$core..ops..drop..Drop$GT$4drop17h75c103dc6ced4686E in host.o ld: symbol(s) not found for architecture x86_64 [nix-shell:~/code/clones/roc]$ ```
|
I root-caused this a little bit more today, sharing my findings: The problem can be reproduced with #[link(name = "AppKit", kind = "framework")]
#[link(name = "CoreGraphics", kind = "framework")]
extern "C" {
pub fn CGDisplayCreateUUIDFromDisplayID(display: u32) -> *const std::os::raw::c_void;
}
fn main() {
unsafe { CGDisplayCreateUUIDFromDisplayID(0) };
}Weirdly enough, if you remove the linking to I think this is actually a bug in
Again, I believe this to be a linker bug because the problem does not happen with |
This used to work on Rust
nightly-2021-11-24:CGDisplayCreateUUIDFromDisplayIDis present in theColorSyncframework (on macOS +10.13; on lower versions, in theCoreGraphicsframework). TheAppKitframework usesColorSyncinternally, so the symbol is available through that (I think).Linking like this worked previously, but since Rust
nightly-2021-11-26it started failing.This is affecting
winit's CI.Meta
OS: macOS Mojave 10.14.6.
rustc --version --verbose:Backtrace
The text was updated successfully, but these errors were encountered: