Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

port tests/run-make/extern-fn-reachable to rmake #128314

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

lolbinarycat
Copy link
Contributor

@lolbinarycat lolbinarycat commented Jul 28, 2024

uses helper functions added in #128147, must not be merged before that PR.

try-job: aarch64-apple
try-job: armhf-gnu
try-job: test-various
try-job: x86_64-msvc
try-job: x86_64-mingw
try-job: i686-msvc
try-job: i686-mingw
try-job: x86_64-gnu-llvm-17

@rustbot
Copy link
Collaborator

rustbot commented Jul 28, 2024

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @jieyouxu (or someone else) some time within the next two weeks.

Please see the contribution instructions for more information. Namely, in order to ensure the minimum review times lag, PR authors and assigned reviewers should ensure that the review label (S-waiting-on-review and S-waiting-on-author) stays updated, invoking these commands when appropriate:

  • @rustbot author: the review is finished, PR author should check the comments and take action accordingly
  • @rustbot review: the author is ready for a review, this PR will be queued again in the reviewer's queue

@rustbot rustbot added A-run-make Area: port run-make Makefiles to rmake.rs A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Jul 28, 2024
@rustbot
Copy link
Collaborator

rustbot commented Jul 28, 2024

The run-make-support library was changed

cc @jieyouxu

This PR modifies tests/run-make/. If this PR is trying to port a Makefile
run-make test to use rmake.rs, please update the
run-make port tracking issue
so we can track our progress. You can either modify the tracking issue
directly, or you can comment on the tracking issue and link this PR.

cc @jieyouxu

@rust-log-analyzer

This comment has been minimized.

@bors
Copy link
Contributor

bors commented Aug 2, 2024

☔ The latest upstream changes (presumably #128147) made this pull request unmergeable. Please resolve the merge conflicts.

@lolbinarycat lolbinarycat force-pushed the extern-fn-reachable-rmake branch from e2335ba to 594e4c0 Compare August 2, 2024 15:15
@lolbinarycat lolbinarycat marked this pull request as ready for review August 2, 2024 15:19
@rust-log-analyzer

This comment has been minimized.

@bors
Copy link
Contributor

bors commented Aug 2, 2024

☔ The latest upstream changes (presumably #127926) made this pull request unmergeable. Please resolve the merge conflicts.

Copy link
Member

@jieyouxu jieyouxu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the PR. I think I noticed a discrepancy between the symbol checking logic between the original Makefile and the rmake.rs version.

@@ -193,6 +193,11 @@ impl Rustc {
self
}

/// Make `rustc` prefere dynamic linking
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit:

Suggested change
/// Make `rustc` prefere dynamic linking
/// Make `rustc` prefer dynamic linking.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ping

/// The symbol names must match exactly.
///
/// Panics if `path` is not a valid object file readable by the current user.
pub fn contains_exact_symbols(path: impl AsRef<Path>, symbol_names: &[&str]) -> bool {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: having this return a bool but then also have output logging is weird. We probably want to have a read_symbols(path) return something like Vec<OsString> or Vec<Vec<u8>> (pretty sure symbols don't need to be UTF-8), and then have an assertion helper based on that:

fn assert_symbols_match_exactly(expected_symbols: &[&OsStr], found_symbols: &[&OsStr]) {
    assert_eq!(expected_symbols, found_symbols, "symbols do not match exactly");
}

or something more generic, like assert_byte_strings_are_equal.

(or some customized equality assertion to not show u8 slice debug repr)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Problem: am I understanding the logic here is that for a given object file, and a given list of expected symbols, we want the object file to contain the symbol at least once? Or does the test actually want exact counts of occurrences of symbols? AFAICT, the logic here only checks that a given expected symbol occurs at least once, whereas the logic in the original Makefile checks that a given expected symbol occurs exactly once, so the logic is not equivalent from what I can tell.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the previous makefile wasn't checking for exact symbol names though.

"accidentally generating duplicate symbols and somehow not encountering a link error" doesn't seem like a very probable failure mode, and if it is likely, then i would rather approach that with a helper that asserts a file does not contain any duplicate symbols, rather than checking the counts of specific symbols.

i should probably move the debug output into assertion helpers though, the current behavior is based off of the cat-and-grep script, but it's a bit confusing, and we can do better.

@rustbot rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 3, 2024
Comment on lines -6 to -14
NM=nm -D

ifeq ($(UNAME),Darwin)
NM=nm -gU
endif

ifdef IS_WINDOWS
NM=nm -g
endif
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: Hm,

  • -D is "display the dynamic symbols rather than the normal symbols. This is only meaningful for dynamic objects, such as certain types of shared libraries"
  • -g is "display only external symbols"
  • -U is "display only defined symbols for each object file. By default both defined and undefined symbols are displayed"

and these flags are passed differently based on platforms. Are we sure with_symbol_iter based on object preserves these behavior?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the test is configured to never run on those platforms anyways.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the test is configured to never run on those platforms anyways.

Hmm? That doesn't seem right, since the Makefile test is only ignore-cross-compile + ignore-windows-msvc, or am I misunderstanding something? This still leaves host linux, host apple and host windows mingw targets?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems you're correct, I must've been thinking of a different rmake test.

Still, it seems strange that it would look for different kinds of symbols on different platforms, shouldn't the symbols have the same linkage regardless of platform?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still, it seems strange that it would look for different kinds of symbols on different platforms, shouldn't the symbols have the same linkage regardless of platform?

Yeah, this is exactly why I noticed it. I'll try to do a little digging from the original PR to see why it had to use different nm flags, this seems very cursed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I looked at the original PR alongside review comments, and unfortunately it was not enlightening as to the why.

nm flags:

-D: --dynamic
	Display the dynamic symbols rather than the normal symbols. This is only meaningful for dynamic objects, such as certain types of shared libraries. 

-g: --extern-only
	Show only external symbols

-gU: --extern-only --defined-only
	Display only defined symbols for each object file. By default both defined and undefined symbols are displayed.

I'm inclined to say ideally we want --dynamic + --extern-only + --defined-only consistently across the platforms (this probably fails, but I'd like to know on which platforms that fails). So maybe something based on Object::exports.

(This is going to take a couple of tries to get right I'm afraid.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there's the ObjectSymbol::is_* family of function, but i'm somewhat confused what it would mean to have a non-dynamic global symbol in a dynamic object.

@jieyouxu jieyouxu added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Aug 6, 2024
@jieyouxu
Copy link
Member

jieyouxu commented Aug 6, 2024

@rustbot author

@rustbot rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 6, 2024
uses helper functions added in rust-lang#128147, must not be merged before that
PR.
@rust-log-analyzer
Copy link
Collaborator

The job aarch64-apple failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)
Updating files:  98% (47999/48978)
Updating files:  99% (48489/48978)
Updating files: 100% (48978/48978)
Updating files: 100% (48978/48978), done.
Switched to a new branch 'try'
branch 'try' set up to track 'origin/try'.
[command]/opt/homebrew/bin/git log -1 --format='%H'
'b5490c82c9d567d58d0cb22905e827343e39ba4e'
##[group]Run src/ci/scripts/setup-environment.sh
src/ci/scripts/setup-environment.sh
---
file:.git/config remote.origin.url=https://github.com/rust-lang-ci/rust
file:.git/config remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
file:.git/config gc.auto=0
file:.git/config http.https://github.com/.extraheader=AUTHORIZATION: basic ***
file:.git/config branch.try.remote=origin
file:.git/config branch.try.merge=refs/heads/try
file:.git/config submodule.library/backtrace.url=https://github.com/rust-lang/backtrace-rs.git
file:.git/config submodule.library/stdarch.active=true
file:.git/config submodule.library/stdarch.url=https://github.com/rust-lang/stdarch.git
file:.git/config submodule.src/doc/book.active=true
---
---- [run-make] tests/run-make/extern-fn-reachable stdout ----

error: rmake recipe failed to complete
status: exit status: 101
command: cd "/Users/runner/work/rust/rust/build/aarch64-apple-darwin/test/run-make/extern-fn-reachable/rmake_out" && env -u RUSTFLAGS AR="ar" CC="/Applications/Xcode_14.3.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang" CC_DEFAULT_FLAGS="-ffunction-sections -fdata-sections -fPIC --target=arm64-apple-darwin -mmacosx-version-min=11 -isysroot /Applications/Xcode_14.3.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk" CXX="/Applications/Xcode_14.3.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++" CXX_DEFAULT_FLAGS="-ffunction-sections -fdata-sections -fPIC --target=arm64-apple-darwin -mmacosx-version-min=11 -isysroot /Applications/Xcode_14.3.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk -stdlib=libc++" DYLD_LIBRARY_PATH="/Users/runner/work/rust/rust/build/aarch64-apple-darwin/stage0-bootstrap-tools/aarch64-apple-darwin/release/deps:/Users/runner/work/rust/rust/build/aarch64-apple-darwin/stage0/lib:/Users/runner/work/rust/rust/build/aarch64-apple-darwin/stage2-tools-bin:/Users/runner/work/rust/rust/build/aarch64-apple-darwin/stage2/lib/rustlib/aarch64-apple-darwin/lib" HOST_RPATH_DIR="/Users/runner/work/rust/rust/build/aarch64-apple-darwin/stage2/lib" LD_LIB_PATH_ENVVAR="DYLD_LIBRARY_PATH" LLVM_BIN_DIR="/Users/runner/work/rust/rust/build/aarch64-apple-darwin/ci-llvm/bin" LLVM_COMPONENTS="aarch64 aarch64asmparser aarch64codegen aarch64desc aarch64disassembler aarch64info aarch64utils aggressiveinstcombine all all-targets analysis arm armasmparser armcodegen armdesc armdisassembler arminfo armutils asmparser asmprinter avr avrasmparser avrcodegen avrdesc avrdisassembler avrinfo binaryformat bitreader bitstreamreader bitwriter bpf bpfasmparser bpfcodegen bpfdesc bpfdisassembler bpfinfo cfguard codegen codegendata codegentypes core coroutines coverage csky cskyasmparser cskycodegen cskydesc cskydisassembler cskyinfo debuginfobtf debuginfocodeview debuginfodwarf debuginfogsym debuginfologicalview debuginfomsf debuginfopdb demangle dlltooldriver dwarflinker dwarflinkerclassic dwarflinkerparallel dwp engine executionengine extensions filecheck frontenddriver frontendhlsl frontendoffloading frontendopenacc frontendopenmp fuzzercli fuzzmutate globalisel hexagon hexagonasmparser hexagoncodegen hexagondesc hexagondisassembler hexagoninfo hipstdpar instcombine instrumentation interfacestub interpreter ipo irprinter irreader jitlink libdriver lineeditor linker loongarch loongarchasmparser loongarchcodegen loongarchdesc loongarchdisassembler loongarchinfo lto m68k m68kasmparser m68kcodegen m68kdesc m68kdisassembler m68kinfo mc mca mcdisassembler mcjit mcparser mips mipsasmparser mipscodegen mipsdesc mipsdisassembler mipsinfo mirparser msp430 msp430asmparser msp430codegen msp430desc msp430disassembler msp430info native nativecodegen nvptx nvptxcodegen nvptxdesc nvptxinfo objcarcopts objcopy object objectyaml option orcdebugging orcjit orcshared orctargetprocess passes powerpc powerpcasmparser powerpccodegen powerpcdesc powerpcdisassembler powerpcinfo profiledata remarks riscv riscvasmparser riscvcodegen riscvdesc riscvdisassembler riscvinfo riscvtargetmca runtimedyld sandboxir scalaropts selectiondag sparc sparcasmparser sparccodegen sparcdesc sparcdisassembler sparcinfo support symbolize systemz systemzasmparser systemzcodegen systemzdesc systemzdisassembler systemzinfo tablegen target targetparser textapi textapibinaryreader transformutils vectorize webassembly webassemblyasmparser webassemblycodegen webassemblydesc webassemblydisassembler webassemblyinfo webassemblyutils windowsdriver windowsmanifest x86 x86asmparser x86codegen x86desc x86disassembler x86info x86targetmca xray xtensa xtensaasmparser xtensacodegen xtensadesc xtensadisassembler xtensainfo" LLVM_FILECHECK="/Users/runner/work/rust/rust/build/aarch64-apple-darwin/ci-llvm/bin/FileCheck" NODE="/opt/homebrew/bin/node" PYTHON="/usr/bin/python3" RUSTC="/Users/runner/work/rust/rust/build/aarch64-apple-darwin/stage2/bin/rustc" RUSTDOC="/Users/runner/work/rust/rust/build/aarch64-apple-darwin/stage2/bin/rustdoc" SOURCE_ROOT="/Users/runner/work/rust/rust" TARGET="aarch64-apple-darwin" TARGET_RPATH_DIR="/Users/runner/work/rust/rust/build/aarch64-apple-darwin/stage2/lib/rustlib/aarch64-apple-darwin/lib" TARGET_RPATH_ENV="/Users/runner/work/rust/rust/build/aarch64-apple-darwin/test/run-make/extern-fn-reachable/rmake_out:/Users/runner/work/rust/rust/build/aarch64-apple-darwin/stage0-bootstrap-tools/aarch64-apple-darwin/release/deps:/Users/runner/work/rust/rust/build/aarch64-apple-darwin/stage0/lib" "/Users/runner/work/rust/rust/build/aarch64-apple-darwin/test/run-make/extern-fn-reachable/rmake"
--- stderr -------------------------------
--- stderr -------------------------------
[/Users/runner/work/rust/rust/tests/run-make/extern-fn-reachable/rmake.rs:8:14] sym = Symbol {
    name: "_fun1",
    address: 16292,
    size: 0,
    section: Section(
        SectionIndex(
            1,
        ),
        ),
    ),
    scope: Dynamic,
    weak: false,
    flags: MachO {
        n_desc: 0,
    },
}
[/Users/runner/work/rust/rust/tests/run-make/extern-fn-reachable/rmake.rs:8:9] dbg!(sym).is_global() = true
[/Users/runner/work/rust/rust/tests/run-make/extern-fn-reachable/rmake.rs:8:41] sym.is_undefined() = false
[/Users/runner/work/rust/rust/tests/run-make/extern-fn-reachable/rmake.rs:8:14] sym = Symbol {
    name: "_fun2",
    address: 16296,
    size: 0,
    section: Section(
        SectionIndex(
            1,
        ),
        ),
    ),
    scope: Dynamic,
    weak: false,
    flags: MachO {
        n_desc: 0,
    },
}
[/Users/runner/work/rust/rust/tests/run-make/extern-fn-reachable/rmake.rs:8:9] dbg!(sym).is_global() = true
[/Users/runner/work/rust/rust/tests/run-make/extern-fn-reachable/rmake.rs:8:41] sym.is_undefined() = false
[/Users/runner/work/rust/rust/tests/run-make/extern-fn-reachable/rmake.rs:8:14] sym = Symbol {
    name: "_fun3",
    address: 16300,
    size: 0,
    section: Section(
        SectionIndex(
            1,
        ),
        ),
    ),
    scope: Dynamic,
    weak: false,
    flags: MachO {
        n_desc: 0,
    },
}
[/Users/runner/work/rust/rust/tests/run-make/extern-fn-reachable/rmake.rs:8:9] dbg!(sym).is_global() = true
[/Users/runner/work/rust/rust/tests/run-make/extern-fn-reachable/rmake.rs:8:41] sym.is_undefined() = false
[/Users/runner/work/rust/rust/tests/run-make/extern-fn-reachable/rmake.rs:8:14] sym = Symbol {
    name: "_fun4",
    address: 16304,
    size: 0,
    section: Section(
        SectionIndex(
            1,
        ),
        ),
    ),
    scope: Dynamic,
    weak: false,
    flags: MachO {
        n_desc: 0,
    },
}
[/Users/runner/work/rust/rust/tests/run-make/extern-fn-reachable/rmake.rs:8:9] dbg!(sym).is_global() = true
[/Users/runner/work/rust/rust/tests/run-make/extern-fn-reachable/rmake.rs:8:41] sym.is_undefined() = false
[/Users/runner/work/rust/rust/tests/run-make/extern-fn-reachable/rmake.rs:8:14] sym = Symbol {
    name: "_fun5",
    address: 16308,
    size: 0,
    section: Section(
        SectionIndex(
            1,
        ),
        ),
    ),
    scope: Dynamic,
    weak: false,
    flags: MachO {
        n_desc: 0,
    },
}
[/Users/runner/work/rust/rust/tests/run-make/extern-fn-reachable/rmake.rs:8:9] dbg!(sym).is_global() = true
[/Users/runner/work/rust/rust/tests/run-make/extern-fn-reachable/rmake.rs:8:41] sym.is_undefined() = false
[/Users/runner/work/rust/rust/tests/run-make/extern-fn-reachable/rmake.rs:8:14] sym = Symbol {
    name: "_rust_metadata_dylib_466e03c52fa389c6",
    address: 16384,
    size: 0,
    kind: Unknown,
        SectionIndex(
            3,
        ),
    ),
    ),
    scope: Dynamic,
    weak: false,
    flags: MachO {
        n_desc: 0,
    },
}
[/Users/runner/work/rust/rust/tests/run-make/extern-fn-reachable/rmake.rs:8:9] dbg!(sym).is_global() = true
[/Users/runner/work/rust/rust/tests/run-make/extern-fn-reachable/rmake.rs:8:41] sym.is_undefined() = false
[/Users/runner/work/rust/rust/tests/run-make/extern-fn-reachable/rmake.rs:8:14] sym = Symbol {
    name: "dyld_stub_binder",
    address: 0,
    size: 0,
    kind: Unknown,
    section: Undefined,
    scope: Unknown,
    weak: false,
    flags: MachO {
        n_desc: 512,
}
}
[/Users/runner/work/rust/rust/tests/run-make/extern-fn-reachable/rmake.rs:8:9] dbg!(sym).is_global() = true
[/Users/runner/work/rust/rust/tests/run-make/extern-fn-reachable/rmake.rs:8:41] sym.is_undefined() = true
dylib.so does not contain symbol(s): 
* fun1
* fun2
* fun3
* fun4
* fun5
missing symbols
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
------------------------------------------

@bors
Copy link
Contributor

bors commented Aug 7, 2024

💔 Test failed - checks-actions

@bors bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 7, 2024
@Oneirical
Copy link
Contributor

Oneirical commented Aug 12, 2024

Symbol names on OSX are listed as _fun4 with a trailing leading underscore. In previous tests, we've been doing the rough and potentially prone to error check of "does this output contain the string fun4". If you want to make this correct, you may consider adding an is_darwin check to your helper function to autoformat symbols.

@lolbinarycat
Copy link
Contributor Author

Symbol names on OSX are listed as _fun4 with a trailing underscore.

do you mean a leading underscore? or do you mean its listed as _fun_?

@Oneirical
Copy link
Contributor

Leading, sorry, it's a typo.

@lolbinarycat
Copy link
Contributor Author

@Oneirical any idea why it does that? is there some piece of documentation saying that's how it works?

it seems odd that a no_mangle symbol would be subject to a form of mangling.

@jieyouxu
Copy link
Member

any idea why it does that? is there some piece of documentation saying that's how it works?

it seems odd that a no_mangle symbol would be subject to a form of mangling.

@lolbinarycat based on https://developer.apple.com/forums/thread/715385

C, and all later languages, add a leading underscore (_) to distinguish their symbols from assembly language symbols.

It might be a macos/apple linker convention that we follow?

Comment on lines +7 to +9
assert_contains_exact_symbols("dylib.so", &["fun1", "fun2", "fun3", "fun4", "fun5"], |sym| {
dbg!(dbg!(sym).is_global()) && !dbg!(sym.is_undefined())
});
Copy link
Member

@jieyouxu jieyouxu Aug 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remark: original Makefile only tested for contains not exact match, I think one of the reasons being the exact naming is different.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm....that still feels pretty fragile to me, maybe we should just add an underscore on macOS

any idea what's going on with the symbol types?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you clarify what you mean by "symbol types"?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: see above, macOS expects dylib symbols to be prefixed with _.

@alex-semenyuk
Copy link
Member

From triage. @lolbinarycat Do you need to address something from above or it's ready for review?

@lolbinarycat
Copy link
Contributor Author

@alex-semenyuk this is mostly blocked on "symbols are really weird on other platforms and i don't fully understand what's going on"

@alex-semenyuk alex-semenyuk added S-blocked Status: Blocked on something else such as an RFC or other implementation work. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Sep 22, 2024
@jieyouxu jieyouxu added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-blocked Status: Blocked on something else such as an RFC or other implementation work. labels Sep 22, 2024
@jieyouxu
Copy link
Member

I need to revisit this and ask the target maintainers and people who know about the symbol situation.

@jieyouxu
Copy link
Member

Dear apple experts, do you know if macOS expects dylib symbol names with an underscore prefix? The closest thing I can find is developer.apple.com/forums/thread/715385 and the sentence "C, and all later languages, add a leading underscore (_) to distinguish their symbols from assembly language symbols." I'm asking as I don't know what is the intended behavior.

@rustbot ping apple

@rustbot rustbot added the O-apple Operating system: Apple (macOS, iOS, tvOS, visionOS, watchOS) label Sep 25, 2024
@rustbot
Copy link
Collaborator

rustbot commented Sep 25, 2024

Hey Apple notification group! This issue or PR could use some Apple-specific
guidance. Could one of you weigh in? Thanks <3

(In case it's useful, here are some instructions for tackling these sorts of
issues).

cc @BlackHoleFox @hkratz @inflation @madsmtm @nvzqz @shepmaster @thomcc

@inflation
Copy link

Dear apple experts, do you know if macOS expects dylib symbol names with an underscore prefix?

Yes. dyld expects anything beyond assembly procedures starts with an underscore prefix.

@jieyouxu
Copy link
Member

jieyouxu commented Sep 25, 2024

Dear apple experts, do you know if macOS expects dylib symbol names with an underscore prefix?

Yes. dyld expects anything beyond assembly procedures starts with an underscore prefix.

Thank you, that's what I expected but I couldn't find relevant "official" docs. Do you know of any official docs or references that describes this? I couldn't find anything in docs for dyld (incl. its man pages) that describes this.

The closest I got is https://opensource.apple.com/source/dyld/dyld-132.13/src/dyldAPIs.cpp.auto.html and

void* dlsym(void* handle, const char* symbolName)
{
	if ( dyld::gLogAPIs )
		dyld::log("%s(%p, %s)\n", __func__, handle, symbolName);

	dlerrorClear();

	const ImageLoader* image;
	const ImageLoader::Symbol* sym;

	// dlsym() assumes symbolName passed in is same as in C source code
	// dyld assumes all symbol names have an underscore prefix

which is almost like "source: I made it up" lol.

///
/// The symbols must also match `pred`.
///
/// The symbol names must match exactly.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: apparently dyld expects dylib symbol names to be prefixed with _, so we should describe this difference.

@@ -193,6 +193,11 @@ impl Rustc {
self
}

/// Make `rustc` prefere dynamic linking
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ping


/// Get a list of symbols that are in `symbol_names` but not the final binary.
///
/// The symbols must also match `pred`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: match pred -> satisfies pred.

.collect();
}

/// Assert that the symbol file contains all of the listed symbols and they all match the given predicate
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: ditto above, we should describe on macOS we expect _ prefix in addition to the non-underscore-prefixed symbol name.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ping

Comment on lines +7 to +9
assert_contains_exact_symbols("dylib.so", &["fun1", "fun2", "fun3", "fun4", "fun5"], |sym| {
dbg!(dbg!(sym).is_global()) && !dbg!(sym.is_undefined())
});
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you clarify what you mean by "symbol types"?

Comment on lines +7 to +9
assert_contains_exact_symbols("dylib.so", &["fun1", "fun2", "fun3", "fun4", "fun5"], |sym| {
dbg!(dbg!(sym).is_global()) && !dbg!(sym.is_undefined())
});
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: see above, macOS expects dylib symbols to be prefixed with _.

@rustbot rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 25, 2024
@jieyouxu
Copy link
Member

@bjorn3 said:

Mach-O states that all exported symbols should have an underscore as prefix. At the same time dlsym will implicitly add it, so outside of compilers, linkers and people writing assembly, nobody needs to be aware of this.

@jieyouxu
Copy link
Member

On macOS we should expect that the dylib symbol begins with an underscore _.

@rustbot author

@bors
Copy link
Contributor

bors commented Oct 8, 2024

☔ The latest upstream changes (presumably #131387) made this pull request unmergeable. Please resolve the merge conflicts.

@JohnCSimon
Copy link
Member

@lolbinarycat

ping from triage - can you post your status on this PR? This PR has not received an update in a few months.

@lolbinarycat
Copy link
Contributor Author

Well, first that was because it was blocked on figuring out what macos is up to, now it's just because i've been busy with other things.

I'll try and finish this up, but if anyone else has more time on their hands and wants to take ownership of this PR, let me know.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-run-make Area: port run-make Makefiles to rmake.rs A-testsuite Area: The testsuite used to check the correctness of rustc O-apple Operating system: Apple (macOS, iOS, tvOS, visionOS, watchOS) S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
Projects
Status: In progress
Development

Successfully merging this pull request may close these issues.

9 participants