ld64.lld: error: too many personalities (4) for compact unwind to encode

[glandium]

(Largely a copy/paste from rust-lang/rust#102754)

When mixing ObjC, C++ and Rust in the same binary on aarch64-apple-darwin targets (e.g. like in Firefox), lld from LLVM 15 complains about "too many personalities". This does not happen with lld from LLVM 14 because AIUI it had limited support for compact unwind info (or no support at all). This does not happen with ld64.

For some reason, since rust-lang/rust#92845, this code in lld is hit:

llvm-project/lld/MachO/UnwindInfoSection.cpp

Line 300 in d30dccd

s = make<Defined>("<internal>", /*file=*/nullptr, referentIsec,

Ultimately, this leads to

llvm-project/lld/MachO/UnwindInfoSection.cpp

Line 378 in d30dccd

if (personalities.size() > 3)

where there are 4 personalities instead of 3:

• ___objc_personality_v0,
• ___gxx_personality_v0,
• _rust_eh_personality
• <internal>

With previous rust versions, only the first three ones would be there. It looks like the relocation that triggers the <internal> is related to the rust_eh_personality function in library/std/src/personality/gcc.rs itself.

STR:

• Create a bar.cc file with content:

#include <stdexcept>

extern "C" void qux();

void bar() { qux(); throw std::runtime_error("foo"); }

• Create a foo.m file with content:

void foo() { @try {} @finally {} }

• Create a qux.rs file with content:

#![crate_type = "staticlib"]

extern "C" {
    fn foo();
}

#[no_mangle]
pub unsafe extern "C" fn qux() {
    println!("foo");
    foo();
}

• Create a Makefile file with content:

TARGET = --target=aarch64-apple-darwin

lib.dylib: bar.o foo.o libqux.a
	clang++ -o $@ $^ -fuse-ld=lld -dynamiclib $(TARGET) -lobjc

bar.o: bar.cc
	clang++ -o $@ -c $^ $(TARGET)

foo.o: foo.m
	clang -o $@ -c $^ $(TARGET)

libqux.a: qux.rs
	rustc $^ $(TARGET)

• Get clang 15 + lld and make sure it's in your $PATH.
• Get rustc beta and make sure it's in your $PATH.
• Set SDKROOT to point to a macOS SDK
• Run make

This does not happen on x86_64-apple-darwin, but interestingly the list of personalities there is:

• ___objc_personality_v0,
• ___gxx_personality_v0,
• <internal>
  (no _rust_eh_personality).

[llvmbot]

@llvm/issue-subscribers-lld-macho

[int3]

Interesting. I believe @oontvoo saw something similar recently too.

ld64 also emits 3 personality entries for arm64. It appears that it links in _rust_eh_personality as a local ("internal") indirect symbol, and does not emit a separate _rust_eh_personality entry. The question then is why we aren't recognizing our <local> and _rust_eh_personality entries as equivalent...

[oontvoo]

I think this also needs a longer term "fix". That is, 2-bit (but 0 is already reserved) isn't going to be enough. (There are at least 3 common personalities: ___gxx_personality_v0, ___gcc_personality_v0, ___objc_personality_v0 already. Now with rust, it's 4)

[int3]

@oontvoo I don't think ld64 supports encoding 4 (distinct) personalities either, so we're fine on that front. Simple repro: https://gist.github.com/int3/7235eb5a7266b3712b055805e3cea634

[int3]

Okay I figured this out. Our personality-dedup logic doesn't account for personalities encoded in EH frames. That also explains why it wasn't an issue in LLVM 14 -- we already had support for compact unwinds then, but not for EH frames. So the personalities in the EH frames were ignored.

[int3]

https://reviews.llvm.org/D135728

[int3]

Also note that the initial test case likely didn't repro the error on x86_64-apple-darwin because different relocations were chosen by MC. Without section relocs in compact unwind, this issue will not repro. However, it is nonetheless possible to construct inputs for x86_64 that cause it to emit those relocs and trigger this issue. In fact, the test in the aforementioned diff targets x86_64.

[int3]

/cherry-pick 7b45dfc

[llvmbot]

/cherry-pick 7b45dfc

Error: Command failed due to missing milestone.

[int3]

/cherry-pick 7b45dfc

[llvmbot]

/branch llvm/llvm-project-release-prs/issue58277

[llvmbot]

/pull-request llvm/llvm-project-release-prs#188

[glandium]

Should the PR be added to the next milestone, since it didn't make the cut for 15.0.3?

[int3]

@tstellar how can I make sure this gets into the next patch release? I'd thought creating the PR would be enough...

[int3]

Reopening so we don't miss pulling this into 15.0.4 :)

[smeenai]

CC @tru, who's been running this release. 15.0.4 wasn't planned originally but we're getting it now: https://discourse.llvm.org/t/llvm-15-0-4-release-schedule-update/66099

[tru]

CC @tru, who's been running this release. 15.0.4 wasn't planned originally but we're getting it now: https://discourse.llvm.org/t/llvm-15-0-4-release-schedule-update/66099

Yep thanks - I was chatting with @int3 over at discord :)