SIGTRAP during compilation on MacOS

[orlp]

Unfortunately I don't have a minimal reproducible example, just a reproducible example. On MacOS Ventura 13.5 on an Apple M2 processor, the following steps reproduce it:

git clone https://github.com/pola-rs/polars --depth 1
cd polars
make build-opt

Said steps cause rustc to exit with a SIGTRAP signal, unfortunately all the way at the end of compilation.

Meta

The above SIGTRAP is a regression, it occurs with channel = "nightly-2024-09-02" in rust-toolchain.toml, as well as with nightly-2024-08-26, but not with nightly-2024-07-26. I have not bisected it further.

The error does not occur when using make build instead of make build-opt. The difference between these two is the profile selected, make build uses the debug profile whereas built-opt uses the following profile:

[profile.debug-release]
inherits = "release"
debug = true
incremental = true
codegen-units = 16
lto = "thin"

Error output

error: could not compile `py-polars` (lib)

Caused by:
  process didn't exit successfully: `/Users/orlp/.rustup/toolchains/nightly-2024-09-02-aarch64-apple-darwin/bin/rustc --crate-name polars --edition=2021 py-polars/src/lib.rs --error-format=json --json=diagnostic-re
ndered-ansi,artifacts,future-incompat --diagnostic-width=214 --crate-type cdylib --emit=dep-info,link -C opt-level=3 -C lto=thin -C codegen-units=16 -C debuginfo=2 -C split-debuginfo=unpacked -C link-arg=-undefined
 -C link-arg=dynamic_lookup -C 'link-args=-Wl,-install_name,@rpath/polars.abi3.so' --cfg 'feature="all"' --cfg 'feature="build_info"' --cfg 'feature="clipboard"' --cfg 'feature="csv"' --cfg 'feature="default"' --cf
g 'feature="ffi_plugin"' --cfg 'feature="ipc"' --cfg 'feature="nightly"' --cfg 'feature="object"' --cfg 'feature="parquet"' --cfg 'feature="polars_cloud"' --cfg 'feature="sql"' --cfg 'feature="trigonometry"' --chec
k-cfg 'cfg(docsrs)' --check-cfg 'cfg(feature, values("all", "array_any_all", "array_count", "asof_join", "avro", "binary_encoding", "build_info", "clipboard", "cloud", "cross_join", "cse", "csv", "cutqcut", "decomp
ress", "default", "dtype-array", "dtype-i16", "dtype-i8", "dtype-u16", "dtype-u8", "dtypes", "extract_groups", "extract_jsonpath", "ffi_plugin", "find_many", "hist", "io", "ipc", "ipc_streaming", "is_in", "json", "
list_any_all", "list_count", "list_drop_nulls", "list_gather", "list_sample", "list_sets", "merge_sorted", "meta", "new_streaming", "nightly", "object", "operations", "optimizations", "parquet", "pct_change", "peak
s", "performant", "pivot", "polars_cloud", "propagate_nans", "regex", "repeat_by", "rle", "search_sorted", "sign", "sql", "streaming", "timezones", "top_k", "trigonometry"))' -C metadata=ea4eeeedc424a3d4 --out-dir
/Users/orlp/programming/rust/polars/target/debug-release/deps -C incremental=/Users/orlp/programming/rust/polars/target/debug-release/incremental -L dependency=/Users/orlp/programming/rust/polars/target/debug-relea
se/deps --extern jemallocator=/Users/orlp/programming/rust/polars/target/debug-release/deps/libjemallocator-ddd1e6f334ef7c27.rlib --extern libc=/Users/orlp/programming/rust/polars/target/debug-release/deps/liblibc-
42e938498c975329.rlib --extern polars_python=/Users/orlp/programming/rust/polars/target/debug-release/deps/libpolars_python-99e3509ba3962109.rlib --extern pyo3=/Users/orlp/programming/rust/polars/target/debug-relea
se/deps/libpyo3-8bd11066bcba163b.rlib -L native=/Users/orlp/programming/rust/polars/target/debug-release/build/jemalloc-sys-b8e69d721500db60/out/build/lib -L native=/Users/orlp/programming/rust/polars/target/debug-
release/build/lz4-sys-e43c007a0372d905/out -L native=/Users/orlp/programming/rust/polars/target/debug-release/build/ring-ff56c916aa4069c8/out -L native=/Users/orlp/programming/rust/polars/target/debug-release/build
/psm-6c1c5b5e6ef90509/out -L native=/Users/orlp/programming/rust/polars/target/debug-release/build/zstd-sys-a95d2f6c0a1c8f6d/out -L native=/Users/orlp/programming/rust/polars/target/debug-release/build/blake3-44100
89b8e8e51f2/out -L native=/Users/orlp/programming/rust/polars/target/debug-release/build/libz-ng-sys-220b3fe3c720fe48/out/lib -L native=/Users/orlp/programming/rust/polars/target/debug-release/build/libz-ng-sys-220
b3fe3c720fe48/out/lib64 --check-cfg 'cfg(allocator, values("default", "mimalloc"))'` (signal: 5, SIGTRAP: trace/breakpoint trap)
💥 maturin failed
  Caused by: Failed to build a native library through cargo
  Caused by: Cargo build finished with "exit status: 101": `env -u CARGO PYO3_ENVIRONMENT_SIGNATURE="cpython-3.11-64bit" PYO3_PYTHON="/Users/orlp/programming/rust/polars/.venv/bin/python" PYTHON_SYS_EXECUTABLE="/Us
ers/orlp/programming/rust/polars/.venv/bin/python" "cargo" "rustc" "--profile" "debug-release" "--message-format" "json-render-diagnostics" "--manifest-path" "/Users/orlp/programming/rust/polars/py-polars/Cargo.tom
l" "--lib" "--" "-C" "link-arg=-undefined" "-C" "link-arg=dynamic_lookup" "-C" "link-args=-Wl,-install_name,@rpath/polars.abi3.so"`
make: *** [build-opt] Error 1

I've tagged this issue as an 'internal compiler error', even though it does not panic but actually SIGTRAPs. I thus don't have a Rust backtrace, but I did attach lldb to rustc before the sigtrap occurred and got the following backtrace:

Backtrace

Process 62117 stopped
* thread #23, name = 'lto polars_parquet-a4434a04818ab16d.el3w6d8nkur84ww5qq4oo2yuv.r', stop reason = EXC_BREAKPOINT (code=1, subcode=0x10f889e50)
    frame #0: 0x000000010f889e50 librustc_driver-5d5f1ff7b417ce42.dylib`llvm::DataLayout::getTypeSizeInBits(llvm::Type*) const
librustc_driver-5d5f1ff7b417ce42.dylib`llvm::DataLayout::getTypeSizeInBits:
->  0x10f889e50 <+460>: brk    #0x1

librustc_driver-5d5f1ff7b417ce42.dylib`std::__1::vector<llvm::Value*, std::__1::allocator<llvm::Value*>>::insert<llvm::Use*>:
    0x10f889e54 <+0>:   stp    x28, x27, [sp, #-0x60]!
    0x10f889e58 <+4>:   stp    x26, x25, [sp, #0x10]
    0x10f889e5c <+8>:   stp    x24, x23, [sp, #0x20]
Target 0: (rustc) stopped.
(lldb) bt
* thread #23, name = 'lto polars_parquet-a4434a04818ab16d.el3w6d8nkur84ww5qq4oo2yuv.r', stop reason = EXC_BREAKPOINT (code=1, subcode=0x10f889e50)
  * frame #0: 0x000000010f889e50 librustc_driver-5d5f1ff7b417ce42.dylib`llvm::DataLayout::getTypeSizeInBits(llvm::Type*) const
    frame #1: 0x000000010e7ae570 librustc_driver-5d5f1ff7b417ce42.dylib`llvm::slpvectorizer::BoUpSLP::computeMinimumValueSizes()
    frame #2: 0x000000010e7b2a84 librustc_driver-5d5f1ff7b417ce42.dylib`llvm::SLPVectorizerPass::vectorizeStoreChain(llvm::ArrayRef<llvm::Value*>, llvm::slpvectorizer::BoUpSLP&, unsigned int, unsigned int, unsigned int&)
    frame #3: 0x000000010e7b44b8 librustc_driver-5d5f1ff7b417ce42.dylib`llvm::SLPVectorizerPass::vectorizeStores(llvm::ArrayRef<llvm::StoreInst*>, llvm::slpvectorizer::BoUpSLP&, llvm::DenseSet<std::__1::tuple<llvm::Value*, llvm::Value*, llvm::Value*, llvm::Value*, unsigned int>, llvm::DenseMapInfo<std::__1::tuple<llvm::Value*, llvm::Value*, llvm::Value*, llvm::Value*, unsigned int>, void>>&)::$_156::operator()(std::__1::set<std::__1::pair<unsigned int, int>, llvm::SLPVectorizerPass::vectorizeStores(llvm::ArrayRef<llvm::StoreInst*>, llvm::slpvectorizer::BoUpSLP&, llvm::DenseSet<std::__1::tuple<llvm::Value*, llvm::Value*, llvm::Value*, llvm::Value*, unsigned int>, llvm::DenseMapInfo<std::__1::tuple<llvm::Value*, llvm::Value*, llvm::Value*, llvm::Value*, unsigned int>, void>>&)::StoreDistCompare, std::__1::allocator<std::__1::pair<unsigned int, int>>> const&) const
    frame #4: 0x000000010e7b3b30 librustc_driver-5d5f1ff7b417ce42.dylib`llvm::SLPVectorizerPass::vectorizeStores(llvm::ArrayRef<llvm::StoreInst*>, llvm::slpvectorizer::BoUpSLP&, llvm::DenseSet<std::__1::tuple<llvm::Value*, llvm::Value*, llvm::Value*, llvm::Value*, unsigned int>, llvm::DenseMapInfo<std::__1::tuple<llvm::Value*, llvm::Value*, llvm::Value*, llvm::Value*, unsigned int>, void>>&)
    frame #5: 0x000000010e7b05d0 librustc_driver-5d5f1ff7b417ce42.dylib`llvm::SLPVectorizerPass::vectorizeStoreChains(llvm::slpvectorizer::BoUpSLP&)
    frame #6: 0x000000010e7afbac librustc_driver-5d5f1ff7b417ce42.dylib`llvm::SLPVectorizerPass::runImpl(llvm::Function&, llvm::ScalarEvolution*, llvm::TargetTransformInfo*, llvm::TargetLibraryInfo*, llvm::AAResults*, llvm::LoopInfo*, llvm::DominatorTree*, llvm::AssumptionCache*, llvm::DemandedBits*, llvm::OptimizationRemarkEmitter*)
    frame #7: 0x000000010e7af614 librustc_driver-5d5f1ff7b417ce42.dylib`llvm::SLPVectorizerPass::run(llvm::Function&, llvm::AnalysisManager<llvm::Function>&)
    frame #8: 0x000000010f960f48 librustc_driver-5d5f1ff7b417ce42.dylib`llvm::PassManager<llvm::Function, llvm::AnalysisManager<llvm::Function>>::run(llvm::Function&, llvm::AnalysisManager<llvm::Function>&)
    frame #9: 0x000000010f963a2c librustc_driver-5d5f1ff7b417ce42.dylib`llvm::ModuleToFunctionPassAdaptor::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&)
    frame #10: 0x000000010f96033c librustc_driver-5d5f1ff7b417ce42.dylib`llvm::PassManager<llvm::Module, llvm::AnalysisManager<llvm::Module>>::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&)
    frame #11: 0x000000010d5ae318 librustc_driver-5d5f1ff7b417ce42.dylib`LLVMRustOptimize
    frame #12: 0x000000010b1d20b4 librustc_driver-5d5f1ff7b417ce42.dylib`rustc_codegen_llvm::back::write::llvm_optimize
    frame #13: 0x000000010b203d04 librustc_driver-5d5f1ff7b417ce42.dylib`rustc_codegen_llvm::back::lto::run_pass_manager
    frame #14: 0x000000010b204af8 librustc_driver-5d5f1ff7b417ce42.dylib`rustc_codegen_llvm::back::lto::optimize_thin_module
    frame #15: 0x000000010b1c5ca4 librustc_driver-5d5f1ff7b417ce42.dylib`std::sys::backtrace::__rust_begin_short_backtrace::<<rustc_codegen_llvm::LlvmCodegenBackend as rustc_codegen_ssa::traits::backend::ExtraBackendMethods>::spawn_named_thread<rustc_codegen_ssa::back::write::spawn_work<rustc_codegen_llvm::LlvmCodegenBackend>::{closure#0}, ()>::{closure#0}, ()>
    frame #16: 0x000000010b1c9f7c librustc_driver-5d5f1ff7b417ce42.dylib`<<std::thread::Builder>::spawn_unchecked_<<rustc_codegen_llvm::LlvmCodegenBackend as rustc_codegen_ssa::traits::backend::ExtraBackendMethods>::spawn_named_thread<rustc_codegen_ssa::back::write::spawn_work<rustc_codegen_llvm::LlvmCodegenBackend>::{closure#0}, ()>::{closure#0}, ()>::{closure#1} as core::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
    frame #17: 0x000000010d4e5404 librustc_driver-5d5f1ff7b417ce42.dylib`std::sys::pal::unix::thread::Thread::new::thread_start::h4746909631f3e791
    frame #18: 0x000000018b05bfa8 libsystem_pthread.dylib`_pthread_start

[nikic]

@DianQK As this is on M2, can you please take a look?

[DianQK]

Reduced:

target datalayout = "e-m:o-i64:64-i128:128-n32:64-S128-Fn32"
target triple = "arm64-apple-macosx11.0.0"

define void @"_ZN91_$LT$core..slice..iter..Iter$LT$T$GT$$u20$as$u20$core..iter..traits..iterator..Iterator$GT$4fold17h482f4bde4fc3470aE"(ptr %0, i64 %1, i64 %2) personality ptr null {
  br label %4

4:                                                ; preds = %"_ZN4core4iter8adapters3map8map_fold28_$u7b$$u7b$closure$u7d$$u7d$17he00eb30a769dd1bfE.exit", %3
  %5 = phi i64 [ %spec.select.i2.i.i, %"_ZN4core4iter8adapters3map8map_fold28_$u7b$$u7b$closure$u7d$$u7d$17he00eb30a769dd1bfE.exit" ], [ 0, %3 ]
  %spec.select.i.i.i1 = phi i64 [ %spec.select.i.i.i, %"_ZN4core4iter8adapters3map8map_fold28_$u7b$$u7b$closure$u7d$$u7d$17he00eb30a769dd1bfE.exit" ], [ 0, %3 ]
  %6 = invoke ptr null(ptr null, ptr null)
          to label %"_ZN4core4iter8adapters3map8map_fold28_$u7b$$u7b$closure$u7d$$u7d$17he00eb30a769dd1bfE.exit" unwind label %7

7:                                                ; preds = %4
  %8 = landingpad { ptr, i32 }
          cleanup
  %9 = getelementptr i8, ptr %0, i64 48
  %10 = getelementptr i8, ptr %0, i64 40
  store i64 %spec.select.i.i.i1, ptr %10, align 8
  store i64 %5, ptr %9, align 8
  resume { ptr, i32 } zeroinitializer

"_ZN4core4iter8adapters3map8map_fold28_$u7b$$u7b$closure$u7d$$u7d$17he00eb30a769dd1bfE.exit": ; preds = %4
  %trunc.i.i.i = trunc i64 %1 to i1
  %spec.select.i.i.i = select i1 %trunc.i.i.i, i64 40, i64 0
  %trunc.i1.i.i = trunc i64 %5 to i1
  %spec.select.i2.i.i = select i1 %trunc.i1.i.i, i64 0, i64 %2
  br label %4
}

https://llvm.godbolt.org/z/P58q1ozsP

If this reduction is too aggressive, I'll try again tomorrow. :)

[DianQK]

Upstream issue: llvm/llvm-project#107036.

[DianQK]

Fixed in LLVM 19 (Probably the next backport).
@rustbot label +llvm-fixed-upstream

[workingjubilee]

Even though this is basically-already-fixed, we need to figure out if this is on the 1.82 beta (it has to be, right?)

[workingjubilee]

Wait, I can just do math. Yes, it is.

[DianQK]

I cannot reproduce this because this requires the nightly version, but the regression has included in 1.82 beta.

The LLVM 19.1.0 final release is planned for Sep 3rd. The rustc 1.82 stable release will be on Oct 17th. #127513

I would expect to integrate the final 19.1.0 version of LLVM to rustc 1.82, although this contains many changes.

[apiraino]

WG-prioritization assigning priority (Zulip discussion).

@rustbot label -I-prioritize +P-high

[nikic]

Yeah, we'll backport at least the 19.1.0 release to the beta branch, possibly also a later point release if needed.

[lqd]

It's not super important but the regression looks to be from #128371.

[workingjubilee]

then reverting that PR is also an option, but I will of course defer to you on exactly what gets backported, just wanted to make sure this was tagged appropriately so we didn't lose track. :^)