"recursive" Dependency Causes "section too large" Error When Compiling for wasm

[berkaysynnada]

Describe the bug

I am encountering a problem when trying to compile wasmtest with the following command:

RUSTFLAGS="-C link-arg=--max-memory=4294967296" cargo build --release --target wasm32-unknown-unknown

The build fails with the following error:

error: failed to build archive at `.../target/wasm32-unknown-unknown/release/deps/libpsm-cc498cff4b514c30.rlib`: LLVM error: section too large

The following warnings were emitted during compilation:

warning: [email protected]: warning: ...

I suspect the issue is related to the psm dependency, and it is coming with recursive crate, which was introduced in #13310. I am unsure if this error can be resolved without removing or replacing this dependency.

To Reproduce

Go in datafusion/wasmtest and run cargo build --target wasm32-unknown-unknown

Expected behavior

It needs to compile

Additional context

I tried again after removing recursive dep, and it works successfully

[Eason0729]

I have try to reproduce the bug by either build wasmtest or build stacker with wasm32-unknown-unknown, but was unable to reproduce the issue.

Can you provide more information? (cargo and rustc version...)

[blaginin]

This obviously doesn't resolve the issue, but rustwasm/wasm-pack#1381 (comment) fixed it for me

[berkaysynnada]

I have applied the @blaginin suggestion:

berkaysahin@Berkays-MacBook-Pro wasmtest % clang --version                                        
Homebrew clang version 19.1.4
Target: arm64-apple-darwin23.4.0
Thread model: posix
InstalledDir: /opt/homebrew/Cellar/llvm/19.1.4/bin
Configuration file: /opt/homebrew/etc/clang/arm64-apple-darwin23.cfg

@Eason0729 these are my cargo and rust versions

berkaysahin@Berkays-MacBook-Pro wasmtest % cargo --version                                        
cargo 1.82.0 (8f40fc59f 2024-08-21)
berkaysahin@Berkays-MacBook-Pro wasmtest % rustc --version                                        
rustc 1.82.0 (f6e511eec 2024-10-15)

When I build with: cargo build --target wasm32-unknown-unknown ,
it successfully compiles and generates the binary.

However, in the release mode: cargo build --release --target wasm32-unknown-unknown,

Compiling psm v0.1.24
   Compiling arrow-ipc v53.3.0
   Compiling arrow-json v53.3.0
   Compiling arrow-csv v53.3.0
   Compiling stacker v0.1.17
   Compiling recursive v0.1.1
error: failed to build archive at `/Users/berkaysahin/Desktop/datafusion-upstream/target/wasm32-unknown-unknown/release/deps/libpsm-cc498cff4b514c30.rlib`: LLVM error: section too large

The following warnings were emitted during compilation:

warning: [email protected]: warning: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: archive library: /Users/berkaysahin/Desktop/datafusion-upstream/target/wasm32-unknown-unknown/release/build/psm-d273ebae732f9287/out/libpsm_s.a the table of contents is empty (no object file members in the library define global symbols)

error: could not compile `psm` (lib) due to 1 previous error
warning: build failed, waiting for other jobs to finish...

I don't know why this still refers to Xcode path.

Since wasm-pack build goes with release flag, we need to solve this (assuming I am not the only one)

[Eason0729]

It compiles under 1.82.0 and 1.83.0(didn't use LLVM 15), I think it's LLVM version related.

In addition, I may need some help on how to use specific version of LLVM?

root@73b0a356b106:/stacker/psm# echo $PATH
/usr/lib/llvm-15/bin:/root/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

root@73b0a356b106:/stacker/psm# clang --version
Debian clang version 15.0.6
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/lib/llvm-15/bin

root@73b0a356b106:/stacker/psm# rustc --version --verbose 
rustc 1.83.0 (90b35a623 2024-11-26)
binary: rustc
commit-hash: 90b35a6239c3d8bdabc530a6a0816f7ff89a0aaf
commit-date: 2024-11-26
host: x86_64-unknown-linux-gnu
release: 1.83.0
LLVM version: 19.1.1

[peter-toth]

@berkaysynnada, is this sill an issue? rustwasm/wasm-pack#1381 (comment) seems to work for me too.

% export PATH=/opt/homebrew/Cellar/llvm/19.1.4/bin/:$PATH

% clang --version
Homebrew clang version 19.1.4
Target: arm64-apple-darwin23.6.0
Thread model: posix
InstalledDir: /opt/homebrew/Cellar/llvm/19.1.4/bin
Configuration file: /opt/homebrew/etc/clang/arm64-apple-darwin23.cfg

% rustc --version --verbose
rustc 1.83.0 (90b35a623 2024-11-26)
binary: rustc
commit-hash: 90b35a6239c3d8bdabc530a6a0816f7ff89a0aaf
commit-date: 2024-11-26
host: aarch64-apple-darwin
release: 1.83.0
LLVM version: 19.1.1

% cargo clean
...

% cargo build --release --target wasm32-unknown-unknown
...
   Compiling datafusion-wasmtest v43.0.0 (/Users/ptoth/git/apache/datafusion/datafusion/wasmtest)
    Finished `release` profile [optimized] target(s) in 1m 36s

[berkaysynnada]

That worked for me now (and passed the tests). Perhaps I should have tried with cargo clean 😞

[robtaylor]

I'm still getting this issue. Mac M4 with Sequoia 15.2.

% rustc --version --verbose
rustc 1.84.0 (9fc6b4312 2025-01-07)
binary: rustc
commit-hash: 9fc6b43126469e3858e2fe86cafb4f0fd5068869
commit-date: 2025-01-07
host: aarch64-apple-darwin
release: 1.84.0
LLVM version: 19.1.5

error: failed to build archive at `[cut]/wasm32-unknown-unknown/release/deps/libpsm-5686a1fe6f354cae.rlib`: LLVM error: section too large

warning: [email protected]: warning: /Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: archive library: [cut]wasm32-unknown-unknown/release/build/psm-2da2e2ac984ce6da/out/libpsm_s.a the table of contents is empty (no object file members in the library define global symbols)
error: could not compile `psm` (lib) due to 1 previous error

[Eason0729]

This should works on Mac, but I didn't have one to test.

We should try figuring out the reason. If we couldn't, maybe just make recursive arch-dependant dependencies(which will be another issue to open).