[build] Some improvements to the LLVM build system

[giordano]

Elaboration of the issue

After #55180 we implicitly require an LLVM built with Zlib support, but compiling Julia with make USE_BINARYBUILDER_LLVM=0 builds an LLVM without Zlib support, despite the fact we attempt to request it at

julia/deps/llvm.mk

Line 97 in 996351f

LLVM_CMAKE += -DLLVM_ENABLE_ZLIB=ON -DZLIB_LIBRARY="$(build_prefix)/lib"

This was first identified in #55337.

Explanation of how configuration of LLVM failed

ZLIB_LIBRARY must be the path to the zlib library, but we currently set it to the libdir where the library is installed (introduced in #42622):

julia/deps/llvm.mk

Line 97 in 996351f

LLVM_CMAKE += -DLLVM_ENABLE_ZLIB=ON -DZLIB_LIBRARY="$(build_prefix)/lib"

which is wrong. However, CMake is actually able to find Zlib correctly, but then the check at https://github.com/llvm/llvm-project/blob/46425b8d0fac3c529aa4a716d19abd7032e452f3/llvm/cmake/config-ix.cmake#L139-L141 uses the value of ZLIB_LIBRARY to list the Zlib to link for the test, but being ZLIB_LIBRARY a directory, CMake doesn't see any valid Zlib and thus tries to run the test without linking any Zlib, and the test silently fails (they're silent only when LLVM_ENABLE_ZLIB=ON), resulting in no usable Zlib available, even if found.

Proposed solution

ZLIB_ROOT is the only hint recommended by the CMake module FindZLIB. This PR replaces a broken ZLIB_LIBRARY with an appropriate ZLIB_ROOT. Also, we set LLVM_ENABLE_ZLIB=FORCE_ON which is the only way to make CMake fail loudly if no usable Zlib is available, and avoid going on with a non-usable build.

Other comments

I debugged this together with @gbaraldi, thanks!.

I confirm this fixes #55337 for me, it should likely address JuliaCI/julia-buildkite#373 as well.

Also, options COMPILER_RT_ENABLE_IOS, COMPILER_RT_ENABLE_WATCHOS, COMPILER_RT_ENABLE_TVOS, and HAVE_HISTEDIT_H don't exist anymore, and they are removed.

[gbaraldi]

#42622 was done so that the nix build worked as expected so @rbvermaa could you test if this breaks the Nix from source workflow?

[rbvermaa]

#42622 was done so that the nix build worked as expected so @rbvermaa could you test if this breaks the Nix from source workflow?

Ran into another issue with curl, but that seems unrelated, and I will dig into it. LLVM seems to build fine, so should be good to go from our side, thanks for the ping!

[NickCao]

#42622 was done so that the nix build worked as expected so @rbvermaa could you test if this breaks the Nix from source workflow?

Nix build works fine! One issue is libunwind also depends on zlib but is not configured with the correct search path?

[giordano]

Can you provide more information? For libunwind we use autotools, this is the configure call:

julia/deps/unwind.mk

Line 51 in 378f192

$(dir $<)/configure $(CONFIGURE_COMMON) CPPFLAGS="$(CPPFLAGS) $(LIBUNWIND_CPPFLAGS)" CFLAGS="$(CFLAGS) $(LIBUNWIND_CFLAGS)" --enable-shared --disable-minidebuginfo --disable-tests --enable-zlibdebuginfo --disable-conservative-checks --enable-per-thread-cache

and CONFIGURE_COMMON includes the prefix setting:

julia/deps/tools/common.mk

Line 7 in 378f192

CONFIGURE_COMMON = --prefix=$(abspath $(build_prefix)) --build=$(BUILD_MACHINE) --libdir=$(abspath $(build_libdir)) --bindir=$(abspath $(build_depsbindir)) $(CUSTOM_LD_LIBRARY_PATH)

Anyway, this is material for a new issue, it isn't good to keep talking in an unrelated merged PR

[NickCao]

See #55617