[C++] Build failure due to orc_ep in IDE

[llama90]

Describe the bug, including details regarding any error messages, version, and platform.

• GH-40507: [C++][ORC] Upgrade ORC to 2.0.0 #40508

Since the addition of the PR, the following error occurs in IDE(e.g., CLion):

I verified the git commit while checking it out.

error message

[27/901] Performing configure step for 'orc_ep'
FAILED: orc_ep-prefix/src/orc_ep-stamp/orc_ep-configure /Users/lama/workspace/arrow-new/cpp/cmake-build-debug/orc_ep-prefix/src/orc_ep-stamp/orc_ep-configure 
cd /Users/lama/workspace/arrow-new/cpp/cmake-build-debug/orc_ep-prefix/src/orc_ep-build && /Applications/CLion.app/Contents/bin/cmake/mac/aarch64/bin/cmake -P /Users/lama/workspace/arrow-new/cpp/cmake-build-debug/orc_ep-prefix/src/orc_ep-stamp/orc_ep-configure-DEBUG.cmake && /Applications/CLion.app/Contents/bin/cmake/mac/aarch64/bin/cmake -E touch /Users/lama/workspace/arrow-new/cpp/cmake-build-debug/orc_ep-prefix/src/orc_ep-stamp/orc_ep-configure
CMake Error at /Users/lama/workspace/arrow-new/cpp/cmake-build-debug/orc_ep-prefix/src/orc_ep-stamp/orc_ep-configure-DEBUG.cmake:37 (message):
  Command failed: 1

   '/Applications/CLion.app/Contents/bin/cmake/mac/aarch64/bin/cmake' '-DCMAKE_C_COMPILER=/Library/Developer/CommandLineTools/usr/bin/cc' '-DCMAKE_CXX_COMPILER=/Library/Developer/CommandLineTools/usr/bin/c++' '-DCMAKE_AR=/Library/Developer/CommandLineTools/usr/bin/ar' '-DCMAKE_RANLIB=/Library/Developer/CommandLineTools/usr/bin/ranlib' '-DBUILD_SHARED_LIBS=OFF' '-DBUILD_STATIC_LIBS=ON' '-DBUILD_TESTING=OFF' '-DCMAKE_BUILD_TYPE=DEBUG' '-DCMAKE_CXX_FLAGS= -Qunused-arguments -fcolor-diagnostics -fPIC' '-DCMAKE_CXX_FLAGS_DEBUG=-g -Werror -O0 -ggdb   -Wno-error' '-DCMAKE_CXX_FLAGS_MISIZEREL=-Os -DNDEBUG ' '-DCMAKE_CXX_FLAGS_RELEASE=-O3 -DNDEBUG -O2  ' '-DCMAKE_CXX_FLAGS_RELWITHDEBINFO=-O2 -g -DNDEBUG -ggdb  ' '-DCMAKE_CXX_STANDARD=17' '-DCMAKE_C_FLAGS= -Qunused-arguments -fPIC' '-DCMAKE_C_FLAGS_DEBUG=-g -Werror -O0 -ggdb   -Wno-error' '-DCMAKE_C_FLAGS_MISIZEREL=-Os -DNDEBUG ' '-DCMAKE_C_FLAGS_RELEASE=-O3 -DNDEBUG -O2  ' '-DCMAKE_C_FLAGS_RELWITHDEBINFO=-O2 -g -DNDEBUG -ggdb  ' '-DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=' '-DCMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY=' '-DCMAKE_INSTALL_LIBDIR=lib' '-DCMAKE_OSX_SYSROOT=/Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk' '-DCMAKE_VERBOSE_MAKEFILE=FALSE' '-DCMAKE_C_COMPILER_LAUNCHER=/opt/homebrew/bin/ccache' '-DCMAKE_CXX_COMPILER_LAUNCHER=/opt/homebrew/bin/ccache' '-DCMAKE_INSTALL_PREFIX=/Users/lama/workspace/arrow-new/cpp/cmake-build-debug/orc_ep-install' '-DSTOP_BUILD_ON_WARNING=OFF' '-DBUILD_LIBHDFSPP=OFF' '-DBUILD_JAVA=OFF' '-DBUILD_TOOLS=OFF' '-DBUILD_CPP_TESTS=OFF' '-DINSTALL_VENDORED_LIBS=OFF' '-DLZ4_HOME=/Users/lama/workspace/arrow-new/cpp/cmake-build-debug/lz4_ep-install' '-DPROTOBUF_EXECUTABLE=/Users/lama/workspace/arrow-new/cpp/cmake-build-debug/protobuf_ep-install/bin/protoc' '-DPROTOBUF_HOME=/Users/lama/workspace/arrow-new/cpp/cmake-build-debug/protobuf_ep-install' '-DPROTOBUF_INCLUDE_DIR=/Users/lama/workspace/arrow-new/cpp/cmake-build-debug/protobuf_ep-install/include' '-DPROTOBUF_LIBRARY=/Users/lama/workspace/arrow-new/cpp/cmake-build-debug/protobuf_ep-install/lib/libprotobuf.a' '-DPROTOC_LIBRARY=/Users/lama/workspace/arrow-new/cpp/cmake-build-debug/protobuf_ep-install/lib/libprotoc.a' '-DSNAPPY_HOME=/Users/lama/workspace/arrow-new/cpp/cmake-build-debug/snappy_ep/src/snappy_ep-install' '-DSNAPPY_LIBRARY=/Users/lama/workspace/arrow-new/cpp/cmake-build-debug/snappy_ep/src/snappy_ep-install/lib/libsnappy.a' '-DLZ4_LIBRARY=/Users/lama/workspace/arrow-new/cpp/cmake-build-debug/lz4_ep-install/lib/liblz4.a' '-DLZ4_STATIC_LIBRARY=/Users/lama/workspace/arrow-new/cpp/cmake-build-debug/lz4_ep-install/lib/liblz4.a' '-DLZ4_INCLUDE_DIR=/Users/lama/workspace/arrow-new/cpp/cmake-build-debug/lz4_ep-install/include' '-DSNAPPY_INCLUDE_DIR=/Users/lama/workspace/arrow-new/cpp/cmake-build-debug/snappy_ep/src/snappy_ep-install/include' '-DZSTD_HOME=/Users/lama/workspace/arrow-new/cpp/cmake-build-debug/zstd_ep-install' '-DZSTD_INCLUDE_DIR=/Users/lama/workspace/arrow-new/cpp/cmake-build-debug/zstd_ep-install/include' '-DZSTD_LIBRARY=/Users/lama/workspace/arrow-new/cpp/cmake-build-debug/zstd_ep-install/lib/libzstd.a' '-GNinja' '-S' '/Users/lama/workspace/arrow-new/cpp/cmake-build-debug/orc_ep-prefix/src/orc_ep' '-B' '/Users/lama/workspace/arrow-new/cpp/cmake-build-debug/orc_ep-prefix/src/orc_ep-build'

  See also

    /Users/lama/workspace/arrow-new/cpp/cmake-build-debug/orc_ep-prefix/src/orc_ep-stamp/orc_ep-configure-*.log

orc_ep-configure-*.log

�[31mCMake Error at /opt/homebrew/lib/cmake/protobuf/protobuf-targets.cmake:71 (set_target_properties):
  The link interface of target "protobuf::libprotobuf" contains:

    absl::absl_check

  but the target was not found.  Possible reasons include:

    * There is a typo in the target name.
    * A find_package call is missing for an IMPORTED target.
    * An ALIAS target is missing.

Call Stack (most recent call first):
  /opt/homebrew/lib/cmake/protobuf/protobuf-config.cmake:16 (include)
  cmake_modules/FindProtobuf.cmake:35 (find_package)
  cmake_modules/ThirdpartyToolchain.cmake:417 (find_package)
  CMakeLists.txt:166 (INCLUDE)

�[0m
�[0mCMake Generate step failed.  Build files cannot be regenerated correctly.�[0m

I have no issues building from the terminal, but the environment within the IDE breaks, making it inconvenient for me to personally review the code.

The build succeeds when I execute it excluding the contents of that commit.

Component(s)

C++

[kou]

I'll explain details tomorrow but here are workarounds:

• Uninstall Protobuf installed by Homebrew
• Use Protobuf installed by Homebrew not bundled Protobuf

Cc: @wgtmac

[llama90]

@kou You are awesome.

I've spent a lot of time trying to solve the problem. 🫠

[wgtmac]

@kou Thanks for pinging me!

@llama90 Did you use the default CMake presets? How can I reproduce this?

[llama90]

Sure, it is my preset.

--preset ninja-debug-maximal \
-G Ninja \
-DCMAKE_INSTALL_PREFIX=build \
-DARROW_AZURE:BOOL=OFF \
-DARROW_CUDA=OFF \
-DARROW_SKYHOOK=OFF \
-DARROW_GANDIVA=ON \
-DARROW_DEPENDENCY_SOURCE=BUNDLED \
-DARROW_BUILD_STATIC=ON \
-DARROW_JEMALLOC=ON \
-DARROW_MIMALLOC=OFF

[wgtmac]

Oh, I recall that I also ran into similar issue when I upgraded the libprotobuf in Homebrew which has bundled absl.

[kou]

The reason why this is happen:

The case that we use bundled Protobuf: We want ORC to use the same bundled Protobuf. But ORC tried system Protobuf before PROTOBUF_INCLUDE_DIR/PROTOBUF_LIBRARY/PROTOC_LIBRARY are checked.
See also: https://github.com/apache/orc/blob/d4f13dc284fc12b7ff109493652473faec8724d3/cmake_modules/FindProtobuf.cmake#L35

But I don't know why absl::absl_check couldn't found... It seems that protobuf-config.cmake finds Abseil:

cat $(brew --prefix)/lib/cmake/protobuf/protobuf-config.cmake
# User options
include("${CMAKE_CURRENT_LIST_DIR}/protobuf-options.cmake")

# Depend packages
if(NOT ZLIB_FOUND)
  find_package(ZLIB)
endif()
if(NOT TARGET absl::strings)
  find_package(absl CONFIG)
endif()
if(NOT TARGET utf8_range)
  find_package(utf8_range CONFIG)
endif()

# Imported targets
include("${CMAKE_CURRENT_LIST_DIR}/protobuf-targets.cmake")

# protobuf-generate function
include("${CMAKE_CURRENT_LIST_DIR}/protobuf-generate.cmake")

# CMake FindProtobuf module compatible file
if(protobuf_MODULE_COMPATIBLE)
  include("${CMAKE_CURRENT_LIST_DIR}/protobuf-module.cmake")
endif()

[wgtmac]

But ORC tried system Protobuf before PROTOBUF_INCLUDE_DIR/PROTOBUF_LIBRARY/PROTOC_LIBRARY are checked.

It seems that this can be fixed by not searching system config if ${PROTOBUF_HOME} is set.

But I don't know why absl::absl_check couldn't found... It seems that protobuf-config.cmake finds Abseil:

When I ran into a similar but different issue, I found my Homebrew has installed a higher version of absl than the bundled protobuf and errors are reported for symbols not found.

[kou]

It seems that this can be fixed by not searching system config if ${PROTOBUF_HOME} is set.

ORC may want to support all of (1) "find a dependency by conan", (2) "find a dependency by CMake package" and (3) "find a dependency by user input parameters". (1) and (2) are already supported but (3) is missing. (FYI: Arrow supports all of them.)

When I ran into a similar but different issue, I found my Homebrew has installed a higher version of absl than the bundled protobuf and errors are reported for symbols not found.

Wow. How did you solve it? brew upgrade?

[wgtmac]

(1) and (2) are already supported but (3) is missing.

Actually (3) is supported but it seems to be broken by https://github.com/apache/orc/pull/1529/files

How did you solve it?

I added Protobuf_SOURCE=BUNDLED;absl_SOURCE=BUNDLED env to build Arrow.

[kou]

Actually (3) is supported but it seems to be broken by https://github.com/apache/orc/pull/1529/files

Oh, I see. You're right.

How did you solve it?

I added Protobuf_SOURCE=BUNDLED;absl_SOURCE=BUNDLED env to build Arrow.

Thanks. Your case was for Arrow. I thought your case was for ORC too.

[wgtmac]

FYI, Apache ORC is having a hard time to upgrade Protobuf due to libhdfspp: apache/orc#1857.