CMake - TEST_BIG_ENDIAN

[boaks]

Describe the bug

Adding

include(TestBigEndian)
TEST_BIG_ENDIAN(IS_BIG_ENDIAN)

to the CMakeLists.txtfails. It reports:

-- Check if the system is big endian
-- Searching 16 bit integer
-- Check size of unsigned short
CMake Error at /usr/share/cmake-3.23/Modules/CheckTypeSize.cmake:146 (try_compile):
Cannot copy output executable

''

to destination specified by COPY_FILE:

'zephyr/coaps-zephyr/coaps-zephyr/build/CMakeFiles/CheckTypeSize/CMAKE_SIZEOF_UNSIGNED_SHORT.bin'

Unable to find the executable at any of:

zephyr/coaps-zephyr/coaps-zephyr/build/CMakeFiles/CMakeTmp/cmTC_13b71.elf
zephyr/coaps-zephyr/coaps-zephyr/build/CMakeFiles/CMakeTmp/Debug/cmTC_13b71.elf
zephyr/coaps-zephyr/coaps-zephyr/build/CMakeFiles/CMakeTmp/Development/cmTC_13b71.elf

Call Stack (most recent call first):
/usr/share/cmake-3.23/Modules/CheckTypeSize.cmake:277 (__check_type_size_impl)
/usr/share/cmake-3.23/Modules/TestBigEndian.cmake:55 (CHECK_TYPE_SIZE)
/usr/share/cmake-3.23/Modules/TestBigEndian.cmake:37 (__TEST_BIG_ENDIAN_LEGACY_IMPL)
zephyr/coaps-zephyr/modules/lib/tinydtls/zephyr/AutoConf.cmake:49 (TEST_BIG_ENDIAN)
zephyr/coaps-zephyr/modules/lib/tinydtls/zephyr/CMakeLists.txt:27 (include)

-- Check size of unsigned int
CMake Error at /usr/share/cmake-3.23/Modules/CheckTypeSize.cmake:146 (try_compile):
Cannot copy output executable

''

to destination specified by COPY_FILE:

'zephyr/coaps-zephyr/coaps-zephyr/build/CMakeFiles/CheckTypeSize/CMAKE_SIZEOF_UNSIGNED_INT.bin'

Unable to find the executable at any of:

zephyr/coaps-zephyr/coaps-zephyr/build/CMakeFiles/CMakeTmp/cmTC_6e8ef.elf
zephyr/coaps-zephyr/coaps-zephyr/build/CMakeFiles/CMakeTmp/Debug/cmTC_6e8ef.elf
zephyr/coaps-zephyr/coaps-zephyr/build/CMakeFiles/CMakeTmp/Development/cmTC_6e8ef.elf

Call Stack (most recent call first):
/usr/share/cmake-3.23/Modules/CheckTypeSize.cmake:277 (__check_type_size_impl)
/usr/share/cmake-3.23/Modules/TestBigEndian.cmake:60 (CHECK_TYPE_SIZE)
/usr/share/cmake-3.23/Modules/TestBigEndian.cmake:37 (__TEST_BIG_ENDIAN_LEGACY_IMPL)
zephyr/coaps-zephyr/modules/lib/tinydtls/zephyr/AutoConf.cmake:49 (TEST_BIG_ENDIAN)
zephyr/coaps-zephyr/modules/lib/tinydtls/zephyr/CMakeLists.txt:27 (include)

To Reproduce

Add

include(TestBigEndian)
TEST_BIG_ENDIAN(IS_BIG_ENDIAN)

to the CMakeLists.txt. Execute

$ west build

Expected behavior

TEST_BIG_ENDIAN(IS_BIG_ENDIAN) sets IS_BIG_ENDIAN according the used processor.

Impact

IS_BIG_ENDIAN must be set manually,

Environment (please complete the following information):

• Ubuntu 18.04
• ncs 1.9.1

[stephanosio]

This seems to be more of a bug in the CMake itself than the Zephyr build system.

Determining if the system is big endian failed with the following output:
Change Dir: /home/stephanos/Dev/zephyrproject/zephyr/samples/hello_world/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/ninja cmTC_5e2a9 && [1/2] Building C object CMakeFiles/cmTC_5e2a9.dir/TestEndianess.c.obj
[2/2] Linking C executable cmTC_5e2a9
FAILED: cmTC_5e2a9
: && /opt/zephyr-sdk-0.14.1/arm-zephyr-eabi/bin/arm-zephyr-eabi-gcc   CMakeFiles/cmTC_5e2a9.dir/TestEndianess.c.obj -o cmTC_5e2a9   && :
/opt/zephyr-sdk-0.14.1/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/10.3.0/../../../../arm-zephyr-eabi/bin/ld: /opt/zephyr-sdk-0.14.1/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/10.3.0/../../../../arm-zephyr-eabi/lib/libc.a(lib_a-exit.o): in function `exit':
exit.c:(.text.exit+0x34): undefined reference to `_exit'
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed

The compilation of TestEndianness.c fails because the toolchain, without any "required arguments," defaults to linking some libraries that expect certain symbols (e.g. _exit in case of arm-zephyr-eabi) to be defined by the application.

Note that the CMake TEST_BIG_ENDIAN implementation calls try_compile without any arguments, causing some required flags like -nostdlib to be not passed when compiling TestEndianness.c:

     try_compile(HAVE_${VARIABLE}
      "${CMAKE_BINARY_DIR}"
      ${_test_file}
      OUTPUT_VARIABLE OUTPUT
      COPY_FILE "${CMAKE_BINARY_DIR}${CMAKE_FILES_DIRECTORY}/TestEndianess.bin" )

For instance, the compilation of CMAKE_SIZEOF_UNSIGNED_SHORT.c, also built as part of the TEST_BIG_ENDIAN function, succeeds because the CHECK_TYPE_SIZE function calls try_compile with the CMAKE_REQUIRED_FLAGS:

  try_compile(HAVE_${var} ${CMAKE_BINARY_DIR} ${src}
    COMPILE_DEFINITIONS ${CMAKE_REQUIRED_DEFINITIONS}
    LINK_OPTIONS ${CMAKE_REQUIRED_LINK_OPTIONS}
    LINK_LIBRARIES ${CMAKE_REQUIRED_LIBRARIES}
    CMAKE_FLAGS
      "-DCOMPILE_DEFINITIONS:STRING=${CMAKE_REQUIRED_FLAGS}"
...

Note that -nostdlib is added to CMAKE_REQUIRED_FLAGS here:

zephyr/cmake/compiler/gcc/target.cmake

Line 121 in 04836af

-nostdlib

Applying the following patch to the CMake TestBigEndian.cmake makes this function run successfully:

--- /usr/share/cmake-3.22/Modules/TestBigEndian.cmake   2022-01-25 22:45:06.000000000 +0900
+++ cmaketest/Modules/TestBigEndian.cmake       2022-05-02 17:45:40.429774864 +0900
@@ -91,6 +91,9 @@
      try_compile(HAVE_${VARIABLE}
       "${CMAKE_BINARY_DIR}"
       ${_test_file}
+      CMAKE_FLAGS
+        "-DCOMPILE_DEFINITIONS:STRING=${CMAKE_REQUIRED_FLAGS}"
+        "-DINCLUDE_DIRECTORIES:STRING=${CMAKE_REQUIRED_INCLUDES}"
       OUTPUT_VARIABLE OUTPUT
       COPY_FILE "${CMAKE_BINARY_DIR}${CMAKE_FILES_DIRECTORY}/TestEndianess.bin" )

Output after applying the patch:

-- The C compiler identification is GNU 10.3.0
-- The CXX compiler identification is GNU 10.3.0
-- The ASM compiler identification is GNU
-- Found assembler: /opt/zephyr-sdk-0.14.1/arm-zephyr-eabi/bin/arm-zephyr-eabi-gcc
-- Check if the system is big endian
-- Searching 16 bit integer
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of unsigned short
-- Check size of unsigned short - done
-- Searching 16 bit integer - Using unsigned short
-- Check if the system is big endian - little endian
-- Configuring done
-- Generating done

Of course, the upstream CMake developers may argue that it is the toolchain's fault for not being able to produce an executable as-is without some special arguments (i.e. -nostdlib), which we can certainly address for the Zephyr SDK; but, other third-party GNU toolchains such as the GNU Arm Embedded also has this problem, and it is not without a good reason (i.e. the toolchain is configured such that it uses the newlib by default, which expects _exit to be defined either by the libgloss or the application).

Nevertheless, apart from the questionable toolchain issues, it is true that CMake has an inconsistency in its internal compiler checks that the CMAKE_REQUIRED_INCLUDES flags are used for some checks while they are not for the rest.

When @tejlmand (or @SebastianBoe) comes back, he can further comment on this and maybe file a bug report to the upstream CMake repository.

[boaks]

FMPOV, the interesting part of this issue seems to be, that the tool-chain is usually used for "cross-compiling". So I'm not sure, if that is a working solution at all, to compile & execute the test.

Maybe, it's possible to provide an other approach, where somewhere in the board definition (or target arch), this could be provided and returned, instead of trying to do the test above.

[stephanosio]

So I'm not sure, if that is a working solution at all, to compile & execute the test.

It only compiles and does not execute the test, so it can and does (as shown above) work for cross compilers.

Note that it literally looks for a string in a compiled binary, without attempting to execute it:
https://github.com/Kitware/CMake/blob/8dbf38eaaf0d465a7b44e337204c3b763915ebb5/Modules/TestBigEndian.cmake#L99-L103

[boaks]

Thanks, then I was wrong, mislead by the "executable" in the error logs.

[boaks]

I tried to apply your patch. Doing so, I found TestBigEndian.cmake - deprecated:: 3.20

Supserseded by the :variable:CMAKE_<LANG>_BYTE_ORDER variable.

I will try that. Maybe that works without patch.

Edited:
CMake 3.23:
Seems, that if CMAKE_<LANG>_BYTE_ORDER is defined, TEST_BIG_ENDIAN(IS_BIG_ENDIAN) will use it.
On my toolchain, it's unfortunately not defined, therefore the test is executed. I patched the "TestBigEndian.cmake" according your proposal, but I still get the same errors.

[tejlmand]

Thanks for raising this.

There are a couple of reasons why this doesn't work out of the box with Zephyr.
First part is that Zephyr doesn't rely on CMakes infrastructure for testing the compiler due to a couple of different reasons.

Secondly, even if trying to use the new endianess detecting for CMake >=3.20, then the next issue that arises is that CMake default in Zephyr context tries to compile for an executable, which fails as we haven't generated a valid linker script for the compiler.

Therefore one must set STATIC_LIBRARY as compile target type.

I have a working hack that correctly detects the endianess with no patching to CMake, as can be seen here:

-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
endianess (test_big_endian)   : 0
endianess (CMAKE_C_BYTE_ORDER): LITTLE_ENDIAN

going to clean up the hack and find a proper location for the code.
So stay tuned for a upcomming PR.

This will mean that both the CMake CMAKE_C_BYTE_ORDER or the test_big_endian can be used to fetch the endianess.

[boaks]

Just to mention:
It seems to be also different, if that is used for the application (library) itself, or for a module (library).
At least that's my final understanding/experience with #45840 (see my last comment).

[github-actions]

This issue has been marked as stale because it has been open (more than) 60 days with no activity. Remove the stale label or add a comment saying that you would like to have the label removed otherwise this issue will automatically be closed in 14 days. Note, that you can always re-open a closed issue at any time.