Add missing package files #195

Jake-Shadle · 2024-05-17T10:08:34Z

Resolves: #194

Byron · 2024-05-17T13:48:18Z

Thanks for your help!

Submodules are checked out on CI, yet the hardcoded file seems to be missing. I checked and it's available locally, strangely enough.

When trying the package commands locally, I also see two failures, both with zlib-ng and zlib-ng-cc. The build of zlib-ng-cc also fails for me, but works for zlib-ng.

I have a feeling there is a CWD difference between what cargo package does and what cargo build does - maybe one should use environment variables to get the manifest-root instead of relying on the CWD.

Here are my logs for completeness.

Logs

❯ cargo package --no-default-features --features zlib-ng-no-cmake-experimental-community-maintained
   Packaging libz-sys v1.1.18 (/Users/byron/dev/github.com/rust-lang/libz-sys)
   Verifying libz-sys v1.1.18 (/Users/byron/dev/github.com/rust-lang/libz-sys)
    Updating crates.io index
   Compiling cc v1.0.97
   Compiling libc v0.2.153
   Compiling pkg-config v0.3.30
   Compiling vcpkg v0.2.15
   Compiling libz-sys v1.1.18 (/Users/byron/dev/github.com/rust-lang/libz-sys/target/package/libz-sys-1.1.18)
The following warnings were emitted during compilation:

warning: [email protected]: Compiler family detection failed due to error: IOError: No such file or directory (os error 2)

error: failed to run custom build command for `libz-sys v1.1.18 (/Users/byron/dev/github.com/rust-lang/libz-sys/target/package/libz-sys-1.1.18)`

Caused by:
  process didn't exit successfully: `/Users/byron/dev/github.com/rust-lang/libz-sys/target/package/libz-sys-1.1.18/target/debug/build/libz-sys-7889a033ca7521fc/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-env-changed=LIBZ_SYS_STATIC
  cargo:rerun-if-changed=build.rs
  cargo:rerun-if-changed=zng/cmake.rs
  cargo:rerun-if-changed=zng/cc.rs
  OPT_LEVEL = Some("0")
  TARGET = Some("aarch64-apple-darwin")
  HOST = Some("aarch64-apple-darwin")
  cargo:rerun-if-env-changed=CC_aarch64-apple-darwin
  CC_aarch64-apple-darwin = None
  cargo:rerun-if-env-changed=CC_aarch64_apple_darwin
  CC_aarch64_apple_darwin = None
  cargo:rerun-if-env-changed=HOST_CC
  HOST_CC = None
  cargo:rerun-if-env-changed=CC
  CC = None
  cargo:warning=Compiler family detection failed due to error: IOError: No such file or directory (os error 2)
  cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
  CRATE_CC_NO_DEFAULTS = None
  DEBUG = Some("true")
  CARGO_CFG_TARGET_FEATURE = Some("aes,crc,dit,dotprod,dpb,dpb2,fcma,fhm,flagm,fp16,frintts,jsconv,lor,lse,neon,paca,pacg,pan,pmuv3,ras,rcpc,rcpc2,rdm,sb,sha2,sha3,ssbs,vh")
  cargo:rerun-if-env-changed=CC_ENABLE_DEBUG_OUTPUT
  cargo:rerun-if-env-changed=CFLAGS_aarch64-apple-darwin
  CFLAGS_aarch64-apple-darwin = None
  cargo:rerun-if-env-changed=CFLAGS_aarch64_apple_darwin
  CFLAGS_aarch64_apple_darwin = None
  cargo:rerun-if-env-changed=HOST_CFLAGS
  HOST_CFLAGS = None
  cargo:rerun-if-env-changed=CFLAGS
  CFLAGS = None

  --- stderr
  thread 'main' panicked at zng/cc.rs:65:10:
  called `Result::unwrap()` on an `Err` value: "failed to read \"src/zlib-ng/gzread.c.in\": No such file or directory (os error 2)"
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
error: failed to verify package tarball

libz-sys ( EmbarkStudios/main:refs/pull/195/head) [$]
❯  cargo package --no-default-features --features zlib-ng
   Packaging libz-sys v1.1.18 (/Users/byron/dev/github.com/rust-lang/libz-sys)
   Verifying libz-sys v1.1.18 (/Users/byron/dev/github.com/rust-lang/libz-sys)
    Updating crates.io index
   Compiling cc v1.0.97
   Compiling libc v0.2.153
   Compiling vcpkg v0.2.15
   Compiling pkg-config v0.3.30
   Compiling cmake v0.1.50
   Compiling libz-sys v1.1.18 (/Users/byron/dev/github.com/rust-lang/libz-sys/target/package/libz-sys-1.1.18)
error: failed to run custom build command for `libz-sys v1.1.18 (/Users/byron/dev/github.com/rust-lang/libz-sys/target/package/libz-sys-1.1.18)`

Caused by:
  process didn't exit successfully: `/Users/byron/dev/github.com/rust-lang/libz-sys/target/package/libz-sys-1.1.18/target/debug/build/libz-sys-0303a1ebf4c2e3d8/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-env-changed=LIBZ_SYS_STATIC
  cargo:rerun-if-changed=build.rs
  cargo:rerun-if-changed=zng/cmake.rs
  cargo:rerun-if-changed=zng/cc.rs
  CMAKE_TOOLCHAIN_FILE_aarch64-apple-darwin = None
  CMAKE_TOOLCHAIN_FILE_aarch64_apple_darwin = None
  HOST_CMAKE_TOOLCHAIN_FILE = None
  CMAKE_TOOLCHAIN_FILE = None
  CMAKE_GENERATOR_aarch64-apple-darwin = None
  CMAKE_GENERATOR_aarch64_apple_darwin = None
  HOST_CMAKE_GENERATOR = None
  CMAKE_GENERATOR = None
  CMAKE_PREFIX_PATH_aarch64-apple-darwin = None
  CMAKE_PREFIX_PATH_aarch64_apple_darwin = None
  HOST_CMAKE_PREFIX_PATH = None
  CMAKE_PREFIX_PATH = None
  CMAKE_aarch64-apple-darwin = None
  CMAKE_aarch64_apple_darwin = None
  HOST_CMAKE = None
  CMAKE = None
  running: cd "/Users/byron/dev/github.com/rust-lang/libz-sys/target/package/libz-sys-1.1.18/target/debug/build/libz-sys-db60ce2f2aa8146d/out/build" && CMAKE_PREFIX_PATH="" "cmake" "/Users/byron/dev/github.com/rust-lang/libz-sys/target/package/libz-sys-1.1.18/src/zlib-ng" "-DCMAKE_OSX_ARCHITECTURES=arm64" "-DBUILD_SHARED_LIBS=OFF" "-DZLIB_COMPAT=ON" "-DZLIB_ENABLE_TESTS=OFF" "-DWITH_GZFILEOP=ON" "-DCMAKE_INSTALL_PREFIX=/Users/byron/dev/github.com/rust-lang/libz-sys/target/package/libz-sys-1.1.18/target/debug/build/libz-sys-db60ce2f2aa8146d/out" "-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -fPIC --target=arm64-apple-darwin -mmacosx-version-min=14.5" "-DCMAKE_C_COMPILER=/usr/bin/cc" "-DCMAKE_CXX_FLAGS= -ffunction-sections -fdata-sections -fPIC --target=arm64-apple-darwin -mmacosx-version-min=14.5" "-DCMAKE_CXX_COMPILER=/usr/bin/c++" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC --target=arm64-apple-darwin -mmacosx-version-min=14.5" "-DCMAKE_ASM_COMPILER=/usr/bin/cc" "-DCMAKE_BUILD_TYPE=Debug"

  --- stderr
  CMake Warning:
    Ignoring extra path from command line:

     "/Users/byron/dev/github.com/rust-lang/libz-sys/target/package/libz-sys-1.1.18/src/zlib-ng"


  CMake Error: The source directory "/Users/byron/dev/github.com/rust-lang/libz-sys/target/package/libz-sys-1.1.18/src/zlib-ng" does not appear to contain CMakeLists.txt.
  Specify --help for usage, or press the help button on the CMake GUI.
  thread 'main' panicked at /Users/byron/.cargo/registry/src/index.crates.io-6f17d22bba15001f/cmake-0.1.50/src/lib.rs:1098:5:

  command did not execute successfully, got: exit status: 1

  build script failed, must exit now
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
error: failed to verify package tarball

libz-sys ( EmbarkStudios/main:refs/pull/195/head) [$] took 2s
❯ l src/zlib-ng/CMakeLists.txt

libz-sys ( EmbarkStudios/main:refs/pull/195/head) [$]
❯ cargo build
   Compiling libc v0.2.153
   Compiling cc v1.0.90
   Compiling vcpkg v0.2.15
   Compiling pkg-config v0.3.30
   Compiling libz-sys v1.1.18 (/Users/byron/dev/github.com/rust-lang/libz-sys)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.78s

libz-sys ( EmbarkStudios/main:refs/pull/195/head) [$]
❯ cargo build  --all-features
   Compiling cmake v0.1.50
   Compiling libz-sys v1.1.18 (/Users/byron/dev/github.com/rust-lang/libz-sys)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.93s

libz-sys ( EmbarkStudios/main:refs/pull/195/head) [$]
❯

libz-sys ( EmbarkStudios/main:refs/pull/195/head) [$]
❯ cargo build  --no-default-features --features zlib-ng-no-cmake-experimental-community-maintained
   Compiling libz-sys v1.1.18 (/Users/byron/dev/github.com/rust-lang/libz-sys)
The following warnings were emitted during compilation:

warning: [email protected]: In file included from src/zlib-ng/arch/arm/adler32_neon.c:9:
warning: [email protected]: src/zlib-ng/arch/arm/neon_intrins.h:31:28: error: expected identifier or '('
warning: [email protected]: static inline uint16x8x4_t vld1q_u16_x4(uint16_t const *a) {
warning: [email protected]:                            ^
warning: [email protected]: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/15.0.0/include/arm_neon.h:10524:28: note: expanded from macro 'vld1q_u16_x4'
warning: [email protected]: #define vld1q_u16_x4(__p0) __extension__ ({ \
warning: [email protected]:                            ^
warning: [email protected]: In file included from src/zlib-ng/arch/arm/adler32_neon.c:9:
warning: [email protected]: src/zlib-ng/arch/arm/neon_intrins.h:40:28: error: expected identifier or '('
warning: [email protected]: static inline uint8x16x4_t vld1q_u8_x4(uint8_t const *a) {
warning: [email protected]:                            ^
warning: [email protected]: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/15.0.0/include/arm_neon.h:10467:27: note: expanded from macro 'vld1q_u8_x4'
warning: [email protected]: #define vld1q_u8_x4(__p0) __extension__ ({ \
warning: [email protected]:                           ^
warning: [email protected]: In file included from src/zlib-ng/arch/arm/adler32_neon.c:9:
warning: [email protected]: src/zlib-ng/arch/arm/neon_intrins.h:49:20: error: expected identifier or '('
warning: [email protected]: static inline void vst1q_u16_x4(uint16_t *p, uint16x8x4_t a) {
warning: [email protected]:                    ^
warning: [email protected]: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/15.0.0/include/arm_neon.h:28042:34: note: expanded from macro 'vst1q_u16_x4'
warning: [email protected]: #define vst1q_u16_x4(__p0, __p1) __extension__ ({ \
warning: [email protected]:                                  ^
warning: [email protected]: 3 errors generated.

error: failed to run custom build command for `libz-sys v1.1.18 (/Users/byron/dev/github.com/rust-lang/libz-sys)`

Caused by:
  process didn't exit successfully: `/Users/byron/dev/github.com/rust-lang/libz-sys/target/debug/build/libz-sys-6de51d426d3975ac/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-env-changed=LIBZ_SYS_STATIC
  cargo:rerun-if-changed=build.rs
  cargo:rerun-if-changed=zng/cmake.rs
  cargo:rerun-if-changed=zng/cc.rs
  OPT_LEVEL = Some("0")
  TARGET = Some("aarch64-apple-darwin")
  HOST = Some("aarch64-apple-darwin")
  cargo:rerun-if-env-changed=CC_aarch64-apple-darwin
  CC_aarch64-apple-darwin = None
  cargo:rerun-if-env-changed=CC_aarch64_apple_darwin
  CC_aarch64_apple_darwin = None
  cargo:rerun-if-env-changed=HOST_CC
  HOST_CC = None
  cargo:rerun-if-env-changed=CC
  CC = None
  cargo:rerun-if-env-changed=CC_ENABLE_DEBUG_OUTPUT
  cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
  CRATE_CC_NO_DEFAULTS = None
  DEBUG = Some("true")
  cargo:rerun-if-env-changed=CFLAGS_aarch64-apple-darwin
  CFLAGS_aarch64-apple-darwin = None
  cargo:rerun-if-env-changed=CFLAGS_aarch64_apple_darwin
  CFLAGS_aarch64_apple_darwin = None
  cargo:rerun-if-env-changed=HOST_CFLAGS
  HOST_CFLAGS = None
  cargo:rerun-if-env-changed=CFLAGS
  CFLAGS = None
  cargo:warning=In file included from src/zlib-ng/arch/arm/adler32_neon.c:9:
  cargo:warning=src/zlib-ng/arch/arm/neon_intrins.h:31:28: error: expected identifier or '('
  cargo:warning=static inline uint16x8x4_t vld1q_u16_x4(uint16_t const *a) {
  cargo:warning=                           ^
  cargo:warning=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/15.0.0/include/arm_neon.h:10524:28: note: expanded from macro 'vld1q_u16_x4'
  cargo:warning=#define vld1q_u16_x4(__p0) __extension__ ({ \
  cargo:warning=                           ^
  cargo:warning=In file included from src/zlib-ng/arch/arm/adler32_neon.c:9:
  cargo:warning=src/zlib-ng/arch/arm/neon_intrins.h:40:28: error: expected identifier or '('
  cargo:warning=static inline uint8x16x4_t vld1q_u8_x4(uint8_t const *a) {
  cargo:warning=                           ^
  cargo:warning=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/15.0.0/include/arm_neon.h:10467:27: note: expanded from macro 'vld1q_u8_x4'
  cargo:warning=#define vld1q_u8_x4(__p0) __extension__ ({ \
  cargo:warning=                          ^
  cargo:warning=In file included from src/zlib-ng/arch/arm/adler32_neon.c:9:
  cargo:warning=src/zlib-ng/arch/arm/neon_intrins.h:49:20: error: expected identifier or '('
  cargo:warning=static inline void vst1q_u16_x4(uint16_t *p, uint16x8x4_t a) {
  cargo:warning=                   ^
  cargo:warning=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/15.0.0/include/arm_neon.h:28042:34: note: expanded from macro 'vst1q_u16_x4'
  cargo:warning=#define vst1q_u16_x4(__p0, __p1) __extension__ ({ \
  cargo:warning=                                 ^
  cargo:warning=3 errors generated.

  --- stderr
  ToolExecError: Command env -u IPHONEOS_DEPLOYMENT_TARGET "cc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-gdwarf-2" "-fno-omit-frame-pointer" "--target=arm64-apple-darwin" "-mmacosx-version-min=14.5" "-std=c11" "-I" "/Users/byron/dev/github.com/rust-lang/libz-sys/target/debug/build/libz-sys-cf968d9e59cebf55/out/include" "-I" "src/zlib-ng" "-Werror-implicit-function-declaration" "-fvisibility=hidden" "-march=armv8-a+crc+simd" "-DZLIB_COMPAT" "-DWITH_GZFILEOP" "-DHAVE_ALIGNED_ALLOC" "-DHAVE_ATTRIBUTE_ALIGNED" "-DHAVE_BUILTIN_CTZ" "-DHAVE_BUILTIN_CTZLL" "-DHAVE_THREAD_LOCAL" "-DHAVE_VISIBILITY_HIDDEN" "-DHAVE_VISIBILITY_INTERNAL" "-D_LARGEFILE64_SOURCE=1" "-D__USE_LARGEFILE64" "-DSTDC" "-D_POSIX_SOURCE" "-DHAVE_POSIX_MEMALIGN" "-D_C99_SOURCE" "-DARM_FEATURES" "-DARM_ACLE" "-DHAVE_ARM_ACLE_H" "-DARM_NEON" "-o" "/Users/byron/dev/github.com/rust-lang/libz-sys/target/debug/build/libz-sys-cf968d9e59cebf55/out/lib/1b0507d0191b23d7-adler32_neon.o" "-c" "src/zlib-ng/arch/arm/adler32_neon.c" with args "cc" did not execute successfully (status code exit status: 1).
  thread 'main' panicked at zng/cc.rs:439:9:
  failed to compile zlib-ng with cc: detected compiler version as
  ---
  Apple clang version 15.0.0 (clang-1500.3.9.4)
  Target: arm64-apple-darwin23.4.0
  Thread model: posix
  InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
  ---
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

libz-sys ( EmbarkStudios/main:refs/pull/195/head) [$]
❯ cargo build  --no-default-features --features zlib-ng
   Compiling libz-sys v1.1.18 (/Users/byron/dev/github.com/rust-lang/libz-sys)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 5.01s

Jake-Shadle · 2024-05-17T15:49:16Z

Opened a PR on cc to fix the issue I found when debugging the packaging, but for now the workaround works. rust-lang/cc-rs#1072

Byron

Thanks so much, great work! I will create a new release ASAP.

zng/cc.rs

Co-authored-by: Sebastian Thiel <[email protected]>

Byron · 2024-05-18T10:53:09Z

Thank you! It's very strange that CI is now failing, it's probably flakiness due to the nightly GNU compiler, but at least today it reproduces when retrying.
Maybe the easiest is to try again tomorrow hoping it's fixed by then.

Byron · 2024-05-18T11:56:04Z

Maybe it's related to this change? The timing would match: https://blog.rust-lang.org/2024/05/17/enabling-rust-lld-on-linux.html

NobodyXu · 2024-05-19T08:38:43Z

cc 1.0.98 has released

Jake-Shadle · 2024-05-19T20:11:31Z

Unfortunately the documented method of disabling either isn't functioning or there is something else going on. See also cross-rs/cross#1496. It might be related to the LLD change, but if it is there is something wrong as targets other than x86_64-unknown-linux-gnu are also affected, which shouldn't be the case according to the post.

Byron · 2024-05-20T07:40:58Z

I see, thanks for letting me know. Maybe a solution arises in the coming days/weeks, so we can finish this fix and finally bring CC-zlib-ng to the people :).

Jake-Shadle · 2024-05-21T14:04:35Z

Upstream rust issue rust-lang/rust#125330

Jake-Shadle · 2024-05-24T14:19:55Z

The nightly upstream fix works and I removed the cc workaround.

Byron · 2024-05-24T15:05:45Z

Fantastic, let me create a new build right away, let's see if it works.

Jake-Shadle added 4 commits May 17, 2024 12:08

Add missing package files

fc0aba6

Add CI check for packaging

e8b8465

Bump version/CI

34e6e40

Oops

66b68f5

Fix packaging issues

9c2bf78

Byron approved these changes May 17, 2024

View reviewed changes

zng/cc.rs Outdated Show resolved Hide resolved

Note PR

530b1a5

Co-authored-by: Sebastian Thiel <[email protected]>

ipetkov mentioned this pull request May 18, 2024

libz-sys 1.1.7 fails to build with current crane ipetkov/crane#613

Closed

Try force disabling lld on nightly

85afe16

Jake-Shadle force-pushed the main branch from 2de5c2e to 85afe16 Compare May 19, 2024 08:06

Oops

3e1c547

Jake-Shadle added 2 commits May 21, 2024 16:11

maybe this works

3ffe600

Use updated cc

e3d7095

Byron merged commit 414ca0a into rust-lang:main May 24, 2024
45 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add missing package files #195

Add missing package files #195

Jake-Shadle commented May 17, 2024

Byron commented May 17, 2024

Jake-Shadle commented May 17, 2024

Byron left a comment

Byron commented May 18, 2024

Byron commented May 18, 2024 •

edited

Loading

NobodyXu commented May 19, 2024

Jake-Shadle commented May 19, 2024

Byron commented May 20, 2024

Jake-Shadle commented May 21, 2024

Jake-Shadle commented May 24, 2024

Byron commented May 24, 2024

Add missing package files #195

Add missing package files #195

Conversation

Jake-Shadle commented May 17, 2024

Byron commented May 17, 2024

Jake-Shadle commented May 17, 2024

Byron left a comment

Choose a reason for hiding this comment

Byron commented May 18, 2024

Byron commented May 18, 2024 • edited Loading

NobodyXu commented May 19, 2024

Jake-Shadle commented May 19, 2024

Byron commented May 20, 2024

Jake-Shadle commented May 21, 2024

Jake-Shadle commented May 24, 2024

Byron commented May 24, 2024

Byron commented May 18, 2024 •

edited

Loading