Rebuild triggered after touching a (git submodule) file under the registry/src directory

[Ben-PH]

Problem

Initial presentation of bug

In our CI pipeline, one of our dependencies (specifically rocksdb) will run a rebuild, despite being cached.

Steps

I am using rocksdb as a representative

1. Make minimalist hello, world! executable
2. Add rocksdb = "0.17.0" as a dependency.
3. install clang or llvm as per the crate requirements
4. cargo build
   5.a. rm -rf ~/.cargo/registry/src
   5.b. (alternate rebuild trigger) touch /path/to/registry/src/directory/librocksdb-sys-6.20.3/rocksdb
5. run cargo build and observe the rebuild being triggered.

To validate this is unique, I followed the steps with other executables using the rm option, without any issues.

Possible Solution(s)

• given the bug is reproduced with a touch, issue [WIP] mtime+content tracking #8623 could fix it
• issue git submodules are not cached #7987 is another issue relating to caching of a submodule
• issue Revert "Disable preserving mtimes on archives" #7935 relates to mtimes, and may offer some insights.

Notes

The re-build appears to be triggered by this line of code

The CI setup is following the advice outlined here

Version

cargo 1.56.0 (4ed5d137b 2021-10-04)
release: 1.56.0
commit-hash: 4ed5d137baff5eccf1bae5a7b2ae4b57efad4a7d
commit-date: 2021-10-04

[alexcrichton]

You can try seeing why Cargo is rebuilding with:

export CARGO_LOG=cargo::core::compiler::fingerprint=info

and that should print out why the rebuild is happening

[Ben-PH]

[2021-12-29T09:57:37Z INFO  cargo::core::compiler::fingerprint] stale: changed "/home/ben/.cargo/registry/src/github.com-1ecc6299db9ec823/librocksdb-sys-6.20.3/rocksdb/"
[2021-12-29T09:57:37Z INFO  cargo::core::compiler::fingerprint]           (vs) "/home/ben/rust/repro/target/debug/build/librocksdb-sys-66a9853c3d7cfb95/output"
[2021-12-29T09:57:37Z INFO  cargo::core::compiler::fingerprint]                FileTime { seconds: 1640771696, nanos: 604901852 } != FileTime { seconds: 1640771857, nanos: 395909002 }
[2021-12-29T09:57:37Z INFO  cargo::core::compiler::fingerprint] fingerprint error for repro v0.1.0 (/home/ben/rust/repro)/Build/TargetInner { name: "repro", doc: true, ..: with_path("/home/ben/rust/repro/src/main.rs", Edition2021) }
[2021-12-29T09:57:37Z INFO  cargo::core::compiler::fingerprint]     err: current filesystem status shows we're outdated
[2021-12-29T09:57:37Z INFO  cargo::core::compiler::fingerprint] fingerprint error for rocksdb v0.17.0/Build/TargetInner { ..: lib_target("rocksdb", ["lib"], "/home/ben/.cargo/registry/src/github.com-1ecc6299db9ec823/rocksdb-0.17.0/src/lib.rs", Edition2018) }
[2021-12-29T09:57:37Z INFO  cargo::core::compiler::fingerprint]     err: current filesystem status shows we're outdated
[2021-12-29T09:57:37Z INFO  cargo::core::compiler::fingerprint] fingerprint error for librocksdb-sys v6.20.3/Build/TargetInner { ..: lib_target("librocksdb-sys", ["lib"], "/home/ben/.cargo/registry/src/github.com-1ecc6299db9ec823/librocksdb-sys-6.20.3/src/lib.rs", Edition2018) }
[2021-12-29T09:57:37Z INFO  cargo::core::compiler::fingerprint]     err: current filesystem status shows we're outdated
[2021-12-29T09:57:37Z INFO  cargo::core::compiler::fingerprint] fingerprint error for librocksdb-sys v6.20.3/RunCustomBuild/TargetInner { ..: custom_build_target("build-script-build", "/home/ben/.cargo/registry/src/github.com-1ecc6299db9ec823/librocksdb-sys-6.20.3/build.rs", Edition2018) }
[2021-12-29T09:57:37Z INFO  cargo::core::compiler::fingerprint]     err: current filesystem status shows we're outdated

[Ben-PH]

the following in our ci.yml file prevents the rebuild:

    - name: Zero mtimes of deps
      run: |
        set -euxo pipefail
        CARGO_DIR="${CARGO_HOME:-$HOME/.cargo}"
        for source in "$CARGO_DIR"/registry/cache/*/; do
          pushd "$source"
          source=$(basename -- "$PWD")
          mkdir -p "$CARGO_DIR/registry/src/$source"
          for crate in *.crate; do
            crate=${crate%.crate}
            mkdir "$CARGO_DIR/registry/src/$source/$crate" || continue
            tar --strip-components=1 -xzf "$crate.crate" -C "$CARGO_DIR/registry/src/$source/$crate"
            echo -n ok > "$CARGO_DIR/registry/src/$source/$crate/.cargo-ok"
          done
          popd
        done
        find "$CARGO_DIR/registry/src" -exec touch -cfht 197001010000 {} +
        ```

[alexcrichton]

Does the rocksdb build script print out a rerun-if-changed pointing at the directory? If so that may be the cause of this.

[epage]

#11613 has us no longer checking the mtime for "immutable" sources. It looks like that should resolve this, so I'm closing this. If there is a reason to reconsider, let us know!