Disabled optional weak dependencies end up in `Cargo.lock` #10801

jonasbb · 2022-06-30T17:31:31Z

Problem

Lets say I have two crates, with these contents of Cargo.toml. The Rust code does not matter here.

# foo/Cargo.toml

[dependencies]
serde = {version = "1.0.133", optional = true, default-features = false}
time = {version = "=0.3.10", optional = true, default-features = false}
chrono = {version = "=0.4.18", optional = true, default-features = false}

[features]
serialization = ["dep:serde", "time?/serde-well-known"]

# bar/Cargo.toml

[dependencies]
foo.features = ["serialization"]
foo.path = "../foo"

# time = "=0.3.11"
# chrono = "=0.4.19"

As you can see, bar depends on foo and enables the serialization feature, but it does not enable the time nor the chrono feature.

This results in the following bar/Cargo.lock file:

bar/Cargo.lock

# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
version = 3

[[package]]
name = "app"
version = "0.1.0"
dependencies = [
 "foo",
]

[[package]]
name = "foo"
version = "0.1.0"
dependencies = [
 "serde",
 "time",
]

[[package]]
name = "itoa"
version = "1.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "112c678d4050afce233f4f2852bb2eb519230b3cf12f33585275537d7e41578d"

[[package]]
name = "libc"
version = "0.2.126"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "349d5a591cd28b49e1d1037471617a32ddcda5731b99419008085f72d5a53836"

[[package]]
name = "num_threads"
version = "0.1.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2819ce041d2ee131036f4fc9d6ae7ae125a3a40e97ba64d04fe799ad9dabbb44"
dependencies = [
 "libc",
]

[[package]]
name = "serde"
version = "1.0.137"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "61ea8d54c77f8315140a05f4c7237403bf38b72704d031543aa1d16abbf517d1"

[[package]]
name = "time"
version = "0.3.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "82501a4c1c0330d640a6e176a3d6a204f5ec5237aca029029d21864a902e27b0"
dependencies = [
 "itoa",
 "libc",
 "num_threads",
 "serde",
]

The relevant snippet is at the end:

[[package]]
name = "time"
version = "0.3.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "82501a4c1c0330d640a6e176a3d6a204f5ec5237aca029029d21864a902e27b0"
dependencies = [
 "itoa",
 "libc",
 "num_threads",
 "serde",
]

Here the dependency time is declared, even though it is an optional dependency and nothing enabled the corresponding feature on foo. The time dependency even has serde listed as one of its dependencies too.

I think this is problematic in a couple of ways.

It bloats the Cargo.lock and makes it imply that changes are larger than they actually are. In the above example adding foo will also add time even if that is not necessary.

Dependency management tools will get confused by this. I mean tools like dependabot or cargo audit, which use the Cargo.lock as source of truth. Since these tools will see an entry for time, even if it is unused, they will create pull requests or warn about security vulnerabilities which do not exists.

This prevents using two crates with conflicting version requirements. You can see what happens if you uncomment the lines in bar/Cargo.toml.
Uncommenting time will fail, since now two conflicting version constraints (=0.3.10 != =0.3.11) exist.
However, uncommenting chrono works with the conflicting versions (=0.4.18 != =0.4.19).

Steps

Create project foo with the above content for Cargo.toml.
Create project bar with the above content for Cargo.toml.
Observe how the Cargo.lock of bar contains an entry for time and even a transitive serde dependency in time.

Possible Solution(s)

Disabled optional dependencies should not be tracked in Cargo.lock, even if there is a weak dependency linking to it. This brings it in line with "normal" optional dependencies.

Notes

While the time entry exists in the Cargo.lock, it appears cargo correctly ignores it when building the code. There is neither a Compiling time ... log line nor are there any artifacts in the target folder associated with time.

I checked various issues and documentation entries, but I couldn't find if this is a bug or intended.

The tracking issue (#8832) and documentation (https://doc.rust-lang.org/nightly/cargo/reference/features.html#dependency-features) do not mention any changes in to Cargo.lock. Neither does the implementation PR (#8818), which does not even seem to have any lock files.

Version

cargo 1.62.0 (a748cf5a3 2022-06-08)
release: 1.62.0
commit-hash: a748cf5a3e666bc2dcdf54f37adef8ef22196452
commit-date: 2022-06-08
host: x86_64-unknown-linux-gnu
libgit2: 1.4.2 (sys:0.14.2 vendored)
libcurl: 7.80.0-DEV (sys:0.4.51+curl-7.80.0 vendored ssl:OpenSSL/1.1.1n)
os: OracleLinux 36.0.0 [64-bit]


FYI: My OS is Fedora 36, but this is the output of `cargo version --verbose`.

The text was updated successfully, but these errors were encountered:

Jake-Shadle · 2022-07-11T08:42:04Z

Note that this bug also affects the output of the metadata subcommand, but not the tree subcommand. Weak dependencies are resolved in the resolve.nodes map even if they haven't been enabled by the crate feature.

cargo metadata resolves all dependencies even ones that are disabled (known as weak dependencies) which cargo build doesn't do (see issue rust-lang/cargo#10801). These unresolved dependencies require network access to load their package metadata. As the `--frozen` flag itself blocks network access (in addition to blocking changes to the Cargo.lock) too, we need to remove both the `--frozen` and `--offline` flags. Doing this allows cargo the change the `Cargo.lock` though, which is undesirable (as we're in the process of building the same dependencies that rely on the `Cargo.lock` in the first place).

trevor-crypto · 2022-08-23T13:26:37Z

I believe I am running into this with target-specific dependencies:

[target.'cfg(target_arch = "wasm32")'.dependencies]
wasm-bindgen = "=0.2.78"

I noticed this when there was a version conflict (since I already had the dependency in the lock file from another dependency).

cargo metadata resolves all dependencies even ones that are disabled (known as weak dependencies) which cargo build doesn't do (see issue rust-lang/cargo#10801). These unresolved dependencies require network access to load their package metadata. As the `--frozen` flag itself blocks network access (in addition to blocking changes to the Cargo.lock) too, we need to remove both the `--frozen` and `--offline` flags. Doing this allows cargo the change the `Cargo.lock` though, which is undesirable (as we're in the process of building the same dependencies that rely on the `Cargo.lock` in the first place).

lopopolo · 2022-09-08T04:15:04Z

I'm not sure but I think I ran into this issue in:

bstr pulls in serde when serde feature is not specified BurntSushi/bstr#130

I found it unexpected that serde appears in the lockfile since I am specifically enabling and disabling features such that I avoid pulling in serde. I was extra confused that serde does not appear in cargo tree.

ljedrz · 2022-10-13T14:23:33Z

I've noticed this as well when cargo update caused the lockfile to balloon more than I expected; it was caused by one of the indirect dependencies having introduced a [target.'cfg(target_os = "haiku")'.dependencies] section.

lopopolo · 2022-10-13T15:53:43Z

@ljedrz that behavior is expected. If you're talking about iana-time-zone that dependency is not optional. It is a required dependency that is only used on a specific platform. Cargo adds all required dependencies to the lockfile even if they are cfg'd.

It's the same reason winapi appears in the lockfile even if you are building on Linux.

ljedrz · 2022-10-13T16:00:46Z

@lopopolo indeed, it's that dependency; however, since I'm not building on an applicable target, I'd expect it to not show up in my lockfile just as much as if it were an optional (and disabled) dependency.

Most of the new dependencies are false positive and are not actually required, but are false dependencies due to a Cargo bug. rust-lang/cargo#10801

A note: this adds serde to the lockfile, but it is not actually used. This is caused by rust-lang/cargo#10801.

Sytten · 2022-11-18T13:01:38Z

This also happens with sea-orm which pulls chrono and time when you use the with-json flag.
https://github.com/SeaQL/sea-orm/blob/master/Cargo.toml#L75
This is really wrong.

memoryruins · 2022-11-18T18:19:24Z

@jonasbb

I checked various issues and documentation entries, but I couldn't find if this is a bug or intended.

I believe this is a known limitation of the lock file generation that is intended to be improved eventually. From #5655 (comment),

lockfile generation requires the assumption that all possible features are enabled. Generating a lockfile without optional features is covered by #5133

While #5133 mentions --minimal-cargo-lock, it doesn't appear to be implemented yet. This should also help with offline usage #10352 and platform specific vendoring #7058 (similarly, target specific is brought up in #11313)

From #7058 (comment),

Cargo currently requires that the lock file is platform-independent, meaning that the lock file (and generating the lock file) always has to locate all crates for all possible platforms. [...] There's a tracking issue for this at #5133 which is a possible way to pass a flag to Cargo and say "please generate a minimal lock file", and with that we can safely delete all the dependencies for other platforms.

jonasbb · 2022-11-18T18:43:41Z

@memoryruins

I checked various issues and documentation entries, but I couldn't find if this is a bug or intended.

I believe this is a known limitation of the lock file generation that is intended to be improved eventually. From #5655 (comment),

This only applies to the root crate (the one you are working on) but not the dependencies. When you depend on serde without extra features, then the optional serde_derive and any proc-macro dependencies are not included, since the feature is not enabled.

Cargo.toml

[package]
name = "foobar"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
serde = "1.0.147"

Cargo.lock

# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
version = 3

[[package]]
name = "foobar"
version = "0.1.0"
dependencies = [
 "serde",
]

[[package]]
name = "serde"
version = "1.0.147"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d193d69bae983fc11a79df82342761dfbf28a99fc8d203dca4c3c1b590948965"

From #7058 (comment),

Cargo currently requires that the lock file is platform-independent, meaning that the lock file (and generating the lock file) always has to locate all crates for all possible platforms. [...] There's a tracking issue for this at #5133 which is a possible way to pass a flag to Cargo and say "please generate a minimal lock file", and with that we can safely delete all the dependencies for other platforms.

This issue is not comparable to platform dependent Cargo.lock. The issue presented here covers dependencies which are never used, on no platform and also not if using --all-features.

In my original posts in the bar/Cargo.lock snippet, you find chrono to be missing, while time is included. Here you see again that optional dependencies are already excluded from Cargo.lock. The only difference between these two is the existence of the weak dependency time?/serde-well-known. So I would put the blame here on the implementation of weak features. But as explained in the original post, I could find no comments on if these Cargo.lock changes are intentional or not.

memoryruins · 2022-11-18T19:46:36Z

@jonasbb

I understand what you mean with weak features now. I'll see if I can find anything mentioned in the past.

This issue is not comparable to platform dependent Cargo.lock.

Apologies for any confusion, I forgot to add an @ mention to the platform specific part of my comment, which was meant to be in response to @ljedrz #10801 (comment)

since I'm not building on an applicable target, I'd expect it to not show up in my lockfile just as much as if it were an optional (and disabled) dependency.

Sytten · 2022-11-18T22:22:55Z

The issue with platforms should be be a separate issue, this is really for the issue where a dependency is truly never used but still included. Let's keep it on topic 👍

memoryruins · 2022-11-19T04:05:08Z

@Sytten

The issue with platforms should be be a separate issue

Indeed, I linked to those earlier in #10801 (comment) so that related discussion can move there.

lilith · 2024-08-22T23:56:13Z

Soo... ring isn't in my 'cargo tree', but it is still trying and failing to compile. I think this bug has worsened.

weihanglo · 2024-08-23T00:02:57Z

Soo... ring isn't in my 'cargo tree', but it is still trying and failing to compile. I think this bug has worsened.

@lilith Optional dependencies shouldn't compile if not activated. It might be something enabling ring and cargo-tree has a bug not reporting it. Please open a separate issue with reproduction and we can investigate then.

Caused by rust-lang/cargo#10801

ErichDonGubler · 2024-10-04T17:15:13Z

The Firefox WebGPU team is running into this issue transitively because of once_cell's optional dependency on the sizable portable-atomic crate (see matklad/once_cell#265 (comment). This is particularly painful for us because of a strict cargo vet policy for dependencies in the Mozilla monorepo.

CC @jimb.

epage · 2024-10-04T17:16:45Z

Would that be a bug to fix in cargo vet?

jimblandy · 2024-10-04T17:36:06Z

@epage We've presented this to the cargo vet maintainers, and they said:

This isn't us, the dependency edge appears in cargo metadata
{
  "id": "registry+https://github.com/rust-lang/crates.io-index#[email protected]",
  "dependencies": [
    "registry+https://github.com/rust-lang/crates.io-index#[email protected]"
  ],
  "deps": [
    {
      "name": "portable_atomic",
      "pkg": "registry+https://github.com/rust-lang/crates.io-index#[email protected]",
      "dep_kinds": [
        {
          "kind": null,
          "target": null
        }
      ]
    }
  ],
  "features": [
    "alloc",
    "default",
    "race",
    "std"
  ]
},
There is perhaps some dependency feature unification across multiple targets which is causing this feature to be enabled
There's nothing we can do about that on the cargo-vet side because of the data we get from cargo metadata

jimblandy · 2024-10-04T17:42:41Z

There are a lot of tools that depend on cargo metadata getting this right. In addition to cargo vet, as previously mentioned, this also causes cargo vendor to incorporate the unused crate into the tree.

est31 · 2024-10-05T14:40:42Z

As explained in my (failed) PR #11938, cargo has two resolvers, and only one of the two has this problem. The other resolver doesn't. I wonder if one could add a mode to cargo metadata to use the "dependency resolver" instead of the "feature resolver".

jonasbb added the C-bug Category: bug label Jun 30, 2022

ehuss added A-lockfile Area: Cargo.lock issues A-features2 Area: issues specifically related to the v2 feature resolver labels Jun 30, 2022

Jake-Shadle mentioned this issue Jul 5, 2022

cargo metadata incorrectly adds weak dependencies EmbarkStudios/krates#41

Closed

N3xed mentioned this issue Jul 31, 2022

Build extra components specified in manifest metadata esp-rs/esp-idf-sys#114

Merged

This was referenced Sep 8, 2022

bstr pulls in serde when serde feature is not specified BurntSushi/bstr#130

Closed

Upgrade to bstr 1.0.0 artichoke/artichoke#2173

Merged

niyaznigmatullin mentioned this issue Oct 17, 2022

Bump bstr 1.0.1 uutils/coreutils#4055

Merged

jonasbb added a commit to jonasbb/c-dns that referenced this issue Oct 30, 2022

Upgrade serde_with to v2

6a0e37c

Most of the new dependencies are false positive and are not actually required, but are false dependencies due to a Cargo bug. rust-lang/cargo#10801

ChrisDenton mentioned this issue Oct 31, 2022

Allow workspaces/crates to limit support to an explicit set of targets #11313

Closed

rpjohnst added a commit to rpjohnst/dejavu that referenced this issue Nov 12, 2022

Update dependencies

9534c25

A note: this adds serde to the lockfile, but it is not actually used. This is caused by rust-lang/cargo#10801.

weihanglo mentioned this issue Nov 26, 2022

The usage of weak dependencies pollutes Cargo.lock #11426

Closed

epage changed the title ~~Optional and disabled dependencies end up in Cargo.lock~~ Optional and disabled weak dependencies end up in Cargo.lock Nov 26, 2022

epage changed the title ~~Optional and disabled weak dependencies end up in Cargo.lock~~ Disabled Optional weak dependencies end up in Cargo.lock Nov 26, 2022

epage changed the title ~~Disabled Optional weak dependencies end up in Cargo.lock~~ Disabled optional weak dependencies end up in Cargo.lock Nov 26, 2022

ehuss mentioned this issue Nov 30, 2022

cargo tree not showing a package #11444

Closed

stormshield-gt mentioned this issue Jul 26, 2024

Ensuring that a provider based on the one built-in is used rustls/rustls#1920

Closed

1 task

xxchan mentioned this issue Aug 7, 2024

feat(cdc): support sql server cdc risingwavelabs/risingwave#17429

Merged

9 tasks

diehuxx mentioned this issue Aug 12, 2024

[Bug] Avoid pulling in ring as a transitive dependency decentralized-identity/web5-rs#281

Open

AkinoKaede mentioned this issue Aug 18, 2024

Failed build on Windows even it is specifically not dependent johalun/sysctl-rs#68

Closed

m4tx mentioned this issue Aug 19, 2024

feat: implement very basic ORM flareon-rs/flareon#9

Merged

bxb100 mentioned this issue Aug 20, 2024

Integrate OpenDAL spring-rs/spring-rs#16

Merged

4 tasks

djordon mentioned this issue Aug 22, 2024

[Feature]: Bump the version of our stacks-core dependency stacks-network/sbtc#411

Closed

1 task

BlackHoleFox mentioned this issue Aug 26, 2024

argon2: only enable password-hash dependency if its explicit RustCrypto/password-hashes#521

Merged

MDM23 mentioned this issue Aug 27, 2024

Rust 1.80.0 breaks some packages NixOS/nixpkgs#332957

Open

epage mentioned this issue Sep 4, 2024

Cargo optional feature (those with ?) is not working as expected #14494

Closed

sameer added a commit to sameer/g-code that referenced this issue Sep 9, 2024

v0.5.1: remove unexpected serialization deps

0e5eb47

Caused by rust-lang/cargo#10801

ParkMyCar mentioned this issue Sep 14, 2024

[RUSTSEC-2024-0363] Support sqlx 0.8 ParkMyCar/compact_str#407

Open

complexspaces mentioned this issue Sep 29, 2024

Request for new version release of autogenerated frameworks madsmtm/objc2#656

Open

taiki-e mentioned this issue Oct 1, 2024

Add portable-atomic feature and disable portable-atomic/critical-section by default matklad/once_cell#265

Merged

ErichDonGubler mentioned this issue Oct 4, 2024

Roll back once_cell to 1.19.0. gfx-rs/wgpu#6370

Merged

taiki-e mentioned this issue Oct 4, 2024

Remove "portable-atomic?/require-cas" from race feature matklad/once_cell#267

Merged

andrewdavidmackenzie mentioned this issue Oct 7, 2024

build(deps): bump once_cell from 1.20.1 to 1.20.2 andrewdavidmackenzie/flow#2314

Merged

cedricschwyter mentioned this issue Oct 7, 2024

chore(deps): bump once_cell from 1.19.0 to 1.20.2 cedricschwyter/serde-crypt#121

Merged

nazar-pc mentioned this issue Oct 10, 2024

tokio feature adds libp2p-quic to dependencies libp2p/rust-libp2p#5629

Closed

paupino mentioned this issue Oct 16, 2024

Should std enable rkyv & others? paupino/rust-decimal#686

Open

dima74 mentioned this issue Oct 22, 2024

fix: Fix cargo audit warnings hyperledger-iroha/iroha#5183

Merged

seanmonstar mentioned this issue Oct 28, 2024

Why does enabling rustls-tls require quinn? seanmonstar/reqwest#2458

Closed

AsakuraMizu mentioned this issue Dec 2, 2024

rustls in default features compio-rs/compio#352

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Disabled optional weak dependencies end up in `Cargo.lock` #10801

Disabled optional weak dependencies end up in `Cargo.lock` #10801

jonasbb commented Jun 30, 2022 •

edited

Loading

Jake-Shadle commented Jul 11, 2022

trevor-crypto commented Aug 23, 2022

lopopolo commented Sep 8, 2022

ljedrz commented Oct 13, 2022

lopopolo commented Oct 13, 2022 •

edited

Loading

ljedrz commented Oct 13, 2022

Sytten commented Nov 18, 2022

memoryruins commented Nov 18, 2022 •

edited

Loading

jonasbb commented Nov 18, 2022

memoryruins commented Nov 18, 2022

Sytten commented Nov 18, 2022

memoryruins commented Nov 19, 2022

lilith commented Aug 22, 2024

weihanglo commented Aug 23, 2024

ErichDonGubler commented Oct 4, 2024

epage commented Oct 4, 2024

jimblandy commented Oct 4, 2024 •

edited

Loading

jimblandy commented Oct 4, 2024 •

edited

Loading

est31 commented Oct 5, 2024

Disabled optional weak dependencies end up in Cargo.lock #10801

Disabled optional weak dependencies end up in Cargo.lock #10801

Comments

jonasbb commented Jun 30, 2022 • edited Loading

Problem

Steps

Possible Solution(s)

Notes

Version

Jake-Shadle commented Jul 11, 2022

trevor-crypto commented Aug 23, 2022

lopopolo commented Sep 8, 2022

ljedrz commented Oct 13, 2022

lopopolo commented Oct 13, 2022 • edited Loading

ljedrz commented Oct 13, 2022

Sytten commented Nov 18, 2022

memoryruins commented Nov 18, 2022 • edited Loading

jonasbb commented Nov 18, 2022

memoryruins commented Nov 18, 2022

Sytten commented Nov 18, 2022

memoryruins commented Nov 19, 2022

lilith commented Aug 22, 2024

weihanglo commented Aug 23, 2024

ErichDonGubler commented Oct 4, 2024

epage commented Oct 4, 2024

jimblandy commented Oct 4, 2024 • edited Loading

jimblandy commented Oct 4, 2024 • edited Loading

est31 commented Oct 5, 2024

Disabled optional weak dependencies end up in `Cargo.lock` #10801

Disabled optional weak dependencies end up in `Cargo.lock` #10801

jonasbb commented Jun 30, 2022 •

edited

Loading

lopopolo commented Oct 13, 2022 •

edited

Loading

memoryruins commented Nov 18, 2022 •

edited

Loading

jimblandy commented Oct 4, 2024 •

edited

Loading

jimblandy commented Oct 4, 2024 •

edited

Loading