I think crane mangles my cargo.toml

[pingiun]

My situation:

I have made modifications to sqlx, they live in my branch. sqlx is a workspace project

In my own project, I depend on sqlx via git. My own project is also a workspace.

I get this error when building my project

Nix build logs

warning: The interpretation of store paths arguments ending in `.drv` recently changed. If this command is now failing try again with '/nix/store/p2c53q2h2419ygdgy4snjrf0klrrsm2h-bibs-server-deps-0.1.0.drv^*'
https://cache.nixos.org/drjna8cp1m2bjphqkra58xpjdqb6s4k1.narinfo
cargoArtifacts not set, will not reuse any cargo artifacts
Running phase: unpackPhase
unpacking source archive /nix/store/21i4hbdcj0kw84b8i8ryraqgfpz1b0kx-source
source root is source
Running phase: patchPhase
Executing configureCargoCommonVars
Running phase: updateAutotoolsGnuConfigScriptsPhase
Running phase: configurePhase
will append /build/source/.cargo-home/config.toml with contents of /nix/store/v1gwn492qw6x8pidrgmnln72a0zrnixm-vendor-cargo-deps/config.toml
default configurePhase, nothing to do
Running phase: buildPhase
++ command cargo --version
cargo 1.75.0 (1d8b05cdd 2023-11-20)
++ command cargo check --profile release-server --package bibs-server --all-targets
error: failed to get `sqlx` as a dependency of package `bibs-server v0.1.0 (/build/source/bibs-server)`

Caused by:
  failed to load source for dependency `sqlx`

Caused by:
  Unable to update https://github.com/jelle-bigbridge/sqlx.git?branch=add-local-infile#6a345589

Caused by:
  failed to update replaced source https://github.com/jelle-bigbridge/sqlx.git?branch=add-local-infile#6a345589

Caused by:
  failed to parse manifest at `/nix/store/v1gwn492qw6x8pidrgmnln72a0zrnixm-vendor-cargo-deps/e4efc4d96eed4dc366eaad252dc8b86b89b113ce9da78099f9fd5f7aba534018/sqlx-mysql-0.7.3/Cargo.toml`

Caused by:
  TOML parse error at line 74, column 14
     |
  74 | [dependencies.
     |              ^
  invalid table header
  expected `.`, `]`

The critical part is transforming the tokio dependency for sqlx-mysql so it's settings are copied from the sqlx workspace.

First, the generated Cargo.toml in full:

Cargo.toml for sqlx-mysql

[package]
name = "sqlx-mysql"
documentation = "https://docs.rs/sqlx"
description = "MySQL driver implementation for SQLx. Not for direct use; see the `sqlx` crate for details."
version= "0.7.3"
license= "MIT OR Apache-2.0"
edition= "2021"
authors= [
    "Ryan Leckey <[email protected]>",
    "Austin Bonander <[email protected]>",
    "Chloe Ross <[email protected]>",
    "Daniel Akhterov <[email protected]>",
]
repository= "https://github.com/launchbadge/sqlx"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[features]
json = ["sqlx-core/json", "serde"]
any = ["sqlx-core/any"]
offline = ["sqlx-core/offline", "serde/derive"]
migrate = ["sqlx-core/migrate"]
_rt-tokio = ["sqlx-core/_rt-tokio", "tokio"]

[dependencies]
sqlx-core = { version = "=0.7.3", path = "sqlx-core" }

# Futures crates
futures-channel = { version = "0.3.19", default-features = false, features = ["sink", "alloc", "std"] }
futures-core = { version = "0.3.19", default-features = false }
futures-io = "0.3.24"
futures-util = { version = "0.3.19", default-features = false, features = ["alloc", "sink", "io"] }

# Cryptographic Primitives
crc = "3.0.0"
digest = { version = "0.10.0", default-features = false, features = ["std"] }
hkdf = "0.12.0"
hmac = { version = "0.12.0", default-features = false }
md-5 = { version = "0.10.0", default-features = false }
rand = { version = "0.8.4", default-features = false, features = ["std", "std_rng"] }
rsa = "0.9"
sha1 = { version = "0.10.1", default-features = false }
sha2 = { version = "0.10.0", default-features = false }

# Type Integrations (versions inherited from `[workspace.dependencies]`)
bigdecimal = { optional = true }
chrono = { version = "0.4.22", default-features = false , optional = true }
rust_decimal = { optional = true }
time = { version = "0.3.14", features = ["formatting", "parsing", "macros"] , optional = true }
uuid = { optional = true }

# Misc
atoi = "2.0"
base64 = { version = "0.21.0", default-features = false, features = ["std"] }
bitflags = { version = "2", default-features = false, features = ["serde"] }
byteorder = { version = "1.4.3", default-features = false, features = ["std"] }
bytes = "1.1.0"
dotenvy = "0.15.5"
either = "1.6.1"
generic-array = { version = "0.14.4", default-features = false }
hex = "0.4.3"
itoa = "1.0.1"
log = "0.4.17"
memchr = { version = "2.4.1", default-features = false }
once_cell = "1.9.0"
percent-encoding = "2.1.0"
smallvec = "1.7.0"
stringprep = "0.1.2"
thiserror = "1.0.35"
tracing = { version = "0.1.37", features = ["log"] }
whoami = { version = "1.2.1", default-features = false }

serde = { version = "1.0.144", optional = true }

[dependencies.
# Support for tokio::io::AsyncWrite in mysql::infile
tokio ]
version = "1"
features = ["time", "net", "sync", "fs", "io-util", "rt"]
default-features = false
optional = true

I believe it tries to transform this:

# Support for tokio::io::AsyncWrite in mysql::infile
tokio = { workspace = true, optional = true }

Into this:

[dependencies.tokio]
# Support for tokio::io::AsyncWrite in mysql::infile
version = "1"
features = ["time", "net", "sync", "fs", "io-util", "rt"]
default-features = false
optional = true

But the comment gets inserted in the wrong place so it ends up with this:

[dependencies.
# Support for tokio::io::AsyncWrite in mysql::infile
tokio ]
version = "1"
features = ["time", "net", "sync", "fs", "io-util", "rt"]
default-features = false
optional = true

I hope this is enough to diagnose the issue and fix it, if not please let me know!

[pingiun]

Update: I found out this worked on 0.14.1, but broke on 0.14.2

I can see a lot of dependencies were updated between those versions, maybe this is actually a bug in toml-edit?

[dpc]

I can see a lot of dependencies were updated between those versions, maybe this is actually a bug in toml-edit?

My initial thought. If you could bisect it it would probably be very helpful.

[pingiun]

@dpc hey I just did the bisect, the first bad commit is: 03e442f

[pingiun]

I also added a failing test for you here (not ran it myself, but it should be correct): https://github.com/pingiun/crane/tree/sqlx-workspace-test

[dpc]

Makes sense. toml_edit had some patch releases, if that doesn't help we could report upstream.

[ipetkov]

Hi @pingiun thanks for the report and the test case! Definitely looks like a bug with toml_edit, would you mind opening an issue upstream for it?

Happy to get it updated and merged here as soon as a new release is published!

[ipetkov]

Oh hmm we're a minor version behind, let me see if bumping it will fix it first

[ipetkov]

Same issue with toml_edit 0.22.6: 50c9f5f

Here's the branch: https://github.com/ipetkov/crane/compare/toml_edit

[pingiun]

Hi @pingiun thanks for the report and the test case! Definitely looks like a bug with toml_edit, would you mind opening an issue upstream for it?

I don't know if I understand this bug well enough. Do you know what toml_edit functions manifest this error? I have not really looked into your code, or toml_edit for that matter, I'm just a user of crane for building projects

[ipetkov]

Looks like there is a report for it here: toml-rs/toml#691

[CobaltCause]

Personally, I've bisected this to f23ae17. The mangled output I see, if it helps, is:

[
# dev-dependencies can't be optional, so this is a regular dependency
dependencies.criterion ]
version = "0.5.0"
optional = true 

A not-so-minimal reproducer is to add this dependency to the project-to-be-built-with-crane:

[dependencies.ruma]
git = "https://github.com/ruma/ruma"
rev = "1a1c61ee1e8f0936e956a3b69c931ce12ee28475"
features = [
    "compat",
    "rand",
    "appservice-api-c",
    "client-api",
    "federation-api",
    "push-gateway-api-c",
    "state-res",
    "unstable-msc2448",
    "unstable-msc3575",
    "unstable-exhaustive-types",
    "ring-compat",
    "unstable-unspecified",
]

[dpc]

Would it be possible to workaround it by not doing the sequence that trigger this bug? :D

[ipetkov]

Looks like this will be fixed with #583

[CobaltCause]

Yep, I no longer get the build failure I was experiencing before, it works now