Unable to use libp2p 0.49.0 plus libp2p-swarm-derive while publishing a library

[joao-paulo-parity]

Summary

When trying to publish a library which uses libp2p 0.49.0 and libp2p-swarm-derive, Cargo will pick libp2p-swarm-derive 0.30.2, which is incompatible with libp2p 0.49.0. It should instead pick libp2p-swarm-derive 0.30.1, but it doesn't do that even though the version is set to 0.31.0 in the Cargo.toml of libp2p 0.49.0. EDIT: Not actually true, see #3176 (comment).

The problem's description is expanded in the README of https://github.com/joao-paulo-parity/libp2p-troubleshoot. That repository contains an example project and sample commands that you can run to reproduce this problem.

Expected behaviour

I should be able to publish a library which uses libp2p 0.49.0 and libp2p-swarm-derive

Actual behaviour

I'm unable to achieve the expected behaviour

Possible Solution

The crux of the problem is that cargo picks libp2p-swarm-derive 0.30.2 instead of libp2p-swarm-derive 0.30.1, even though the latter is set in the Cargo.toml of libp2p 0.49.0. Therefore the solutions entail somehow making cargo resolve 0.30.1 for libp2p 0.49.0.

Option 1: Perhaps (i.e. I haven't actually verified this) cargo thinks that it's fine to use libp2p-swarm-derive 0.30.2 because it's a patch bump from 0.30.1, i.e. theoretically semver-compatible with libp2p 0.49.0. Thus if we were to turn the current libp2p-swarm-derive 0.30.2 release into 0.31.0 and yank 0.30.2, that would maybe cause cargo to choose 0.30.1, which would effectively solve the problem.

Option 2: A workaround would be to include an [[example]] within the library to be published. That would cause cargo to not ignore the Cargo.lock and therefore make it correctly pick libp2p-swarm-derive 0.30.1 during publish. This is mentioned in https://doc.rust-lang.org/cargo/reference/manifest.html:

A minimized Cargo.lock is automatically included if the package contains a binary or example target

While that would fix the version resolution during publish, I'm still unsure if it would be effective during consumption of the library. I might test this later and report it back here.

I think Option 1 would be preferrable, but as already mentioned I don't know yet if it would work in practice. I might also try this out later.

[dvdplm]

but it doesn't do that even though the version is set to 0.31.0 in the Cargo.toml

Even if the entry uses =0.30.1 to specify the exact patch version?

Can we yank 0.30.2 and publish it as 0.31.0 instead? @mxinden ?

[joao-paulo-parity]

but it doesn't do that even though the version is set to 0.31.0 in the Cargo.toml

Even if the entry uses =0.30.1 to specify the exact patch version?

Yes, the Cargo.toml of libp2p 0.49.0 has the exact version. You can verify this locally:

$ wget -O libp2p-0.49.0.tar.gz https://crates.io/api/v1/crates/libp2p/0.49.0/download
$ tar -zxf libp2p-0.49.0.tar.gz
$ grep -A 1 "libp2p-swarm-derive" libp2p-0.49.0/Cargo.toml
[dependencies.libp2p-swarm-derive]
version = "0.30.1"

cargo still tries to use 0.30.2 all the same ¯\_(ツ)_/¯

[thomaseizinger]

but it doesn't do that even though the version is set to 0.31.0 in the Cargo.toml

Even if the entry uses =0.30.1 to specify the exact patch version?

Yes, the Cargo.toml of libp2p 0.49.0 has the exact version. You can verify this locally:

Your manifest doesn't have the exact version. What you specified is actually a version range. "0.30.1" is equivalent to "^0.30.1" for cargo and means "any compatible version from '0.30.1' upwards", hence also every patch version. To pin a version you need to use =:

[dependencies.libp2p-swarm-derive]
version = "=0.30.1"

Nevertheless, you are right that the version bump was incorrect. #3011 was actually a breaking change because it requires libp2p-swarm v0.41.0. (cc @jxs).

Can we yank 0.30.2 and publish it as 0.31.0 instead? @mxinden ?

We should yank it yes. Ideally, we also publish a 0.30.3 patch release that reverts the changes made to libp2p-swarm-derive in #3011.

[mxinden]

My bad! Thanks @joao-paulo-parity for reporting this. See #3178 as a first step.

[joao-paulo-parity]

Your manifest doesn't have the exact version. What you specified is actually a version range. "0.30.1" is equivalent to "^0.30.1" for cargo and means "any compatible version from '0.30.1' upwards", hence also every patch version.

I didn't know this. Thanks for the explanatoin.

Using =0.30.1 I was able to work around https://github.com/joao-paulo-parity/libp2p-troubleshoot. I'll see if that also works in another project which is affected by this. After 0.30.2 is yanked I can revert that patch and try again.

[mxinden]

• Released the new version through fix(derive): Release breaking change as v0.31.0 #3178
• Yanked libp2p-swarm-derive v0.30.2

We should yank it yes. Ideally, we also publish a 0.30.3 patch release that reverts the changes made to libp2p-swarm-derive in #3011.

Why would that be needed @thomaseizinger? I would hope that those that tried to update their libp2p v0.49.0 based app to libp2p-swarm-derive v0.30.2 never actually did (compilation failure) and thus there is no need to cut libp2p-swarm-derive v0.30.3. Am I missing something?

[joao-paulo-parity]

I can confirm that the problem is fixed after 0.30.2 was yanked. Thanks! Feel free to close the ticket whenever

[thomaseizinger]

• Released the new version through fix(derive): Release breaking change as v0.31.0 #3178
• Yanked libp2p-swarm-derive v0.30.2

We should yank it yes. Ideally, we also publish a 0.30.3 patch release that reverts the changes made to libp2p-swarm-derive in #3011.

Why would that be needed @thomaseizinger? I would hope that those that tried to update their libp2p v0.49.0 based app to libp2p-swarm-derive v0.30.2 never actually did (compilation failure) and thus there is no need to cut libp2p-swarm-derive v0.30.3. Am I missing something?

It is not strictly needed, yes. It is possible to depend on a yanked version by using either = or 0.30.2. The latter case will never compile again unless we release a 0.30.3.

It is highly unlikely in our case I think so we don't need to do it :)

[mxinden]

Closing here. Thanks to everyone involved, especially @joao-paulo-parity.