Allow running with different RUSTFLAGS and take "max" of all runs

[wmmc88]

Some crates use cfgs to define non-additive features in order to not break Cargo's assumption that features are additive. Some examples include tokio, uuid, windows-rs.

What if cargo-semver-checks could run with each of the cfg flags individually, and take the "max" version of all the runs?

[obi1kenobi]

Hi! Thanks for the suggestion.

Could you say more about how such a feature would be useful to users? I'm having a bit of trouble visualizing it — I've never used cfg clauses in this way, and two of the three examples you provided seem to be around explicitly-unstable features where I'm not sure cargo-semver-checks would be that valuable.

[wmmc88]

I think running it 3 times with separate cfg flags would be similar, but:

• it would require parsing out the new suggested version from the output 3 times (since csc has no option for some machine-readable output related Report "minimum new version" in output #56, Export findings in machine-readable format (e.g. JSON) #6 )
• it would require an external script to take the min new version output of the 3 runs, parse the semver versions, and select the highest one

AFAIU, csc only uses the output of rustdoc(and not rustc), so setting RUSTDOCFLAGS="--cfg A" and similar should work: https://doc.rust-lang.org/rustdoc/command-line-arguments.html?highlight=cfg#--cfg-passing-configuration-flags, except that RUSTDOCFLAGS gets overridden currently. If this were changed to respect existing RUSTDOCFLAGS, then its conceivable that someone could just write their own binary using the csc library to run N times and find the minimum version floor for a new release.

[obi1kenobi]

As of the upcoming release, we actually also use cargo metadata so we can lint package manifests. The announcement for that will go out shortly, so there's no way you could have known this though :)

(EDIT: As far as I can tell, RUSTFLAGS and RUSTDOCFLAGS don't affect cargo metadata, so it's just rustdoc that we need to tell about --cfg A style settings.)

#1016 makes it so that RUSTDOCFLAGS is preserved. It also includes a test case to prove that the new code path works with --cfg A syntax. Thanks for the suggestion on that!

I think building the infra for making multiple runs would be interesting, and would also help for scanning different feature configurations to find non-additive features! Unfortunately, it would take quite a bit of design, implementation, and testing work.

Due to the scope, it's not something I can take on right now as part of my regular work on this project. If this functionality would be useful to your team, I'd love to discuss developing it as part of a commercial development and support contract. Is that something you or your team would be open to?

[wmmc88]

Thanks for the fix for flag override behavior! I think at this time, we're going to prototype some things using the existing functionalities

[obi1kenobi]

Neat! I'd love to know what you figure out — if there's a system you build and find pleasant to use, I'd love to take a look and possibly even a PR.

[obi1kenobi]

The v0.37 release is currently being built btw, it should be out within a few minutes: https://github.com/obi1kenobi/cargo-semver-checks/actions/runs/12172704503