Tracking Issue for cargo report future-incompat

[pnkfelix]

This is a tracking issue for the RFC "Cargo report future-incompat" (rust-lang/rfcs#2834).
There is no feature gate for the issue (the changes are not language visible).

About tracking issues

Tracking issues are used to record the overall progress of implementation.
They are also uses as hubs connecting to other relevant issues, e.g., bugs or open design questions.
A tracking issue is however not meant for large scale discussion, questions, or bug reports about a feature.
Instead, open a dedicated issue for the specific matter and add the relevant feature gate label.

Steps

• Implement the RFC (cc @rust-lang/cargo -- but pnkfelix will either implement or write up mentoring instructions)
• Adjust documentation (see instructions on rustc-dev-guide)
• Stabilization PR (see instructions on rustc-dev-guide)

Unresolved Questions

• There are future-incompatibility warnings emitted by cargo itself, such as discussed on rust-lang/cargo#6313 (comment). It shouldn't require much effort to incorporate support for such lints, but pnkfelix has not explicitly addressed nor seriously investigated this.
• This doesn't work in some editors. Some editor integrations don't display warnings from Cargo itself, because those warnings are not emitted as JSON (see Provide Cargo messages as JSON messages cargo#8283). I'm concerned that users may not see these warnings.
• There is a concern about the performance impact in Cargo where it needs to "replay" the output cache of every dependency for every command. In the normal case, only dependencies that generate future incompatible warnings will generate these files, and it is hoped that the number of dependencies that produce these is very small (preferably zero). However, if it becomes normal to have a large number of dependencies with warnings, this can introduce a nontrivial performance impact, particularly on Windows or slow hard drives.
• Color handling is awkward. One of the use cases is that the user runs cargo in an editor, sees the message that they need to run cargo describe-future-incompatibilities, they do that, and the message isn't colored. Ideally I would think the report data would be saved to the file in a more raw form, and then just re-render it as requested (instead of capturing the color at build time). DONE: Updates to future-incompatible reporting. cargo#9606
• I think the command should save at least a few reports in the on-disk file, so that if there are things like background jobs (like cargo watch), it doesn't make it difficult to view a report. I think it should save the reports as an array, and just trim it to at most N reports. It could also use monotonic numeric IDs instead of the strange random IDs (the next ID could be computed by 1+ the max ID in the file). Then the report command could default to showing the report with the greatest ID when not given an ID. DONE: Updates to future-incompatible reporting. cargo#9606
• The message to "run cargo describe-future-incompatibilities" could also include the target directory if using a non-default one.
• The Cargo Team is discussing introducing a cargo report subcommand that would subsume all user-report functionality (rather than having a bunch of top-level subcommands). This could replace describe-future-incompatibilities with something like cargo report future-incompatibilities. Add report subcommand. cargo#9438
• The RFC discussed a mechanism to silence the warnings (Annoyance modulation). I'm not sure exactly what that should look like, but un-silenceable warnings can be frustrating, especially if it is not possible to update the offending dependency (or no new version is available).
  • Related to above, will need to think about how these can be denied.
• With --future-incompat-report, if there are zero warnings, it should probably say so. Emit note when --future-incompat-report had nothing to report cargo#9263
• There is a concern that the current warnings and reports may be confusing or un-actionable for most users. See Tracking Issue for cargo report future-incompat #71249 (comment) and Tracking Issue for cargo report future-incompat #71249 (comment).

Implementation history

• Implement rustc side of report-future-incompat #75534: Implement rustc side of report-future-incompat
• Implement future incompatibility report support cargo#8825: Implement cargo side of report-future-incompat
• Emit note when --future-incompat-report had nothing to report cargo#9263: Emit note when --future-incompat-report had nothing to report
• Add report subcommand. cargo#9438: Rename to cargo report future-incompatibilities
• Updates to future-incompatible reporting. cargo#9606

[Aaron1011]

I'm interested in working on this.

Would it be better to add additional data to each JSON diagnostic message (e.g. future_compat: { breakage_release: "1.47.0" }), or emit a single 'summary' message at the end of compilation?

[pnkfelix]

Hi @Aaron1011 ; come on over to the zulip thread for the MCP and we can discuss more. (In the MCP I went with "single summary" at the end, but we can talk more about the options on zulip.)

[Aaron1011]

I have a pull request up at #75534

[Aaron1011]

The rustc side of this was implemented in #75534

[ehuss]

@Aaron1011 I was wondering if you are interested in stabilizing this. I don't think there are any major blockers, and at least on the Cargo side we feel comfortable moving forward as long as the compiler team is very conservative with enabling new lints until we've had some more real-world experience.

[Aaron1011]

@ehuss: I'm currently working on a PR that improves the output, based on @LukasKalbertodt's suggestion in #71249 (comment). Once that's done, I think this feature should be ready to stabilize.

[Aaron1011]

@ehuss: Now that rust-lang/cargo#9953 has been merged, I think this should be ready for stabilization.

[ehuss]

Sure, let's go ahead and do an FCP on this issue with both teams.

@rfcbot fcp merge

Stabilization Report

This is a proposal to stabilize the future-incompat feature.

Example usage

To test it out (please do, this has had little testing), create a Cargo project with:

[dependencies]
rental = "=0.5.5"

And run cargo build -Zfuture-incompat-report, and you should see something like:

   Compiling ...
    Finished ...
warning: the following packages contain code that will be rejected by a future version of Rust: rental v0.5.5
note: to see what the problems were, use the option `--future-incompat-report`, or run `cargo report future-incompatibilities --id 1`

To view the report, run cargo report future-incompatibilities --id 1. It's a bit long, but the output should look something like this:

Report Output

The following warnings were discovered during the build. These warnings are an
indication that the packages contain code that will become an error in a
future release of Rust. These warnings typically cover changes to close
soundness problems, unintended or undocumented behavior, or critical problems
that cannot be fixed in a backwards-compatible fashion, and are not expected
to be in wide use.

Each warning should contain a link for more information on what the warning
means and how to resolve it.


To solve this problem, you can try the following approaches:


- Some affected dependencies have newer versions available.
You may want to consider updating them to a newer version to see if the issue has been fixed.

rental v0.5.5 has the following newer versions available: 0.5.6


- If the issue is not solved by updating the dependencies, a fix has to be
implemented by those dependencies. You can help with that by notifying the
maintainers of this problem (e.g. by creating a bug report) or by proposing a
fix to the maintainers (e.g. by creating a pull request):

  - rental:0.5.5
  - Repository: https://github.com/jpernst/rental
  - Detailed warning command: `cargo report future-incompatibilities --id 1 --package rental:0.5.5`

- If waiting for an upstream fix is not an option, you can use the `[patch]`
section in `Cargo.toml` to use your own version of the dependency. For more
information, see:
https://doc.rust-lang.org/cargo/reference/overriding-dependencies.html#the-patch-section

The package `rental v0.5.5` currently triggers the following future incompatibility lints:
> warning: using `procedural-masquerade` crate
>    --> /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/rental-0.5.5/src/lib.rs:94:8
>     |
> 94  |         enum ProceduralMasqueradeDummyType {
>     |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> ...
> 117 |     define_rental_traits!(32);
>     |     ------------------------- in this macro invocation
>     |
>     = note: `#[allow(proc_macro_back_compat)]` on by default
>     = warning: this was previously accepted by the compiler but is being phased out; it will become a hard error in a future release!
>     = note: for more information, see issue #83125 <https://github.com/rust-lang/rust/issues/83125>
>     = note: The `procedural-masquerade` crate has been unnecessary since Rust 1.30.0. Versions of this crate below 0.1.7 will eventually stop compiling.
>     = note: this warning originates in the macro `define_rental_traits` (in Nightly builds, run with -Z macro-backtrace for more info)
>
> warning: using `procedural-masquerade` crate
>    --> /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/rental-0.5.5/src/lib.rs:258:9
>     |
> 258 |               enum ProceduralMasqueradeDummyType {
>     |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> ...
> 285 | / rental! {
> 286 | |     /// Example types that demonstrate the API generated by the rental macro.
> 287 | |     pub mod examples {
> 288 | |         use std::sync;
> ...   |
> 345 | |     }
> 346 | | }
>     | |_- in this macro invocation
>     |
>     = warning: this was previously accepted by the compiler but is being phased out; it will become a hard error in a future release!
>     = note: for more information, see issue #83125 <https://github.com/rust-lang/rust/issues/83125>
>     = note: The `procedural-masquerade` crate has been unnecessary since Rust 1.30.0. Versions of this crate below 0.1.7 will eventually stop compiling.
>     = note: this warning originates in the macro `rental` (in Nightly builds, run with -Z macro-backtrace for more info)
>
> warning: using `procedural-masquerade` crate
>    --> /Users/eric/.cargo/registry/src/github.com-1ecc6299db9ec823/rental-0.5.5/src/lib.rs:258:9
>     |
> 258 |               enum ProceduralMasqueradeDummyType {
>     |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> ...
> 350 | / rental! {
> 351 | |     /// Premade types for the most common use cases.
> 352 | |     pub mod common {
> 353 | |         use std::ops::DerefMut;
> ...   |
> 484 | |     }
> 485 | | }
>     | |_- in this macro invocation
>     |
>     = warning: this was previously accepted by the compiler but is being phased out; it will become a hard error in a future release!
>     = note: for more information, see issue #83125 <https://github.com/rust-lang/rust/issues/83125>
>     = note: The `procedural-masquerade` crate has been unnecessary since Rust 1.30.0. Versions of this crate below 0.1.7 will eventually stop compiling.
>     = note: this warning originates in the macro `rental` (in Nightly builds, run with -Z macro-backtrace for more info)
>

Running with --future-incompat-report is similar, but does not show the exact compiler warnings.

future-incompat is available as an alias in cargo report for those whose fingers get twisted while typing the full future-incompatibilities.

cargo report future-incompat without --id will display the most recent report.

The report command also supports the --package option to only show the warnings for a single package.

When stabilized, the -Z flags will no longer be necessary.

Report interface

Currently, rustc will emit special JSON data when the -Z emit-future-incompat-report flag is included. When stabilized, it is intended for this to become --json future-incompat. The JSON structure looks like:

{
    /* An array of objects describing a warning that will become a hard error 
       in the future. 
    */
    "future_incompat_report":
    [
        {
            /* A diagnostic structure as defined in 
               https://doc.rust-lang.org/rustc/json.html#diagnostics 
            */
            "diagnostic": {...},
        }
    ]
}

This structure is somewhat bare now, but is intended to support additions in the future.

This is emitted towards the end of compilation.

Cargo intercepts these messages and stores them on disk, and reports a message to the console before exiting.

Warning silencing

In situations where a user cannot update the offending code, they can silence the warning using a cargo config value:

[future-incompat-report]
# Value is "never" or "always"
frequency = "never"

Warning opt-in

Currently there are only two warnings which will trigger a report: proc_macro_derive_resolution_fallback and proc_macro_back_compat.

A lint must explicitly opt-in to triggering a report using the FutureIncompatibilityReason::FutureReleaseErrorReportNow option.

Implementation

Internally, rustc buffers any lint that fires that is marked with FutureReleaseErrorReportNow here, even if the code has explicitly tried to allow it (allow override here). At the end of compilation, the report is emitted here. The JSON formatting code is here.

Cargo intercepts these messages here. It has a module here for generating reports and saving them to disk and displaying them.

Warning transitions

The Cargo team is willing to stabilize this with the understanding from the compiler team that they will be very conservative in adding new lints to be reported as this is rolled out. We would like to get some experience with users for a while on the stable channel so that we can respond to any changes that are needed to the reporting experience. This also has the potential to be a sensitive issue, as Rust emphasizes being stable, but this is explicitly breaking that stability (though hopefully for justified reasons).

Future incompatible lints should have a lifecycle, perhaps something like the following:

1. Future-incompat (non-reporting) lint is introduced as a warning.
2. After some period of time, the lint is elevated to "deny".
3. The lint graduates to be FutureReleaseErrorReportNow so that it gets more visibility.
4. The lint migrates to a hard error.

I think it is up to the compiler team what the timeframes in-between steps should be, but hopefully conservative (and depending on the severity and urgency of making the change).

RFC deviations

• The "deadline" from the report was removed (the indication of a date when it will become a hard error). Cargo doesn't need that in a structured form, and I am skeptical that we will be able to accurately predict when changes will be made in the future. If a date is indeed wanted, the text can be added to the lint.
• The "annoyance modulation" is relatively basic, and does not support any sort of temporary silencing. This can be added in the future if desired.

Known bugs

• Color support in cargo report future-incompatibilities is not detected properly, and will emit color escape codes when piping the output to a file (cargo report future-incompatibilities doesn't check for color support correctly cargo#9960).

Unresolved questions

The questions above that aren't resolved:

• Cargo itself emitting its own incompatibility reports: This can be added later if needed.
• Interaction with editors: In some circumstances, a user may not see these warnings since some editors only display JSON messages, and this warning is text only. I think this shouldn't be too bad, as I think most users eventually look at console output. This is a long-running issue with Cargo not emitting its own messages as JSON.
• Performance impact: It is not expected that this should have any performance impact. In the vast majority of cases, there is no cached output for dependencies because of cap-lints. It is also expected that these warnings should be exceedingly rare.
• Telling the user the CARGO_TARGET_DIR to use: I think if a user is using a custom target directory, they should know how that works. One example where this may not be obvious is rustc's own x.py build system. It is not known how common complex build environments like x.py are. I think this is something we can iterate on in the future.
• Reports can be confusing or unactionable: We have been iterating on the format of the report to try to make it clearer, with actionable steps. However, this is definitely a major concern, and something we should be careful about. I think we can iterate more based on real-world user reports of any confusion or frustration.

Other notes

• The currently isn't a convenient way to forbid only these warnings that are in the FutureReleaseErrorReportNow category. You can do something like RUSTFLAGS="-F future-incompatible", but that will forbid all future-incompatible warnings. Perhaps this is something to consider in the future? I imagine either adding a dedicated lint group, or a setting in the [future-incompat-report] config section are options.

[rfcbot]

Team member @ehuss has proposed to merge this. The next step is review by the rest of the tagged team members:

• @Aaron1011
• @Eh2406
• @alexcrichton
• @davidtwco
• @eddyb
• @ehuss
• @estebank
• @joshtriplett
• @lcnr
• @matthewjasper
• @michaelwoerister
• @nagisa
• @nikomatsakis
• @oli-obk
• @petrochenkov
• @pnkfelix
• @wesleywiser

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[Aaron1011]

cc @estebank @michaelwoerister @nagisa @nikomatsakis @pnkfelix @wesleywiser - when you get a chance, can you review the FCP?

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[rfcbot]

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

This will be merged soon.

[Aaron1011]

This has now been stabilized in both rustc and cargo!

[ehuss]

🎉 This is now available in the latest nightly release (2021-12-09). Thanks @Aaron1011!

Closing as this is now complete.