Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add cleanup crew #556

Merged
merged 7 commits into from
Jan 23, 2020
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@
- [`LintStore`](./diagnostics/lintstore.md)
- [Diagnostic Codes](./diagnostics/diagnostic-codes.md)
- [ICE-breaker teams](ice-breaker/about.md)
- ["Cleanup Crew" ICE-breakers](ice-breaker/cleanup-crew.md)
- [LLVM ICE-breakers](ice-breaker/llvm.md)
- [Part 2: How rustc works](./part-2-intro.md)
- [High-level overview of the compiler source](./high-level-overview.md)
Expand Down
113 changes: 113 additions & 0 deletions src/ice-breaker/cleanup-crew.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
# Cleanup Crew

**Github Label:** [ICEBreaker-Cleanup-Crew]

[ICEBreaker-Cleanup-Crew]: https://github.com/rust-lang/rust/labels/ICEBreaker-Cleanup-Crew

The "Cleanup Crew" are focused on improving bug reports. Specifically,
the goal is to try to ensure that every bug report has all the
information that will be needed for someone to fix it:

* a minimal, standalone example that shows the problem
* links to duplicates or related bugs
* if the bug is a regression (something that used to work, but no longer does),
then a bisection to the PR or nightly that caused the regression

This kind of cleanup is invaluable in getting bugs fixed. Better
still, it can be done by anybody who knows Rust, without any
particularly deep knowledge of the compiler.

Let's look a bit at the workflow for doing "cleanup crew" actions.

## Finding a minimal, standalone example

Here the ultimate goal is to produce an example that reproduces the same
problem but without relying on any external crates. Such a test ought to contain
as little code as possible, as well. This will make it much easier to isolate the problem.

However, even if the "ultimate minimal test" cannot be achieved, it's
still useful to post incremental minimizations. For example, if you
can eliminate some of the external dependencies, that is helpful, and
so forth.

It's particularly useful to reduce to an example that works
in the [Rust playground](https://play.rust-lang.org/), rather than
requiring people to checkout a cargo build.

There are many resources for how to produce minimized test cases. Here
are a few:

* The [rust-reduce](https://github.com/jethrogb/rust-reduce) tool can try to reduce
code automatically.
* The [C-reduce](https://embed.cs.utah.edu/creduce/) tool also works
on Rust code, though it requires that you start from a single
file. (XXX link to some post explaining how to do it?)
* pnkfelix's [Rust Bug Minimization Patterns] blog post
* This post focuses on "heavy bore" techniques, where you are
starting with a large, complex cargo project that you wish to
narrow down to something standalone.

[Rust Bug Minimization Patterns]: http://blog.pnkfx.org/blog/2019/11/18/rust-bug-minimization-patterns/

## Links to duplicate or related bugs

If you are on the "Cleanup Crew", you will sometimes see multiple bug
reports that seem very similar. You can link one to the other just by
mentioning the other bug number in a Github comment. Sometimes it is
useful to close duplicate bugs. But if you do so, you should always
copy any test case from the bug you are closing to the other bug that
remains open, as sometimes duplicate-looking bugs will expose
different facets of the same problem.

## Bisecting regressions

For regressions (something that used to work, but no longer does), it
is super useful if we can figure out precisely when the code stopped
working. The gold standard is to be able to identify the precise
**PR** that broke the code, so we can ping the author, but even
narrowing it down to a nightly build is helpful, especially as that
then gives us a range of PRs. (One other challenge is that we
sometimes land "rollup" PRs, which combine multiple PRs into one.)

### cargo-bisect-rustc

To help in figuring out the cause of a regression we have a tool
called [cargo-bisect-rustc]. It will automatically download and test
various builds of rustc. For recent regressions, it is even able to
use the builds from our CI to track down the regression to a specific
PR; for older regressions, it will simply identify a nightly.

To learn to use [cargo-bisect-rustc], check out [this blog
post][learn], which gives a quick introduction to how it works. You
can also ask questions at the Zulip stream
`#t-compiler/cargo-bisect-rustc`, or help in improving the tool.
nikomatsakis marked this conversation as resolved.
Show resolved Hide resolved

[cargo-bisect-rustc]: https://github.com/rust-lang/cargo-bisect-rustc/
[learn]: https://blog.rust-lang.org/inside-rust/2019/12/18/bisecting-rust-compiler.html

### identifying the range of PRs in a nightly

If the regression occurred more than 90 days ago, then
cargo-bisect-rustc will not able to identify the particular PR that
caused the regression, just the nightly build. In that case, we can
identify the set of PRs that this corresponds to by using the git
history.

The command `rustc +nightly -vV` will cause rustc to output a number
of useful bits of version info, including the `commit-hash`. Given the
commit-hash of two nightly versions, you can find all of PRs that have
landed in between by taking the following steps:

1. Go to an update checkout of the [rust-lang/rust] repository
2. Execute the command `git log --author=bors --format=oneline SHA1..SHA2`
* This will list out all of the commits by bors, which is our merge bot
* Each commit corresponds to one PR, and information about the PR should be in the description
3. Copy and paste that information into the bug report

Often, just eye-balling the PR descriptions (which are included in the
commit messages) will give you a good idea which one likely caused the
problem. But if you're unsure feel free to just ping the compiler team
(`@rust-lang/compiler`) or else to ping the authors of the PR
themselves.
spastorino marked this conversation as resolved.
Show resolved Hide resolved

[rust-lang/rust]: https://github.com/rust-lang/rust/