Refactor auto trait handling in librustdoc to be accessible from librustc.

[ibabushkin]

These commits transfer some of the functionality introduced in #47833 to librustc with the intention of making the tools to work with auto traits accessible to third-party code, for example rust-semverver.

Some rough edges remain, and I'm certain some of the FIXMEs introduced will need some discussion, most notably the fairly ugly overall approach to pull out the core logic into librustc, which was previously fairly tightly coupled with various bits and bobs from librustdoc.

cc @Aaron1011

[petrochenkov]

Hmm, r? @nikomatsakis perhaps?

[GuillaumeGomez]

👍

[nikomatsakis]

Left a few nits, but the big one is a comment explaining the purpose of this function -- it's hard to review without that.

[nikomatsakis]

Nit: can we rustfmt this file?

[ibabushkin]

Sure, will do!

[nikomatsakis]

Why is this public?

[ibabushkin]

I suppose this is a leftover from a previous iteration. Good catch.

[nikomatsakis]

Nit: using impl trait and '_, this can be:

auto_trait_callback: impl for<'infcx> Fn(&InferCtxt<'_, 'tcx, 'infcx>, AutoTraitInfo<'infcx>) -> A

[nikomatsakis]

This is hard to review without understanding what it does...can you write a comment explaining the purpose of this function? Thanks! <3

[ibabushkin]

I agree. To be honest, I am also quite unhappy with the actual implementation in this spot -- it is quite awkward (and hence awkward to describe). The idea is basically to offload some of the computation to the auto_trait_callback (because it, for example, requires some types outside rustc) after the main work the function's equivalent in librustdoc performed without this changeset is done.

[TimNN]

Your PR failed on Travis. Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

Resolving deltas: 100% (612539/612539), completed with 4866 local objects.
---
[00:00:51] configure: rust.quiet-tests     := True
---
[00:04:48] tidy error: /checkout/src/librustc/traits/auto_trait.rs:76: line longer than 100 chars
[00:04:49]
[00:04:49]
[00:04:49] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/src" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "--no-vendor" "--quiet"
---
[00:04:49] make: *** [tidy] Error 1
[00:04:49] Makefile:79: recipe for target 'tidy' failed

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN.

[ibabushkin]

I think the new comment in the file somewhat goes in the right direction, but to be honest, I find it hard making it more precise.

[topecongiro]

@ibabushkin Looks like you may have used an older version of rustfmt. Would you mind reformatting the file with the latest version? FWIW rustup is the easiest way to install it.

[rust-highfive]

Your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

Resolving deltas: 100% (612921/612921), completed with 4877 local objects.
---
[00:00:47] configure: rust.quiet-tests     := True
---
[00:05:03] tidy error: /checkout/src/librustc/traits/auto_trait.rs:76: line longer than 100 chars
[00:05:04] some tidy checks failed
[00:05:04]
[00:05:04]
[00:05:04] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/src" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "--no-vendor" "--quiet"
[00:05:04] expected success, got: exit code: 1
[00:05:04]
[00:05:04]
[00:05:04] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test src/tools/tidy
[00:05:04] Build completed unsuccessfully in 0:02:01
[00:05:04] make: *** [tidy] Error 1
[00:05:04] Makefile:79: recipe for target 'tidy' failed
---
$ ls -lat $HOME/Library/Logs/DiagnosticReports/
ls: cannot access /home/travis/Library/Logs/DiagnosticReports/: No such file or directory
travis_time:end:01a9d426:start=1523527631091517032,finish=1523527631097636959,duration=6119927
travis_fold:end:after_failure.2
travis_fold:start:after_failure.3
travis_time:start:08a6e160
$ find $HOME/Library/Logs/DiagnosticReports -type f -name '*.crash' -not -name '*.stage2-*.crash' -not -name 'com.apple.CoreSimulator.CoreSimulatorService-*.crash' -exec printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" {} \; -exec head -750 {} \; -exec echo travis_fold":"end:crashlog \; || true
find: `/home/travis/Library/Logs/DiagnosticReports': No such file or directory
travis_time:end:08a6e160:start=1523527631103495261,finish=1523527631110054064,duration=6558803
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:0fd884f9
$ dmesg | grep -i kill
[   10.907031] init: failsafe main process (1096) killed by TERM signal

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

[ibabushkin]

@topecongiro That's what happens when I run rustfmt (even in a newer version) on then code. Tidy checks fail because a line is too long (which I edited by hand before).

[topecongiro]

@ibabushkin Thanks, that is interesting...would you mind checking the version of rustfmt? It should be rustfmt 0.4.1-nightly (e784712 2018-04-09) or 0.3.8-nightly (346238f 2018-02-04), depending on the toolchain you use.

[ibabushkin]

@topecongiro Surprisingly, a rustfmt installed/updated via rustup component add rustfmt-preview --toolchain nightly, as per the instructions, outputs the following:

$ rustfmt --version
0.8.4 ()

However, I suggest we move this discussion somewhere else, as this somewhat derails the discussion in this issue :)

[topecongiro]

@ibabushkin Ok, looks like you already have rustfmt which has been installed via cargo install rustfmt. Please remove ~/.cargo/bin/rustfmt and ~/.cargo/bin/cargo-fmt, then run the rustup command. If this does not work, would you mind opening an issue in rustfmt repo? I could help you there.

My appologies for derailing the discussion in this issue.

[nikomatsakis]

Why does this exist, given that Region has a case for inference variables?

[ibabushkin]

The following enum is defined in librustdoc, and used in the code I moved over:

#[derive(Eq, PartialEq, Hash, Copy, Clone, Debug)]
enum RegionTarget<'tcx> {
    Region(Region<'tcx>),
    RegionVid(RegionVid)
}

I did not find any explanation or reasoning there, so I duplicated the enumeration (as stated in the TODO comment). I suppose we could eliminate this type altogether given what you said about Region, but that would require further testing. Maybe @Aaron1011 can tell us more about the bits of code here that use this type.

[nikomatsakis]

So I have sort of a meta question:

Is the goal here to just kind of "rubber stamp" moving some code out of rustdoc into rustc, or to really design a nice API?

If the latter, obviously, I'd want to think over more carefully what's going on here. If the former, we can add some comments like "stay away from this code" and land with lighter review, I imagine.

Overall, I do want to add more stable APIs for working with traits that consumers like rustdoc etc can use. This particular API is going in precisely the opposite direction of what I had in mind, though, in terms of exposing internal concepts like vtable that I'd like to remove. (It's kind of interesting, though, since many of the concepts I'd like to remove are basically those that rustdoc wants -- i.e., those that are tightly tied to the source code structure.)

[ibabushkin]

Well, the point you bring up is an important one -- I initially was working on adding support for auto traits to rust-semverver, and was pointed at this code that was supposedly trivial to move. I was quickly convinced that moving it over in a clean fashion would require significantly more design and implementation work. So yes, I am aware this module is terribly designed, but given that the original code was fairly complex and deeply entangled with rustdoc's internals, I still went with this route.

To conclude: I agree that a properly designed refactor for this machinery would be much nicer, but that would come at a cost of a time penalty for what I was working on, and require some work (I'd love to help, but I sadly am somewhat short on time).

[nikomatsakis]

OK, I skimmed in a bit more depth. I think I see now what this code is trying to do -- it seems like it's trying to find, if there is an auto trait impl, what the transitive conditions are on this impl and report those back to the user? That's interesting. Is there an example of the resulting output?

(Also, I was wrong to say it exposes VtableImpl, which is good.)

[ibabushkin]

As far as I am aware, auto trait instances are being generated in rustdoc output. I've not seen examples of that however (and couldn't find any right now). The original PR shows the functionality in a quite detailed way though.

There is also a rust-semverver branch which uses some of the functionality in a rudimentary way.

Regarding the inner working of the code: Yes, your description pretty much coincides with my mental model of it.

[bors]

☔ The latest upstream changes (presumably #49939) made this pull request unmergeable. Please resolve the merge conflicts.

[ibabushkin]

Do we have a definite conclusion on this? Any further changes needed? A proper, clean rewrite from scratch of the mechanics underneath?

[Aaron1011]

Well, the point you bring up is an important one -- I initially was working on adding support for auto traits to rust-semverver, and was pointed at this code that was supposedly trivial to move. I was quickly convinced that moving it over in a clean fashion would require significantly more design and implementation work.

@ibabushkin: I'm sorry about that - I should have taken a closer look at what work would be required before telling you that.

I'd be happy to help out with this in any way I can, especially if parts of my code are unclear or poorly written.

[ibabushkin]

On 2018-04-21, Aaron Hill wrote: @ibabushkin: I'm sorry about that - I should have taken a closer look at what work would be required before telling you that. I'd be happy to help out with this in any way I can, especially if parts of my code are unclear or poorly written.

Well, I'd be glad to take part in a "clean" rewrite of the functionality in librustc if this current PR still causes too many headaches. No feedback yet on the definite state of things.

[nikomatsakis]

OK. So, I'm of mixed minds here. First off, a question:

What does this code do for a type like this:

struct Foo<T: Iterator> {
  data: <T as Iterator>::Item
}

i.e., what where clauses does it give back?

In general, the "always correct" answer for send impls is actually way simpler than what this code does. The correct answer is just to enumerate the field types. So for Foo, the right impl is:

impl<T: Iterator> Send for Foo
  where <T as Iterator>::Item: Send
{ }

This is possible only because Send is an autotrait, and therefore treated "coinductively" -- in short, cycles are ok, as long as those cycles consist only of other autotraits. However, the transformation that this code does produces a more "minimal" result, better suited for display to end users for sure.

I don't really feel like I have the bandwidth to think carefully about whether this API is the kind we would like to support longer term. I suspect something like it would be fine, but we'd want to make changes.

I think I'm inclined therefore to r+ this, since it's basically just moving existing code around it, but with the warning that as the work on chalk etc integration proceeds we may want to make radical changes to the API.

[nikomatsakis]

@ibabushkin

I think I'm inclined therefore to r+ this, since it's basically just moving existing code around it, but with the warning that as the work on chalk etc integration proceeds we may want to make radical changes to the API.

I specifically wanted to know how you felt about that =)

[ibabushkin]

I'm in the same boat there -- the API as-is, after the move, certainly isn't pretty. Given that only two consumers of it are to be expected in the foreseeable future, and that I'd like to get back to this at some point in the future, I am totally fine with it being a temporary state of things.

[ibabushkin]

Am I mistaken or are the build failures unrelated to the changes?

[nikomatsakis]

@ibabushkin you'd think so... but it seems repeatable. =)

[nikomatsakis]

Huh, re-reading the code I can't really see how those failures could come about. It feels like they have to be spurious.

[nikomatsakis]

@bors retry

[nikomatsakis]

@bors r- -- well, let me check

[TimNN]

@bors r- -- well, let me check

Ping from triage, @nikomatsakis, any updates?

[nikomatsakis]

Argh I forgot I had left this on that note. No, I've no idea how this could be anything but spurious. Shall we give it another try?

[nikomatsakis]

@bors r+

[bors]

📌 Commit 8901392 has been approved by nikomatsakis

[bors]

⌛ Testing commit 8901392 with merge 4725417...

[bors]

☀️ Test successful - status-appveyor, status-travis
Approved by: nikomatsakis
Pushing 4725417 to master...