async block with futures::stream::Buffered is not Send

[mikeyhew]

This issue was originally reported in rust-lang/futures-rs#2636, but it looks like a bug in the compiler.

Here is the example from that issue (posted by @Tuetuopay):

use futures::stream::{empty, StreamExt};
use std::future::ready;

fn send<T: Send>(_: T) {}

fn main() {
    send(async {
        empty().map(ready::<&()>).buffered(0).next().await
    });
}

This should compile without issues, however rustc outputs this error message:

error: higher-ranked lifetime error
 --> src/main.rs:7:5
  |
7 | /     send(async {
8 | |         empty().map(ready::<&()>).buffered(0).next().await
9 | |     });
  | |______^
  |
  = note: could not prove `impl futures::Future<Output = Option<&'a ()>>: std::marker::Send`

This occurs on current nightly (2022-11-11) and stable. Playground link: https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=43484ffb5887c99e0845737c0241c853

There appears to be some issue in how the compiler checks the async block. If you call .boxed() on the stream before calling .buffered() on it, the error goes away:

use futures::stream::{empty, StreamExt};
use std::future::ready;

fn send<T: Send>(_: T) {}

fn main() {
    send(async {
        empty().map(ready::<&()>).boxed().buffered(0).next().await
    });
}

This compiles without errors on current nightly and stable. Playground link: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=9c086487d1ff76b309796efa45846f67

[compiler-errors]

This comment isn't particularly helpful except for other contributors, but this is another manifestation of the common "futures have trouble with higher-ranked lifetimes and auto traits" issue that comes from lifetime erasure in generator interiors, I think...

[mikeyhew]

@compiler-errors out of curiosity are there any GitHub issues or Zulip chat histories worth reading that describe the general problem you're referring to?

[zirconium-n]

@compiler-errors out of curiosity are there any GitHub issues or Zulip chat histories worth reading that describe the general problem you're referring to?

#64552
#92449

[eholk]

Out of curiosity, does the issue still show up if you do rustc -Z drop-tracking?

I would guess yes since drop tracking doesn't do much with lifetime issues, but it'd still be interesting to know.

[khuey]

-Z drop-tracking crashes the compiler.

thread '<unnamed>' panicked at 'type variables should not be hashed: _#11t', /rustc/b7bc90fea3b441234a84b49fdafeb75815eebbab/compiler/rustc_ type_ir/src/lib.rs:679:17

[compiler-errors]

@khuey I've got a fix for that ICE coming up.

[vincenzopalazzo]

Now, this issue can be close with #104753 ?

cc @compiler-errors

[compiler-errors]

No, #104753 only fixes this ICE (#104382 (comment)) mentioned by khuey.

[khuey]

After #104753 -Z drop-tracking makes the error message a little more specific, but is otherwise unchanged.

khuey@minbar:~/dev/scratch/rust-scratch$ RUSTFLAGS="-Z drop-tracking" cargo build
   Compiling rust-scratch v0.1.0 (/home/khuey/dev/scratch/rust-scratch)
error: implementation of `std::marker::Send` is not general enough
 --> src/main.rs:7:5
  |
7 | /     send(async {
8 | |         empty().map(ready::<&()>).buffered(0).next().await;
9 | |     });
  | |______^ implementation of `std::marker::Send` is not general enough
  |
  = note: `std::marker::Send` would have to be implemented for the type `&'0 ()`, for any lifetime `'0`...
  = note: ...but `std::marker::Send` is actually implemented for the type `&'1 ()`, for some specific lifetime `'1`

error: could not compile `rust-scratch` due to previous error

[jbg]

rustc also seems to have trouble identifying the location of the error. In a large application with a top-level async fn that branches out to call many others, if you introduce code similar to that in the first comment in a function deep within the application, the error: higher-ranked lifetime error points at the top-level fn (the 'root' of the 'tree of futures'), making debugging difficult.