Unclear compiler error when impl Trait return value captures non-'static argument

[ramosbugs]

(cc @estebank)

I tried this code (the explicit 'static lifetime is unnecessary but emphasizes the implicit 'static requirement of impl Trait):

fn foo<T>(a: T) -> impl Iterator<Item = String> + 'static
where
    T: std::fmt::Display
{
    std::iter::once(a.to_string())
}

fn bar(a: &str) -> impl Iterator<Item = String> + 'static {
    foo(a)
}

See playground.

I expected to see this happen:

Successful compilation or a clear error message suggesting that impl Trait can't be used in this way due to the non-'static lifetime of foo's argument a.

Instead, this happened:

error[E0759]: `a` has an anonymous lifetime `'_` but it needs to satisfy a `'static` lifetime requirement
 --> src/lib.rs:9:9
  |
8 | fn bar(a: &str) -> impl Iterator<Item = String> + 'static {
  |           ---- this data with an anonymous lifetime `'_`...
9 |     foo(a)
  |         ^ ...is captured here...
  |
note: ...and is required to live as long as `'static` here
 --> src/lib.rs:8:20
  |
8 | fn bar(a: &str) -> impl Iterator<Item = String> + 'static {
  |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
help: consider changing the `impl Trait`'s explicit `'static` bound to the lifetime of argument `a`
  |
8 | fn bar(a: &str) -> impl Iterator<Item = String> + '_ {
  |                                                   ^^
help: alternatively, add an explicit `'static` bound to this reference
  |
8 | fn bar(a: &'static str) -> impl Iterator<Item = String> + 'static {
  |           ^^^^^^^^^^^^

foo's return value isn't dependent on a's lifetime since to_string converts it to an owned String. Returning Box<dyn Iterator<Item = String>> instead of using impl Trait fixes the error.

Neither of the compiler's suggestions are the "right" fix here, since I'd like the return value to have a 'static lifetime but don't want to require a to have one.

I think the ideal fix (to the error message) would be something along the lines of:

• a short explanation of why impl Trait doesn't allow the return value not to depend on the lifetime of a (maybe with a tracking issue if this limitation is being addressed)
• an additional suggestion to use Box<dyn Trait> and/or a concrete type

Meta

rustc --version --verbose:

rustc 1.50.0 (cb75ad5db 2021-02-10)
binary: rustc
commit-hash: cb75ad5db02783e8b0222fee363c5f63f7e2cf5b
commit-date: 2021-02-10
host: x86_64-apple-darwin
release: 1.50.0

[jyn514]

This seems similar to #79415.

[estebank]

an additional suggestion to use Box<dyn Trait> and/or a concrete type

That might be reasonable, it'll just be tricky to identify the specific cases where this action would be appropriate to try and avoid a diagnostic that is too verbose for uncommon cases.

This seems similar to #79415.

Good catch! It is a similar case, but will likely need to be handled independently. Let's not close either :)

[estebank]

There's no way currently to express that an impl Trait doesn't capture all the input lifetimes. Once type Ret<'a> = impl Trait + 'a; is stabilized you could express the behavior you're looking for, but for now the only option are the ones identified in the report (explicit type and Box<dyn Trait>).

[rami3l]

@ramosbugs @estebank

A friend of mine just encountered something similar, and I found this workaround from #34511, which seems to fit your problem here. (However, I haven't personally figured out why this works... Any help will be appreciated!)

I tried to fix your code using this method:

fn foo<T>(a: T) -> impl Iterator<Item = String> + 'static
where
    T: std::fmt::Display,
{
    std::iter::once(a.to_string())
}

// What are those?
trait Captures<'a> {}
impl<'a, T: ?Sized> Captures<'a> for T {}

fn bar1<'a>(a: &'a str) -> impl Iterator<Item = String> + 'static + Captures<'a> {
    foo(a)
}

---

Update: Now this workaround can be automatically applied via the fix_hidden_lifetime_bug.rs library.

Update: This could be resolved in language edition 2024 which will, as currently planned, allow you to use TAIT to avoid over-capturing: #115822, HackMD

[gh2o]

I'm having a similar issue here: I can't seem to force the returned impl Animal type to be 'static no matter what I do. Unfortunately, the workaround above doesn't work either because I am unable to name the lifetimes contained in the FnOnce closure.

trait Animal {}

struct Horse;
impl Animal for Horse {}

// Using `impl Trait` here to avoid unnecessary heap allocation if `k < 10`
fn make_animal(set: impl FnOnce(u32)) -> impl Animal {
    set(3);
    Horse // alternatively, return value of complex/unnamed type that impls Animal
}

fn expand_zoo(zoo: &mut Vec<Box<dyn Animal>>) {
    let mut k = 0;
    let x = make_animal(|r| k = r);
    if k >= 10 {
        zoo.push(Box::new(x));
    }
}

[estebank]

@gh2o Given what you're trying to do, you might have to resort to leveraging interior mutability and runtime reference counting.

[estebank]

Triage, the current output for the original report is

error[E0700]: hidden type for `impl Iterator<Item = String> + 'static` captures lifetime that does not appear in bounds
 --> src/lib.rs:9:5
  |
8 | fn bar(a: &str) -> impl Iterator<Item = String> + 'static {
  |           ---- hidden type `impl Iterator<Item = String> + 'static` captures the anonymous lifetime defined here
9 |     foo(a)
  |     ^^^^^^

[t-cadet]

I also ran into this issue, my use case is a function that takes an iterator, reads it to build & send a request, and returns an iterator on the response. Something like:

fn lifetime_issue() {
    let v = Vec::new();
    let _foo = foo(&v);
    drop(v); // do something with v
}

fn foo<'output, 'input_item, 'input_iter>(_: impl IntoIterator<Item = &'input_item String> + 'input_iter) -> impl Iterator<Item = String> + 'output
{
    std::iter::empty()
}

error[E0505]: cannot move out of `v` because it is borrowed
 --> src/main.rs:6:10
  |
5 |     let _foo = foo(&v);
  |                    -- borrow of `v` occurs here
6 |     drop(v); // do something with v
  |          ^ move out of `v` occurs here
7 | }
  | - borrow might be used here, when `_foo` is dropped and runs the destructor for type `impl Iterator<Item = String> + '_`

For more information about this error, try `rustc --explain E0505`.
error: could not compile `lf` due to previous error

A minimal version of the issue:

struct NotCopy;

trait Marker {}
impl<'a> Marker for &'a NotCopy {}
impl Marker for () {}

fn lifetime_issue() {
    let nc = NotCopy;
    let _foo = foo(&nc);
    drop(nc); // do something with nc
}

fn foo(_: impl Marker) -> impl Marker {}

error[E0505]: cannot move out of `nc` because it is borrowed
  --> src/main.rs:13:10
   |
12 |     let _foo = foo(&nc);
   |                    --- borrow of `nc` occurs here
13 |     drop(nc); // do something with nc
   |          ^^ move out of `nc` occurs here
14 | }
   | - borrow might be used here, when `_foo` is dropped and runs the destructor for type `impl Marker`

For more information about this error, try `rustc --explain E0505`.
error: could not compile `lf2` due to previous error

Rustc version:

$ rustc --version --verbose
rustc 1.67.1 (d5a82bbd2 2023-02-07)
binary: rustc
commit-hash: d5a82bbd26e1ad8b7401f6a718a9c57c96905483
commit-date: 2023-02-07
host: x86_64-unknown-linux-gnu
release: 1.67.1
LLVM version: 15.0.6

[rami3l]

@estebank Regarding your reply (#82171 (comment)):

There's no way currently to express that an impl Trait doesn't capture all the input lifetimes. Once type Ret<'a> = impl Trait + 'a; is stabilized you could express the behavior you're looking for, but for now the only option are the ones identified in the report (explicit type and Box<dyn Trait>).

I still can't understand how TAIT could help convey the idea that in the original example (#82171 (comment)), bar's output should be 'static regardless of the input &str's lifetime 'a (Captures<'a> just captures 'a for nothing at all):

trait Captures<'a> {}
impl<'a, T: ?Sized> Captures<'a> for T {}

fn bar1<'a>(a: &'a str) -> impl Iterator<Item = String> + 'static + Captures<'a> {
    foo(a)
}

If I understand correctly, it seems to me that, with TAIT:

type Bar<'a> = impl Iterator<Item = String> + 'a;
fn bar2(a: &str) -> Bar {
    foo(a)
}

... should be equivalent to:

fn bar3<'a>(a: &'a str) -> impl Iterator<Item = String> + 'a {
    foo(a)
}

... which has a weaker bound in the return type.

Could you please elaborate a bit? Many thanks!

[aliemjay]

I still can't understand how TAIT could help convey the idea that in the original example (#82171 (comment)), bar's output should be 'static regardless of the input &str's lifetime 'a (Captures<'a> just captures 'a for nothing at all):

TAIT can be used to indicate that the return type of foo is not generic over T:

#![feature(type_alias_impl_trait)]

mod mod_foo {
    pub type ReturnTy = impl Iterator<Item = String> + 'static;
    pub fn foo<T>(a: T) -> ReturnTy
    where
        T: std::fmt::Display,
    {
        std::iter::once(a.to_string())
    }
}

fn bar(a: &str) -> impl Iterator<Item = String> + 'static {
    mod_foo::foo(a)
}

The need for a separate mod is unfortunate, but hopefully it shouldn't be necessary when TAIT is stabilized.

The default behavior for impl-trait in return position is to be generic over all type parameters in scope. Desugared to TAIT, it is like:

pub type ReturnTy<T> = impl Iterator<Item = String> + 'static;
pub fn foo<T>(a: T) -> ReturnTy<T>
// ...

[rami3l]

@aliemjay Wow, that's somehow surprising to me.

Apart from having to add a new module (which seems more like a current limitation), I was also surprised by having to change the callee rather than the caller, since the compile time error is usually found on the caller's side:

13 | fn bar(a: &str) -> impl Iterator<Item = String> + 'static {
   |           ---- hidden type `impl Iterator<Item = String> + 'static` captures the anonymous lifetime defined here

... and since the "wrong" callee code compiles alright in isolation, this incorrectness might never be exposed to a lib author if (s)he hasn't included the right kind of test cases.

It seems to me that it has always been the API consumers' responsibility to perform the Captures hack. After all, they are the ones who introduce the 'a in fn bar<'a>, so it makes more sense to me to let them clarify that 'a will be ignored in the return type.

Anyway, thanks a lot for your detailed explanation!