-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refined trait implementations #3245
Conversation
I'm nominating this for @rust-lang/lang attention -- this is a principle that we have frequently enunciated internally, so I encouraged @tmandry to make an RFC for it. We don't necessarily need to rush to approve it, but I wanted folks to see it and give it a read. |
So, two big questions for this:
|
I'm going to answer the above question assuming we have return-position impl Trait in traits and support dynamic dispatch on those traits, but keep in mind that neither of those features have been through the RFC process yet.
Dynamic dispatch of return-position impl Trait wouldn't be affected by this proposal. It's like you said. More concretely, if you have something like this: impl Foo {
fn stuff(&self) -> impl Bar;
} then the compiler can generate an impl that looks something like this: impl Foo for dyn Foo {
fn stuff(&self) -> impl Bar;
} and that impl is free to choose whatever concrete type it wants for the return type of
This doesn't currently interact with dynamic dispatch because of the |
Oh right, I completely forgot about the In that case, I would be very interested in seeing The first case would be to allow trait implementations to provide their own types. For example, it would be nice to let iterators return However, the second case is to explicitly remove types from the implementation so that they aren't exposed to users of the API. And with the current proposal, the second case is always true for the default implementations; you can't choose to expose the type of an Note that the same is true for all forms of refinement, not just the defaults -- for example, if you want to not require |
I don't think having provided methods with a refined API is a good idea. That would mean you could break existing code that uses your impl by overriding a provided method to return a different type. But that's antithetical to the idea of this RFC, which is to let impls add to existing guarantees, not take them away. So the provided method API should always match the trait, in my opinion. We could allow this but say it's a breaking change to refine a method on an existing impl in a way that doesn't agree with the provided method API. This would mean that new impls are allowed to override it, as well as impls that don't care about semver guarantees. But this violates the principle of least surprise – users expect the API of a method provided by a trait to match the API of the trait. It would complicate the mental model of the feature in addition to introducing a new semver hazard for library authors. So, on second thought I'll have to revise my earlier statement. I don't think we can backward-compatibly change |
I completely didn't consider the effect my proposal had on breaking changes and now understand completely why it would not work that way. It really does seem like the type-alias-impl-trait feature would be required in these sorts of situations, but that will probably be finished before this one anyway. |
We discussed this in the @rust-lang/lang meeting today -- we were thinking, @tmandry, that it would be better to make the I would personally be in favor of having a lint warning if the signature in the impl diverges from the trait and there is no This rule would also amend the existing unsafe RFC to require |
I think using So if we go this direction, I think we should consider tweaking the formulation slightly. In the current edition it would work just as you said. In future editions the refined signature should always be available, but there is a deny-by-default or warn-by-default lint in cases where there is no |
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
I think this RFC may exacerbate existing problems with name resolution. If a refined impl is semantically equivalent to an inherent impl with the refined signature plus a trait impl that calls into the refined impl. With those semantics, every refined impl runs into the same unintuitive name resolution that inherent functions have but with the added problem that there's no way to explicitly call the trait method with the original signature. It all depends on the context in which the method is called, which means that the "correct" form can change just by moving code from place to place. I think that refined impls are a good idea, but that we should tackle this name resolution problem head-on before introducing language features that provide more opportunities to cause this kind of confusion. For example: in the case where a user calls an unsafe trait method that has been refined to be safe, would that trigger an unnecessary unsafe block lint? Here's an example of this situation right now using a name collision between an inherent impl and a trait method: trait Print {
unsafe fn print(&self);
}
struct Foo;
impl Foo {
fn print(&self) {
println!("Inherent");
}
}
impl Print for Foo {
unsafe fn print(&self) {
println!("Trait");
}
}
fn main() {
let x = Foo;
unsafe {
x.print();
}
fn foo<T: Print>(x: T) {
unsafe {
x.print();
}
}
foo(x);
} This triggers a lint on the first Foo::print(&x);
// or
<Foo as Print>::print(&x); And to be clear, I think that It's worth noting that the community generally avoids naming collisions between inherent impls and trait methods for this exact reason. If the intention is to encourage users to use impl refinement when possible, then I think we should at least give users a way to refer to the unrefined method in concrete contexts. However I think that defaulting to the refined signature in these contexts makes concrete rust code difficult to refactor by default. |
I wouldn't say it's semantically equivalent. It's really even stronger than that: The trait method itself has the refined signature when you know the type of the receiver. That's why the RFC says that you will get the refined signature even when using UFCS.
Good question. I think either behavior could be reasonable: the lint could fire, or the lint could have an exception for trait methods that are marked unsafe. To some extent it's an empirical question what's more useful vs noise in the lint. We could also have a "sub-lint" for this exact situation so people could customize the behavior themselves, but I'm not sure it's worth it.
I don't have that expectation: when moving from a non-generic to a generic context you are always losing some information (indeed that is the point), which can require modifying your code a bit. For example, we already have this problem with associated types today. Concrete code gets to assume associated type values, while generic code only gets the bounds of the type on the trait itself (plus bounds from any where clauses on your function). I'd also like to make it possible to bound method return values in the same way you can bound associated types, which would recover some of the ability to "just add a where clause" and make the problem go away. There is even a version of that idea that allows you to bound the argument types that are accepted. But I think all this is out of scope for this particular RFC. Maybe it's worth digging in more to some example scenarios that this proposal would make harder. My sense is that the benefits of the feature are overall worth it. But it sounds like you might have experience with situations that give you a different impression.
I can think of another strong reason to avoid this, which is that shadowing makes it unclear which implementation is actually being called. That's not the case in this proposal.
Perhaps we should, but I can't think of a scenario where people would actually use this. UFCS is generally only there to reach for when you need it. Would users proactively write their code in a less convenient fashion just so they can avoid changing it if they were to make it generic later? I think not. It seems more reasonable to write your code using Also, we were discussing this some yesterday and realized you can implement this "ultra-UFCS" in a macro, so if people actually want it they could do that. Finally, I wouldn't say the intention is to encourage users to use impl refinement whenever possible. It's a feature that is there if you need it. Refining an impl creates a stronger contract than the one in the trait, which entails a tradeoff that implementers can choose to make or not.
What alternative would you propose? I agree it has this tradeoff, but that seems lower cost than requiring that users use a special syntax to invoke a refined API. I don't anticipate that this feature will get used everywhere. In fact with the version that @nikomatsakis mentioned, implementers would always have to opt in by adding the |
Here's one example where I think refined return types cause confusion. Starting from a relatively egregious example that has nothing to do with trait refinement: Click to expandtrait Iterator {
fn next(&mut self) -> bool;
fn len(&mut self) -> usize {
println!("Slow");
let mut len = 0;
while self.next() {
len += 1;
}
len
}
}
trait LenIterator: Iterator {
fn len(&self) -> usize;
}
#[derive(Clone)]
struct BasicIterator(usize);
impl Iterator for BasicIterator {
fn next(&mut self) -> bool {
if self.0 == 0 {
false
} else {
self.0 -= 1;
true
}
}
}
impl LenIterator for BasicIterator {
fn len(&self) -> usize {
println!("Fast");
self.0
}
}
trait IntoIterator {
type Iterator: Iterator;
fn into_iter(self) -> Self::Iterator;
}
impl IntoIterator for usize {
type Iterator = BasicIterator;
fn into_iter(self) -> Self::Iterator {
BasicIterator(self)
}
} These five calls to
fn call_len<T: Iterator>(mut iter: T) {
iter.next();
iter.len();
}
fn call_len_2<T: LenIterator>(mut iter: T) {
iter.next();
iter.len();
}
fn main() {
10.into_iter().len();
Iterator::len(&mut 10.into_iter());
LenIterator::len(&mut 10.into_iter());
call_len(10.into_iter());
call_len_2(10.into_iter());
} This is very similar to the existing This demonstrates how unintuitive name resolution can already be. In this case, I'd argue that it's unintuitive but manageable. However, with the introduction of refined return types and RPITIT we can cause additional problems by changing trait IntoIterator {
fn into_iter(self) -> impl Iterator;
}
impl IntoIterator for usize {
fn into_iter(self) -> impl Iterator;
}
impl IntoIterator for isize {
fn into_iter(self) -> impl Iterator + LenIterator;
} We can end up in situations where the Obviously RPITIT is not part of this RFC, but it's explicitly mentioned in the text so I feel that it's worth addressing. Refined trait implementations may also lead to somewhat perverse incentives when implementors write trait impls. For example: trait Invert {
/// # Safety
///
/// All of the items in the container must sum to 1.
unsafe fn invert(&mut self);
}
impl Invert for [f32] {
fn invert(&mut self) {
// We want to be able to call invert safely, so we don't rely on the precondition
let total = self.iter().sum::<f32>();
for i in self.iter_mut() {
*i = total / *i;
}
}
} Let's assume that the user is implementing some foreign trait and can't change its signature. That trait may have some unsafe methods with some preconditions. Taking advantage of those preconditions would result in a faster implementation at the cost of being unsafe. However, the user can also write a totally safe implementation that doesn't take advantage of any preconditions. In exchange for performance, they can call their code safely and avoid writing any unsafe code. I think it's fair to say that many users would choose those tradeoffs when writing their own code. However, this subverts the entire purpose of the trait - there's not much point in guaranteeing preconditions if implementors stand to benefit from ignoring them. I would argue that not allowing users to call unsafe methods as safe prevents this kind of perverse incentive. In this particular case, it's typically best to implement the trait unsafely and provide an inherent impl that checks preconditions and delegates to the trait. I also don't see any situations in the motivation that aren't arguably better solved by adding an inherent impl with a more permissive signature. In fact, a proc macro could probably achieve most of these goals without any new language features since inherent impls are still accessible and prioritized when the type is known concretely. Are there any cases where an inherent impl with the more permissive signature is less useful? |
@rfcbot fcp merge I think we've got consensus on most of this here -- the main question is whether, in the next edition, |
Team member @nikomatsakis has proposed to merge this. The next step is review by the rest of the tagged team members: Concerns:
Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up! See this document for info about what commands tagged team members can give me. |
@rfcbot concern unresolved-question-for-next-edition Let's add an official "unresovled question" about the behavior in the next edition. I think the options to choose between are:
|
(those commands were posted earlier, but since there was no T-lang label, rfcbot ignored me) |
I wanted to note a scenario in Rust today where a programmer must write a refined implementation without the expectation it can be called, or perhaps even the expectation it cannot be called. That is, a scenario wherein programmers must write an implementation they do not want exposed in their APIs. The scenario is that mentioned in #2829: trait items must be implemented even if their bounds cannot be met. Say we have pub trait Example {
type Item;
fn f(&self) -> Self::Item where Self::Item: Copy;
} And we attempt: impl Example for String {
type Item = Self;
} It errors; we must supply the method. So we add: fn f(&self) -> Self::Item where Self::Item: Copy {
String::new()
} But this does not compile either. However if we remove the bound: fn f(&self) -> Self::Item {
String::new()
} Then everything compiles, but you can rest assured the method can't be called. (Maybe your dummy method does something undesirable, or maybe those bounds were crucial for some other reason.) Allowing unsatisfiably bounded items to be omitted is not required to improve the situation (although I would personally love to see that). If the trivial bounds RFC was stabilized, you wouldn't need the refined method in order to compile. (Probably you would get a warning, which is fine; I'm not sure a trivial bounds lint exists yet.) This does imply that unsatisfiable bounds on implementations must be respected (i.e. the implementation in the last link should continue to error as the "looks refined but isn't" version on stable does today). |
Emphasize the possibility of continuing to use `#[refine]` in future editions, and list out all the options under Unresolved Questions. Include the attribute in examples. Simplify the usage of `#[refine]` in existing editions so that it is needed to refine any aspect of an API, instead of new features being refine-able by default and old ones needing the attribute. Move any discussion of *not* doing a soft transition to Unresolved Questions to simplify the Reference Level Explanation a bit.
I've updated the RFC with options for how to handle |
From #3245 (comment):
Yes, it is surprising. I found the behavior documented in the reference, which is an improvement from when this was only documented in compiler source code :). The example you gave specifically depends on the fact that we have a The need for a trait like Even without this feature I think it's a bad idea to define a method with the same name as another method on another common trait (especially on a supertrait). That would be confusing for readers in any case. So I think you want to avoid situations like this anyway where possible. To the extent that this feature interacts negatively with the already-confusing method dispatch rules, I think we should focus on improving those rules.
Under this proposal the statement "the trait promises that the return type will be opaque and only implement You cite associated types – this proposal makes associated functions and consts work more like associated types. In all cases you get more information if you know the concrete type. If all you know is the trait, then you get only what the trait says. I don't really see how this is inherently confusing or problematic. |
From #3245 (comment):
How is allowing a user to make this tradeoff a perverse incentive? Just because a method can be implemented safely does not necessarily make it slower. In those cases where it's not you always want to call the same implementation, and making it safe means you have fewer In the cases where you have a safe but slower implementation, yes, using an inherent method with a different name might be best. But if a user knows Rust well enough to consider writing ...but anyway, now we are arguing about RFC #2316 which was already accepted! |
Thanks to @djkoloski and @QuineDot for your feedback. I've updated the RFC with new drawbacks section, Refactoring, and a corresponding future possibilities section, Bounding refined items. I think these are worth considering when evaluating the RFC. I also added interactions with Method dispatch and Unsatisfiable trait members. Since I already added the unresolved question on |
@rfcbot resolve unresolved-question-for-next-edition |
🔔 This is now entering its final comment period, as per the review above. 🔔 |
The final comment period, with a disposition to merge, as per the review above, is now complete. As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed. This will be merged soon. |
@tmandry Want to file a tracking issue, and add that to the RFC? |
The RFC says it “generalizes the safe_unsafe_trait_methods RFC”, then describes the status quo as “This allows code that knows it is calling a safe implementation of an unsafe trait method to do so without using an unsafe block. In other words, this works today” with a code example without a My point being: The RFC as written right now is awefully ambiguous about whether or not |
It doesn't seem realistic that we will avoid doing a soft transition at all. Since it was decided that we would definitely adopt #[refine] in the current edition, I don't think this question is relevant anymore.
Would it make more sense to simply retcon rust-lang/rust#87919 as being the tracking issue for this RFC? Since this is technically just a broader version of RFC #2316. |
@joshtriplett Tracking issue has been opened as rust-lang/rust#100706. @steffahn Thanks, I clarified in the text that I also removed the unresolved question around whether we need a soft transition from the current behavior. With it being decided to use @clarfonthey Sorry, race condition! |
…jackh726 Stabilize `async fn` and return-position `impl Trait` in trait # Stabilization report This report proposes the stabilization of `#![feature(return_position_impl_trait_in_trait)]` ([RPITIT][RFC 3425]) and `#![feature(async_fn_in_trait)]` ([AFIT][RFC 3185]). These are both long awaited features that increase the expressiveness of the Rust language and trait system. Closes rust-lang#91611 [RFC 3185]: https://rust-lang.github.io/rfcs/3185-static-async-fn-in-trait.html [RFC 3425]: https://rust-lang.github.io/rfcs/3425-return-position-impl-trait-in-traits.html ## Updates from thread The thread has covered two major concerns: * [Given that we don't have RTN, what should we stabilize?](rust-lang#115822 (comment)) -- proposed resolution is [adding a lint](rust-lang#115822 (comment)) and [careful messaging](rust-lang#115822 (comment)) * [Interaction between outlives bounds and capture semantics](rust-lang#115822 (comment)) -- This is fixable in a forwards-compatible way via rust-lang#116040, and also eventually via ATPIT. ## Stabilization Summary This stabilization allows the following examples to work. ### Example of return-position `impl Trait` in trait definition ```rust trait Bar { fn bar(self) -> impl Send; } ``` This declares a trait method that returns *some* type that implements `Send`. It's similar to writing the following using an associated type, except that the associated type is anonymous. ```rust trait Bar { type _0: Send; fn bar(self) -> Self::_0; } ``` ### Example of return-position `impl Trait` in trait implementation ```rust impl Bar for () { fn bar(self) -> impl Send {} } ``` This defines a method implementation that returns an opaque type, just like [RPIT][RFC 1522] does, except that all in-scope lifetimes are captured in the opaque type (as is already true for `async fn` and as is expected to be true for RPIT in Rust Edition 2024), as described below. [RFC 1522]: https://rust-lang.github.io/rfcs/1522-conservative-impl-trait.html ### Example of `async fn` in trait ```rust trait Bar { async fn bar(self); } impl Bar for () { async fn bar(self) {} } ``` This declares a trait method that returns *some* [`Future`](https://doc.rust-lang.org/core/future/trait.Future.html) and a corresponding method implementation. This is equivalent to writing the following using RPITIT. ```rust use core::future::Future; trait Bar { fn bar(self) -> impl Future<Output = ()>; } impl Bar for () { fn bar(self) -> impl Future<Output = ()> { async {} } } ``` The desirability of this desugaring being available is part of why RPITIT and AFIT are being proposed for stabilization at the same time. ## Motivation Long ago, Rust added [RPIT][RFC 1522] and [`async`/`await`][RFC 2394]. These are major features that are widely used in the ecosystem. However, until now, these feature could not be used in *traits* and trait implementations. This left traits as a kind of second-class citizen of the language. This stabilization fixes that. [RFC 2394]: https://rust-lang.github.io/rfcs/2394-async_await.html ### `async fn` in trait Async/await allows users to write asynchronous code much easier than they could before. However, it doesn't play nice with other core language features that make Rust the great language it is, like traits. Support for `async fn` in traits has been long anticipated and was not added before due to limitations in the compiler that have now been lifted. `async fn` in traits will unblock a lot of work in the ecosystem and the standard library. It is not currently possible to write a trait that is implemented using `async fn`. The workarounds that exist are undesirable because they require allocation and dynamic dispatch, and any trait that uses them will become obsolete once native `async fn` in trait is stabilized. We also have ample evidence that there is demand for this feature from the [`async-trait` crate][async-trait], which emulates the feature using dynamic dispatch. The async-trait crate is currently the rust-lang#5 async crate on crates.io ranked by recent downloads, receiving over 78M all-time downloads. According to a [recent analysis][async-trait-analysis], 4% of all crates use the `#[async_trait]` macro it provides, representing 7% of all function and method signatures in trait definitions on crates.io. We think this is a *lower bound* on demand for the feature, because users are unlikely to use `#[async_trait]` on public traits on crates.io for the reasons already given. [async-trait]: https://crates.io/crates/async-trait [async-trait-analysis]: https://rust-lang.zulipchat.com/#narrow/stream/315482-t-compiler.2Fetc.2Fopaque-types/topic/RPIT.20capture.20rules.20.28capturing.20everything.29/near/389496292 ### Return-position `impl Trait` in trait `async fn` always desugars to a function that returns `impl Future`. ```rust! async fn foo() -> i32 { 100 } // Equivalent to: fn foo() -> impl Future<Output = i32> { async { 100 } } ``` All `async fn`s today can be rewritten this way. This is useful because it allows adding behavior that runs at the time of the function call, before the first `.await` on the returned future. In the spirit of supporting the same set of features on `async fn` in traits that we do outside of traits, it makes sense to stabilize this as well. As described by the [RPITIT RFC][rpitit-rfc], this includes the ability to mix and match the equivalent forms in traits and their corresponding impls: ```rust! trait Foo { async fn foo(self) -> i32; } // Can be implemented as: impl Foo for MyType { fn foo(self) -> impl Future<Output = i32> { async { 100 } } } ``` Return-position `impl Trait` in trait is useful for cases beyond async, just as regular RPIT is. As a simple example, the RFC showed an alternative way of writing the `IntoIterator` trait with one fewer associated type. ```rust! trait NewIntoIterator { type Item; fn new_into_iter(self) -> impl Iterator<Item = Self::Item>; } impl<T> NewIntoIterator for Vec<T> { type Item = T; fn new_into_iter(self) -> impl Iterator<Item = T> { self.into_iter() } } ``` [rpitit-rfc]: https://rust-lang.github.io/rfcs/3425-return-position-impl-trait-in-traits.html ## Major design decisions This section describes the major design decisions that were reached after the RFC was accepted: - EDIT: Lint against async fn in trait definitions - Until the [send bound problem](https://smallcultfollowing.com/babysteps/blog/2023/02/01/async-trait-send-bounds-part-1-intro/) is resolved, the use of `async fn` in trait definitions could lead to a bad experience for people using work-stealing executors (by far the most popular choice). However, there are significant use cases for which the current support is all that is needed (single-threaded executors, such as those used in embedded use cases, as well as thread-per-core setups). We are prioritizing serving users well over protecting people from misuse, and therefore, we opt to stabilize the full range of functionality; however, to help steer people correctly, we are will issue a warning on the use of `async fn` in trait definitions that advises users about the limitations. (See [this summary comment](rust-lang#115822 (comment)) for the details of the concern, and [this comment](rust-lang#115822 (comment)) for more details about the reasoning that led to this conclusion.) - Capture rules: - The RFC's initial capture rules for lifetimes in impls/traits were found to be imprecisely precise and to introduce various inconsistencies. After much discussion, the decision was reached to make `-> impl Trait` in traits/impls capture *all* in-scope parameters, including both lifetimes and types. This is a departure from the behavior of RPITs in other contexts; an RFC is currently being authored to change the behavior of RPITs in other contexts in a future edition. - Major discussion links: - [Lang team design meeting from 2023-07-26](https://hackmd.io/sFaSIMJOQcuwCdnUvCxtuQ?view) - Refinement: - The [refinement RFC] initially proposed that impl signatures that are more specific than their trait are not allowed unless the `#[refine]` attribute was included, but left it as an open question how to implement this. The stabilized proposal is that it is not a hard error to omit `#[refine]`, but there is a lint which fires if the impl's return type is more precise than the trait. This greatly simplified the desugaring and implementation while still achieving the original goal of ensuring that users do not accidentally commit to a more specific return type than they intended. - Major discussion links: - [Zulip thread](https://rust-lang.zulipchat.com/#narrow/stream/213817-t-lang/topic/.60.23.5Brefine.5D.60.20as.20a.20lint) [refinement RFC]: https://rust-lang.github.io/rfcs/3245-refined-impls.html ## What is stabilized ### Async functions in traits and trait implementations * `async fn` are now supported in traits and trait implementations. * Associated functions in traits that are `async` may have default bodies. ### Return-position impl trait in traits and trait implementations * Return-position `impl Trait`s are now supported in traits and trait implementations. * Return-position `impl Trait` in implementations are treated like regular return-position `impl Trait`s, and therefore behave according to the same inference rules for hidden type inference and well-formedness. * Associated functions in traits that name return-position `impl Trait`s may have default bodies. * Implementations may provide either concrete types or `impl Trait` for each corresponding `impl Trait` in the trait method signature. For a detailed exploration of the technical implementation of return-position `impl Trait` in traits, see [the dev guide](https://rustc-dev-guide.rust-lang.org/return-position-impl-trait-in-trait.html). ### Mixing `async fn` in trait and return-position `impl Trait` in trait A trait function declaration that is `async fn ..() -> T` may be satisfied by an implementation function that returns `impl Future<Output = T>`, or vice versa. ```rust trait Async { async fn hello(); } impl Async for () { fn hello() -> impl Future<Output = ()> { async {} } } trait RPIT { fn hello() -> impl Future<Output = String>; } impl RPIT for () { async fn hello() -> String { "hello".to_string() } } ``` ### Return-position `impl Trait` in traits and trait implementations capture all in-scope lifetimes Described above in "major design decisions". ### Return-position `impl Trait` in traits are "always revealing" When a trait uses `-> impl Trait` in return position, it logically desugars to an associated type that represents the return (the actual implementation in the compiler is different, as described below). The value of this associated type is determined by the actual return type written in the impl; if the impl also uses `-> impl Trait` as the return type, then the value of the associated type is an opaque type scoped to the impl method (similar to what you would get when calling an inherent function returning `-> impl Trait`). As with any associated type, the value of this special associated type can be revealed by the compiler if the compiler can figure out what impl is being used. For example, given this trait: ```rust trait AsDebug { fn as_debug(&self) -> impl Debug; } ``` A function working with the trait generically is only able to see that the return value is `Debug`: ```rust fn foo<T: AsDebug>(t: &T) { let u = t.as_debug(); println!("{}", u); // ERROR: `u` is not known to implement `Display` } ``` But if a function calls `as_debug` on a known type (say, `u32`), it may be able to resolve the return type more specifically, if that implementation specifies a concrete type as well: ```rust impl AsDebug for u32 { fn as_debug(&self) -> u32 { *self } } fn foo(t: &u32) { let u: u32 = t.as_debug(); // OK! println!("{}", t.as_debug()); // ALSO OK (since `u32: Display`). } ``` The return type used in the impl therefore represents a **semver binding** promise from the impl author that the return type of `<u32 as AsDebug>::as_debug` will not change. This could come as a surprise to users, who might expect that they are free to change the return type to any other type that implements `Debug`. To address this, we include a [`refining_impl_trait` lint](rust-lang#115582) that warns if the impl uses a specific type -- the `impl AsDebug for u32` above, for example, would toggle the lint. The lint message explains what is going on and encourages users to `allow` the lint to indicate that they meant to refine the return type: ```rust impl AsDebug for u32 { #[allow(refining_impl_trait)] fn as_debug(&self) -> u32 { *self } } ``` [RFC rust-lang#3245](rust-lang/rfcs#3245) proposed a new attribute, `#[refine]`, that could also be used to "opt-in" to refinements like this (and which would then silence the lint). That RFC is not currently implemented -- the `#[refine]` attribute is also expected to reveal other details from the signature and has not yet been fully implemented. ### Return-position `impl Trait` and `async fn` in traits are opted-out of object safety checks when the parent function has `Self: Sized` ```rust trait IsObjectSafe { fn rpit() -> impl Sized where Self: Sized; async fn afit() where Self: Sized; } ``` Traits that mention return-position `impl Trait` or `async fn` in trait when the associated function includes a `Self: Sized` bound will remain object safe. That is because the associated function that defines them will be opted-out of the vtable of the trait, and the associated types will be unnameable from any trait object. This can alternatively be seen as a consequence of rust-lang#112319 (comment) and the desugaring of return-position `impl Trait` in traits to associated types which inherit the where-clauses of the associated function that defines them. ## What isn't stabilized (aka, potential future work) ### Dynamic dispatch As stabilized, traits containing RPITIT and AFIT are **not dyn compatible**. This means that you cannot create `dyn Trait` objects from them and can only use static dispatch. The reason for this limitation is that dynamic dispatch support for RPITIT and AFIT is more complex than static dispatch, as described on the [async fundamentals page](https://rust-lang.github.io/async-fundamentals-initiative/evaluation/challenges/dyn_traits.html). The primary challenge to using `dyn Trait` in today's Rust is that **`dyn Trait` today must list the values of all associated types**. This means you would have to write `dyn for<'s> Trait<Foo<'s> = XXX>` where `XXX` is the future type defined by the impl, such as `F_A`. This is not only verbose (or impossible), it also uniquely ties the `dyn Trait` to a particular impl, defeating the whole point of `dyn Trait`. The precise design for handling dynamic dispatch is not yet determined. Top candidates include: - [callee site selection][], in which we permit unsized return values so that the return type for an `-> impl Foo` method be can be `dyn Foo`, but then users must specify the type of wide pointer at the call-site in some fashion. - [`dyn*`][], where we create a built-in encapsulation of a "wide pointer" and map the associated type corresponding to an RPITIT to the corresponding `dyn*` type (`dyn*` itself is not exposed to users as a type in this proposal, though that could be a future extension). [callee site selection]: https://smallcultfollowing.com/babysteps/blog/2022/09/21/dyn-async-traits-part-9-callee-site-selection/ [`dyn*`]: https://smallcultfollowing.com/babysteps/blog/2022/03/29/dyn-can-we-make-dyn-sized/ ### Where-clause bounds on return-position `impl Trait` in traits or async futures (RTN/ART) One limitation of async fn in traits and RPITIT as stabilized is that there is no way for users to write code that adds additional bounds beyond those listed in the `-> impl Trait`. The most common example is wanting to write a generic function that requires that the future returned from an `async fn` be `Send`: ```rust trait Greet { async fn greet(&self); } fn greet_in_parallel<G: Greet>(g: &G) { runtime::spawn(async move { g.greet().await; //~ ERROR: future returned by `greet` may not be `Send` }) } ``` Currently, since the associated types added for the return type are anonymous, there is no where-clause that could be added to make this code compile. There have been various proposals for how to address this problem (e.g., [return type notation][rtn] or having an annotation to give a name to the associated type), but we leave the selection of one of those mechanisms to future work. [rtn]: https://smallcultfollowing.com/babysteps/blog/2023/02/13/return-type-notation-send-bounds-part-2/ In the meantime, there are workarounds that one can use to address this problem, listed below. #### Require all futures to be `Send` For many users, the trait may only ever be used with `Send` futures, in which case one can write an explicit `impl Future + Send`: ```rust trait Greet { fn greet(&self) -> impl Future<Output = ()> + Send; } ``` The nice thing about this is that it is still compatible with using `async fn` in the trait impl. In the async working group case studies, we found that this could work for the [builder provider API](https://rust-lang.github.io/async-fundamentals-initiative/evaluation/case-studies/builder-provider-api.html). This is also the default approach used by the `#[async_trait]` crate which, as we have noted, has seen widespread adoption. #### Avoid generics This problem only applies when the `Self` type is generic. If the `Self` type is known, then the precise return type from an `async fn` is revealed, and the `Send` bound can be inferred thanks to auto-trait leakage. Even in cases where generics may appear to be required, it is sometimes possible to rewrite the code to avoid them. The [socket handler refactor](https://rust-lang.github.io/async-fundamentals-initiative/evaluation/case-studies/socket-handler.html) case study provides one such example. ### Unify capture behavior for `-> impl Trait` in inherent methods and traits As stabilized, the capture behavior for `-> impl Trait` in a trait (whether as part of an async fn or a RPITIT) captures all types and lifetimes, whereas the existing behavior for inherent methods only captures types and lifetimes that are explicitly referenced. Capturing all lifetimes in traits was necessary to avoid various surprising inconsistencies; the expressed intent of the lang team is to extend that behavior so that we also capture all lifetimes in inherent methods, which would create more consistency and also address a common source of user confusion, but that will have to happen over the 2024 edition. The RFC is in progress. Should we opt not to accept that RFC, we can bring the capture behavior for `-> impl Trait` into alignment in other ways as part of the 2024 edition. ### `impl_trait_projections` Orthgonal to `async_fn_in_trait` and `return_position_impl_trait_in_trait`, since it can be triggered on stable code. This will be stabilized separately in [rust-lang#115659](rust-lang#115659). <details> If we try to write this code without `impl_trait_projections`, we will get an error: ```rust #![feature(async_fn_in_trait)] trait Foo { type Error; async fn foo(&mut self) -> Result<(), Self::Error>; } impl<T: Foo> Foo for &mut T { type Error = T::Error; async fn foo(&mut self) -> Result<(), Self::Error> { T::foo(self).await } } ``` The error relates to the use of `Self` in a trait impl when the self type has a lifetime. It can be worked around by rewriting the impl not to use `Self`: ```rust #![feature(async_fn_in_trait)] trait Foo { type Error; async fn foo(&mut self) -> Result<(), Self::Error>; } impl<T: Foo> Foo for &mut T { type Error = T::Error; async fn foo(&mut self) -> Result<(), <&mut T as Foo>::Error> { T::foo(self).await } } ``` </details> ## Tests Tests are generally organized between return-position `impl Trait` and `async fn` in trait, when the distinction matters. * RPITIT: https://github.com/rust-lang/rust/tree/master/tests/ui/impl-trait/in-trait * AFIT: https://github.com/rust-lang/rust/tree/master/tests/ui/async-await/in-trait ## Remaining bugs and open issues * rust-lang#112047: Indirection introduced by `async fn` and return-position `impl Trait` in traits may hide cycles in opaque types, causing overflow errors that can only be discovered by monomorphization. * rust-lang#111105 - `async fn` in trait is susceptible to issues with checking auto traits on futures' generators, like regular `async`. This is a manifestation of rust-lang#110338. * This was deemed not blocking because fixing it is forwards-compatible, and regular `async` is subject to the same issues. * rust-lang#104689: `async fn` and return-position `impl Trait` in trait requires the late-bound lifetimes in a trait and impl function signature to be equal. * This can be relaxed in the future with a smarter lexical region resolution algorithm. * rust-lang#102527: Nesting return-position `impl Trait` in trait deeply may result in slow compile times. * This has only been reported once, and can be fixed in the future. * rust-lang#108362: Inference between return types and generics of a function may have difficulties when there's an `.await`. * This isn't related to AFIT (rust-lang#108362 (comment)) -- using traits does mean that there's possibly easier ways to hit it. * rust-lang#112626: Because `async fn` and return-position `impl Trait` in traits lower to associated types, users may encounter strange behaviors when implementing circularly dependent traits. * This is not specific to RPITIT, and is a limitation of associated types: rust-lang#112626 (comment) * **(Nightly)** rust-lang#108309: `async fn` and return-position `impl Trait` in trait do not support specialization. This was deemed not blocking, since it can be fixed in the future (e.g. rust-lang#108321) and specialization is a nightly feature. #### (Nightly) Return type notation bugs RTN is not being stabilized here, but there are some interesting outstanding bugs. None of them are blockers for AFIT/RPITIT, but I'm noting them for completeness. <details> * rust-lang#109924 is a bug that occurs when a higher-ranked trait bound has both inference variables and associated types. This is pre-existing -- RTN just gives you a more convenient way of producing them. This should be fixed by the new trait solver. * rust-lang#109924 is a manifestation of a more general issue with `async` and auto-trait bounds: rust-lang#110338. RTN does not cause this issue, just allows us to put `Send` bounds on the anonymous futures that we have in traits. * rust-lang#112569 is a bug similar to associated type bounds, where nested bounds are not implied correctly. </details> ## Alternatives ### Do nothing We could choose not to stabilize these features. Users that can use the `#[async_trait]` macro would continue to do so. Library maintainers would continue to avoid async functions in traits, potentially blocking the stable release of many useful crates. ### Stabilize `impl Trait` in associated type instead AFIT and RPITIT solve the problem of returning unnameable types from trait methods. It is also possible to solve this by using another unstable feature, `impl Trait` in an associated type. Users would need to define an associated type in both the trait and trait impl: ```rust! trait Foo { type Fut<'a>: Future<Output = i32> where Self: 'a; fn foo(&self) -> Self::Fut<'_>; } impl Foo for MyType { type Fut<'a> where Self: 'a = impl Future<Output = i32>; fn foo(&self) -> Self::Fut<'_> { async { 42 } } } ``` This also has the advantage of allowing generic code to bound the associated type. However, it is substantially less ergonomic than either `async fn` or `-> impl Future`, and users still expect to be able to use those features in traits. **Even if this feature were stable, we would still want to stabilize AFIT and RPITIT.** That said, we can have both. `impl Trait` in associated types is desireable because it can be used in existing traits with explicit associated types, among other reasons. We *should* stabilize this feature once it is ready, but that's outside the scope of this proposal. ### Use the old capture semantics for RPITIT We could choose to make the capture rules for RPITIT consistent with the existing rules for RPIT. However, there was strong consensus in a recent [lang team meeting](https://hackmd.io/sFaSIMJOQcuwCdnUvCxtuQ?view) that we should *change* these rules, and furthermore that new features should adopt the new rules. This is consistent with the tenet in RFC 3085 of favoring ["Uniform behavior across editions"](https://rust-lang.github.io/rfcs/3085-edition-2021.html#uniform-behavior-across-editions) when possible. It greatly reduces the complexity of the feature by not requiring us to answer, or implement, the design questions that arise out of the interaction between the current capture rules and traits. This reduction in complexity – and eventual technical debt – is exactly in line with the motivation listed in the aforementioned RFC. ### Make refinement a hard error Refinement (`refining_impl_trait`) is only a concern for library authors, and therefore doesn't really warrant making into a deny-by-default warning or an error. Additionally, refinement is currently checked via a lint that compares bounds in the `impl Trait`s in the trait and impl syntactically. This is good enough for a warning that can be opted-out, but not if this were a hard error, which would ideally be implemented using fully semantic, implicational logic. This was implemented (rust-lang#111931), but also is an unnecessary burden on the type system for little pay-off. ## History - Dec 7, 2021: [RFC rust-lang#3185: Static async fn in traits](https://rust-lang.github.io/rfcs/3185-static-async-fn-in-trait.html) merged - Sep 9, 2022: [Initial implementation](rust-lang#101224) of AFIT and RPITIT landed - Jun 13, 2023: [RFC rust-lang#3425: Return position `impl Trait` in traits](https://rust-lang.github.io/rfcs/3425-return-position-impl-trait-in-traits.html) merged <!--These will render pretty when pasted into github--> Non-exhaustive list of PRs that are particularly relevant to the implementation: - rust-lang#101224 - rust-lang#103491 - rust-lang#104592 - rust-lang#108141 - rust-lang#108319 - rust-lang#108672 - rust-lang#112988 - rust-lang#113182 (later made redundant by rust-lang#114489) - rust-lang#113215 - rust-lang#114489 - rust-lang#115467 - rust-lang#115582 Doc co-authored by `@nikomatsakis,` `@tmandry,` `@traviscross.` Thanks also to `@spastorino,` `@cjgillot` (for changes to opaque captures!), `@oli-obk` for many reviews, and many other contributors and issue-filers. Apologies if I left your name off 😺
Stabilize `async fn` and return-position `impl Trait` in trait # Stabilization report This report proposes the stabilization of `#![feature(return_position_impl_trait_in_trait)]` ([RPITIT][RFC 3425]) and `#![feature(async_fn_in_trait)]` ([AFIT][RFC 3185]). These are both long awaited features that increase the expressiveness of the Rust language and trait system. Closes #91611 [RFC 3185]: https://rust-lang.github.io/rfcs/3185-static-async-fn-in-trait.html [RFC 3425]: https://rust-lang.github.io/rfcs/3425-return-position-impl-trait-in-traits.html ## Updates from thread The thread has covered two major concerns: * [Given that we don't have RTN, what should we stabilize?](rust-lang/rust#115822 (comment)) -- proposed resolution is [adding a lint](rust-lang/rust#115822 (comment)) and [careful messaging](rust-lang/rust#115822 (comment)) * [Interaction between outlives bounds and capture semantics](rust-lang/rust#115822 (comment)) -- This is fixable in a forwards-compatible way via #116040, and also eventually via ATPIT. ## Stabilization Summary This stabilization allows the following examples to work. ### Example of return-position `impl Trait` in trait definition ```rust trait Bar { fn bar(self) -> impl Send; } ``` This declares a trait method that returns *some* type that implements `Send`. It's similar to writing the following using an associated type, except that the associated type is anonymous. ```rust trait Bar { type _0: Send; fn bar(self) -> Self::_0; } ``` ### Example of return-position `impl Trait` in trait implementation ```rust impl Bar for () { fn bar(self) -> impl Send {} } ``` This defines a method implementation that returns an opaque type, just like [RPIT][RFC 1522] does, except that all in-scope lifetimes are captured in the opaque type (as is already true for `async fn` and as is expected to be true for RPIT in Rust Edition 2024), as described below. [RFC 1522]: https://rust-lang.github.io/rfcs/1522-conservative-impl-trait.html ### Example of `async fn` in trait ```rust trait Bar { async fn bar(self); } impl Bar for () { async fn bar(self) {} } ``` This declares a trait method that returns *some* [`Future`](https://doc.rust-lang.org/core/future/trait.Future.html) and a corresponding method implementation. This is equivalent to writing the following using RPITIT. ```rust use core::future::Future; trait Bar { fn bar(self) -> impl Future<Output = ()>; } impl Bar for () { fn bar(self) -> impl Future<Output = ()> { async {} } } ``` The desirability of this desugaring being available is part of why RPITIT and AFIT are being proposed for stabilization at the same time. ## Motivation Long ago, Rust added [RPIT][RFC 1522] and [`async`/`await`][RFC 2394]. These are major features that are widely used in the ecosystem. However, until now, these feature could not be used in *traits* and trait implementations. This left traits as a kind of second-class citizen of the language. This stabilization fixes that. [RFC 2394]: https://rust-lang.github.io/rfcs/2394-async_await.html ### `async fn` in trait Async/await allows users to write asynchronous code much easier than they could before. However, it doesn't play nice with other core language features that make Rust the great language it is, like traits. Support for `async fn` in traits has been long anticipated and was not added before due to limitations in the compiler that have now been lifted. `async fn` in traits will unblock a lot of work in the ecosystem and the standard library. It is not currently possible to write a trait that is implemented using `async fn`. The workarounds that exist are undesirable because they require allocation and dynamic dispatch, and any trait that uses them will become obsolete once native `async fn` in trait is stabilized. We also have ample evidence that there is demand for this feature from the [`async-trait` crate][async-trait], which emulates the feature using dynamic dispatch. The async-trait crate is currently the #5 async crate on crates.io ranked by recent downloads, receiving over 78M all-time downloads. According to a [recent analysis][async-trait-analysis], 4% of all crates use the `#[async_trait]` macro it provides, representing 7% of all function and method signatures in trait definitions on crates.io. We think this is a *lower bound* on demand for the feature, because users are unlikely to use `#[async_trait]` on public traits on crates.io for the reasons already given. [async-trait]: https://crates.io/crates/async-trait [async-trait-analysis]: https://rust-lang.zulipchat.com/#narrow/stream/315482-t-compiler.2Fetc.2Fopaque-types/topic/RPIT.20capture.20rules.20.28capturing.20everything.29/near/389496292 ### Return-position `impl Trait` in trait `async fn` always desugars to a function that returns `impl Future`. ```rust! async fn foo() -> i32 { 100 } // Equivalent to: fn foo() -> impl Future<Output = i32> { async { 100 } } ``` All `async fn`s today can be rewritten this way. This is useful because it allows adding behavior that runs at the time of the function call, before the first `.await` on the returned future. In the spirit of supporting the same set of features on `async fn` in traits that we do outside of traits, it makes sense to stabilize this as well. As described by the [RPITIT RFC][rpitit-rfc], this includes the ability to mix and match the equivalent forms in traits and their corresponding impls: ```rust! trait Foo { async fn foo(self) -> i32; } // Can be implemented as: impl Foo for MyType { fn foo(self) -> impl Future<Output = i32> { async { 100 } } } ``` Return-position `impl Trait` in trait is useful for cases beyond async, just as regular RPIT is. As a simple example, the RFC showed an alternative way of writing the `IntoIterator` trait with one fewer associated type. ```rust! trait NewIntoIterator { type Item; fn new_into_iter(self) -> impl Iterator<Item = Self::Item>; } impl<T> NewIntoIterator for Vec<T> { type Item = T; fn new_into_iter(self) -> impl Iterator<Item = T> { self.into_iter() } } ``` [rpitit-rfc]: https://rust-lang.github.io/rfcs/3425-return-position-impl-trait-in-traits.html ## Major design decisions This section describes the major design decisions that were reached after the RFC was accepted: - EDIT: Lint against async fn in trait definitions - Until the [send bound problem](https://smallcultfollowing.com/babysteps/blog/2023/02/01/async-trait-send-bounds-part-1-intro/) is resolved, the use of `async fn` in trait definitions could lead to a bad experience for people using work-stealing executors (by far the most popular choice). However, there are significant use cases for which the current support is all that is needed (single-threaded executors, such as those used in embedded use cases, as well as thread-per-core setups). We are prioritizing serving users well over protecting people from misuse, and therefore, we opt to stabilize the full range of functionality; however, to help steer people correctly, we are will issue a warning on the use of `async fn` in trait definitions that advises users about the limitations. (See [this summary comment](rust-lang/rust#115822 (comment)) for the details of the concern, and [this comment](rust-lang/rust#115822 (comment)) for more details about the reasoning that led to this conclusion.) - Capture rules: - The RFC's initial capture rules for lifetimes in impls/traits were found to be imprecisely precise and to introduce various inconsistencies. After much discussion, the decision was reached to make `-> impl Trait` in traits/impls capture *all* in-scope parameters, including both lifetimes and types. This is a departure from the behavior of RPITs in other contexts; an RFC is currently being authored to change the behavior of RPITs in other contexts in a future edition. - Major discussion links: - [Lang team design meeting from 2023-07-26](https://hackmd.io/sFaSIMJOQcuwCdnUvCxtuQ?view) - Refinement: - The [refinement RFC] initially proposed that impl signatures that are more specific than their trait are not allowed unless the `#[refine]` attribute was included, but left it as an open question how to implement this. The stabilized proposal is that it is not a hard error to omit `#[refine]`, but there is a lint which fires if the impl's return type is more precise than the trait. This greatly simplified the desugaring and implementation while still achieving the original goal of ensuring that users do not accidentally commit to a more specific return type than they intended. - Major discussion links: - [Zulip thread](https://rust-lang.zulipchat.com/#narrow/stream/213817-t-lang/topic/.60.23.5Brefine.5D.60.20as.20a.20lint) [refinement RFC]: https://rust-lang.github.io/rfcs/3245-refined-impls.html ## What is stabilized ### Async functions in traits and trait implementations * `async fn` are now supported in traits and trait implementations. * Associated functions in traits that are `async` may have default bodies. ### Return-position impl trait in traits and trait implementations * Return-position `impl Trait`s are now supported in traits and trait implementations. * Return-position `impl Trait` in implementations are treated like regular return-position `impl Trait`s, and therefore behave according to the same inference rules for hidden type inference and well-formedness. * Associated functions in traits that name return-position `impl Trait`s may have default bodies. * Implementations may provide either concrete types or `impl Trait` for each corresponding `impl Trait` in the trait method signature. For a detailed exploration of the technical implementation of return-position `impl Trait` in traits, see [the dev guide](https://rustc-dev-guide.rust-lang.org/return-position-impl-trait-in-trait.html). ### Mixing `async fn` in trait and return-position `impl Trait` in trait A trait function declaration that is `async fn ..() -> T` may be satisfied by an implementation function that returns `impl Future<Output = T>`, or vice versa. ```rust trait Async { async fn hello(); } impl Async for () { fn hello() -> impl Future<Output = ()> { async {} } } trait RPIT { fn hello() -> impl Future<Output = String>; } impl RPIT for () { async fn hello() -> String { "hello".to_string() } } ``` ### Return-position `impl Trait` in traits and trait implementations capture all in-scope lifetimes Described above in "major design decisions". ### Return-position `impl Trait` in traits are "always revealing" When a trait uses `-> impl Trait` in return position, it logically desugars to an associated type that represents the return (the actual implementation in the compiler is different, as described below). The value of this associated type is determined by the actual return type written in the impl; if the impl also uses `-> impl Trait` as the return type, then the value of the associated type is an opaque type scoped to the impl method (similar to what you would get when calling an inherent function returning `-> impl Trait`). As with any associated type, the value of this special associated type can be revealed by the compiler if the compiler can figure out what impl is being used. For example, given this trait: ```rust trait AsDebug { fn as_debug(&self) -> impl Debug; } ``` A function working with the trait generically is only able to see that the return value is `Debug`: ```rust fn foo<T: AsDebug>(t: &T) { let u = t.as_debug(); println!("{}", u); // ERROR: `u` is not known to implement `Display` } ``` But if a function calls `as_debug` on a known type (say, `u32`), it may be able to resolve the return type more specifically, if that implementation specifies a concrete type as well: ```rust impl AsDebug for u32 { fn as_debug(&self) -> u32 { *self } } fn foo(t: &u32) { let u: u32 = t.as_debug(); // OK! println!("{}", t.as_debug()); // ALSO OK (since `u32: Display`). } ``` The return type used in the impl therefore represents a **semver binding** promise from the impl author that the return type of `<u32 as AsDebug>::as_debug` will not change. This could come as a surprise to users, who might expect that they are free to change the return type to any other type that implements `Debug`. To address this, we include a [`refining_impl_trait` lint](rust-lang/rust#115582) that warns if the impl uses a specific type -- the `impl AsDebug for u32` above, for example, would toggle the lint. The lint message explains what is going on and encourages users to `allow` the lint to indicate that they meant to refine the return type: ```rust impl AsDebug for u32 { #[allow(refining_impl_trait)] fn as_debug(&self) -> u32 { *self } } ``` [RFC #3245](rust-lang/rfcs#3245) proposed a new attribute, `#[refine]`, that could also be used to "opt-in" to refinements like this (and which would then silence the lint). That RFC is not currently implemented -- the `#[refine]` attribute is also expected to reveal other details from the signature and has not yet been fully implemented. ### Return-position `impl Trait` and `async fn` in traits are opted-out of object safety checks when the parent function has `Self: Sized` ```rust trait IsObjectSafe { fn rpit() -> impl Sized where Self: Sized; async fn afit() where Self: Sized; } ``` Traits that mention return-position `impl Trait` or `async fn` in trait when the associated function includes a `Self: Sized` bound will remain object safe. That is because the associated function that defines them will be opted-out of the vtable of the trait, and the associated types will be unnameable from any trait object. This can alternatively be seen as a consequence of rust-lang/rust#112319 (comment) and the desugaring of return-position `impl Trait` in traits to associated types which inherit the where-clauses of the associated function that defines them. ## What isn't stabilized (aka, potential future work) ### Dynamic dispatch As stabilized, traits containing RPITIT and AFIT are **not dyn compatible**. This means that you cannot create `dyn Trait` objects from them and can only use static dispatch. The reason for this limitation is that dynamic dispatch support for RPITIT and AFIT is more complex than static dispatch, as described on the [async fundamentals page](https://rust-lang.github.io/async-fundamentals-initiative/evaluation/challenges/dyn_traits.html). The primary challenge to using `dyn Trait` in today's Rust is that **`dyn Trait` today must list the values of all associated types**. This means you would have to write `dyn for<'s> Trait<Foo<'s> = XXX>` where `XXX` is the future type defined by the impl, such as `F_A`. This is not only verbose (or impossible), it also uniquely ties the `dyn Trait` to a particular impl, defeating the whole point of `dyn Trait`. The precise design for handling dynamic dispatch is not yet determined. Top candidates include: - [callee site selection][], in which we permit unsized return values so that the return type for an `-> impl Foo` method be can be `dyn Foo`, but then users must specify the type of wide pointer at the call-site in some fashion. - [`dyn*`][], where we create a built-in encapsulation of a "wide pointer" and map the associated type corresponding to an RPITIT to the corresponding `dyn*` type (`dyn*` itself is not exposed to users as a type in this proposal, though that could be a future extension). [callee site selection]: https://smallcultfollowing.com/babysteps/blog/2022/09/21/dyn-async-traits-part-9-callee-site-selection/ [`dyn*`]: https://smallcultfollowing.com/babysteps/blog/2022/03/29/dyn-can-we-make-dyn-sized/ ### Where-clause bounds on return-position `impl Trait` in traits or async futures (RTN/ART) One limitation of async fn in traits and RPITIT as stabilized is that there is no way for users to write code that adds additional bounds beyond those listed in the `-> impl Trait`. The most common example is wanting to write a generic function that requires that the future returned from an `async fn` be `Send`: ```rust trait Greet { async fn greet(&self); } fn greet_in_parallel<G: Greet>(g: &G) { runtime::spawn(async move { g.greet().await; //~ ERROR: future returned by `greet` may not be `Send` }) } ``` Currently, since the associated types added for the return type are anonymous, there is no where-clause that could be added to make this code compile. There have been various proposals for how to address this problem (e.g., [return type notation][rtn] or having an annotation to give a name to the associated type), but we leave the selection of one of those mechanisms to future work. [rtn]: https://smallcultfollowing.com/babysteps/blog/2023/02/13/return-type-notation-send-bounds-part-2/ In the meantime, there are workarounds that one can use to address this problem, listed below. #### Require all futures to be `Send` For many users, the trait may only ever be used with `Send` futures, in which case one can write an explicit `impl Future + Send`: ```rust trait Greet { fn greet(&self) -> impl Future<Output = ()> + Send; } ``` The nice thing about this is that it is still compatible with using `async fn` in the trait impl. In the async working group case studies, we found that this could work for the [builder provider API](https://rust-lang.github.io/async-fundamentals-initiative/evaluation/case-studies/builder-provider-api.html). This is also the default approach used by the `#[async_trait]` crate which, as we have noted, has seen widespread adoption. #### Avoid generics This problem only applies when the `Self` type is generic. If the `Self` type is known, then the precise return type from an `async fn` is revealed, and the `Send` bound can be inferred thanks to auto-trait leakage. Even in cases where generics may appear to be required, it is sometimes possible to rewrite the code to avoid them. The [socket handler refactor](https://rust-lang.github.io/async-fundamentals-initiative/evaluation/case-studies/socket-handler.html) case study provides one such example. ### Unify capture behavior for `-> impl Trait` in inherent methods and traits As stabilized, the capture behavior for `-> impl Trait` in a trait (whether as part of an async fn or a RPITIT) captures all types and lifetimes, whereas the existing behavior for inherent methods only captures types and lifetimes that are explicitly referenced. Capturing all lifetimes in traits was necessary to avoid various surprising inconsistencies; the expressed intent of the lang team is to extend that behavior so that we also capture all lifetimes in inherent methods, which would create more consistency and also address a common source of user confusion, but that will have to happen over the 2024 edition. The RFC is in progress. Should we opt not to accept that RFC, we can bring the capture behavior for `-> impl Trait` into alignment in other ways as part of the 2024 edition. ### `impl_trait_projections` Orthgonal to `async_fn_in_trait` and `return_position_impl_trait_in_trait`, since it can be triggered on stable code. This will be stabilized separately in [#115659](rust-lang/rust#115659). <details> If we try to write this code without `impl_trait_projections`, we will get an error: ```rust #![feature(async_fn_in_trait)] trait Foo { type Error; async fn foo(&mut self) -> Result<(), Self::Error>; } impl<T: Foo> Foo for &mut T { type Error = T::Error; async fn foo(&mut self) -> Result<(), Self::Error> { T::foo(self).await } } ``` The error relates to the use of `Self` in a trait impl when the self type has a lifetime. It can be worked around by rewriting the impl not to use `Self`: ```rust #![feature(async_fn_in_trait)] trait Foo { type Error; async fn foo(&mut self) -> Result<(), Self::Error>; } impl<T: Foo> Foo for &mut T { type Error = T::Error; async fn foo(&mut self) -> Result<(), <&mut T as Foo>::Error> { T::foo(self).await } } ``` </details> ## Tests Tests are generally organized between return-position `impl Trait` and `async fn` in trait, when the distinction matters. * RPITIT: https://github.com/rust-lang/rust/tree/master/tests/ui/impl-trait/in-trait * AFIT: https://github.com/rust-lang/rust/tree/master/tests/ui/async-await/in-trait ## Remaining bugs and open issues * #112047: Indirection introduced by `async fn` and return-position `impl Trait` in traits may hide cycles in opaque types, causing overflow errors that can only be discovered by monomorphization. * #111105 - `async fn` in trait is susceptible to issues with checking auto traits on futures' generators, like regular `async`. This is a manifestation of #110338. * This was deemed not blocking because fixing it is forwards-compatible, and regular `async` is subject to the same issues. * #104689: `async fn` and return-position `impl Trait` in trait requires the late-bound lifetimes in a trait and impl function signature to be equal. * This can be relaxed in the future with a smarter lexical region resolution algorithm. * #102527: Nesting return-position `impl Trait` in trait deeply may result in slow compile times. * This has only been reported once, and can be fixed in the future. * #108362: Inference between return types and generics of a function may have difficulties when there's an `.await`. * This isn't related to AFIT (rust-lang/rust#108362 (comment)) -- using traits does mean that there's possibly easier ways to hit it. * #112626: Because `async fn` and return-position `impl Trait` in traits lower to associated types, users may encounter strange behaviors when implementing circularly dependent traits. * This is not specific to RPITIT, and is a limitation of associated types: rust-lang/rust#112626 (comment) * **(Nightly)** #108309: `async fn` and return-position `impl Trait` in trait do not support specialization. This was deemed not blocking, since it can be fixed in the future (e.g. #108321) and specialization is a nightly feature. #### (Nightly) Return type notation bugs RTN is not being stabilized here, but there are some interesting outstanding bugs. None of them are blockers for AFIT/RPITIT, but I'm noting them for completeness. <details> * #109924 is a bug that occurs when a higher-ranked trait bound has both inference variables and associated types. This is pre-existing -- RTN just gives you a more convenient way of producing them. This should be fixed by the new trait solver. * #109924 is a manifestation of a more general issue with `async` and auto-trait bounds: #110338. RTN does not cause this issue, just allows us to put `Send` bounds on the anonymous futures that we have in traits. * #112569 is a bug similar to associated type bounds, where nested bounds are not implied correctly. </details> ## Alternatives ### Do nothing We could choose not to stabilize these features. Users that can use the `#[async_trait]` macro would continue to do so. Library maintainers would continue to avoid async functions in traits, potentially blocking the stable release of many useful crates. ### Stabilize `impl Trait` in associated type instead AFIT and RPITIT solve the problem of returning unnameable types from trait methods. It is also possible to solve this by using another unstable feature, `impl Trait` in an associated type. Users would need to define an associated type in both the trait and trait impl: ```rust! trait Foo { type Fut<'a>: Future<Output = i32> where Self: 'a; fn foo(&self) -> Self::Fut<'_>; } impl Foo for MyType { type Fut<'a> where Self: 'a = impl Future<Output = i32>; fn foo(&self) -> Self::Fut<'_> { async { 42 } } } ``` This also has the advantage of allowing generic code to bound the associated type. However, it is substantially less ergonomic than either `async fn` or `-> impl Future`, and users still expect to be able to use those features in traits. **Even if this feature were stable, we would still want to stabilize AFIT and RPITIT.** That said, we can have both. `impl Trait` in associated types is desireable because it can be used in existing traits with explicit associated types, among other reasons. We *should* stabilize this feature once it is ready, but that's outside the scope of this proposal. ### Use the old capture semantics for RPITIT We could choose to make the capture rules for RPITIT consistent with the existing rules for RPIT. However, there was strong consensus in a recent [lang team meeting](https://hackmd.io/sFaSIMJOQcuwCdnUvCxtuQ?view) that we should *change* these rules, and furthermore that new features should adopt the new rules. This is consistent with the tenet in RFC 3085 of favoring ["Uniform behavior across editions"](https://rust-lang.github.io/rfcs/3085-edition-2021.html#uniform-behavior-across-editions) when possible. It greatly reduces the complexity of the feature by not requiring us to answer, or implement, the design questions that arise out of the interaction between the current capture rules and traits. This reduction in complexity – and eventual technical debt – is exactly in line with the motivation listed in the aforementioned RFC. ### Make refinement a hard error Refinement (`refining_impl_trait`) is only a concern for library authors, and therefore doesn't really warrant making into a deny-by-default warning or an error. Additionally, refinement is currently checked via a lint that compares bounds in the `impl Trait`s in the trait and impl syntactically. This is good enough for a warning that can be opted-out, but not if this were a hard error, which would ideally be implemented using fully semantic, implicational logic. This was implemented (#111931), but also is an unnecessary burden on the type system for little pay-off. ## History - Dec 7, 2021: [RFC #3185: Static async fn in traits](https://rust-lang.github.io/rfcs/3185-static-async-fn-in-trait.html) merged - Sep 9, 2022: [Initial implementation](rust-lang/rust#101224) of AFIT and RPITIT landed - Jun 13, 2023: [RFC #3425: Return position `impl Trait` in traits](https://rust-lang.github.io/rfcs/3425-return-position-impl-trait-in-traits.html) merged <!--These will render pretty when pasted into github--> Non-exhaustive list of PRs that are particularly relevant to the implementation: - #101224 - #103491 - #104592 - #108141 - #108319 - #108672 - #112988 - #113182 (later made redundant by #114489) - #113215 - #114489 - #115467 - #115582 Doc co-authored by `@nikomatsakis,` `@tmandry,` `@traviscross.` Thanks also to `@spastorino,` `@cjgillot` (for changes to opaque captures!), `@oli-obk` for many reviews, and many other contributors and issue-filers. Apologies if I left your name off 😺
This RFC generalizes the
safe_unsafe_trait_methods
RFC, allowing implementations of traits to add type information about the API of their methods and constants which then become part of the API for that type. Specifically, lifetimes and where clauses are allowed to extend beyond what the trait provides.Rendered RFC
This RFC was authored as part of the async fundamentals initiative.