Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC] externally implementable functions #3632

Open
wants to merge 17 commits into
base: master
Choose a base branch
from
Open
140 changes: 140 additions & 0 deletions text/0000-externally-implementable-functions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
- Feature Name: `extern_impl_fn`
- Start Date: 2024-05-10
- RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/0000)
- Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000)

# Summary

A mechanism for defining a function whose implementation can be defined (or overridden) in another crate.

Example 1:

```rust
// core::panic:

extern impl fn panic_handler(_: &PanicInfo) -> !;

// user:

impl fn core::panic::panic_handler(_: &PanicInfo) -> ! {
loop {}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like this particular syntax very much. It is too close to existing impl $t:ty { } syntax when $t is an fn type.

#![feature(rustc_attrs)]
impl fn(_: &core::panic::PanicInfo) -> ! {
    #[rustc_allow_incoherent_impl]
    pub fn what() {}
}

Granted, there is no ambiguity at the moment for the actual syntax proposed here since an fn type can't specify a name (right?) plus you aren't really allowed to impl on an fn type anyway unless you are the standard library.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that users aren't allowed to have an impl block for fn types anyway, and that the syntax is unambiguous regardless, I'm not too worried about this.

But I also don't care that much about the syntax. We can consider other syntaxes before stabilization of course.

```

Example 2:

```rust
// log crate:

extern impl fn logger() -> Logger {
Logger::default()
}

// user:

impl fn log::logger() -> Logger {
Logger::to_stdout().with_colors()
}
```

# Motivation

We have several items in the standard library that are overridable/definable by the user crate.
For example, the (no_std) `panic_handler`, the global allocator for `alloc`, and so on.

Each of those is a special lang item with its own special handling.
Having a general mechanism simplifies the language and makes this functionality available for other crates, and potentially for more use cases in core/alloc/std.

# Explanation

A function can be defined as "externally implementable" using `extern impl` as follows:

```rust
// In crate `x`:

// Without a body:
extern impl fn a();

// With a body:
extern impl fn b() {
println!("default impl");
}
```

Another crate can then provide (or override) the implementation of these functions using `impl fn` syntax (using their path) as follows:
m-ou-se marked this conversation as resolved.
Show resolved Hide resolved

```rust
// In another crate:

impl fn x::a() {
println!("my implementation of a");
}

impl fn x::b() {
println!("my implementation of b");
}
```

# Details
m-ou-se marked this conversation as resolved.
Show resolved Hide resolved

## Signature
m-ou-se marked this conversation as resolved.
Show resolved Hide resolved

It is an error to have a different signature for the `impl fn` item
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this use the same rules as trait Def Vs impls where the implementor can write a more general signature?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably, yes.


## No impl

It is an error to have no `impl fn` item (in any crate) for an `extern impl fn` item without a body.

## Duplicates

It is an error to have multiple `impl fn` items (across all crates) for the same `extern impl fn` item.
m-ou-se marked this conversation as resolved.
Show resolved Hide resolved
m-ou-se marked this conversation as resolved.
Show resolved Hide resolved

## Visibility

`extern impl fn` items can have a visibility specifier (like `pub`), which determines who can *call* the function (or create pointers to it, etc.).

*Implementing* the function can be done by any crate that can name the item.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you clarify whether this is intended to also tie into visibility? For instance, a pub(crate) extern impl fn can only have an implementation provided by the crate, right?

Copy link
Member Author

@m-ou-se m-ou-se May 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea is that if you do:

pub mod a {
    pub(crate) extern impl fn x();
}

The, other crates can provide an implementation (because a is pub, allowing them to name a::x), but they cannot call it (because the function is not public itself).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@m-ou-se That seems potentially confusing. Is there some way to set the visibility of being able to implement it? Is there value in being able to do so?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, because the point of this feature is allowing other crates to implement it. So unless you want to propose a kind of visibility that includes some crates but not others, that just implies full public.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So if the extern impl fn is in a private module, it is impossible to implement?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@m-ou-se

The idea is that if you do:

pub mod a {
    pub(crate) extern impl fn x();
}

The, other crates can provide an implementation (because a is pub, allowing them to name a::x), but they cannot call it (because the function is not public itself).

Yeah this is very confusing. Could you make it respect the normal privacy rule and reuse e.g. #3323 to explicitly deny call/ref permission from dependencies?

pub mod a {
    pub restrict_use(crate) extern impl fn x();
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So if the extern impl fn is in a private module, it is impossible to implement?

It could be re-exported, but that reexport then presumably allows both calling and implementing the item. The magic behavior where these two permissions are different can only be obtain via the original definition of the item, IIUC.


# Implementation

The implementation will be based on the same mechanisms as used today for the `panic_handler` and `#[global_allocator]` features.

The compiler of the root crate will find the implementation of all externally implementable functions and give an error
if more than one implementation is found for any of them.
If none are found, the result is either an error, or, if the `extern impl fn` has a default body, an implementation
is generated that calls that default body.
m-ou-se marked this conversation as resolved.
Show resolved Hide resolved
joshtriplett marked this conversation as resolved.
Show resolved Hide resolved

# Drawbacks

- It encourages globally defined behaviour.
- Counterargument: We are already doing this anyway, both inside the standard library (e.g. panic_handler, allocator)
and outside (e.g. global logger). This just makes it much easier (and safer) to get right.

m-ou-se marked this conversation as resolved.
Show resolved Hide resolved
# Rationale and alternatives

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we allow grouping multiple functions together like global_allocator in this RFC? Or should that be left as future potential improvement?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you could work around that with a TAIT:

pub trait MyFunctions {
    fn fn1() -> String;
    fn fn2(a: String, b: u32);
}

pub type MyFunctionsImpl = impl MyFunctions;

fn f(v: Infallible) -> MyFunctionsImpl {
    my_functions(v)
}

pub extern impl fn my_functions(v: Infallible) -> impl MyFunctions;

pub fn fn3() -> String {
    MyFunctionsImpl::fn1()
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that'd be part of a potential future (more compplicated) RFC, such as #2492

Copy link
Member

@RalfJung RalfJung May 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possibly global_alloc could at least use the same internal mechanism, even if it's not visible to the user?

#[global_alloc]
static ALLOC: MyAlloc = ...;

could expand to something like

static ALLOC: MyAlloc = ...;

impl fn alloc::alloc::alloc(layout: Layout) -> *mut u8 {
  ALLOC.alloc(layout)
}
impl fn alloc::alloc::dealloc(layout: Layout) -> *mut u8 {
  ALLOC.dealloc(layout)
}
// ...

Then codegen and Miri would only have to support one such mechanism. :)

- The syntax re-uses existing keywords. Alternatively, we could:
- Use the `override` reserved keyword.
- Add a new (contextual) keyword (e.g. `existential fn`).
- Use an attribute (e.g. `#[extern_impl]`) instead.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should the following alternative be mentioned / discussed:

  • multiple impl's are allowed
  • the root crate must import the impl they want
  • the normal-default impl is imported via the prelude

also just had the thought: does use crateA::different_name as panic_handler work similar to how (I believe) it works for main?


# Prior art

[RFC 2494 "Existential types with external definition"](https://github.com/rust-lang/rfcs/pull/2492)
has been proposed before, which basically does this for *types*. Doing this for functions (as a start) saves a lot of complexity.

# Unresolved questions

- What should the syntax be once we stabilize this?
m-ou-se marked this conversation as resolved.
Show resolved Hide resolved
- How should this work in dynamic libraries?
m-ou-se marked this conversation as resolved.
Show resolved Hide resolved

# Future possibilities

- Doing this for `static` items too. (Perhaps all items that can appear in an `extern "Rust" { … }` block.)
joshtriplett marked this conversation as resolved.
Show resolved Hide resolved
- Using this for existing overridable global behavior in the standard library, like the panic handler, global allocator, etc.
m-ou-se marked this conversation as resolved.
Show resolved Hide resolved
- Using this mechanism in the standard library to make more parts overridable. For example:
- Allowing custom implementations of `panic_out_of_bounds` and `panic_overflowing_add`, etc.
(The Rust for Linux project would make use of this.)
- Allowing overriding `write_to_stdout` and `write_to_stderr`.
(This enables custom testing frameworks to capture output. It is also extremely useful on targets like wasm.)
- This could possibly be extended to groups of functions in the form of a `trait` that can be globally implemented.
(E.g. `extern impl AsyncRuntime`, to say that there must be a global implementation of that trait.)
m-ou-se marked this conversation as resolved.
Show resolved Hide resolved