Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extern types v2 #3396

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions text/1861-extern-types.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,10 @@
- RFC PR: [rust-lang/rfcs#1861](https://github.com/rust-lang/rfcs/pull/1861)
- Rust Issue: [rust-lang/rust#43467](https://github.com/rust-lang/rust/issues/43467)

> This RFC has been superseded by [RFC 0000](https://rust-lang.github.io/rfcs/0000-extern-types-v2.md).
>
> This RFC omits details such as the alignment of extern types which proved essential for implementing it.

# Summary
[summary]: #summary

Expand Down
249 changes: 249 additions & 0 deletions text/3396-extern-types-v2.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,249 @@
- Feature Name: `extern_types_v2`
- Start Date: 2023-02-19
- RFC PR: [rust-lang/rfcs#3396](https://github.com/rust-lang/rfcs/pull/3396)
- Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000)


# Summary
[summary]: #summary

Define types external to Rust and introduce syntax to declare them.
This additionally introduces the `MetaSized` trait to allow these types to be interacted with in a generic context.
This supersedes [RFC 1861].


# Motivation
[motivation]: #motivation

The motivation from [RFC 1861] for why we want extern types at all still applies.

The summary of that is that when writing FFI bindings, the foreign code may give you a pointer to a type but not provide the layout for that type.
The current approach for representing these types in Rust, suggested by the 'nomicon, is with a ZST like the following.
```rust
#[repr(C)]
pub struct Opaque {
_data: [u32; 0],
_marker: PhantomData<(*mut u8, PhantomPinned)>
}
```
The type is `repr(C)` so that it can be used in FFI.
The `_data` field provides the alignment of the type, and the `_marker` field causes the correct set of automatically implemented traits to be implemented, in this case `!Send`, `!Sync`, `!Unpin`, but `Freeze` (`UnsafeCell` can be used to remove the last one of those).
This works, however the compiler believes this type is statically sized and aligned which may not be true.
This means that the type can be easily misused by placing it on the stack, allocating it on the heap, calling `ptr::read`, calling `mem::size_of`, etc.
This proposal introduces extern types, a way to accurately represent these opaque types in the Rust type system such that the compiler prevents you from misusing it.

The motivation for replacing [RFC 1861] is around computing the offset of fields in aggregate types.
Extern types do not have an alignment known to the Rust compiler and therefore cannot be included as a field in an aggregate type as it is impossible to correctly calculate their offset, however that RFC did not address this.
This implies that extern types cannot be included in any (non-`repr(transparent)`) structs and so must be prevented.

Extern types should be able to be used in generic contexts so that they receive blanket trait implementations and can be used inside generic wrapper types.
Prior to this, all generic types could be included as fields of structs and it was possible to obtain the size and alignment of any type.
Extern types do not have a size or alignment so this RFC allows users to specify generic bounds sufficiently to statically prevent the above issues.


# Guide-level explanation
[guide-level-explanation]: #guide-level-explanation

When writing bindings to foreign code, any types that are opaque to the Rust type system should be declared as follows:
```rust
extern {
type Foo;
}
Comment on lines +49 to +51
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the extern type Foo shorthand still allowed under this proposal or do all extern types have to be inside an extern block?

Similarly, how do extern blocks interact with ABI tags? For example, if I have an extern "C" block, is a type declaration inside it invalid, or just a warning? I would say that adding an ABI to these should be a deny-by-default lint that states that the tag is unused, with the potential for them having some meaning in the future.

Copy link
Member

@thomcc thomcc Feb 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that extern { type Blah; } will currently be reformatted by rustfmt to extern "C" { type Blah; }. So this question should probably have t-style weigh in.

I believe we've guaranteed the default is "C" anyway, so it's not clear to me that this makes sense.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This question gets at one of the niggles I have with this proposal. The ABI doesn't really mean anything as we're using extern to mean something slightly different to what it normally means. I think this is a point in favour of repr(unsized) but extern makes so much sense in that it heavily suggests what this feature is useful for.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When it comes to bikeshedding, what about something like the following?

#[repr(opaque)]
struct Foo;

I personally don't like the use of type and I think opaque captures the meaning better than either extern or unsized. I also think it just makes more sense as a repr, and it would go nicely with specifying alignment (eg. repr(opaque, align(16))) if we allow that, which I think we should (though that could be added later if necessary).

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's your reasoning for preferring opaque over unsized? I'm thinking of the common header use case where you may want:

#[repr(C, unsized)]
struct Header {
    field1: u8,
    field2: usize,
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are described as opaque types throughout the RFC, which I think describes them well. The thing that makes them unique isn't that you don't know their size, but that you don't know (nor care about) their composition. This is in contrast to types like [T], str, and dyn Trait, which are unsized, but not opaque.

Regarding the example you gave, if I didn't already know what it meant, I would find it very confusing. I wouldn't understand what unsized meant here, especially considering that the struct would appear to be composed entirely of sized types. It would be a lot less confusing, though, if the struct had no fields.

To create a type with a non-opaque header followed by opaque data, I would imagine creating two separate types: one for the opaque data itself, and a separate type for the opaque data with a header (though that would seem to require repr(align) on the opaque type). Using the word opaque in this context (where the opaque type has no fields) would clearly indicate not only that we don't know the size, but that we don't know (nor care about) the composition of the type. Here's how I would imagine doing that:

#[repr(opaque, align(4)]
struct Opaque;

#[repr(C)]
struct OpaqueWithHeader {
    field1: u8,
    field2: usize,
    opaque: Opaque,
}

That said, I'm not opposed to either extern type T nor the repr(unsized) syntax. I just wanted to express my view that repr(opaque) might be a better fit. However, regardless of the syntax used, I would discourage the idea of allowing fields within the opaque type itself, as that would seem quite confusing to me.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One issue I see with the repr attribute is that it looks too much like it's a unit struct declaration, if we're using a struct declaration I wonder if we could have native syntax for saying there are unknown fields (maybe with the repr too)

#[repr(opaque)]
struct Opaque {
  ..
}

Though, this has been proposed as syntax for #[non_exhaustive] previously. I think it fits opaqueness better since the fields are unknown even locally, whereas #[non_exhaustive] only affects metadata and the visible struct declaration is what it is known to be locally.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's one place where extern "C" { type X; } and extern "Rust" { type X; } could differ: the former obviously should be FFI-safe, but the latter doesn't necessarily have to be. If and only if it's not, then it's presumably semver-compatible to replace it with a standard struct in the future.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I see "opaque (type)", I think of RPIT (return-position impl trait) types, and soon TAIT, RPITIT, etc.

```
This is called an extern type and is `!Sized`, `!MetaSized` (explained below), `!Send`, `!Sync`, and is FFI-safe.
Unlike other dynamically-sized types (DSTs), currently only slices and trait objects, pointers to it (`&Foo`, `&mut Foo`, `*const Foo`, `*mut Foo`, etc) are thin, that is they are 1 `usize` wide not 2.

These types cannot be included in structs as Rust is not able to compute the offset of the field because it does not know the alignment.
```rust
struct Header {
data: u8,
tail: Foo, // Error due to unknown offset
}
```

However, these types may be placed in `repr(transparent)` structs:
```rust
#[repr(transparent)]
struct Wrapper<T> {
inner: Foo,
_marker: PhantomData<T>,
}
```

The lack of the `Sized` and `MetaSized` traits on these structs prevent you from calling `ptr::read`, `mem::size_of_val`, etc, which are not meaningful for opaque types.

In the 2021 edition and earlier, these types cannot be used in generic contexts as `T: Sized` and `T: ?Sized` both imply that `T` has a computable size and alignment.
Copy link
Member

@RalfJung RalfJung Sep 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if a new-edition trait has an associated type with ?Sized bound? If now old-edition code is generic over that trait, Trait::T would be a not-meta-sized type! IOW, old-edition code might actually have not-meta-sized types in a generic context.

Similarly, what if a new-edition trait has a generic function with T: ?Sized bound. If now old-edition code implements this with a T: ?Sized bound, this should raise an error since in the old edition, T: ?Sized actually means T: ?Sized+MetaSized and this the implementation is less generic than the trait requires. This means having generic functions with ?Sized bounds in a new-edition trait makes the trait impossible to implement in old editions.

(Both of these examples came out of boats's recent exploration into adding a Leak trait.)


In the 2024 edition and later, `T: ?Sized` no longer implies any knowledge of the size and alignment so opaque types can be used in generic contexts.
If you require your generic type to have a computable size and alignment you can use the bound `T: ?Sized + MetaSized`, which will enable you to store the type in a struct.

The automated tooling for migrating from the 2021 edition to the 2024 edition will replace `?Sized` bounds with `?Sized + MetaSized` bounds.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tooling should only do the replacement if Metasized is required, or a lot of declarations will get a lot noisier.

I feel it would be useful to have an idea on how much of an impact this will have on the ecosystem -- how common will it be to need a Metasized bound.

Doing the review of the standard library mentioned later would be a start.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tooling should only do the replacement if Metasized is required, or a lot of declarations will get a lot noisier.

How do you propose deciding which case is which? Note that (usually) relaxing the bound is non-breaking but adding it in later is breaking so a human decision is often required for what promises the API wants to make.



# Reference-level explanation
[reference-level-explanation]: #reference-level-explanation

Some nomenclature to assist the rest of this explanation:

Types have a size and alignment that is known:
* statically - the size/alignment is known by the Rust compiler at compile time. This is the current `Sized` trait.
Most types in Rust are statically sized and aligned, like `u32`, `String`, `&[u8]`.
Comment on lines +89 to +90
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In a way, this is the same as knowing from metadata -- the metadata is just ().

(I think that this way of wording is still clearer than omitting it, but figured it's worth pointing out somehow.)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I possibly should clarify that all statically sized things are metadata sized and all metadata sized things are dynamically sized (and similarly for alignment). Note there is still a meaningful distinction between these things: you can put as many statically sized&aligned things in a struct but only one metadata aligned thing in.

* from metadata - the size/alignment can be derived purely from pointer metadata without having to inspect or dereference the pointer.
All remaining types fit in this category and are DSTs.
`[u8]` has a statically known alignment but the size can only be determined from the pointer metadata, `dyn Debug`'s size and alignment are both obtained from the vtable in the pointer metadata.
* dynamically - the size/alignment can only be determined at run time.
There are no types currently expressible in the language with dynamically known size or alignment.
The most discussed potential type in this category is `CStr`, which has a statically known alignment but it's size can only be determined by iterating over it's contents to find the position of the null byte.
Note that these types are odd, for example determining the size of a `Mutex<CStr>` requires taking a lock on the mutex.
* unknown - the size/alignment is not able to be determined at compile time or run time.
This is the category that opaque types fall in (and no other existing types occupy), without any additional domain specific knowledge.
Therefore extern types will occupy this category to allow the most flexibility.

The rest of this document will refer to types as "statically sized", "metadata aligned", etc.

"dynamically aligned" (or "unknown aligned") types cannot be placed as the last field of a struct as their offset cannot be determined, without already having a pointer to the field.

In the Rust 2021 edition and earlier `T: Sized` implies that `T` is "statically sized" and "statically aligned" and `T: ?Sized` implies that `T` is "metadata sized" and "metadata aligned".

In the Rust 2024 edition and later `T: Sized` means the same but `T: ?Sized` implies that `T` is "unknown sized" and "unknown aligned".
A `MetaSized` bound can be introduced to regain the previous meaning.

## `MetaSized` trait
[metasized-trait]: #metasized-trait

This introduces a new trait `core::marker::MetaSized` that represents a type that is "metadata sized" and "metadata aligned":
```rust
#[lang = "meta_sized"]
trait MetaSized {}
```
This trait is automatically implemented for all types except extern types.

All types that implement `MetaSized` can be placed as the last field in a struct because the compiler can emit code to determine the offset of the field purely from the pointer metadata.
At the time of writing all `MetaSized` types are either statically aligned, or are trait objects and have their alignment in their vtable.
This RFC does not propose changing codegen for computing the offset as it does not introduce any new `MetaSized` types.

All locations in the standard library that use `?Sized` bounds need to be reviewed when migrating to the 2024 edition and replaced with either `?Sized` or `?Sized + MetaSized` as appropriate. Some examples:
```rust
pub fn size_of_val<T: ?Sized + MetaSized>(val: &T) -> usize
```
This requires `MetaSized` because the size must be known at run time.

```rust
pub struct Box<T: ?Sized + MetaSized, A: Allocator = Global>(Unique<T>, A);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without implied bounds, this will require the bound be mentioned everywhere Box (or Arc etc) are mentioned and can take a non-Sized parameter, including in traits.

pub trait Trait {
    fn foo(self: Box<Self>) where Self: MetaSized;
    fn bar(self: Arc<Self>) where Self: MetaSized;
}

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting but probably bad idea: MetaSized as a default bound for all trait, unless they opt out.

It's worth noting that since receivers are special, we could add special rules to make a Box<Self> receiver imply Self: MetaSized in traits. On the other hand, we'd probably prefer to make receivers less special.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implied bounds could solve this if they are ever implemented…

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really sad. It feels like a blocker that can't be easily skipped. How reasonable would it be to make receivers special until implied bounds is implemented? Or is the only option here to go and implement implied bounds?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just had a thought, doesn't implied bounds for self types arguably already exist given that you can write this:

trait Trait {
    fn foo(self);
}

You don't have to specify where Self: Sized on that. Granted, you can't implement it for unsized types but that feels like the correct behaviour.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well, technically:

trait Trait {
    fn foo(self);
}

doesn't actually imply where Self: Sized, just that you need unstable features to implement it without where Self: Sized. for a good example, see FnOnce which can be used as Box<dyn FnOnce(...)> to call fn call_once(self, ...) with Self = dyn FnOnce(...)

```
This requires `MetaSized` because the `Box` must be able to determine the Layout of the type at run time to allocate and free the backing memory.

```rust
impl<A: ?Sized, B: ?Sized> const PartialEq<&B> for &A
where
A: ~const PartialEq<B>,
```
This does not require `MetaSized` because the references to `A` and `B` always remain behind a pointer and the size and alignment is not required to call a function on the type.

## Extern types
[extern-types]: #extern-types

Extern types are defined as above, they are thin DSTs, that is their metadata is `()`. They cannot ever exist except behind a pointer, and so attempts to dereference them fail at compile time similar to trait objects.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume &*x is allowed with such types. This means that dereferencing (using the * operator) is actually fine, it's only the place-to-value coercion which is forbidden. That matches trait objects (and slices).

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, exactly. I hadn't considered that distinction, I'll try to clarify that later.


`repr` attributes are not permitted on extern types as none of the existing representations are applicable.

Extern types do not automatically implement any traits (except `Pointee`), but users can manually implement, for example, `Send`.
This does mean that all extern types will be `!Freeze` as it is private, however this is a compiler internal that is only used for optimisations, so this only causes some potential missed optimisations.

An extern type can be included in a `repr(transparent)` struct as it is always at offset 0.
The transparent struct is then also a thin DST and inherits traits as normal.


# Drawbacks
[drawbacks]: #drawbacks
Copy link
Member

@RalfJung RalfJung Sep 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another drawback is that this RFC rejects a type that is used widely inside rustc:

extern "C" {
    type OpaqueListContents;
}

pub struct List<T> {
    len: usize,
    data: [T; 0],
    opaque: OpaqueListContents,
}

This must definitely be mentioned in the RFC. Ideally there is some kind of proposal for what to do with this type...

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A simple suggestion: allow structs with extern tails, but don't allow computing the offset of (or taking a reference of) the extern tail. This is sufficient for the rustc impl, which uses the data field rather than the opaque field.

A less simple suggestion is a way to specify a known alignment to use for an extern type; this could either be via repr(align = N) or unsafe impl marker::Aligned { const ALIGN: usize = N; }.


This introduces language complexity for a feature that most users will not use directly.

(Copied from [RFC 1861])
The syntax has the potential to be confused with introducing a type alias, rather than a new nominal type. The use of extern here is also a bit of a misnomer as the name of the type does not refer to anything external to Rust.


# Rationale and alternatives
[rationale-and-alternatives]: #rationale-and-alternatives

## Do nothing

Doing nothing is not a compelling option here, [RFC 1861] is merged but is not implementable.
At minimum we should mark that RFC as deprecated/unimplemented and remove extern types from the compiler and docs.
However, the fact that RFC was merged is a very strong indication that something is needed here, and the current workaround with sized types is not good enough.

## No generics

This design aims to be minimal while still allowing extern types to feel like a fully supported part of the language, namely retaining the ability for them to be used in generics.
A simpler alternative would be to not allow extern types in generics, this would mean not adding `MetaSized` and retaining the existing meaning of `?Sized`.
This would severely restrict the utility of extern types and would prevent them from implementing many useful traits that have blanket implementations in the standard library.

## More traits

We could introduce a large portion of the traits implied by the "dynamically/metadata/statically sized/aligned" nomenclature above.
This would allow the most flexibility about exactly what a generic needs but existing usage suggests that this is not necessary and would be a significant amount of language complexity.

## Post-monomorphisation errors

We could allow extern types to be used in generics without the `MetaSized` machinery and simply generate a post-monomorphisation error when the compiler is not able to lay out a type or a function attempts to call `size_of_val` or `align_of_val`.
This would be a significant departure from the compilers approach to generics, but is not entirely unprecedented as these sorts of errors can be generated by const evaluation.

## Opt-out trait bound - `?MetaSized`

This change could be made in an entirely backwards compatible way by leaving the existing meaning of `?Sized` alone and allowing relaxing the implied `MetaSized` bound with `T: ?Sized + ?MetaSized` (or possibly just `T: ?MetaSized`).
This is not suggested by this document because the lang team has historically said that they do not wish to add any more opt-out bounds.

## Trait methods

`MetaSized` could include `size_of_val` and `align_of_val` rather than relying on the free functions:
``` rust
trait MetaSized: Pointee {
fn size_of_val(metadata: <Self as Pointee>::Metadata) -> usize;
fn align_of_val(metadata: <Self as Pointee>::Metadata) -> Alignment;
}
```
This is the more obvious way of writing the trait, however it would prevent splitting `MetaSized` and `MetaAligned` in the future and adding these functions later would be hard but probably not impossible.


# Prior art
[prior-art]: #prior-art

There is a lot of prior art in rust itself from previous attempts at custom DSTs, the `DynSized` trait, and some other things related to "exotically sized types".
- [Lang team design notes on exotically sized types](https://github.com/rust-lang/lang-team/blob/master/src/design_notes/dynsized_constraints.md).
This document contains notes from the lang team about what `?Sized + DynSized` needs to imply.
This document outlines `DynSized`, `MetaSized`, and `Sized` which inspired the "metadata sized" and friends in this RFC.
I believe this RFC satisfies the constraints outlined, mostly by dropping `DynSized` as an available bound, which limits expressiveness in favour of simplicity.
- [Custom DSTs](https://github.com/rust-lang/rfcs/pull/2594).
This postponed RFC attempts to introduce a generic framework for DSTs with arbitrary metadata.
This RFC aims to be compatible with future attempts at custom DSTs, as it is in essence a very restricted form.
- [DynSized without ?DynSized](https://github.com/rust-lang/rfcs/pull/2310).
This contains a lot of very useful analysis of what exactly commonly used crates want when they state `?Sized`.
However, this RFC aims to be simpler than the lint-based solution presented there.
Additionally, this deals with not being able to place extern types as fields in structs, rather than solely on `size_of_val` and `align_of_val`.
- [More implicit bounds](https://github.com/rust-lang/rfcs/issues/2255).
This discusses whether we should be adding more implicit bounds into the language, and there associated relaxations (like `?MetaSized`).
This RFC aims to sidestep this issue by utilising the edition system to change the definition of `?Sized`.


# Unresolved questions
[unresolved-questions]: #unresolved-questions

- Should "metadata sized" imply "metadata aligned" or should we be adding the `MetaAligned` trait rather than `MetaSized`?
- Should `MetaSized` be a supertrait of `Sized`? All `Sized` things are `MetaSized` but `Sized` doesn't semantically require `MetaSized`.
- Should users be able to slap a `#[repr(align(n))]` attribute onto opaque types to give them an alignment?
This would allow us to represent `CStr` properly but would necessitate splitting `MetaSized` and `MetaAligned` as it is only "dynamically sized" but "statically aligned".
(We may be able to get away with the [Aligned trait](https://github.com/rust-lang/rfcs/pull/3319))
- Should the `extern type` syntax exist, or should there just be a `repr(unsized)`?
This would allow headers with opaque tails (which are very common in C code) but is a more significant departure from the original RFC, and looks more like custom DSTs.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you need repr(unsized) for this? Couldn't you just allow the last field of a type to be ?MetaSized? (Perhaps requiring the struct to be repr(C)?)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Imagine we have a setup like the following:

#[repr(C)]
struct Foo {
    a: u32,
    b: u8,
    foo: String,
}
#[repr(C)]
struct Bar {
    a: u32,
    b: u8,
    bar: u8,
}
extern type OpaqueTail;
#[repr(C)]
struct HeaderWithTail {
    a: u32,
    b: u8,
    tail: OpaqueTail,
}
#[repr(C, unsize)]
struct HeaderUnsized {
    a: u32,
    b: u8,
}

What alignment does OpaqueTail have? &header.tail doesn't really have a well defined meaning as it's either pointing at bar directly (assuming an alignment of one) or some random padding bytes before foo starts. However, you can freely convert between a &Foo (or &Bar) and a &HeaderUnsize without worrying about that.
This alignment issue is why this RFC doesn't propose allowing "dynamically aligned" or "unknown aligned" (?MetaSized) types as fields of structs (except as the only non-ZST field of a #[repr(transparent)] struct).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I suppose you could forbid referencing the field entirely…



# Future possibilities
[future-possibilities]: #future-possibilities

The most obvious future possibility is custom DSTs.
This would provide a mechanism for allowing users to implement types with entirely custom metadata and therefore custom implementations of `size_of_val` and `align_of_val`.
This would likely mean that users could implement types that acted like extern types without using the `extern type` syntax.
This should not be an issue as `extern type` communicates the intent of these types well, and guarantees FFI-safety.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One other valuable extension would be to make MaybeUninit accept ?Sized + MetaSized types.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MaybeUninit currently takes a sized T, why would this enable relaxing that bound? T: ?Sized currently means the same as I'm proposing T: ?Sized + MetaSized will mean, so I don't see how this helps.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I misread the proposal and assumed that ?Sized would opt out of MetaSized too.

Without a notion of MetaSized, MaybeUninit has to be sized, since there's no good way to determine the layout of an unsized value. Adding MetaSized means we can now do this with just metadata, making it valid again.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry to be clear the proposal is that ?Sized will opt out of MetaSized after the edition change. So far I've assumed that, because all types in current Rust are MetaSized, that introducing this won't help with issues like this. It's possible that's not true and this is an exception.

How would you create a MaybeUninit<T: ?Sized + MetaSized>, where would you get the metadata from? Can it be uninit? Aren't those the same issues you'd face if you opened an RFC to make MaybeUninit take ?Sized today?


[RFC 1861]: https://rust-lang.github.io/rfcs/1861-extern-types.html