Add DynSized trait (rebase of #44469) #46108

mikeyhew · 2017-11-20T05:46:22Z

I managed to rebase @plietar's PR #44469, and get all the tests passing.

This unfortunately conflicts with #44917 (by @Zoxc) in a lot of places, so one of us will have to do a rebase after the other gets merged.

r? @pnkfelix since you were reviewing #44469
cc @arielb1

Zoxc · 2017-11-20T20:11:11Z

Letting auto trait imply ?Move and ?DynSized might be a good idea. It would at least avoid all the changes to the tests.

@nikomatsakis Any chance that bounds on Self will become sound for auto traits in the future?

mikeyhew · 2017-11-20T20:21:52Z

A couple of interesting points that weren't mentioned in the conversation on #44469:

DynSized is an implicit supertrait of all traits, including Sized. This is in contrast to Sized, which is only an implicit bound on type parameters and associated types.
As a result, this messes up auto traits, which may not have super traits. After this PR, all auto traits must be updated to explicitly remove DynSized as a super trait:
```
#![feature(optin_builtin_traits, dynsized)]
use core::marker::DynSized;
unsafe auto trait Auto: ?DynSized {}
```
Whenever another implicit supertrait is added, auto traits will break again. This will happen when Immovable types prototype where the Move trait is builtin #44917 lands, which adds an implicit supertrait Move.

Because of the potential for future breakage, it may make sense to special-case auto traits, so that they don't have any implicit supertraits. As long as auto traits can't have items, it hopefully should be possible to add or remove implicit supertraits without breaking. Or alternatively, if auto traits could be allowed to have supertraits, we could allow that, as @Zoxc has just mentioned

mikeyhew · 2017-11-20T20:22:52Z

cc @bluss

mikeyhew · 2017-11-20T20:25:59Z

@Zoxc I saw your comment just before I posted mine, just didn't want to rewrite everything. You basically said everything that I was saying, but in way fewer words :)

mikeyhew · 2017-11-20T22:29:38Z

ugh... src/test/rustdoc/foreigntype.rs is failing, but I don't have that file in my local branch... guess I need to rebase

The DynSized trait is implemented by all types which have a size and alignment known at runtime. This includes every type other than extern types, introduced in RFC 1861 and implemented in rust-lang#44295, which are completely opaque. The main motivation for this trait is to prevent the use of !DynSized types as struct tails. Consider for example the following types : ```rust extern { type foo; } struct A<T: ?Sized> { a_x: u8 a_y: T } struct B<T: ?Sized> { b_x: u8 b_y: T } ``` Before this change, the type `A<B<foo>>` is considered well-formed. However, the alignment of `B<foo>` and thus the offset of the `a_y` field depends on the alignment of `foo`, which is unknown. By introducing this new trait, struct tails are now required to implement `DynSized`, such that their alignment is known. The trait is an implicit bound, making `A<B<foo>>` ill-formed. Just like the `Sized` trait, the default bound can be opted-out by using `?DynSized`.

auto traits cannot have bounds, so they must remove the implicit DynSized bound with `Self: ?DynSized`

This fixes a bunch of ui tests. It now either prints out `T`, `T: ?Sized`, or `T: ?DynSized`.

There are other types that can be in a DST struct’s tail: other DST structs, DST tuples, and possibly more. The only one we don’t want to allow is extern types, so I added a check for that

had to add the dynsized lang item, and add ?DynSized bound to auto traits

Had to add ?DynSized to auto-traits

Add a `?DynSized` unbound to the auto trait in the example, and explain why it is necessary

extern types do not implement DynSized, so to implement a trait for one you need to explicitly remove the DynSized bound for that trait

liigo · 2017-11-21T03:24:37Z

dynamically (at runtime) sized types are DynSized, that's ok.
but statically (known at compiletime) sized types are DynSized?? that's a bit weird.
Maybe you should rename DynSized. (ClearSized? VS Opaque types)

mikeyhew · 2017-11-21T04:55:41Z

@liigo anything that is Sized is trivially also DynSized. If you know the size of something at compile time, then you know it a runtime. That's why DynSized is a supertrait of Sized, though that's not immediately obvious from the declaration of Sized because the DynSized bound is implicit

mikeyhew · 2017-11-21T06:08:54Z

@iigo I'll try to explain that a bit better: DynSized (in this interpretation) doesn't mean "not statically sized", but rather "has a size that can be calculated at run time".

shepmaster · 2017-11-24T20:56:05Z

Ping from triage @pnkfelix — will you be able to review this shortly?

nikomatsakis · 2018-01-10T22:14:19Z

So @withoutboats and I were talking and we had what I think is an interesting insight. We were thinking about when one might want T vs T: ?Sized vs T: ?DynSized and how to relate those to the concepts of ownership / borrowing. We came up with the idea that:

T -- will use this "by value" (no indirection)
T: ?Sized -- may take ownership (or at least partial ownership), but only use through a "smart pointer" (e.g., Box, Arc)
- this falls out because, at least with our current allocators, we can't free memory unless we know its size, so you must have size_of_val
T: ?DynSized -- no ownership (e.g., only used via &T or *mut T)
- if something is borrowed, you don't need to know its size

I'm not sure yet what this means but I find it to be an interesting correspondence. Are there exceptions?

mikeyhew · 2018-01-11T02:13:19Z

T: ?Dynsized can still be owned, just not using Box. People will probably implement Drop for T by calling some C api function, and use a different owning pointer type that just calls ptr::drop_in_place when dropped.
The distinction between T and T: ?Sized will be clouded a bit if/when we have by-value DST.

nikomatsakis · 2018-01-11T16:41:24Z

@mikeyhew

just not using Box

Yes; not using the Rust allocator, basically. Not sure how important that is. (Sometimes I regret designing our allocators so that they supply the size of the value being freed, but I guess it's awfully late to try and reverse course there.)

The distinction between T and T: ?Sized will be clouded a bit if/when we have by-value DST.

Yes, true.

withoutboats · 2018-01-11T23:01:23Z

(Sometimes I regret designing our allocators so that they supply the size of the value being freed, but I guess it's awfully late to try and reverse course there.)

It seems like the use cases for size_of_val diminish significantly without this requirement. I wonder if we couldn't revisit this?

nikomatsakis · 2018-01-23T18:39:05Z

OK, I feel bad that this PR is sitting here with no viable means to go forward or backward yet. I remain highly nervous about ? traits like ?DynSized. It still feels to me like this is overall a bad direction for Rust to be going in. But I don't want to get into a litany of these concerns (though it makes sense to write those out and work through them). I would rather focus on the fate of this PR.

i.e., do we want to move forward with ?DynSized even in an experimental capacity? Or should we close the PR?

(It seems clear we have to ensure it is 100% backwards compatible; I've kind of lost track of whether that is the case now.)

@mikeyhew if your main motivation is to explore custom DST, I'd rather just explore that directly, while leaving the questions around DynSized slightly unresolved. I continue to feel they are somewhat orthogonal questions.

mikeyhew · 2018-01-24T00:41:50Z

@nikomatsakis and @withoutboats, and anyone else with concerns about ?DynSized and other ?-traits, a good place to discuss them is in More implicit bounds (?Sized, ?DynSized, ?Move) rfcs#2255
Regarding backward compatibility, this PR is in theory backward compatible, but there is still this issue, which will cause some regressions. Back in October, @Zoxc listed crates that were affected by that issue in this comment
about custom DST and related stuff, I would also like to move ahead and experiment with that. What I can do is open a thread on internals and discuss the design that I have in mind, using the DynSized trait, and then you can poke and prod and we can discuss alternatives

nikomatsakis · 2018-01-24T17:42:09Z

@mikeyhew

a good place to discuss them is in rust-lang/rfcs#2255

agreed, this PR doesn't feel like the right place!

this PR is in theory backward compatible, but there is still this issue, which will cause some regressions

OK, I see, I remember that now. Thanks for the clarification.

What I can do is open a thread on internals and discuss the design that I have in mind, using the DynSized trait, and then you can poke and prod and we can discuss alternatives

Sounds like a plan.

kennytm · 2018-01-25T18:55:55Z

FYI I've submitted rust-lang/rfcs#2310 as an alternative.

nikomatsakis · 2018-01-26T21:01:38Z

@rfcbot fcp close

We discussed this in the @rust-lang/lang meeting today. As a result of that conversation, I'm going to move to close this PR. The general feeling was that the "jury is still out" on what approach to use for handling extern types and the related problems -- as @withoutboats and I have shared here, there is a general concern about the complexity costs of a ?DynSized-based solution (and ?-trait-bounds in general), particularly when weighed against the relative obscurity of extern types (it's a lot of language baggage for a corner case). On the other hand, @mikeyhew and others make a strong case for static guarantees.

Given this uncertainty, I at least am reluctant to land deep changes to the compiler at this time, and in particular not to the trait system (which I think is due for some overhauls). Past experience (e.g., with the ! type) shows that this often has unexpected consequences, causing regressions and other maintenance burden, which then distracts from higher priority goals.

Speaking personally: I do intend to leave a comment on rust-lang/rfcs#2255 "real soon now" trying to spell out some of the specific confusion that can arise from ?DynSized. I am also still intrigued by the possibility of finding an "ownership-based" way of thinking about optional traits along the lines I outlined before, though I don't know if that really leads anywhere.

rfcbot · 2018-01-26T21:01:39Z

shepmaster · 2018-02-03T01:58:00Z

Ping for ticky boxes @Zoxc, @arielb1, @jseyfried, @michaelwoerister!

nikomatsakis · 2018-02-05T17:10:24Z

I tagged @arielb1 (who should be moved to alumni status, really) as well as @michaelwoerister (who is on vacation afaik) and @jseyfried (who probably also should be moved to alumni status).

rfcbot · 2018-02-05T17:10:26Z

🔔 This is now entering its final comment period, as per the review above. 🔔

aturon · 2018-02-06T18:54:16Z

@nikomatsakis you can modify the FCP list here: https://github.com/anp/rfcbot-rs/blob/master/teams.toml

@arielb1

Removed @arielb1 and @jseyfried (rust-lang/rust#46108 (comment)). Added @nagisa and @oli-obk (https://internals.rust-lang.org/t/please-welcome-nagisa-and-oli-obk-to-the-compiler-team/6712).

rfcbot · 2018-02-15T17:11:25Z

The final comment period is now complete.

nikomatsakis · 2018-02-15T17:18:04Z

OK, going to close. Thanks @mikeyhew

Ericson2314 · 2018-03-29T13:37:43Z

@nikomatsakis The language team decision against ?DynSize looked to me like "jurry is still out, so the fact that the code is in flux (chalk, etc) we use as a tie-breaker". That sounds to me more like "not now" than "no never": is there a possibility for landing this on an experimental basis once the type system overhauls are done?

The problem seems to be ?-traits syntax more than semantics that people want. I say landing this on an experimental basis is best way to find a different syntax / explore downstream problems like custom DSTs. Otherwise too many chained problems are hard to untangle.

mikeyhew · 2018-03-29T19:32:26Z

@Ericson2314 I agree with you, I think that ultimately having a trait or traits like DynSized is desired, and that the decision to close this PR wasn't a rejection of the underlying concept, but rather a "let's postpone this for now, and get a better idea of what we want to do first".

I've been meaning to comment on https://internals.rust-lang.org/t/pre-erfc-lets-fix-dsts/6663 and keep the discussion going, and I still think that is the best way forward – we should get an eRFC merged and start working on things experimentally.

Ericson2314 · 2018-04-04T15:32:06Z

@mikeyhew See the end of #43467 . Looks like DynSized is off the table for now, but hopefully it could come back as a non-default bound with {size,align}_of_bound being deprecated.

rust-highfive assigned pnkfelix Nov 20, 2017

kennytm added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Nov 20, 2017

plietar and others added 17 commits November 20, 2017 17:29

Make <*const T>::is_null() and al work with T: ?DynSized.

959e988

Fix tests for DynSized

e9639d1

Add DynSized related tests

ec6809d

Fix tidy errors

537da97

Fix typo by me

76c6636

add DynSized query variants to plumbing

136cd10

fix some auto_trait tests by removing the DynSized bound

0531eb5

auto traits cannot have bounds, so they must remove the implicit DynSized bound with `Self: ?DynSized`

allow ?Trait bounds in auto traits

fe40804

don't print DynSized in conflicting trait impl error messages

38efc95

This fixes a bunch of ui tests. It now either prints out `T`, `T: ?Sized`, or `T: ?DynSized`.

don't bug! if the unsized struct tail isn't [T], str, or a trait object

76b2086

There are other types that can be in a DST struct’s tail: other DST structs, DST tuples, and possibly more. The only one we don’t want to allow is extern types, so I added a check for that

shorten line for tidy

83cffc8

fix run-make/simd-ffi test

76ab610

had to add the dynsized lang item, and add ?DynSized bound to auto traits

add ?Dynsized support to rustic

d308bee

fix some rustdoc tests

a36e2fe

Had to add ?DynSized to auto-traits

Update auto traits in the unstable book

62b4c91

Add a `?DynSized` unbound to the auto trait in the example, and explain why it is necessary

Fix src/test/rustdoc/foreigntype.rs

b9a69c7

extern types do not implement DynSized, so to implement a trait for one you need to explicitly remove the DynSized bound for that trait

mikeyhew force-pushed the add-dynsized branch from 27133cc to b9a69c7 Compare November 21, 2017 00:11

shepmaster added the T-lang Relevant to the language team, which will review and decide on the PR/issue. label Dec 1, 2017

nikomatsakis added the I-nominated label Jan 25, 2018

rfcbot added the proposed-final-comment-period Proposed to merge/close by relevant subteam, see T-<team> label. Will enter FCP once signed off. label Jan 26, 2018

nikomatsakis removed the I-nominated label Jan 29, 2018

rfcbot added the final-comment-period In the final comment period and will be merged soon unless new substantive objections are raised. label Feb 5, 2018

rfcbot removed the proposed-final-comment-period Proposed to merge/close by relevant subteam, see T-<team> label. Will enter FCP once signed off. label Feb 5, 2018

This was referenced Feb 6, 2018

Update compiler team rust-lang/rfcbot-rs#183

Merged

Move @arielb1 and @jseyfried to alumni rust-lang/prev.rust-lang.org#1037

Merged

nikomatsakis closed this Feb 15, 2018

SimonSapin mentioned this pull request Mar 28, 2018

Tracking issue for RFC 1861: Extern types #43467

Open

7 tasks

nox mentioned this pull request Sep 10, 2020

Introduce Pointee and DynSized rust-lang/rfcs#2984

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add DynSized trait (rebase of #44469) #46108

Add DynSized trait (rebase of #44469) #46108

mikeyhew commented Nov 20, 2017

Zoxc commented Nov 20, 2017

mikeyhew commented Nov 20, 2017

mikeyhew commented Nov 20, 2017

mikeyhew commented Nov 20, 2017 •

edited

Loading

mikeyhew commented Nov 20, 2017

liigo commented Nov 21, 2017 •

edited

Loading

mikeyhew commented Nov 21, 2017 •

edited

Loading

mikeyhew commented Nov 21, 2017

shepmaster commented Nov 24, 2017

nikomatsakis commented Jan 10, 2018

mikeyhew commented Jan 11, 2018

nikomatsakis commented Jan 11, 2018

withoutboats commented Jan 11, 2018

nikomatsakis commented Jan 23, 2018

mikeyhew commented Jan 24, 2018

nikomatsakis commented Jan 24, 2018 •

edited

Loading

kennytm commented Jan 25, 2018

nikomatsakis commented Jan 26, 2018

rfcbot commented Jan 26, 2018 •

edited by nikomatsakis

Loading

shepmaster commented Feb 3, 2018

nikomatsakis commented Feb 5, 2018

rfcbot commented Feb 5, 2018

aturon commented Feb 6, 2018

rfcbot commented Feb 15, 2018

nikomatsakis commented Feb 15, 2018

Ericson2314 commented Mar 29, 2018

mikeyhew commented Mar 29, 2018

Ericson2314 commented Apr 4, 2018

Add DynSized trait (rebase of #44469) #46108

Add DynSized trait (rebase of #44469) #46108

Conversation

mikeyhew commented Nov 20, 2017

Zoxc commented Nov 20, 2017

mikeyhew commented Nov 20, 2017

mikeyhew commented Nov 20, 2017

mikeyhew commented Nov 20, 2017 • edited Loading

mikeyhew commented Nov 20, 2017

liigo commented Nov 21, 2017 • edited Loading

mikeyhew commented Nov 21, 2017 • edited Loading

mikeyhew commented Nov 21, 2017

shepmaster commented Nov 24, 2017

nikomatsakis commented Jan 10, 2018

mikeyhew commented Jan 11, 2018

nikomatsakis commented Jan 11, 2018

withoutboats commented Jan 11, 2018

nikomatsakis commented Jan 23, 2018

mikeyhew commented Jan 24, 2018

nikomatsakis commented Jan 24, 2018 • edited Loading

kennytm commented Jan 25, 2018

nikomatsakis commented Jan 26, 2018

rfcbot commented Jan 26, 2018 • edited by nikomatsakis Loading

shepmaster commented Feb 3, 2018

nikomatsakis commented Feb 5, 2018

rfcbot commented Feb 5, 2018

aturon commented Feb 6, 2018

rfcbot commented Feb 15, 2018

nikomatsakis commented Feb 15, 2018

Ericson2314 commented Mar 29, 2018

mikeyhew commented Mar 29, 2018

Ericson2314 commented Apr 4, 2018

mikeyhew commented Nov 20, 2017 •

edited

Loading

liigo commented Nov 21, 2017 •

edited

Loading

mikeyhew commented Nov 21, 2017 •

edited

Loading

nikomatsakis commented Jan 24, 2018 •

edited

Loading

rfcbot commented Jan 26, 2018 •

edited by nikomatsakis

Loading