Tracking issue for dyn upcasting coercion

[alexreg]

This is a tracking issue for RFC3324. Corresponding MCP is here.
The feature gate for the issue is #![feature(trait_upcasting)].

STATUS UPDATE

We are in the process of stabilizing.

About tracking issues

Tracking issues are used to record the overall progress of implementation.
They are also used as hubs connecting to other relevant issues, e.g., bugs or open design questions.
A tracking issue is however not meant for large scale discussion, questions, or bug reports about a feature.
Instead, open a dedicated issue for the specific matter and add the relevant feature gate label.

Steps

• Implement the experimental feature behind a feature gate.
• Define the semantics explicitly.
• Gathering design docs into initiative repo: https://github.com/rust-lang/dyn-upcasting-coercion-initiative
• Adjust documentation ([see instructions on rustc-dev-guide][doc-guide]) Add trait_upcasting related languages changes reference#1259
• Gather data about memory usage? add feature gate to permit measuring memory usage of dyn upcasting #112355
• Stabilization PR ([see instructions on rustc-dev-guide][stabilization-guide]) Stablize trait_upcasting feature #101718, Stabilize RFC3324 dyn upcasting coercion #118133
• Fix soundness problems.
  • Make casts of pointers to trait objects stricter #120248
• Restabilize.

Previous discussions

• RFC401: Type conversions rfcs#401
• RFC982: DST custom coercions. rfcs#982
• RFC-issue2765: Objects should be upcastable to supertraits rfcs#2765
• MCP: Trait upcasting coercion lang-team#98 (Current lang-team MCP)
• Initiative: Dyn upcasting coercion initiative
• RFC: Support upcasting of dyn Trait values rfcs#3324

Unresolved Questions

• Currently CoerceUnsized trait cannot express this case: a smart pointers that wrap a raw pointer and don't guarantee via a custom invariant that it is valid. Maybe a separate CoerceUnsizedUnsafe trait is needed. (see Trait upcasting coercion (part4) #88010 (comment)).
• Should we make upcasting opt-in in some form to limit vtable size by default? The current inclination of the lang-team is "no", but it would be useful to gather data on how much supporting upcasting contributors to overall binary size.
• Before stabilizing it we should check that libs-api is ok with upcasting for all dyn-allowed traits in the library, since those we can't change. (addressed by Tracking issue for dyn upcasting coercion #65991 (comment))
• Should we add an opt-out mechanism, and extend library stabilization checklist with "do we want to opt-out for now"?

Implementation history

• Initial attempt: Trait upcasting #60900
• Second attempt:
  • Preparation refactoring: Refactor vtable codegen #86291
  • Unify vtable building: Change vtable memory representation to use tcx allocated allocations. #86475
  • Update vtable format: Refactor vtable format for upcoming trait_upcasting feature. #86461
  • Feature gate implementation: Trait upcasting coercion (part1) #86264
  • Pointer conversion adjustments: Trait upcasting coercion (part2) #87515
  • Support multiple occurrence of a target trait with different substsdefs: Trait upcasting coercion (part 3) #88135
  • Unsafety checking for raw dyn pointers: Trait upcasting coercion (part4) #88010 (not prefered approach, likely will need to do nothing here, but need a design meeting discussion with lang-team) Solution: https://rust-lang.zulipchat.com/#narrow/stream/326132-t-types.2Fmeetings/topic/meeting.202022-09-02.20planning/near/296826711, Commit to safety rules for dyn trait upcasting #101336
  • Lint for easing migration: Tracking issue for future-incompatibility lint deref_into_dyn_supertrait #89460
  • Reference changes
    • Add trait_upcasting related languages changes reference#1259
    • Add trait_upcasting related languages changes reference#1622
  • Mark this feature no longer incomplete and stablize: Stablize trait_upcasting feature #101718
  • Make trait upcasting work with supertraits with fewer associated types: Rework upcasting confirmation to support upcasting to fewer projections in target bounds #114036

[nikomatsakis]

This is currently blocked on resolving the safety conditions for upcasting. The following updated document describes my current thinking, looking for feedback:

https://rust-lang.github.io/dyn-upcasting-coercion-initiative/design-discussions/upcast-safety-2.html

[dimpolo]

I was playing around with this and discovered that the feature gate (and its incomplete_features warning) can be bypassed using #![feature(unsize)] instead.

This will upcast a &dyn Bar where Foo is a supertrait of Bar.

fn upcast<Dyn: ?Sized + Unsize<dyn Foo>>(bar: &Dyn) -> &dyn Foo { bar }

Playground

Would this mean the unsize feature should be incomplete as well?

[danielhenrymantilla]

Came to comment the same observation that @dimpolo has made: I think the impl of Unsize<dyn '_ + Super> for dyn '_ + Sub ought, itself, to be feature-gated by trait_upcasting.

[nikomatsakis]

That makes sense, although really I just want to stabilize this.

[Mark-Simulacrum]

cc #89460, which is tracking a future-compat lint that seems related to this feature.

[PoignardAzur]

Can we have a status update with regards to the reverted stabilization? The current status update says "We are in the process of stabilizing" since January, with no mention of the recent soundness issues.

[WaffleLapkin]

The stabilization is currently blocked on the fix of the unsoundness bug tracked in #120222. The fix is in #120248, but needs a bit more work. I've been occupied by the never type stuff which is urgent because some of the changes are edition based. I hope I'll have time soon to finish #120248.

This is also blocked on rust reference PR, which someone needs to take over: rust-lang/reference#1259 (comment).

[Kish29]

It looks like the blocking bug #120222 have been fixed out, and what's next for this stabilization?

[WaffleLapkin]

@Kish29 the next steps are changing the reference (cc rust-lang/reference#1259) and doing a new stabilization PR :)