Heap allocations in constants

[fee1-dead]

Proposal

Summary and problem statement

Support allocating and deallocating heap memory in constants.

Motivation, use-cases, and solution sketches

In order to totally outdo any other constant evaluators out there, it is desirable to allow things like using serde to deserialize e.g. json or toml files into constants. In order to not duplicate code between const eval and runtime, this will require types like Vec and String. Otherwise every type with a String field would either need to be generic and support &str and String in that field, or just outright have a mirror struct for const eval. Both ways seem too restrictive and not in the spirit of "const eval that just works".

We need to forbid constants where the final value is a non-empty buffer, because we might try to deallocate static memory if used:

const FOO: String = String::from("foo");
let x = FOO;
drop(x);

One proposed solution could be auto marker traits that decide what types constants can be. We can use ConstSafe and ConstRefSafe (bikeshed):

• &T: ConstSafe where T: ConstRefSafe
• &mut T: !ConstSafe
• *const T: !ConstSafe
• *mut T: !ConstSafe
• UnsafeCell<T>: !ConstSafe
• String: ConstRefSafe

*const T isn't ConstSafe, since it might point to the const heap, but a null pointer is perfectly fine and can be created in constants, therefore we need another way to track if something is allocated in the heap.

We can use a const(heap) effect to signify that the returned type contains an allocation in the heap. My current approach is an attribute: #[const_heap] instead of modifying the syntax. The exact semantics are as follows:

• All const fn not marked with #[const_heap] has a return type of TotallyConstSafe<T> where T is the annotated return type of the function.
  • TotallyConstSafe cannot be named. It is an invisible type that only exists in the type system.
• const FOO: T also has the type TotallyConstSafe<T>.
• The return value of a const body coerce into TotallyConstSafe<T> if either:
  • T: ConstSafe; or
  • There are no calls to #[const_heap] functions in the body.

This would forbid const S: String = String::from("foo"); because String::from is #[const_heap] and String: !ConstSafe.

const A: String = String::new(); // Ok since String::new isn't #[const_heap]
const B: String = String::from("foo"); // Not OK because of reasons above
const C: &String = &String::from("foo"); // Ok because &String is ConstSafe 
const D: &str = &String::from("foo"); // Ok because &str is ConstSafe

This approach is totally backwards compatible as there are no const functions currently marked with #[const_heap].

Links and related work

Some text of this proposal are copied from rust-lang/const-eval#20.

Initial people involved

• Owner, if known: @fee1-dead
• Liaison: TBD

What happens now?

This issue is part of the lang-team initiative process. Once this issue is filed, a Zulip topic will be opened for discussion, and the lang-team will review open proposals in its weekly triage meetings. You should receive feedback within a week or two.

This issue is not meant to be used for technical discussion. There is a Zulip stream for that. Use this issue to leave procedural comments, such as volunteering to review, indicating that you second the proposal (or third, etc), or raising a concern that you would like to be addressed.

[rustbot]

This issue is not meant to be used for technical discussion. There is a Zulip stream for that. Use this issue to leave procedural comments, such as volunteering to review, indicating that you second the proposal (or third, etc), or raising a concern that you would like to be addressed.

[nikomatsakis]

cc @rust-lang/wg-const-eval

[joshtriplett]

We could discuss this on the December 15th design meeting, if wg-const-eval is able to present it on that date.

[fee1-dead]

I would be partly unavailable - but since most of this is based on @RalfJung and @oli-obk's ideas, I think it is better for them to present this :) - I would be available on Zulip for questions until ~20-30 minutes into the meeting.

[RalfJung]

I assume you mean @oli-obk? I did not spend a lot of time thinking about heap allocation, I just played 'rubber duck' with Oli so he could bounce his ideas off of someone.

What time are design meetings again? Maybe this should be prominently mentioned in the issue template or on https://github.com/rust-lang/lang-team, I have that same question every single time a design meeting is scheduled where I might be involved...

EDIT: Okay I found the calendar (just 4 clicks away or so^^) and it looks like it's 1pm. I will be in Europe again next week so this should be possible but I have to talk to my family about adjusting dinner time.

[RalfJung]

When and where will it be announced whether the meeting is scheduled for tmr or not?

[nikomatsakis]

@RalfJung I believe the plan was to possibly do this on Dec 22nd not 15th, sorry, that was a mistake, but we hadn't definitely decided. We would need to somebody to kind of lead it. I'm not sure who would be best to write a document to guide that discussion -- perhaps @oli-obk ?

I'm also not clear on whether this particular proposal would be the best focus or if there are other priorities (an overview of the roadmap might be useful, for example).

[oli-obk]

I'll write something up. I think starting with a roadmap overview and then digging into this would be best

[RalfJung]

Okay -- I will probably not be able to attend that day. But I have not invested a lot of thought into const heap support anyway and didn't plan to be deeply involved. I am happy to give feedback or play rubber duck for concrete proposals though.

[nikomatsakis]

@oli-obk prepared this document, so we will try to review it tomorrow at the usual time (1pm US Eastern time)

[nikomatsakis]

I propose we close this proposal. I think we are not ready to endorse this particular step at this particular time, although I would very much like to see progress around const eval (and I'm sympathetic to the goals).

[nikomatsakis]

Discussed in the @rust-lang/lang meeting and decided that we are going to close this issue for now. This is a deferral in the sense that we're not opposed to the concept, but it seems like we are not ready to address this particular question right now, given the state of consteval. Thanks to @fee1-dead for raising it!

[nikomatsakis]

In terms of when we'd like to see this re-opened, I would expect that to be part of a more general "compile time computation" initiative that lays out a complete roadmap for const-eval and const-generics.