Unmanaged Constants

[tannergooding]

Summary

Provide a general-purpose and safe way for declaring constant values for unmanaged structs.

Motivation

Today, users can only create constants for a limited selection of "primitive" types. However, users often want or need to declare constant values for their own types.

Design

The runtime currently provides support for Data declarations [II.16.3.1], which I believe can be used to facilitate this support without requiring changes to the underlying runtime.

A user should then be able to declare something similar to the following:

public unmanaged struct Vector4
{
    private float _x;
    private float _y;
    private float _z;
    private float _w;

    public Vector4(float x, float y, float z, float w)
    {
        _x = x;
        _y = y;
        _z = z;
        _w = w;
    }

    public const Vector4 One = new Vector4(1.0f, 1.0f, 1.0f, 1.0f);
}

This would get emitted as IL similar to the following:

.class public sequential ansi sealed beforefieldinit Vector4
       extends [mscorlib]System.ValueType
{
    .custom instance void UnmanagedAttribute::.ctor() = ( 01 00 00 00 )

    .field private float32 _x
    .field private float32 _y
    .field private float32 _z
    .field private float32 _w

    .method public hidebysig specialname rtspecialname 
        instance void  .ctor(float32 x,
                             float32 y,
                             float32 z,
                             float32 w) cil managed
    {
        .maxstack  8
        nop
        ldarg.0
        ldarg.1
        stfld      float32 Vector4::_x
        ldarg.0
        ldarg.2
        stfld      float32 Vector4::_y
        ldarg.0
        ldarg.3
        stfld      float32 Vector4::_z
        ldarg.0
        ldarg.s    w
        stfld      float32 Vector4::_w
        ret
    }

    .data OneData = {
        float32 (1.0),
        float32 (1.0),
        float32 (1.0),
        float32 (1.0)
    }

    .field public static initonly Vector4 One = at OneData
}

IL Grammar

The IL grammar for a "data declaration" is:

DataDecl ::= [ DataLabel ‘=’ ] DdBody

DdBody ::= DdItem 
         | ‘{’ DdItemList ‘}’

DdItemList ::= DdItem [ ‘,’ DdItemList ]

DdItem ::= ‘&’ ‘(’ Id ‘)’
         | bytearray ‘(’ Bytes ‘)’
         | char ‘*’ ‘(’ QSTRING ‘)’
         | float32 [ ‘(’ Float64 ‘)’ ] [ ‘[’ Int32 ‘]’ ]
         | float64 [ ‘(’ Float64 ‘)’ ] [ ‘[’ Int32 ‘]’ ]
         | int8 [ ‘(’ Int32 ‘)’ ] [‘[’ Int32 ‘]’ ]
         | int16 [ ‘(’ Int32 ‘)’ ] [ ‘[’ Int32 ‘]’ ]
         | int32 [ ‘(’ Int32 ‘)’ ] [‘[’ Int32 ‘]’ ]
         | int64 [ ‘(’ Int64 ‘)’ ] [ ‘[’ Int32 ‘]’ ]

Accessing the Data

Accessing the data is the defined as:

The data stored in a PE File using the .data directive can be accessed through a static variable, either global or a member of a type, declared at a particular position of the data

FieldDecl ::= FieldAttr* Type Id at DataLabel

The data is then accessed by a program as it would access any other static variable, using instructions such as ldsfld, ldsflda, and so on.

The ability to access data from within the PE File can be subject to platform-specific rules, typically related to section access permissions within the PE File format itself.

Additional Details

It was suggested that automatically converting constructors may not be the best and providing some separate syntax for this would be beneficial.

Alternatives

Users can use static-readonly to solve "some" of these problems. However, this incurs a one-time runtime initialization cost (per declaration)

The compiler could provide general use of the CustomConstantAttribute to support a sort of psuedo-constant (this is what it does for DateTime and Decimal).

Using Field Initializers was originally suggested but this may have issues with machines using different endian formats.

Data Declarations

There is also another relevant section Data declarations [II.16.3.1]. Specifically, this would allow users to embed data into an assembly that the runtime could reference (both constant and modifiable data). An interesting scenario where this could be used is UTF8 string-literals.

[fanoI]

This could cover these scenarios too?

BigInteger bint = 420000000000000000000000000000000bi;
Int128 octaword = 12800000000ow;

[tannergooding]

It would not cover custom C# literal expressions, so you would still need to declare:

BigInteger bint = new BigInteger()
Int128 octaword = new Int128()

This just covers being able to emit constants (runtime literals) for blittable structures.

[tannergooding]

Its probably worth noting that if, for some reason, there are memory safety concerns/etc, allowing users to declare byte array constants is likely useful in itself (and users could then use the System.Rutime.CompilerServices.Unsafe class to do their own "magic").

There is also a separate section to these parts that may be useful, which is allowing users to declare their own Data declarations [II.16.3.1]. I'm considering opening a separate issue for this. Data declarations are similar to literals, except that the CLI can/does use it directly (where-as literals are just metadata that the compiler uses).

[tannergooding]

Data declarations (specifically) could also be used for embedding UTF-8 strings

[tannergooding]

@jaredpar, I think you had specifically mentioned that some people were interested in an existing mechanism for embedding UTF-8 strings, could you tag them as appropriate?

[tannergooding]

Tagging @KrzysztofCwalina who may be interested

[jkotas]

    .field public static literal valuetype Vector4 One = bytearray(00 00 80 3F
                                                                   00 00 80 3F
                                                                   00 00 80 3F
                                                                   00 00 80 3F)

This assumes that the machine is little endian. It won't work on bigendian machines.

[tannergooding]

@jkotas, string literals and all other data embedded in the PE file are also stored in little endian format.

I'm sure the distinction is important in some scenarios if/when CoreCLR even properly supports running on big-endian platforms. However, last I checked, we still had multiple algorithms that didn't take big-endian into consideration and had even decided to drop comments indicating that they would need to be updated to support such systems in some cases.

Storing as byte-arrays would never correctly work on big-endian platforms. However, Data Declarations would, provided they list the individual fields (which is supported). The caveat being that referencing the data directly becomes impossible (if you want to work on both big/little endian machines) and that the code becomes non-performant, due to needing to load each field individually (to support byte-swapping where required).

I think for the case of supporting a much requested feature and ensuring it is performant, storing and referencing as little-endian is reasonable. However, I'll leave the ultimate decision up to the language-design team.

[jkotas]

string literals and all other data embedded in the PE file are also stored in little endian format

The runtime understands these and can swap them as necessary. They are not raw data, like in your example.

CoreCLR even properly supports running on big-endian platforms

Mono does support running on big-endian platforms. We have been fixing the code shared with Mono as necessary to be compatible with bigendian, e.g.: dotnet/corefx@c3d802a

[tannergooding]

@jkotas, do you know if the runtime also understands and can swap data declarations?

Grammar

The grammar for them is defined as:

DataDecl ::= [ DataLabel ‘=’ ] DdBody

DdBody ::= DdItem 
         | ‘{’ DdItemList ‘}’

DdItemList ::= DdItem [ ‘,’ DdItemList ]

DdItem ::= ‘&’ ‘(’ Id ‘)’
         | bytearray ‘(’ Bytes ‘)’
         | char ‘*’ ‘(’ QSTRING ‘)’
         | float32 [ ‘(’ Float64 ‘)’ ] [ ‘[’ Int32 ‘]’ ]
         | float64 [ ‘(’ Float64 ‘)’ ] [ ‘[’ Int32 ‘]’ ]
         | int8 [ ‘(’ Int32 ‘)’ ] [‘[’ Int32 ‘]’ ]
         | int16 [ ‘(’ Int32 ‘)’ ] [ ‘[’ Int32 ‘]’ ]
         | int32 [ ‘(’ Int32 ‘)’ ] [‘[’ Int32 ‘]’ ]
         | int64 [ ‘(’ Int64 ‘)’ ] [ ‘[’ Int32 ‘]’ ]

Accessing Data

Accessing the data is then defined as:

The data stored in a PE File using the .data directive can be accessed through a static variable, either global or a member of a type, declared at a particular position of the data

FieldDecl ::= FieldAttr* Type Id at DataLabel

The data is then accessed by a program as it would access any other static variable, using instructions such as ldsfld, ldsflda, and so on.

The ability to access data from within the PE File can be subject to platform-specific rules, typically related to section access permissions within the PE File format itself.

Example

An example given is that you can declare:

.data theInt = int32(123) 

A static variable is then declared for the data:

.field public static int32 myInt at theInt

You can then load the variable:

ldsfld int32 myInt
// data on stack 

Additional Details

One of the interesting scenarios from the grammar is that you can declare structured data:

.data ZeroData = {
    float32 (0.0),
    float32 (0.0),
    float32 (0.0),
    float32 (0.0)
}

// could also declare .data ZeroData = { float32 (0.0) [4] }

The static field accessing the data:

.field public static initonly Vector4 Zero at ZeroData

Loading the data:

ldsfld Vector4 Zero

This enables data which is known at link time to be embedded directly into the PE, incurring no startup cost and allowing it to be referenced in other places which use constants/default parameters (this is assuming that constants are required to be readonly).

[jkotas]

@jkotas, do you know if the runtime also understands and can swap data declarations?

I do not know whether Mono does that. I would expect that it does not do it today.

[michaelcheers]

@tannergooding, I think it shouldn't automatically convert from constructors to const, I think it should be something of automatic constructors.

[tannergooding]

@michaelcheers, I agree that automatically converting constructors probably aren't the best way to support this (this should probably be done 'properly' with constexpr support, which is tracked by a separate issue).

[alrz]

Is it possible to have a blittable struct that cannot be a constant? in other words, do these requirements overlap each other? If they do, I feel like const struct is a perfect choice there. ;)

[tannergooding]

@alrz, I would be worried about that being confused with a struct being immutable (roughly equivalent to the readonly struct feature)

[alrz]

I would be worried about that being confused with a struct being immutable (roughly equivalent to the readonly struct feature)

All I'm saying is that we could take a feature that has a requirement of a type being blittable, so we don't need to bring the bittable concept directly to the language, rather maybe just a paragraph in the spec.

[tannergooding]

Updated the OP to use the term "Unmanaged", rather than blittable.

Also updated the OP to suggest Data Declarations as the primary mechanism, since they appear to work with big or little endian machines (given that they are "structured data", rather than "raw bytes").

[alrz]

I suppose the unmanaged modifier is not required at the declaration site. The compiler can already determine if a type is unmanaged from metadata (including blittable structs). (though, I think it could be helpful to guarantee that a struct always satisfies all the constraints).

That being said, I think when we have this mechanism, we can also optimize array/stackalloc initializers on all non-trivial blittable types as well (regardless if they are marked as unmanaged).

[briandrennan]

@tannergooding, I would like to propose readonly local variables, but perhaps this proposal covers what I'm looking for. Today, it's not possible to write code like this:

public static string GetString() {
    var value = new { GivenName = "John", SurName = "Doe" };
    readonly string SomeValue = $"{value.GivenName} {value.SurName}";
    return SomeValue;
}

I'm okay with not being able to declare the value as const, but I would like for the compiler to raise an error for re-assignment. I'd prefer const, but expect that there are some issues with doing this.

What's the best way to proceed with this request? I didn't see my exact request as an issue already, but this proposal looks like it's along the lines of what I'm looking for personally.

Edit: I know that SomeValue in my example isn't a string - I was just trying to show what my desired _syntax_ was.

[tannergooding]

@briandrennan, I believe #188 tracks that feature.

[briandrennan]

Ah, so it does. Thanks!

[akrsmv]

Hi Guys, sorry if not relevant (not red entire thread) - I was searching for way to define const structs as well, but mainly because I want to tight several constants to a single "unit of meaning". My question is - would it be possible instead of const struct, to have at least const tupples? I.e:

public const (string DICTIONARY_KEY, string i18NValue) s = ("a","b");

[Ultrahead]

Any news about this?

[spixy]

Would unmanaged be braking change to existing .net structs? Can they be moved to unmanaged?

I want to do

const date = new DateTime(42);
...
[TestCase(date, true)]
...