Implement MVP part of const generics for CoreCLR

[hez2010]

Const Generics

"Const Generics" stands for allowing constant value to be used in a type parameter.

This PR contains an MVP (Minimum Viable Product) part of the implementation to enable const generics for CoreCLR, and it only contains changes for the native part (type loader, JIT and etc.), with no meaningful changes to the managed part.

Contributes to #89730.

Design and Implementation

Wording

• Const type parameter: a type parameter that carries a const value.
• Const type argument: the constant value for a type parameter in the instantiation.

Const Type Parameter

To support const generics, we need a way to declare a const type parameter that will carry the const value after instantiation.
Due to the fact that a const type parameter behaves no difference than a normal type parameter until instantiation, here we treat the type of a const type parameter as a special generic constraint.

We want to emit the type of a const type parameter as TypeSpec, but in order to distinguish this type token from other generic constraints, we introduced a mdtGenericParamType and then emit the type of const type parameter with mdtGenericParamType, and make sure it will always be the first entry in generic constraints.

To load the type of a type parameter, we simply look up the first entry in generic constraints and see if it's mdtGenericParamType. If yes, then replace it with mdtTypeSpec using (token & ~mdtGenericParamType) | mdtTypeSpec. When loading generic constraints, if we see a generic constraint has type mdtGenericParamType, we can skip it directly.

Const Type Argument

A const type argument contains the actual constant value in the instantiation.
Here we introduced a new element type ELEMENT_TYPE_CTARG which stands for const type argument.

A const type argument is encoded as follows:

ELEMENT_TYPE_CTARG <element type of const value> <const value>

Note that the size of const value is determined by its element type.
For example, an int 42 will be encoded as:

 ELEMENT_TYPE_CTARG ELEMENT_TYPE_I4     42
|      1 byte      |     1 byte    | 4 bytes |

While a double 3.1415926 will be encoded as:

 ELEMENT_TYPE_CTARG ELEMENT_TYPE_R8 3.1415926
|      1 byte      |     1 byte    | 8 bytes |

IL Parser

Reused the keyword literal in IL to indicate the type argument contains a const value. Particularly, we use the keyword literal to differentiate a const type argument from a type argument. For example, literal int32 T.

For a const type argument, we simply use int32 (42) to express an int constant with the value 42.

This is following how we represent a const field in IL today.

We changed the parser to parse "literal" type typeName as a const type parameter, and type '(' value ')' as a const type argument. See changes in asmparser.y for details.

Type Desc

A const type parameter has no more difference than the additional type token, so we reuse the TypeVarTypeDesc and add a field m_type to save the type of const type if it's a const type parameter.

A const type argument is exactly a constant value, so we need a separate TypeDesc for it.
Therefore, a ConstValueTypeDesc has been added to save the type and the value of a const type argument.

We support up to 8 bytes of constant value, so we use uint64_t as the storage.

class ConstValueTypeDesc : TypeDesc {
    TypeHandle m_type;
    uint64_t m_value;
};

To read the constant value from a ConstValueTypeDesc, we need to reinterpret the storage based on the type of constant value. For example, while reading a constant value which is a float, we can simply use *(float*)&m_value.

Method Table

Similar to function pointers, we don't need a MethodTable for const value.

Type Loader

We always load constant values in the CoreLib module because a constant value is independent from the assembly, a same constant value can be served from any assembly.
To avoid loading the same constant value other than once, once we load a constant value, we will save it into a hash table m_pAvailableParamTypes.
Whenever we load a constant value, we first lookup in the hash table, if found then we load the TypeHandle from the hash table directly, otherwise we allocate a new ConstValueTypeDesc for it.

Value Loading

We may need to use the const value from a type parameter, here we reuse the ldtoken instruction to achieve this.
Instead of loading the TypeHandle of the type parameter, we load the constant value and push it to the stack directly when we see the type parameter is a const type parameter.

JIT

We only need to handle ldtoken here, so we changed the impResolveToken to resolve the information about the const value as well, and then use the information to determine whether we should load a type handle or a const value to the stack.

Generic Sharing

We don't share the implementation among const generic type parameters. Each const type argument get specialized so we can always import const type argument as a real type-rich constant value anytime.

Generic on Const Generic Type Parameter

Added support for generic on const generic type parameter.

For example,

.class public auto ansi beforefieldinit Test`2<T, literal !T N>
       extends [System.Runtime]System.Object
{
  .method public hidebysig specialname rtspecialname 
        instance void .ctor () cil managed 
    {
        .maxstack 8

        ldarg.0
        call instance void [System.Runtime]System.Object::.ctor()
        ret
    }

  .method public hidebysig newslot virtual 
        instance void M<U, literal !!U V> () cil managed
    {
        .maxstack 8

        ldstr "Test`2::M"
        call void [System.Console]System.Console::WriteLine(string)
        ldtoken !1
        box !T
        call void [System.Console]System.Console::WriteLine(object)
        ldtoken !!1
        box !!U
        call void [System.Console]System.Console::WriteLine(object)
        ret
    }
}

Type Validation

We validate type during checking the generic constraints.
When we meet a const value, we simply check whether the const value type is equivalent to the type saved in generic param props.

Examples

A basic example

.assembly _ {}

.class public auto ansi beforefieldinit Foo`2<T, literal int32 N>
       extends [System.Runtime]System.Object
{
  .method public hidebysig specialname rtspecialname 
        instance void .ctor () cil managed 
    {
        .maxstack 8

        ldarg.0
        call instance void [System.Runtime]System.Object::.ctor()
        ret
    }

  .method public hidebysig newslot virtual 
        instance void M<literal int32 V, literal int32 W> () cil managed 
    {
        .maxstack 1
        .locals init (
            [0] int32 v
        )


        newobj instance void class Foo`2<string, int32 (42)>::.ctor()
        call instance void class Foo`2<string, int32 (42)>::M<!!V, !!V>()
        newobj instance void class Foo`2<string, !!V>::.ctor()
        call instance void class Foo`2<string, !!V>::M<!N, int32 (42)>()
        newobj instance void class Foo`2<string, !N>::.ctor()
        call instance void class Foo`2<string, !N>::M<!!V, !!W>()

        ldtoken !!V
        call void [System.Console]System.Console::WriteLine(int32)
        ldtoken !!W
        call void [System.Console]System.Console::WriteLine(int32)
        ldtoken !N
        call void [System.Console]System.Console::WriteLine(int32)
        ret
    }
}

This can be interpreted to the following dummy C# code:

class Foo<T, int N>
{
    public void M<int V, int W>()
    {
        new Foo<string, 42>().M<V, V>();
        new Foo<string, V>().M<N, 42>();
        new Foo<string, N>().M<V, W>();
        Console.WriteLine(V);
        Console.WriteLine(W);
        Console.WriteLine(N);
    }
}

Generic Virtual Method with Const Type Parameters

.assembly _ {}

.class private auto ansi beforefieldinit Program
    extends [System.Runtime]System.Object
{
    .method private hidebysig static 
        void Main (
            string[] args
        ) cil managed 
    {

        .maxstack 8
        .entrypoint

        newobj instance void class Bar`2<string, int32( 42 )>::.ctor()
        call instance void class Bar`2<string, int32( 42 )>::N<int32( 42 ), int32( 42 )>()

        ret
    }

    .method public hidebysig specialname rtspecialname 
        instance void .ctor () cil managed 
    {
        .maxstack 8

        ldarg.0
        call instance void [System.Runtime]System.Object::.ctor()
        ret
    }

}

.class public auto ansi beforefieldinit Foo`2<T, literal int32 N>
       extends [System.Runtime]System.Object
{
  .method public hidebysig specialname rtspecialname 
        instance void .ctor () cil managed 
    {
        .maxstack 8

        ldarg.0
        call instance void [System.Runtime]System.Object::.ctor()
        ret
    }

  .method public hidebysig newslot virtual 
        instance void M<literal int32 V, literal int32 W> () cil managed 
    {
        .maxstack 8

        ldstr "From Foo::M"
        call void [System.Console]System.Console::WriteLine(string)
        ldtoken !!V
        call void [System.Console]System.Console::WriteLine(int32)
        ldtoken !!W
        call void [System.Console]System.Console::WriteLine(int32)
        ldtoken !N
        call void [System.Console]System.Console::WriteLine(int32)
        ret
    }

  .method public hidebysig newslot virtual 
        instance void N<literal int32 V, literal int32 W> () cil managed 
    {
        .maxstack 8

        newobj instance void class Foo`2<string, int32( 42 )>::.ctor()
        call instance void class Foo`2<string, int32 (42)>::M<!!V, !!V>()
        newobj instance void class Foo`2<string, !!V>::.ctor()
        call instance void class Foo`2<string, !!V>::M<!N, int32 (42)>()
        newobj instance void class Foo`2<string, !N>::.ctor()
        call instance void class Foo`2<string, !N>::M<!!V, !!W>()
        ret
    }
}


.class public auto ansi beforefieldinit Bar`2<T, literal int32 N>
       extends class Foo`2<!T, int32 (128)>
{
  .method public hidebysig specialname rtspecialname 
        instance void .ctor () cil managed 
    {
        .maxstack 8

        ldarg.0
        call instance void class Foo`2<!T, int32 (128)>::.ctor()
        ret
    }

  .method public hidebysig virtual 
        instance void M<literal int32 V, literal int32 W> () cil managed 
    {
        .maxstack 8
        .locals init (
            [0] string v
        )
        ldstr "From Bar::M"
        call void [System.Console]System.Console::WriteLine(string)
        ldtoken !!V
        call void [System.Console]System.Console::WriteLine(int32)
        ldtoken !!W
        call void [System.Console]System.Console::WriteLine(int32)
        ldtoken !N
        call void [System.Console]System.Console::WriteLine(int32)

        ret
    }

  .method public hidebysig virtual 
        instance void N<literal int32 V, literal int32 W> () cil managed 
    {
        .maxstack 8

        ldarg.0
        call instance void class Foo`2<!T, int32 (128)>::M<!!V, !!W>()

        ldarg.0
        callvirt instance void class Foo`2<!T, !N>::M<!!V, !!W>()

        ret
    }
}

This will yield the below execution result:

From Foo::M
42
42
128
From Bar::M
42
42
42

List of Not-Yet-Implemented

• Constant string and bytearray support
• Const Arithmetic
• Generic constraints
• Managed Type System, crossgen2 (ReadyToRun) and ilc (NativeAOT) support
• Reflection and Type APIs
• A built-in type ValueArray<T, int Length>

While I have another branch and have all those managed stuff implemented, so if you are looking for a more complete implementation, please refer to the "Prototype" section in #89730 to get the SDK I built and then you are ready to use const generics in C#.

[hez2010]

I discarded the changes on GitHub by accident. I will reopen the PR after I restore the changes.

[hez2010]

I pushed a commit to reflect the latest design. Now we no longer have any breaking changes to the existing v2.0 metadata.

[sgf]

This feature is very important and can realize many functions that cannot be realized at the current C# language level. I hope it will be taken seriously as soon as possible.

[tannergooding]

I hope it will be taken seriously as soon as possible.

Const generics would be nice, but they are not strictly more or strictly less important than other highly requested features such as:

• ref structs implementing interfaces
• using ref structs or pointers in generic types
• bridging generic constraints
• discriminated unions
• extension everything
• user-defined constants for unmanaged types
• constant expressions
• associated and/or higher kinded types
• --insert the other 100+ features people have been asking for years here--
• etc

There are many features that many different people view as "very important". A lot of the same features, another significant portion of the community might view as "incredibly unimportant". The actual impact a feature will have, ease of implementation, and general targeted themes for a given release will impact when (and if) each happens.

[sgf]

I hope it will be taken seriously as soon as possible.

Const generics would be nice, but they are not strictly more or strictly less important than other highly requested features such as:

• ref structs implementing interfaces
• using ref structs or pointers in generic types
• bridging generic constraints
• discriminated unions
• extension everything
• user-defined constants for unmanaged types
• constant expressions
• associated and/or higher kinded types
• --insert the other 100+ features people have been asking for years here--
• etc

There are many features that many different people view as "very important". A lot of the same features, another significant portion of the community might view as "incredibly unimportant". The actual impact a feature will have, ease of implementation, and general targeted themes for a given release will impact when (and if) each happens.

If this feature is implemented, many pending features will be easily implemented.
For example:
direct implement: custom struct fixed size buffer
helpfull for implementing bit field struct

Although the features you listed are very important,
But I don't think they can be implemented quickly.

[tannergooding]

If this feature is implemented, many pending features will be easily implemented.

That is making assumptions about the design of the feature which haven't been properly vetted, documented, or tracked.

But I don't think they can be implemented quickly.

Nor can this one. While hez2010 has done the work of implementing a proof of concept, or minimum viable product, that is actually a minority of the overall work required.

Almost any feature of this level requires significant design work to ensure that it is handling all or at least most of the scenarios that users are expecting it to handle. To ensure that it is considering both forwards and backwards compatibility. To ensure that it correctly integrates and doesn't hit bugs or edge cases with other language features. To ensure that it doesn't block or hinder future language directions desired. To ensure that existing code can correctly migrate, consume, or otherwise integrate with the new feature. To ensure that the relevant tests covering edge cases are handled. To ensure that tooling is fully supported. To ensure things like IntelliSense work end to end and within the performance goals required by IDEs Roslyn is shipped with.

Several of the features listed above have actually had some 1-2 week hackathon done on them, and have had something similar to this PR achieved. They haven't been completed yet because that is just the tip of the iceberg in terms of actual implementation cost. These features aren't trivial, even the smallest that end up looking like 1 line of code often have significant cost behind them.

[ghost]

Draft Pull Request was automatically closed for 30 days of inactivity. Please let us know if you'd like to reopen it.