Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Collection expressions: use inline array for ReadOnlySpan<T> argument to builder method #69227

Merged
merged 14 commits into from
Aug 1, 2023

Conversation

cston
Copy link
Member

@cston cston commented Jul 26, 2023

Use inline array type instances as the storage for spans passed to collection expression builder methods. The inline array types are synthesized by the compiler and emitted as internal types in the assembly.

Relates to test plan #66418

@dotnet-issue-labeler dotnet-issue-labeler bot added Area-Compilers untriaged Issues and PRs which have not yet been triaged by a lead labels Jul 26, 2023
@cston cston force-pushed the collections-ia-arg branch 4 times, most recently from f975cc1 to 4176dcd Compare July 27, 2023 21:56
@cston cston marked this pull request as ready for review July 27, 2023 22:12
@cston cston requested a review from a team as a code owner July 27, 2023 22:12
@CyrusNajmabadi
Copy link
Member

CyrusNajmabadi commented Jul 27, 2023

Will review tomorrow!


internal override FileIdentifier? AssociatedFileIdentifier => null;

internal override bool MangleName => false; // PROTOTYPE: Is this correct?
Copy link
Member

@jcouv jcouv Jul 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// PROTOTYPE: Is this correct?

PROTOTYPE comments are not allowed in main branch #Closed

var sideEffects = ArrayBuilder<BoundExpression>.GetInstance();
BoundExpression span;

// PROTOTYPE: Check also ReadOnlySpan<T> parameter is not [UnscopedRef].
Copy link
Member

@jcouv jcouv Jul 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PROTOTYPE comment (also elsewhere) #Closed

@RikkiGibson RikkiGibson self-assigned this Jul 27, 2023

string typeName = PrivateImplementationDetails.GetInlineArrayTypeName(arrayLength);
var privateImplClass = GetPrivateImplClass(syntaxNode, diagnostics);
var typeAdapter = privateImplClass.GetType(typeName);
Copy link
Member

@jcouv jcouv Jul 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GetType

nit: consider calling this TryGetType since it can fail to return a type. Then maybe consider using a TryGet API pattern (return bool, out parameter). #Closed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went with GetType() to match the existing GetMethod() method used in the other Ensure methods here.


var typeSymbol = new SynthesizedInlineArrayTypeSymbol(SourceModule, typeName, arrayLength, attributeConstructor);
privateImplClass.TryAddSynthesizedType(typeSymbol.GetCciAdapter());
typeAdapter = privateImplClass.GetType(typeName)!;
Copy link
Member

@jcouv jcouv Jul 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

!

Do we need this suppression? #Closed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're dereferencing typeAdapter below.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: consider doing Debug.Assert(typeAdapter!.Name == typeName); instead. That assertion should also ensure the nullable state of typeAdapter is not-null.

internal static string GetInlineArrayTypeName(int arrayLength)
{
Debug.Assert(arrayLength > 0);
return $"$InlineArray{arrayLength}";
Copy link
Member

@jcouv jcouv Jul 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: the other nested types we add in private implementation details have a different naming convention. Consider aligning. Here's one that was recently added:

        public string Name => _alignment == 1 ?
            $"__StaticArrayInitTypeSize={_size}" :
            $"__StaticArrayInitTypeSize={_size}_Align={_alignment}";
``` #Closed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to __InlineArrayN<T> but perhaps __InlineArrayTypeSize=N<T> might be better, or at least more consistent. Feel free to suggest specific names.


namespace Microsoft.CodeAnalysis.CSharp.Symbols
{
internal sealed class SynthesizedInlineArrayTypeSymbol : NamedTypeSymbol
Copy link
Member

@jcouv jcouv Jul 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SynthesizedInlineArrayTypeSymbol

nit: doc? #Closed


public override bool IsReadOnly => true;

public override Symbol? ContainingSymbol => null;
Copy link
Member

@jcouv jcouv Jul 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is ContainingSymbol null? #Closed


internal override ObsoleteAttributeData? ObsoleteAttributeData => null;

public override ImmutableArray<Symbol> GetMembers() => ImmutableArray<Symbol>.Empty;
Copy link
Member

@jcouv jcouv Jul 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we return the field here? #Closed


public override ImmutableArray<Symbol> GetMembers() => ImmutableArray<Symbol>.Empty;

public override ImmutableArray<Symbol> GetMembers(string name) => GetMembers().WhereAsArray(m => m.Name == name);
Copy link
Member

@jcouv jcouv Jul 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be specialized to just return the name of the field #Closed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current implementation seems more robust, and WhereAsArray() should avoid any allocations since the result should be either the entire array or an empty array.


public override ImmutableArray<Location> Locations => ImmutableArray<Location>.Empty;

public override ImmutableArray<SyntaxReference> DeclaringSyntaxReferences => ImmutableArray<SyntaxReference>.Empty;
Copy link
Member

@jcouv jcouv Jul 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Why return a result here, but throw on SynthesizedInlineArrayTypeSymbol.DeclaringSyntaxReferences? #Closed

BoundExpression span;

// PROTOTYPE: Check also ReadOnlySpan<T> parameter is not [UnscopedRef].
if (elements.Length > 0
Copy link
Member

@jcouv jcouv Jul 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should there be any kind of upper limit/heuristic? #Closed

Copy link
Member Author

@cston cston Jul 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now, we don't think there should be a limit (see notes), because the elements are already explicit in the source and because the work around to use the heap is to use C# 11 syntax.

Debug.Assert(inlineArrayType.HasInlineArrayAttribute(out int inlineArrayLength) && inlineArrayLength == arrayLength);

var intType = _factory.SpecialType(SpecialType.System_Int32);
var elementRef = _factory.ModuleBuilderOpt.EnsureInlineArrayElementRefExists(syntax, intType, _diagnostics.DiagnosticBag);
Copy link
Member

@jcouv jcouv Jul 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

elementRef

What's the type of this? #Closed


var intType = _factory.SpecialType(SpecialType.System_Int32);
var elementRef = _factory.ModuleBuilderOpt.EnsureInlineArrayElementRefExists(syntax, intType, _diagnostics.DiagnosticBag);
elementRef = elementRef.Construct(ImmutableArray.Create(TypeWithAnnotations.Create(inlineArrayType), elementType));
Copy link
Member

@jcouv jcouv Jul 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

elementRef =

nit: consider doing a chain call to .Construct as couple of lines above #Closed

for (int i = 0; i < arrayLength; i++)
{
var element = VisitExpression(elements[i]);
var call = _factory.Call(null, elementRef, inlineArrayLocal, _factory.Literal(i), useStrictArgumentRefKinds: true);
Copy link
Member

@jcouv jcouv Jul 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add pseudo-code comments corresponding to what is generated #Closed

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding. It makes things much easier to follow :-)

var inlineArrayAsSpan = _factory.ModuleBuilderOpt.EnsureInlineArrayAsSpanExists(syntax, spanType, intType, _diagnostics.DiagnosticBag);
inlineArrayAsSpan = inlineArrayAsSpan.Construct(ImmutableArray.Create(TypeWithAnnotations.Create(inlineArrayType), elementType));

spanType = spanType.Construct(ImmutableArray.Create(elementType));
Copy link
Member

@jcouv jcouv Jul 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was already constructed by EnsureInlineArrayAsSpanExists/SynthesizedInlineArrayAsSpanMethod. Consider grabbing the return type from there. #Closed

@cston
Copy link
Member Author

cston commented Jul 31, 2023

    public void CollectionBuilder_MissingSpanMembers(bool useCompilationReference)

There is nothing to test here after switching to InlineArrayAsReadOnlySpan() since the Span<T> members are no longer referenced. (And if we test missing ReadOnlySpan<T> members instead, then binding will report an error that the collection cannot be constructed because the builder method uses ReadOnlySpan<T> as the parameter type.)


Refers to: src/Compilers/CSharp/Test/Emit2/Semantics/CollectionExpressionTests.cs:9263 in 4cfe4e1. [](commit_id = 4cfe4e1, deletion_comment = True)

// synthesized inline array types
private readonly ConcurrentDictionary<string, Cci.INamedTypeDefinition> _synthesizedInlineArrayTypes =
new ConcurrentDictionary<string, Cci.INamedTypeDefinition>();

// field types for different block sizes.
Copy link
Member

@jcouv jcouv Jul 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment seems out-of-date #Closed

Construct(ImmutableArray.Create(TypeWithAnnotations.Create(inlineArrayType), elementType));

// Create an inline array and assign to a local.
// var tmp = new __InlineArrayN<T>();
Copy link
Member

@jcouv jcouv Jul 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

T

nit: ElementType instead of T (also below)? #Closed


public override bool IsReadOnly => true;

public override Symbol? ContainingSymbol => _containingModule.GlobalNamespace;
Copy link
Member

@jcouv jcouv Jul 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the right containing symbol? I thought this is a nested type inside the private implementation type (based on _orderedNestedTypes = orderedProxyTypes.Concat(orderedInlineArrayTypes)) #Closed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The types are actually top-level types even though they are visited in emit through PrivateImplementationDetails.GetNestedTypes().

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's test that they can be referenced in any way in source (when loading from metadata), not merely inaccessible.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From offline discussion, let's leave a comment here if the symbol cannot reference the proper containing symbol. Let's try to emit those types are nested types in the private implementation.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed inline array types to be top-level types with names that cannot be used in source.

@jcouv
Copy link
Member

jcouv commented Jul 31, 2023

        var verifier = CompileAndVerify(new[] { source, s_collectionExtensionsWithSpan }, targetFramework: TargetFramework.Net70, verify: Verification.Skipped, expectedOutput: IncludeExpectedOutput("[], [1, 2], [3, 4, 5], [null, 7], "));

nit: consider breaking line (also applies below)


In reply to: 1658987553


Refers to: src/Compilers/CSharp/Test/Emit2/Semantics/CollectionExpressionTests.cs:2691 in 75ead79. [](commit_id = 75ead79, deletion_comment = False)

if (elements.Length > 0
&& !elements.Any(i => i is BoundCollectionExpressionSpreadElement)
&& _compilation.Assembly.RuntimeSupportsInlineArrayTypes
&& (!constructMethod.ReturnType.IsRefLikeType || constructMethod.Parameters[0].EffectiveScope == ScopedKind.ScopedValue))
Copy link
Member

@jcouv jcouv Jul 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did we add tests to cover those conditions, or were those getting hit by existing scenarios? #Closed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Covered by CollectionBuilder_RefStructCollection.

var inlineArrayType = _factory.ModuleBuilderOpt.EnsureInlineArrayTypeExists(syntax, _factory, arrayLength, _diagnostics.DiagnosticBag).Construct(ImmutableArray.Create(elementType));
Debug.Assert(inlineArrayType.HasInlineArrayAttribute(out int inlineArrayLength) && inlineArrayLength == arrayLength);

var intType = _factory.SpecialType(SpecialType.System_Int32);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have a test where type System.Int32 is missing?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. I believe we're handling that correctly here, and I don't think we'd get this far without an error in that case.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have tests for various other missing types, but not this one. Let's cover it too.

var privateImplClass = GetPrivateImplClass(syntaxNode, diagnostics);
var typeAdapter = privateImplClass.GetType(typeName);

if (typeAdapter is null)
Copy link
Member

@jcouv jcouv Jul 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have a test with two collection expressions of the same size? (the InlineArray type will be re-used/shared) #Closed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CollectionBuilder_InlineArrayTypes

Copy link
Member

@jcouv jcouv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done with review (iteration 7)

@@ -320,6 +333,18 @@ public override IEnumerable<Cci.IMethodDefinition> GetMethods(EmitContext contex
return method;
}

internal Cci.INamespaceTypeDefinition? GetType(string name)
Copy link
Member

@jcouv jcouv Aug 1, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GetType

nit: naming this GetSynthesizedInlineArrayType would make more sense. GetType is confusing/misleading. Same comment for TryAddSynthesizedType. Alternatively, we can rename _synthesizedInlineArrayTypes to something less specific like _additionalTopLevelTypes. #Closed

Debug.Assert(Compilation.Assembly.RuntimeSupportsInlineArrayTypes);
Debug.Assert(arrayLength > 0);

string typeName = $"<>y__InlineArray{arrayLength}"; // see GeneratedNameKind.InlineArrayType
Copy link
Member

@jcouv jcouv Aug 1, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GeneratedNameKind.InlineArrayType

Consider $"<>{GeneratedNameKind.InlineArrayType}__InlineArray{arrayLength}" or adding an assertion Debug.Assert("y" == GeneratedNameKind.InlineArrayType)

That way, GoToDefinition on GeneratedNameKind.InlineArrayType leads somewhere useful and that we don't accidentally modify it.
#Closed

Copy link
Member

@jcouv jcouv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done with review pass (iteration 12)

Copy link
Member

@jcouv jcouv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM Thanks (iteration 13)

Copy link
Member

@jcouv jcouv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM Thanks (iteration 14)

@cston cston merged commit 7183a8a into dotnet:main Aug 1, 2023
@cston cston deleted the collections-ia-arg branch August 1, 2023 20:58
@ghost ghost added this to the Next milestone Aug 1, 2023
@dibarbet dibarbet modified the milestones: Next, 17.8 P2 Aug 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area-Compilers untriaged Issues and PRs which have not yet been triaged by a lead
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants