[API Proposal]: Add CollectionsMarshal.AsMemory(List<T>)

[Neme12]

Background and motivation

I previously proposed this in #90141 and it was rejected for the reason that it would prevent List<T>'s backing store from ever being changed from a T[] to some new type of array, like some sort of resizable array. Or rather, it wouldn't prevent such a change, but it would require the returned Memory<T> to now be backed by a MemoryManager, which is an additional allocation.

@stephentoub said:

While AsSpan(List) baked in the notion of List being backed by contiguous memory, AsMemory(List) would effectively bake in the notion of it being backed by a T[]. That's something we'd like to avoid doing right now. (Such an AsMemory could return a Memory backed by a MemoryManager, but that would then either require that AsMemory to start allocating or it would require the list to grow to be able to cache the MemoryManager reference, neither of which we'd want.)

However, after thinking about this some more, I don't think this is the case. Unfortunately, I cannot add a comment to the original proposal, so I'm opening a new issue.

Memory<T> can be backed by a GC object that stores the data inline in the object - currently (afaik), that's only T[] and string. But if there is ever a new type of array added to the runtime, like a resizable array, then Memory<T> could just as well be backed by this new type of array - it would store a reference to this new array-like object. Or even if List<T> is ever specialized by the runtime and changed to store the data inline in the object, like string does, as opposed to going through an indirection, then the Memory<T> could store the reference to the List<T> itself. That's why I don't think returning Memory<T> would bake in the notion of it being backed by a T[]. It would only bake in the notion of it being backed by a GC object. Such a GC object could very well be a new kind of array, or even the list itself.

Am I wrong in this conclusion?

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-collections
See info in area-owners.md if you want to be subscribed.

[stephentoub]

But if there is ever a new type of array added to the runtime, like a resizable array, then Memory could just as well be backed by this new type of array - it would store a reference to this new array-like object.

That's not pay for play. It would make existing use of members like memory.Span more expensive.

I don't think anything meaningful has changed since the last time this was discussed and rejected.

[huoyaoyuan]

#87210

[Neme12]

That's not pay for play. It would make existing use of members like memory.Span more expensive.

How so? In this snippet, ObjectHasComponentSize would return true for a hypothetical new kind of array. And this new kind of array could store Length in the same location in memory as a regular array, so getting it could use the same code.

runtime/src/libraries/System.Private.CoreLib/src/System/Memory.cs

Lines 293 to 309 in 71abc26

	else if (RuntimeHelpers.ObjectHasComponentSize(tmpObject))
	{
	// We know the object is not null, it's not a string, and it is variable-length. The only
	// remaining option is for it to be a T[] (or a U[] which is blittable to T[], like int[]
	// and uint[]). As a special case of this, ROM<T> allows some amount of array variance
	// that Memory<T> disallows. For example, an array of actual type string[] cannot be turned
	// into a Memory<object> or a Span<object>, but it can be turned into a ROM/ROS<object>.
	// We'll assume these checks succeeded because they're performed during Memory<T> construction.
	// It's always possible for somebody to use private reflection to bypass these checks, but
	// preventing type safety violations due to misuse of reflection is out of scope of this logic.
	// 'tmpObject is T[]' below also handles things like int[] <-> uint[] being convertible
	Debug.Assert(tmpObject is T[]);
	refToReturn = ref MemoryMarshal.GetArrayDataReference(Unsafe.As<T[]>(tmpObject));
	lengthOfUnderlyingSpan = Unsafe.As<T[]>(tmpObject).Length;
	}

I think it would be a real shame if some new kind of resizable array was added, and it had no AsMemory() method. These days, I would personally consider that a requirement for any kind of new array type, along with AsSpan().

[Neme12]

But okay, it seems there is no interest in this. I'll just keep using my own implementation of this. Thanks for entertaining the idea.

[theodorzoulias]

Unfortunately, I cannot add a comment to the original proposal, so I'm opening a new issue.

I don't think that the goddess Fortuna has anything to do with this. Locking GitHub issues after a month of inactivity is a deliberate policy implemented by Microsoft. It didn't just occurred by accident. The reason that this policy exists continues to baffle me. As far as I can think, it only serves at depriving Microsoft from feedback about their products, from the knowledgeable engineers who use them.

[Neme12]

@theodorzoulias It's only here as far as I can tell, not Microsoft in general. I had been contributing to Roslyn quite a bit in the past, and they never had any policy like this (and they still do not). I was fixing issues no matter how old.

[danmoseley]

@theodorzoulias

As far as I can think, it only serves at depriving Microsoft from feedback about their products, from the knowledgeable engineers who use them.

I believe the bot locks closed issues and PRs after a month. Whereas open issues are only closed after 4-5 years of inactivity, and even then only after a notice saying that any comment will restart the clock for another several years.

Locking closed issues is not done to avoid community comments. It is to direct them to open issues so that we actually see the comments. Before this we would often get comments on closed issues which we would not see, as practically we only monitor open issues. Often times those comments should have gone in a new issue anyway - for example the original problem was fixed, and they are experiencing the same symptoms for a different reason. I don't think this policy of locking closed issues is unique to this repo.

If it really makes sense to reopen an issue rather than start a new one, what I suggest is asking for a maintainers to do that using the Discord channel or by creating a discussion topic here or possibly asking in a related issue.

Does that help? This is not about suppressing comment.