Make GC stack walking robust in the presence of unmanaged byrefs with extended lifetime

[jkotas]

Consider this example:

byte* ptr = Marshal.AllocHGlobal(...);
ref byte b = ref Unsafe.AsRef<byte>(ptr);
Use(ref b);
Marshal.FreeHGlobal(ptr);

// `b` never used again in the code. Let's assume JIT decided to extend the `b` variable lifetime till end of the method.

object o = new object(); // Let's assume that the GC run out of space, allocated a new memory block using virtual alloc, and the OS memory manager decides to give it the memory block that was just freed by `Marshal.FreeHGlobal`.

GC.Collect(); // The GC may assert or crash because of it will see `byref b` pointing into middle of the segment where no valid byrefs are supposed to point to

Context: #75857 (comment)

[ghost]

Tagging subscribers to this area: @dotnet/gc
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Consider this example:

byte* ptr = Marshal.AllocHGlobal(...);
ref byte b = ref Unsafe.AsRef<byte>(ptr);
Use(ref b);
Marshal.FreeHGlobal(ptr);

// `b` never used again in the code. Let's assume JIT decided to extend the `b` variable lifetime till end of the method.

object o = new object(); // Let's assume that the GC run out of space, allocated a new memory block using virtual alloc, and the OS memory manager decides to give it the memory block that was just freed by `Marshal.FreeHGlobal`.

GC.Collect(); // The GC may assert or crash because of it will see `byref b` pointing into middle of the segment where no valid byrefs are supposed to point to

Context: #75857 (comment)

Author:	jkotas
Assignees:	-
Labels:	area-GC-coreclr
Milestone:	-

[jkotas]

@davidwrighton Do you have thoughts about this scenario? It is related to the clarification made in #71794.

We may want to make GCConfig::GetConservativeGC to be on by default to fix part of this problem.

[Maoni0]

with regions you don't have this problem since we reserve a big range for regions GC will be acquiring from so you'll never had such overlap.

[ghost]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Consider this example:

byte* ptr = Marshal.AllocHGlobal(...);
ref byte b = ref Unsafe.AsRef<byte>(ptr);
Use(ref b);
Marshal.FreeHGlobal(ptr);

// `b` never used again in the code. Let's assume JIT decided to extend the `b` variable lifetime till end of the method.

object o = new object(); // Let's assume that the GC run out of space, allocated a new memory block using virtual alloc, and the OS memory manager decides to give it the memory block that was just freed by `Marshal.FreeHGlobal`.

GC.Collect(); // The GC may assert or crash because of it will see `byref b` pointing into middle of the segment where no valid byrefs are supposed to point to

Context: #75857 (comment)

Author:	jkotas
Assignees:	-
Labels:	area-CodeGen-coreclr, area-GC-coreclr, untriaged
Milestone:	-

[jkotas]

Are you saying that you never ever call virtual alloc to reserve more memory with regions? Or that it is unlikely to run into this with regions?

Also, we still have 32-bit platforms where regions are not enabled.

I do not think codegen is the right area label for this. There is nothing that codegen team can do about this. If there is a change to do for this, it would need to be in the GC.

[ghost]

Tagging subscribers to this area: @dotnet/gc
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Consider this example:

byte* ptr = Marshal.AllocHGlobal(...);
ref byte b = ref Unsafe.AsRef<byte>(ptr);
Use(ref b);
Marshal.FreeHGlobal(ptr);

// `b` never used again in the code. Let's assume JIT decided to extend the `b` variable lifetime till end of the method.

object o = new object(); // Let's assume that the GC run out of space, allocated a new memory block using virtual alloc, and the OS memory manager decides to give it the memory block that was just freed by `Marshal.FreeHGlobal`.

GC.Collect(); // The GC may assert or crash because of it will see `byref b` pointing into middle of the segment where no valid byrefs are supposed to point to

Context: #75857 (comment)

Author:	jkotas
Assignees:	-
Labels:	area-CodeGen-coreclr, area-GC-coreclr, untriaged
Milestone:	-

[Maoni0]

I'm saying with regions that'll never happen.

yes there'll still be 32-bit and segments will continue to be used there which makes it much lower priority.