[NativeAOT] Stackoverflow reporting on Linux

[jkotas]

Repro

Recursion(1);

void Recursion(int x)
{
    Recursion(x+1);
    Recursion(x+1);
}

Actual result

Segmentation fault

Expected result

Process is terminating due to StackOverflowException

(Reported by partner team.)

[ghost]

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Repro

Recursion(1);

void Recursion(int x)
{
    Recursion(x+1);
    Recursion(x+1);
}

Actual result

Segmentation fault

Expected result

Process is terminating due to StackOverflowException

(Reported by partner team.)

Author:	jkotas
Assignees:	-
Labels:	area-NativeAOT-coreclr
Milestone:	8.0.0

[jkotas]

The change was reverted by #95415

[jtschuster]

It looks like all the crashes occurred when the SIGSEGV was hit while the GC was trying to suspend all threads. I'm not sure why that was causing crashes with the alternate stack but isn't causing crashes when it uses the regular stack.

[janvorli]

My (wild) guess is that it might be due to libunwind not being able to walk over the SIGSEGV frame when the handler is running on a different stack from the code where the sigsegv occured. In coreclr, we actually don't rely on libunwind over that boundary, we explicitly skip it using a context that we store in the sigsegv handler. See

runtime/src/coreclr/pal/src/exception/seh-unwind.cpp

Lines 669 to 679 in 62dcb12

	// Check if the PC is the return address from the SEHProcessException.
	// If that's the case, extract its local variable containing a pointer to the windows style context of the hardware
	// exception and return that. This skips the hardware signal handler trampoline that the libunwind
	// cannot cross on some systems. On macOS, it skips a similar trampoline we create in HijackFaultingThread.
	if ((void*)curPc == g_SEHProcessExceptionReturnAddress)
	{
	CONTEXT* exceptionContext = *(CONTEXT**)(CONTEXTGetFP(context) + g_hardware_exception_context_locvar_offset);
	memcpy_s(context, sizeof(CONTEXT), exceptionContext, sizeof(CONTEXT));
	return TRUE;
	}