Skip to content

Commit

Permalink
[mono] Implement synch block fast paths in managed (#81380)
Browse files Browse the repository at this point in the history
* [mono] Implement synch block fast paths in managed

   Investigate one incremental alternative from #48058

   Rather than switching to NativeAOT's Monitor implementation in one step, first get access to Mono's synchronization block from managed, and implement the fast paths for hash code and monitor code in C#.

* Implement TryEnterFast and TryEnterInflatedFast

   This should help the interpreter particularly, since it doesn't have an intrinsic path here

* Add replacements for test_owner and test_synchronized icalls

   ObjectHeader.IsEntered and ObjectHeader.HasOwner, respectively

* access sync block through helpers

* Get a pointer to the object header another way

   Unsafe.As<TFrom,TTo>() was converting a object** into a header*.

   Go through Unsafe.AsPointer to get a Header** and then deref it once to get a header

* logical shifts, not arithmetic

   on wasm32, IntPtr and int are both the same size. so when we store a hash code into _lock_word that ends up with the 30th high bit set, the right shift to get it back out needs to be arithmetic or else we get incorrect hash codes

* force inlining

* move fast path into ReliableEnterTimeout

* Don't pass lockTaken to ObjectHolder.TryEnterFast

   set it in the caller if we succeeded

* TryEnterFast: handle flat recursive locking, too

* Implement Monitor.Exit fast path in managed

   If the LockWord is flat, there has been no contention, so there is noone to signal when we exit

* Don't call MonoResolveUnmanagedDll for the default ALC

   it would always return null anyway.

   But also this avoids a circular dependency between the managed Default ALC constructor and Monitor icalls (the ALC base constructor locks allContexts).

* [interp] don't lookup internal calls that were replaced by an opcode

   If it was replaced by an intrinsic, we're not really going to call the wrapper, no need to look for it

* Add Interlocked.CompareExchange interp intrinsics for I4 and I8

* more inlining: LockWordCompareExchange

* Use a ref struct to pass around the address of a MonoObject* on the stack

   This is similar to ObjectHandleOnStack except with a getter that lets us view the object header.

   Get rid of calls to GC.KeepAlive.

   With this, the fast path (locking a flat unowned object) doesn't have any calls on it besides argument validation

* Add fast path for Monitor.Exit for inflated monitors with no waiters

* Check for obj == null before ThrowIfNull

   In the interp it's cheaper to do a null check followed by a call than an unconditional call

* Inline RuntimeHelpers.GetHashCode

* inline TryEnterInflatedFast; nano-opts

   - change SyncBlock.HashCode to return the value, not a ref - we don't have managed code that writes into the sync block.
   - change TryEnterInflatedFast to avoid looping - if the CompareExchange fails, fall back to the slow path

* disable the TryGetHashCode fast path helper

   It was actually making things slower on the interpreter.

   In the JIT it made object.GetHashCode about 50% faster for an object with a pre-computed hash, but that might mean we need the same kind of "no-wrapper; then wrapper" fastpath/slowpath that we have for the Monitor.Enter intrinsic in mini.

* Fix last exit from inflated monitor

   When nest == 1 the caller is responsible for setting the owner to 0 to indicate that the monitor is unlocked.  (But we leave the nest count == 1, so that it is correct next time the monitor is entered)

* avoid repeated calls in TryEnterInflatedFast

* Combine TryExit and IsEntered into TryExitChecked

   avoid duplicated lockword and sync block manipulations

* Re-enable managed GetHashCode in JIT; intrinsify in interp

   Instead of treating InternalGetHashCode/InternalTryGetHashCode as intrinsics in the interpreter, intrinsify RuntimeHelpers.GetHashCode and RuntimeHelpers.TryGetHashCode and avoid the managed code entirely when possible

* add CMPXCHG opcodes to the jiterpreter

   For the i64 version, pass the address of the i64 values to the helper function.

   The issue is that since JS doesn't have i64, EMSCRIPTEN_KEEPALIVE messes with the function's signature and we get import errors in the JITed wasm module.  Passing by address works around the issue since addresses are i32.

* fix whitespace

* [jiterp] Allow null obj in hashcode intrinsics

   The underlying C functions are null-friendly, and the managed intrinsics are allowed to C null inputs.  The assert is unnecessary.
  • Loading branch information
lambdageek authored Feb 15, 2023
1 parent db5dfad commit 0202b24
Show file tree
Hide file tree
Showing 14 changed files with 594 additions and 65 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -266,6 +266,7 @@
<Compile Include="$(BclSourcesRoot)\System\Security\DynamicSecurityMethodAttribute.cs" />
<Compile Include="$(BclSourcesRoot)\System\Threading\Interlocked.Mono.cs" />
<Compile Include="$(BclSourcesRoot)\System\Threading\Monitor.Mono.cs" />
<Compile Include="$(BclSourcesRoot)\System\Threading\ObjectHeader.Mono.cs" />
<Compile Include="$(BclSourcesRoot)\System\Threading\Thread.Mono.cs" />
<Compile Include="$(BclSourcesRoot)\System\Threading\ThreadPool.Mono.cs" />
</ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,12 @@ public static int OffsetToStringData
[MethodImplAttribute(MethodImplOptions.InternalCall)]
private static extern int InternalGetHashCode(object? o);

[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static int GetHashCode(object? o)
{
// NOTE: the interpreter does not run this code. It intrinsifies the whole RuntimeHelpers.GetHashCode function
if (Threading.ObjectHeader.TryGetHashCode (o, out int hash))
return hash;
return InternalGetHashCode(o);
}

Expand All @@ -55,8 +59,12 @@ public static int GetHashCode(object? o)
/// The advantage of this over <see cref="GetHashCode" /> is that it avoids assigning a hash
/// code to the object if it does not already have one.
/// </remarks>
[MethodImpl(MethodImplOptions.AggressiveInlining)]
internal static int TryGetHashCode(object? o)
{
// NOTE: the interpreter does not run this code. It intrinsifies the whole RuntimeHelpers.TryGetHashCode function
if (Threading.ObjectHeader.TryGetHashCode (o, out int hash))
return hash;
return InternalTryGetHashCode(o);
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,17 @@ public static void Enter(object obj, ref bool lockTaken)
}

[MethodImplAttribute(MethodImplOptions.InternalCall)]
public static extern void Exit(object obj);
private static extern void InternalExit(object obj);

public static void Exit(object obj)
{
if (obj == null)
ArgumentNullException.ThrowIfNull(obj);
if (ObjectHeader.TryExitChecked(obj))
return;

InternalExit(obj);
}

public static bool TryEnter(object obj)
{
Expand Down Expand Up @@ -57,7 +67,7 @@ public static void TryEnter(object obj, int millisecondsTimeout, ref bool lockTa
public static bool IsEntered(object obj)
{
ArgumentNullException.ThrowIfNull(obj);
return IsEnteredNative(obj);
return ObjectHeader.IsEntered(obj);
}

[UnsupportedOSPlatform("browser")]
Expand All @@ -79,15 +89,12 @@ public static void PulseAll(object obj)
ObjPulseAll(obj);
}

[MethodImplAttribute(MethodImplOptions.InternalCall)]
private static extern bool Monitor_test_synchronised(object obj);

[MethodImplAttribute(MethodImplOptions.InternalCall)]
private static extern void Monitor_pulse(object obj);

private static void ObjPulse(object obj)
{
if (!Monitor_test_synchronised(obj))
if (!ObjectHeader.HasOwner(obj))
throw new SynchronizationLockException();

Monitor_pulse(obj);
Expand All @@ -98,7 +105,7 @@ private static void ObjPulse(object obj)

private static void ObjPulseAll(object obj)
{
if (!Monitor_test_synchronised(obj))
if (!ObjectHeader.HasOwner(obj))
throw new SynchronizationLockException();

Monitor_pulse_all(obj);
Expand All @@ -111,7 +118,7 @@ private static bool ObjWait(int millisecondsTimeout, object obj)
{
if (millisecondsTimeout < 0 && millisecondsTimeout != (int)Timeout.Infinite)
throw new ArgumentOutOfRangeException(nameof(millisecondsTimeout));
if (!Monitor_test_synchronised(obj))
if (!ObjectHeader.HasOwner(obj))
throw new SynchronizationLockException();

return Monitor_wait(obj, millisecondsTimeout, true);
Expand All @@ -120,22 +127,22 @@ private static bool ObjWait(int millisecondsTimeout, object obj)
[MethodImplAttribute(MethodImplOptions.InternalCall)]
internal static extern void try_enter_with_atomic_var(object obj, int millisecondsTimeout, bool allowInterruption, ref bool lockTaken);

[MethodImpl(MethodImplOptions.AggressiveInlining)]
private static void ReliableEnterTimeout(object obj, int timeout, ref bool lockTaken)
{
ArgumentNullException.ThrowIfNull(obj);
if (obj == null)
ArgumentNullException.ThrowIfNull(obj);

if (timeout < 0 && timeout != (int)Timeout.Infinite)
throw new ArgumentOutOfRangeException(nameof(timeout));

try_enter_with_atomic_var(obj, timeout, true, ref lockTaken);
}

[MethodImplAttribute(MethodImplOptions.InternalCall)]
private static extern bool Monitor_test_owner(object obj);
// fast path
if (ObjectHeader.TryEnterFast(obj)) {
lockTaken = true;
return;
}

private static bool IsEnteredNative(object obj)
{
return Monitor_test_owner(obj);
try_enter_with_atomic_var(obj, timeout, true, ref lockTaken);
}

public static extern long LockContentionCount
Expand Down
Loading

0 comments on commit 0202b24

Please sign in to comment.