Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[browser] WASM sidecar - multi-threading proposal #91731

Closed
wants to merge 1 commit into from
Closed

[browser] WASM sidecar - multi-threading proposal #91731

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
158 changes: 158 additions & 0 deletions docs/design/mono/wasm-threads.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
# Multi-threading on browser

## Goals
- CPU intensive workloads on dotnet thread pool
- enable blocking .Wait APIs from C# user code
- Current public API throws PNSE for it
- This is core part on MT value proposition.
- If people want to use existing MT code-bases, most of the time, the code is full of locks. People want to use existing code as is.
- allow HTTP and WS C# APIs to be used from any thread
- Underlying JS object have thread affinity
- don't change/break single threaded build. †
- don't try to block on UI thread.

<sub><sup>† Note: all the text below discusses MT build only, unless explicit about ST build.</sup></sub>

## Design proposal TL;DR - Alternative 10
20. MAUI/BlazorWebView ... is the same thing
21. execute whole WASM runtime on a worker. Blocking is fine there.
22. The UI thread will only have `blazor.server.js` with small modifications.
23. This needs new marketing name! -> `WASM server` ???

# Detailed design

## Blazor - what changes vs MAUI server
- it still needs to compile for WASM target
- it will have threadpool and blocking C# `.Wait` and `lock()`
- it would not have Socket or DB connection

## Blazor - what changes vs "Blazor WASM"
- "onClick" would be asynchronous, same way as in Blazor server
- Blazor's [`IJSInProcessRuntime.Invoke`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.jsinterop.ijsinprocessruntime.invoke) would not be available
- Blazor's [`IJSUnmarshalledRuntime`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.jsinterop.ijsunmarshalledruntime) would not be available
- no JavaScript APIs on `globalThis.Blazor` or `globalThis.MONO`
- no runtime JS interop
- no runtime on UI thread
- Blazor united: progressive switch from remote server to WASM server, after WASM binaries arrived

## Blazor startup
- we will need to start emscripten on a worker and `postMessage` there all the configuration.
- this is possibly the biggest effort
- probably no `Blazor.start`, `start.withResourceLoader` with returning `Response` object
- we may keep `dotnet.js` loader to run in the UI
- TODO: prototype

### Blazor `renderBatch`
- streaming bytes - as Blazor server does
- we use [RenderBatchWriter](https://github.com/dotnet/aspnetcore/blob/045afcd68e6cab65502fa307e306d967a4d28df6/src/Components/Shared/src/RenderBatchWriter.cs) in the WASM
- we use `blazor.server.js` to render it
- preferred if fast enough
- could be ok to render batch by another thread while UI is rendering
- plan b) use existing `SharedArrayBuffer`
- WASM would:
- stop the GC
- send the "read-memory-now"
- await for "done" message
- enable GC
- UI could duplicate `conv_string` and `unbox_mono_obj` dark magic and use it

## Blazor JS interop
- will work as usual on "Blazor server"

## runtime JS Interop
- would be available only to workers, but it would not have access to UI thread and DOM
- could call sync and async calls both directions, promises and callbacks

## HTTP & WS
- would be running on a worker, will be accessible from all C# threads
- we will overcome thread affinity internally

## C# Thread
- could block on synchronization primitives
- without JS interop. calling JSImport will PNSE.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think leaving out JSImport makes sense as a 1.0 version of this, but it's meaningful to support JSImport so that the "server" here can utilize browser APIs, to compensate for the lack of native APIs. Like for example if the "server" is able to use an offscreen canvas it would be able to access hardware accelerated rendering, which would enable developers to use stuff like browser machine learning APIs.

Copy link
Member Author

@pavelsavara pavelsavara Sep 22, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be able to do that we need JSImport/JSExport interop on the UI thread.

There are 4 choices how to do that
A) dispatch via managed as described in deputy-worker proposal

B) dispatch via just JS - double proxy

  • Which means we will have JS proxy in UI thread of C# proxy in side-car worker.
  • Comlink style.
  • We could write out own and make in also spin-blocking the UI.
  • But this is double dispatch on each call. Hopping over 2 threads and their main loop. This is in cases when caller was not on side-car.
  • It will be slow and difficult to GC.

C) dispatch just the mono_wasm_bind_js_function and mono_wasm_bind_cs_function and then do the JS side of the marshaling in the UI thread. But the implementation is heavily dependent on

  • memory (that's easy one)
  • shared code invoke-cs.ts, invoke-js.ts etc
  • but that is dependent on Mono C methods and emscripten methods.
  • Most of them are synchronous and need to be fast. Because some of them are called per argument.
  • But this proposal assumes that emscripten is not on UI thread!
    • emscripten: stack alllocation, memory views (growing)
    • GC and JS handles (I guess those should be UI thread local)
    • various JS helpers (logging, exception handling, asserts)
    • mono: string marshaling, gc roots
    • mono: call dispatch to managed code: instantiate TaskCompletionSource etc

Copy link
Member Author

@pavelsavara pavelsavara Sep 22, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

D) we could further move the boundary and have it between full dotnet.runtime.js on the UI thread and emscripten+MONO on the side-car. That would narrow it down to

  • memory view update events
  • per assembly
    • mono_wasm_runtime_run_module_cctor
  • per method binding
    • BindJSFunction
    • BindCSFunction
    • mono_wasm_assembly_find_class
    • mono_wasm_assembly_find_method
    • mono_wasm_invoke_method_ref
    • free
  • per method invoke
    • InvokeJSFunction
    • InvokeImport
    • mono_wasm_invoke_method_bound
    • stackalloc
  • per parameter when array
    • mono_wasm_deregister_root/DeregisterGCRoot
    • mono_wasm_register_root/RegisterGCRoot
    • malloc
  • per parameter instance when proxy
    • release_js_owned_object_by_gc_handle for proxy of C# object
    • mono_wasm_release_cs_owned_object/ReleaseCSOwnedObject for JSObject proxy of C# object
    • get_managed_stack_trace_method
  • per parameter instance when string
    • mono_wasm_string_get_data_ref
    • mono_wasm_string_from_utf16_ref
    • mono_wasm_write_managed_pointer_unsafe
    • mono_wasm_deregister_root
  • per parameter instance when promise/task/function/delegate
    • create_task_callback_method
    • complete_task_method
    • call_delegate_method
    • MarshalPromise

most of the C methods above need GC boundary and Mono registered thread

perhaps we could marshal strings and other value types already in side-car

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was proposing that the sidecar worker be able to access JS inside the sidecar context, to be clear. Not JS objects from the main thread. No remoting, so it can still be synchronous.

Copy link
Member Author

@pavelsavara pavelsavara Sep 22, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need some JS interop anyway for Blazor, the current surface is

This is about startup, loading and embedding

  • INTERNAL.loadLazyAssembly - async, string
  • INTERNAL.loadSatelliteAssemblies - async, string
  • Blazor._internal.getApplicationEnvironment, string
  • receiveHotReloadAsync - async void

This needs to hit UI thread (but the payload is string/bytes, not objects)

  • Blazor._internal.endInvokeDotNetFromJS
  • Blazor._internal.invokeJSJson
  • Blazor._internal.receiveByteArray

This needs to hit UI thread

  • Blazor._internal.getPersistedState -> sync string

Could be on sidecar

  • globalThis.console.debug
  • globalThis.console.error
  • globalThis.console.info
  • globalThis.console.warn
  • Blazor._internal.dotNetCriticalError

This is related to renderBatch (which we could do the same way as Blazor server does and skip this)

  • MONO.getI16
  • MONO.getI32
  • MONO.getF32
  • BINDING.js_string_to_mono_string
  • BINDING.conv_string
  • Blazor._internal.renderBatch
  • BINDING.unbox_mono_obj - this one could be tricky

I hope this is out of scope

  • ICall InvokeJS


## C# Threadpool Thread
- could block on synchronization primitives
- without JS interop. calling JSImport will PNSE.

## Blocking problem
1. If you have multithreading, any thread might need to block while waiting for any other to release a lock.
- locks are in the user code, in nuget packages, in Mono VM itself
- there are managed and un-managed locks
- in single-threaded build of the runtime, all of this is NOOP. That's why it works on UI thread.
2. UI thread in the browser can't synchronously block
- you can spin-lock but it's bad idea.
- Deadlock: when you spin-block, the JS timer loop and any messages are not pumping. But code in other threads may be waiting for some such event to resolve.
- It eats your battery
- Browser will kill your tab at random point (Aw, snap).
- It's not deterministic and you can't really test your app to prove it harmless.
- all the other threads/workers could synchronously block
3. JavaScript engine APIs and objects have thread affinity. The DOM and few other browser APIs are only available on the main UI "thread"
- and so, you need to have some way how to talk to UI

This is the main reason why we can't run MT dotnet also on UI thread.

## Alternatives
- "deputy thread" proposal https://github.com/dotnet/runtime/pull/91696

## Debugging
- VS debugger would work as usual
- Chrome dev tools would only see the events coming from `postMessage`
- Chrome dev tools debugging C# could be bit different, it possibly works already. The C# code would be in different node of the "source" tree view

## Non-blazor
- does Uno have similar "render from distance" architecture ?

## Open questions
- when MT emscripten starts on a WebWorker, does it know that it doesn't have to spin-block there ?

# Further improvements

## JSWebWorker with JS interop
- is C# thread created and disposed by new API for it
- could block on synchronization primitives
- there is JSSynchronizationContext installed on it
- so that user code could dispatch back to it, in case that it needs to call JSObject proxy (with thread affinity)

## Promise, Task, Task<T>
- passing Promise should work everywhere.
- when marshaled to JS they bind to specific `Promise` and have affinity
- the `Task.SetResult` need to be marshaled on thread of the Promise.

## JSObject proxy
- has thread affinity, marked by private ThreadId.
- in deputy worker, it will be always UI thread Id
- the JSHandle always belongs to UI thread
- `Dispose` need to be called on the right thread.
- how to do that during GC/finalizer ?
- should we run finalizer per worker ?
- is it ok for `SynchronizationContext` to be public API
- because it could contain UI thread SynchronizationContext, which user code should not be dispatched on.

## should we hide `SynchronizationContext` inside of the interop generated code.
- needs to be also inside generated nested marshalers
- is solution for deputy's SynchronizationContext same as for JSWebWorker's SynchronizationContext, from the code-gen perspective ?
- how could "HTTP from any C# thread" redirect this to the thread of fetch JS object affinity ?
- should generated code or the SynchronizationContext detect it from passed arguments ?
- TODO: figure out backward compatibility of already generated code. Must work on single threaded
- why not make user responsible for doing it, instead of changing generator ?
- I implemented MT version of HTTP and WS by calling `SynchronizationContext.Send` and it's less than perfect. It's difficult to do it right: Error handling, asynchrony.

## SynchronizationContext
- we will need public C# API for it, `JSHost.xxxSynchronizationContext`
- maybe `JSHost.Post(direction, lambda)` without exposing the `SynchronizationContext` would be better.
- we could avoid it by generating late bound ICall. Very ugly.
- on a JSWebWorker
- to dispatch any calls of JSObject proxy members
- to dispatch `Dispose()` of JSObject proxy
- to dispatch `TaskCompletionSource.SetResult` etc

### dispatch alternatives
- we could use emscripten's `emscripten_dispatch_to_thread_async` or JS `postMessage`
- the details on how to interleave that with calls to `ToManaged` and `ToJS` for each argument may be tricky.

Related Net8 tracking https://github.com/dotnet/runtime/issues/85592

# TODO
- [ ] experiment with `test-main.js` to do the same
- [ ] experiment with Blazor to try easy version without JS extensibility concerns