poll_oneoff - is async io support possible? #14
Comments
|
Should be fine to implement. It can't be actual async io, but that isn't needed anyway as an in memory VFS is used, so every operation finishes pretty much instantly. |
|
Would you mind posting an example wasm binary that hits this error? That would help with testing the implementation. |
|
Good to hear! You can get a reproducer from https://tristancacqueray.github.io/tiny-game-haskell-wasm/ , where the shim is patched locally to remove the throw. The expected result is that pressing enter should clear and redraw the terminal. The client source is from https://github.com/TristanCacqueray/tiny-game-haskell-wasm/blob/main/client.js |
|
Yes, please, I am having the same issue. |
|
I was busy for the past couple of week, but I can probably work on this tomorrow. |
|
Appreciated! |
|
I just took a closer look and while diff --git a/src/wasi.js b/src/wasi.js
index 3a1060f..1c341ae 100644
--- a/src/wasi.js
+++ b/src/wasi.js
@@ -426,6 +426,7 @@ export default class WASI {
}
},
poll_oneoff(in_, out, nsubscriptions) {
+ return 0;
throw "async io not supported";
},
proc_exit(exit_code/*: number*/) {fixes the crash, it now gets stuck in an infinite loop waiting for input that will never come as It should be possible to run the wasm code in a web worker and then in this web worker do an For now if you want to implement this yourself you can overwrite |
|
I'm giving the |
|
For me making |
|
@bjorn3 sure, I am using this branch: https://github.com/TristanCacqueray/browser_wasi_shim/tree/worker-example Please note that I've never used web worker and atomics before. When browsing the dist, in the console I see an infinite loop between |
|
Looks like diff --git a/examples/worker.html b/examples/worker.html
index 10f2918..27a4d8c 100644
--- a/examples/worker.html
+++ b/examples/worker.html
@@ -4,8 +4,8 @@
<meta charset="utf-8">
</head>
<body>
- <script type="module">
- const worker = new Worker("worker.js");
+ <script>
+ const worker = new Worker("./worker.js", {type:"module"});
worker.onmessage = e => {
console.log('Main: onMessage', e.data)
}
diff --git a/examples/worker.js b/examples/worker.js
index 37794e8..68c9f83 100644
--- a/examples/worker.js
+++ b/examples/worker.js
@@ -1,22 +1,26 @@
+// @flow
+
import { Fd } from "../src/fd.js";
import { File, Directory } from "../src/fs_core.js";
-import { Fdstat } from "../src/wasi_defs.js";
+import { Fdstat, FILETYPE_UNKNOWN, FDFLAGS_APPEND } from "../src/wasi_defs.js";
import { PreopenDirectory } from "../src/fs_fd.js";
import WASI from "../src/wasi.js";
import { strace } from "../src/strace.js"
class XTermStdio extends Fd {
- constructor(term) {
+ /*::term: { write: (Uint8Array) => mixed }*/
+
+ constructor(term/*: { write: (Uint8Array) => mixed }*/) {
super();
this.term = term;
}
- fd_read(x, y) {
- console.log("Reading!", x, y)
- return { ret: 0 }
+ fd_read(view8, iovs) {
+ console.log("Reading!", view8, iovs);
+ return { ret: 0, nread: 0 };
}
fd_fdstat_get() {
console.log("FDSTAT")
- return { ret: 0, fdstat: new Fdstat() };
+ return { ret: 0, fdstat: new Fdstat(FILETYPE_UNKNOWN, FDFLAGS_APPEND) };
}
fd_write(view8, iovs) {
console.log("Writing!")
@@ -38,7 +42,11 @@ onmessage = function(e) {
(async function () {
const wasm = await WebAssembly.compileStreaming(fetch("tiny-brot.wasm"));
- const term = {}
+ const term = {
+ write: (buf) => {
+
+ }
+ }
const fds = [
new XTermStdio(term),
new XTermStdio(term),
diff --git a/src/wasi.js b/src/wasi.js
index d4cd18a..5b6ae8f 100644
--- a/src/wasi.js
+++ b/src/wasi.js
@@ -433,8 +433,39 @@ export default class WASI {
return wasi.ERRNO_BADF;
}
},
- poll_oneoff(in_, out, nsubscriptions) {
- return 0
+ poll_oneoff(in_ptr, out_ptr, nsubscriptions) {
+ // in_: *const subscription
+ // out: *mut event
+ // nsubscription: usize
+ let buffer = new DataView(self.inst.exports.memory.buffer);
+ let in_ = wasi.Subscription.read_bytes_array(
+ buffer,
+ in_ptr,
+ nsubscriptions,
+ );
+ console.log("poll_oneoff", in_, out_ptr, nsubscriptions);
+ let events = [];
+ for (let sub of in_) {
+ if (sub.u.tag.variant == "fd_read") {
+ let event = new wasi.Event();
+ event.userdata = sub.userdata;
+ event.error = 0;
+ event.type = new wasi.EventType("fd_read");
+ event.fd_readwrite = new wasi.EventFdReadWrite(1n, new wasi.EventRwFlags());
+ events.push(event);
+ }
+ if (sub.u.tag.variant == "fd_write") {
+ let event = new wasi.Event();
+ event.userdata = sub.userdata;
+ event.error = 0;
+ event.type = new wasi.EventType("fd_write");
+ event.fd_readwrite = new wasi.EventFdReadWrite(1n, new wasi.EventRwFlags());
+ events.push(event);
+ }
+ }
+ console.log(events);
+ wasi.Event.write_bytes_array(buffer, out_ptr, events);
+ return events.length;
throw "async io not supported";
},
proc_exit(exit_code/*: number*/) {
diff --git a/src/wasi_defs.js b/src/wasi_defs.js
index a2f8322..7f34a0e 100644
--- a/src/wasi_defs.js
+++ b/src/wasi_defs.js
@@ -348,3 +348,153 @@ export class Prestat {
this.inner.write_bytes(view, ptr + 4);
}
}
+
+/*::declare type UserData = BigInt */ // u64
+
+export class EventType {
+ /*:: variant: "clock" | "fd_read" | "fd_write"*/
+
+ constructor(variant/*: "clock" | "fd_read" | "fd_write"*/) {
+ this.variant = variant;
+ }
+
+ static from_u8(data/*: number*/)/*: EventType*/ {
+ switch (data) {
+ case EVENTTYPE_CLOCK:
+ return new EventType("clock");
+ case EVENTTYPE_FD_READ:
+ return new EventType("fd_read");
+ case EVENTTYPE_FD_WRITE:
+ return new EventType("fd_write");
+ default:
+ throw "Invalid event type " + String(data);
+ }
+ }
+
+ to_u8()/*: number*/ {
+ switch (this.variant) {
+ case "clock":
+ return EVENTTYPE_CLOCK;
+ case "fd_read":
+ return EVENTTYPE_FD_READ;
+ case "fd_write":
+ return EVENTTYPE_FD_WRITE;
+ default:
+ throw "unreachable";
+ }
+ }
+}
+
+export class EventRwFlags {
+ /*:: hangup: bool*/
+
+ static from_u16(data/*: number*/)/*: EventRwFlags*/ {
+ let self = new EventRwFlags();
+ if ((data & EVENTRWFLAGS_FD_READWRITE_HANGUP) == EVENTRWFLAGS_FD_READWRITE_HANGUP) {
+ self.hangup = true;
+ } else {
+ self.hangup = false;
+ }
+ return self;
+ }
+
+ to_u16()/*: number*/ {
+ let res = 0;
+ if (self.hangup) {
+ res = res | EVENTRWFLAGS_FD_READWRITE_HANGUP;
+ }
+ return res;
+ }
+}
+
+export class EventFdReadWrite {
+ /*:: nbytes: BigInt*/
+ /*:: flags: EventRwFlags*/
+
+ constructor(nbytes/*: BigInt*/, flags/*: EventRwFlags*/) {
+ this.nbytes = nbytes;
+ this.flags = flags;
+ }
+
+ write_bytes(view/*: DataView*/, ptr/*: number*/) {
+ view.setBigUint64(ptr, this.nbytes, true);
+ view.setUint16(ptr + 8, this.flags.to_u16(), true);
+ }
+}
+
+export class Event {
+ /*:: userdata: UserData*/
+ /*:: error: number*/
+ /*:: type: EventType*/
+ /*:: fd_readwrite: EventFdReadWrite | null*/
+
+ write_bytes(view/*: DataView*/, ptr/*: number*/) {
+ view.setBigUint64(ptr, this.userdata, true);
+ view.setUint32(ptr + 8, this.error, true);
+ view.setUint8(ptr + 12, this.type.to_u8());
+ if (this.fd_readwrite) {
+ this.fd_readwrite.write_bytes(view, ptr + 16);
+ }
+ }
+
+ static write_bytes_array(view/*: DataView*/, ptr/*: number*/, events/*: Array<Event>*/) {
+ for (let i = 0; i < events.length; i++) {
+ events[i].write_bytes(view, ptr + 32 * i);
+ }
+ }
+}
+
+export class SubscriptionClock {
+ // FIXME
+}
+
+export class SubscriptionFdReadWrite {
+ /*:: fd: number*/
+
+ static read_bytes(view/*: DataView*/, ptr/*: number*/)/*: SubscriptionFdReadWrite*/ {
+ let self = new SubscriptionFdReadWrite();
+ self.fd = view.getUint32(ptr, true);
+ return self;
+ }
+}
+
+export class SubscriptionU {
+ /*:: tag: EventType */
+ /*:: data: SubscriptionClock | SubscriptionFdReadWrite */
+
+ static read_bytes(view/*: DataView*/, ptr/*: number*/)/*: SubscriptionU*/ {
+ let self = new SubscriptionU();
+ self.tag = EventType.from_u8(view.getUint8(ptr));
+ switch (self.tag.variant) {
+ case "clock":
+ break; // FIXME implement
+ case "fd_read":
+ case "fd_write":
+ self.data = SubscriptionFdReadWrite.read_bytes(view, ptr + 4);
+ break;
+ default:
+ throw "unreachable";
+ }
+ return self;
+ }
+}
+
+export class Subscription {
+ /*:: userdata: UserData */
+ /*:: u: SubscriptionU */
+
+ static read_bytes(view/*: DataView*/, ptr/*: number*/)/*: Subscription*/ {
+ let subscription = new Subscription();
+ subscription.userdata = view.getBigUint64(ptr, true);
+ subscription.u = SubscriptionU.read_bytes(view, ptr + 8);
+ return subscription;
+ }
+
+ static read_bytes_array(view/*: DataView*/, ptr/*: number*/, len/*: number*/)/*: Array<Subscription>*/ {
+ let subscriptions = [];
+ for (let i = 0; i < len; i++) {
+ subscriptions.push(Subscription.read_bytes(view, ptr + 12 * i));
+ }
+ return subscriptions;
+ }
+}I might clean it up at a later time, but there are currently some things going on that have priority. |
|
Thank you, that's super helpful. It seems like returning |
|
I updated the According to ghc errno.js, this happens because of the E2BIG errno, perhaps @TerrorJack or @hsyl20 would know what could be the cause? |
|
@TristanCacqueray I haven't taken a closer look at the examples yet; what kind of I think it may be a low effort to enhance |
|
@TerrorJack first I'd like to setup xtermjs to handle interactive wasm instance, so |
|
@TristanCacqueray So your Haskell logic is a plain loop that polls |
|
@TerrorJack well the goal is to enable running the tiny-game-hs in the browser. It seems like some of them already work unmodified with wasmtime, and perhaps they could also work with @bjorn3 wasi shim, e.g. using the worker+atomics solution discussed above. |
|
Looks like I messed up the abi of poll_oneoff. The event count needs to be written to the memory referenced by a pointer passed as 4th argument and a value of 0 needs to be returned on success. Instead I directly returned the event count, which is interpreted as E2BIG for a certain event count. |
|
@bjorn3 ok, I think I managed to fix the abi, but this results in the |
|
I’ve been running into this same issue. An easy workaround (as pointed out on the Haskell Discourse by @brandonchinn178) is to just avoid stdin/stdout altogether and compile the application as a WASI reactor instead. However, I’ve since realised that this issue blocks any kind of asynchronicity on the Haskell side at all (e.g. halting a computation on a timeout). It would be really nice to get this fixed! |
|
I had the same problem. In my case, I compiled (part of) pandoc as wasm32-wasi, and the entire input is available before running the module. So I implemented I wish this helps someone with a similar use case! |
|
https://gitlab.haskell.org/ghc/ghc/-/merge_requests/11697 should hopefully mitigate the need for |
It did indeed! Is there any chance you could repost this as a pull request, @igrep? Or would you mind if I did? To be quite honest, I don't fully grasp what's happning in your patch but it looks well thought-through and it fixed my simple usage of |
|
@ansemjo My patch is useful only if the entire data written to stdin is available before running |
|
@igrep Okay, I see. Thanks for the explanation. I'm on mobile now, so I can't find the exact.location of the Edit: upon closer inspection, it does busy-loop by calling |
Hello, thank you for this great library.
When using it for program built with https://gitlab.haskell.org/ghc/ghc-wasm-meta, it is failing early on because of the poll_oneoff "async io not supported" exception. Using
return 0instead makes the program go a bit further, but then it crash again when reading the stdin. Would it be possible to implement this api?The text was updated successfully, but these errors were encountered: