Design: Concurrency and Asynchrony, or, why does the real world have to be happening all at once?

[sethp]

Ok, rubber duck text box a-go-go. I have two loose areas of concern:

1. Tasks/threading/sequential composition
2. Efficiency/timing/real-time bounds

(this also feels related to the remaining work on #15 that I'm a little unsure about how to implement. Roughly: that's got the sense of a task priority system to me, but both the tasks and priorities are like shouting through tin can connected via string to a megaphone).

I notice that modifying either of those is pretty hard and requires a relatively complete "picture" of what all the moving parts are. Like: adding a new "task", especially one that ought to be concurrent with the existing 4 (serial input + output, keyboard input, frame emission) is pretty hard to do in our current model; let's consider say, tickling the I2S peripheral for emitting a bell sound? That'll have some schedule of (low frequency, but real time) updates that's ~naturally independent of everything else once it's triggered, right? (if a bell sound fits in a single I2S peripheral payload, let's imagine we're bit-banging it or we decide we want to play background music or we want to support inline animated gifs like iTerm2 or something). It seems like we'd have to spend a lot of time finessing the existing stuff to find a place to "fit." Or, maybe the first thing we'd try would work, but we'd be eating up some unknown amount of safety margin on the other pieces, which is, I think scarier to me: when something does eventually fail, it'll have a more complex cause accumulated over weeks or months of by-then-hard-to-access context. I have this vision in mind of sort of shaking a jigsaw puzzle box, hoping that all the pieces end up landing together in a way that fits.

Some constraints are physical: if we have two hard real-time systems that both need the same slice of CPU time, we're out of luck. But in general we've got a lot more cycles than we need, so adding latency is an excellent way to "stretch" out around our real-time bounds; we ought to be able to keep playing around and adding stuff until we're either:

1. So far out of time that the latency becomes intolerable (e.g. 100ms or something per keystroke)
2. Colliding on hard real-time constraints (but I would really like to know this, precisely, and also how far away it is)

Also, I have this vague sense that we're spending some amount of tokens from a limited resource, but would really like a more concrete sense of that. Something like a one second timeline that fills up with colored blocks as we slot stuff in so we can "see" how much e.g. time we're spending in a real-time handler vs. in general processing vs. wfi ("idle")? (Something like humility trace seems ~relevant?). We've talked about a lot of the stuff we can do to put things back into the supply, too (e.g. optimizing the channel implementation), but I've found its generally a better time to optimize 1) with guidance measurements and 2) a specific goal in mind to hit.

Some very interesting prior work in this space is hubris, which to me has the feel of some very smart engineers staring down exactly this problem set and coming up with an exquisitely form-and-function-fitting solution (at least to me). Some example tasks that might be worth a peek?

• Ping: https://github.com/oxidecomputer/hubris/blob/master/task/ping/src/main.rs
• Pong: https://github.com/oxidecomputer/hubris/blob/master/task/pong/src/main.rs

There's also embassy that exposes a more traditional async/await approach, which is also worth considering (even if personally I'm not such a huge fan of the style). That would have the benefit, if we adopted the approach (or the framework, whichever) of allowing us to write things like:

#[some::task]
fn play_bell() {
   i2s.send(&bell_sound[..HALF]).await;
   i2s.send(&bell_sound[HALF..]);
}

(also there's at least some support in the esp-rs hal for embassy-related things; I haven't looked at it at all, but I've seen the word floating around upstream, which might make a difference to us)

Looking into the Wikipedia "See Also" section of Async/Await I see:

• Coroutines — potentially interesting, rust does have some (early) language support that'd be helpful
• Continuation-passing style — I'm fairly sure in our context this is a highfalutin term for "callback hell"
• Direct style — this redirects to CPS, because it's sort of the usual unnamed default except when CPS needs to define what it is by opposition
• Cooperative multitasking — a broader umbrella term, mostly useful in that it mentions "event loop", a broad sort of implementation strategy (e.g. I usually think of an event loop as being at the heart of coroutines and async/await runtimes and probably even continuations in practice)

Sorta helpful? At least in the idea-generating sense? There's a lot more content and diagrams to dig into under Event Loop too. I think a "preemptable Event Loop" is probably the most accurate description of how our program is currently structured, what would it mean to lean more strongly into that?