API Change Proposal: Events Loop 2.0

[Osspial]

UPDATE 8/23/18:

A PR for Windows is available at #638

UPDATE 7/12/18:

I've adapted the agreed-upon changes into a changelog here: https://gist.github.com/Osspial/d3a3390807fa95a469d9c9b677db4715. The original proposal is below.

---

Events Loop 2.0

This issue proposes a series of changes to winit's EventsLoop API, based around the experiences I've had with creating a GUI library that uses winit as a window creation backend. The main goal is to give winit complete control over the event loop, both for event polling and event waiting. These changes should let us fix various implementation bugs on the win32 backend, as well as fix what I believe are design bugs in the current API.

Technical Motivation:

EDIT: The technical motivation for the Windows backend is included below. However, pretty much every platform winit supports is hacky or broken in some way by necessity of the current API we expose.

Right now, the win32 backend creates a background thread, in which the events loop resides. The biggest problem with this is that it introduces various unexpected synchronization bugs that require a good amount of effort to fix and increase the complexity of the backend (see #415 and #391). However, it's necessary to cleanly handle win32's various modal event loops.

So, what are these modal loops, and why do they matter? On Windows, when a user begins resizing a window, Windows suspends the program's main event loop and begins its own loop, inside of the resize event's handler. Unfortunately, this freezes any processing that's going on outside the Windows event loop, which is currently worked around by having the loop exist in its own thread, introducing quite a few bugs as mentioned above. Inside the loop, Windows does a bunch of additional processing, most notably handling window snapping (although it probably does other stuff too that isn't obvious).

Windows doesn't publicly expose the APIs used to handle snapping, so re-implementing the modal loops ourselves so that they don't suspend the main event loop isn't really an option. However, Windows should still process events we push to the Windows events queue, letting us implement event polling inside the Windows event loop with custom events (using the same mechanisms used for wakeup messages), as long as winit's API lets it assume complete control over program control flow. Right now, it doesn't, but this change would let us remove the second thread and simplify much of the implementation.

Actually removing the second thread isn't that difficult (see #445), but these bugs present pretty major blockers that should be resolved before that's merged.

Design Details

EDIT: The original design has been removed from this post, as it is outdated and may cause confusion for new readers. However, it does contain some useful information about the rationale behind the changes, and can be accessed here: https://gist.github.com/Osspial/53efa87797899872650f6bece14736c0

[elinorbgr]

I don't have much to say about the API itself (I'm not that much used to writing apps with winit tbh), just commenting to say I don't see any strong incompatibility / difficulty with implementing this regarding to the wayland backend.

The most tricky part will be handling WaitWithTimeout, which will require to do some manual epoll, that's all I see.

[francesca64]

I think it's too opaque to have Refresh mean both "event processing finished" and "OS requested redraw", and if we were to go through with this, the name Refresh should probably be changed altogether. I can see some users thinking it represents vertical refresh, and it's just not very descriptive in general.

My grasp of your intended ControlFlow enum is a little unclear. Do you want users to be able to switch between these strategies at will during the event loop? Is that something we can implement effectively?

I'm curious how you'd say this proposal is better than #231, as your Refresh machinery seems to exist to solve the same problem (and get_event seems cleaner to me). I definitely agree that the current API has problems, and I'm starting to get the impression that the current model is incompatible with most platforms. At this point, I'm wondering if we should just invert the control flow, since unless I'm missing something, it seems like the most straightforward solution.

Additionally, since this is an area that has such an immediate and substantial impact on users, before acting on anything I'd want to see a lot of feedback. When we've achieved a good enough internal consensus, I think it would be a good idea to discuss it in other places in the community, and we'd hopefully be able to deliver an API change that people are hyped for, rather than one that unceremoniously breaks their application one morning.

[willglynn]

I think a breaking API change is necessary, but it must require the user to entirely surrender their thread. A callback-only API would fix Windows, it would fix MacOS (which has nearly identical resizing gotchas described in #219), it would fix the web (which needs to return to the browser to deliver more events), it would fix iOS and Android (which likewise expect to run the event loop), etc.

Going callback-only is a massive breaking change, but it's the right move. Most platforms expect to call user code from the UI thread when things happen, while winit tells its users that they can keep the UI thread and occasionally call into the platform. That is why this is hard.

Dropping run_forever() in favor of a new API that takes over your thread would be a massive breaking change, but it is possible to glue both models back together again using the pattern from #237. We could make a winit-compat crate that provides run_forever() on top of a new callback-only API via coroutines, and it could work on the major platforms.

Having said all that, while I am convinced that this is the right way forward, I am not in a position to be the champion of this approach. I no longer use winit and can't justify investing the required time and effort into making this happen.

[francesca64]

@willglynn I'd say I'm in a good position to be the champion of this approach. If necessary, I'd even implement it for every platform, though for my sake it hopefully wouldn't come to that.

At this point, I'm quite convinced that this is of great importance. Interfacing with platform APIs the correct way is already hard enough, and our current approach doesn't seem to be doing any favors for the users, contributors, or maintainers.

I'm not sure how necessary the winit-compat idea is. Good migration documentation and coordinating with projects could go a long way.

Anyway, now we just need to get the approval of... basically everyone.

[elinorbgr]

I would be in principle in favor of such a change on the winit API (going callback-only), though I'm not sure how much my opinion count on that, the fact remaining that wayland as a platform remains pretty accommodating regarding control-flow handling.

Though, if such a thing starts going, I'd like to emphasize the need for good planning ahead (maybe going through an RFC-like process ?), in order to clearly define the contents and guarantees of the API, in order to allow easier coherence in backend implementations.

Typical questions that in my opinion need to be clearly addressed are for example:

• guarantees and requirements regarding thread-safety (what is Send and/or Sync ?)
• if winit takes control of the thread, do we insert a mechanism allowing users to insert other kind of sources of events in the event loop?

[kvark]

@francesca64

I'd say I'm in a good position to be the champion of this approach. If necessary, I'd even implement it for every platform, though for my sake it hopefully wouldn't come to that.

viva de revolution! I'd be happy to provide limited assistance (got all platforms by hand).

Anyway, now we just need to get the approval of... basically everyone.

You have my seal of approval. Can also speak about some of the clients of winit:

• gfx-rs/examples and vange-rs: need a way to schedule a frame (at a timeout, or just on idle), no other issues are expected
• gfx-hal/examples: will benefit from NSAutoreleasePool being automatic on OSX/Metal
• three-rs: will need to own the main loop in order to give that ownership to winit (similarly, any middleware using winit would have to own the main loop after the change), also needs to schedule a frame, otherwise should be good
• webrender/wrench: already using run_forever

[Osspial]

@vberger My instinct is to make Windows Send and Sync, and then have the EventsLoop structure itself be locked to a single thread. I know this would work on Windows, as it's essentially what's already there, but I don't know how it would work on other platforms.

@francesca64 I can see your point with having Refresh mark the end of event processing, but I'm not sure if I agree with it. An alternative would be to have queue_refresh immediately send a Refresh event to the user, but I don't see a situation arising where someone would want to redraw immediately inside of the event loop. Regardless, there probably should be an event that communicates when the event loop is going to resume waiting - WaitBegin, perhaps? Also, even if this isn't the API we settle on, I do feel like we should rename Refresh to Redraw, as that communicates its purpose better.

My goal with the ControlFlow enum is to have it tell winit when it should call back into user code, and to allow the application to cleanly switch between updating constantly, updating intermittently and being suspended. This should be possible on Windows by switching between calling GetMessage and PeekMessage, and I'm assuming there's equivalent code to do this on other platforms, but I couldn't say with any certainty.

@willglynn basically explained the advantages of this API over the get_event API - while get_event is cleaner, and I'd rather use it if the underlying platforms allowed for it, the majority of platforms assume the OS is driving the event loop. A callback-based API fits into the OS event loop way better than the iterative design you get with get_event.

[Osspial]

I'd be able to implement this on Windows, at least, as it should be a fairly simple change to my current single-threaded win32 branch.

[willglynn]

A common pattern that pops up in this space is sending yourself a message, particularly to communicate things from other threads onto a UI thread. This can usually be expressed directly using platform-specific hooks, and when it can't, it can be synthesized.

This pattern often separates concerns: you end up writing event handling code to handle the message you sent from some other context and do whatever it asks you to do.

An alternative to consider:

fn run_in_event_loop<F>(f: F) where F: FnOnce(), F: Send + 'static

Instead of sending custom events to the event loop, send closures to run within the event loop. (Platforms could implement this by sending themselves a message – though there are a few alternatives on Mac/iOS and a native closure queueing mechanism on the web.) This ability to easily execute code within the event loop from arbitrary contexts could substantially reduce the need for other thread-safe interfaces.

[Osspial]

@willglynn An issue I can see with that is that any closures you pass through that function don't gain access to the data in the main UI closure, which would limit pretty heavily what you could do with the passed closure. Passing custom events which are handled by the main event closure gives you access to that data. How would that be handled? The best solution I can see would be to wrap all the data in the main UI closure in a Rc<RefCell<...>>, but that doesn't seem ideal.

[elinorbgr]

@Osspial

The best solution I can see would be to wrap all the data in the main UI closure in a Rc<RefCell<...>>, but that doesn't seem ideal.

An other possibility would be to add a type argument to the event loop : EventLoop<Data>, and require the user to provide an instance of this type at creation, and then all closures run on the event loop would take an additional &mut Data argument.

However, I suspect several of the platform backend would implement it by using an Rc<RefCell<Data>> internally (or some other more costly mechanism), so...

For what it's worth, I've kind of struggled with this question for quite some time in wayland-client, for a very similar mechanism (allow closures run by the event loop to share data without using Rc<RefCell<_>>), and in the end, it appeared to me that in all, Rc<RefCell<_>> was actually not that bad, and I suspect its runtime cost should be quite negligible in the context of UI events dispatching.

[willglynn]

@Osspial Sure; there's definitely API design to do here.

It's generally not sufficient just to be on the right thread, you also need the ability to use resources which are exclusive to that thread. That relationship – "there are things which are exclusive to this thread, but you can interact with them until your function returns" – should be described via the type system. This sounds winit should provide a &mut T where T is neither Send nor Sync gating access to the windowing system.

The user probably has application-wide state which is synchronized with the windowing system. Parameterizing winit around the user's state seems like a valid approach, e.g. this sketch in #231.

Thing is: this same thing (providing thread-local exclusive access to the platform and exclusive access to some application data) is needed for the event handling callback too. Whatever solution is used there should inform the parameter(s) to the run_in_event_loop() closure. I just wanted to plant the idea that the ability to asynchronously execute a closure within the event loop could greatly reduce the scope of things which need to be made thread-safe.

[Osspial]

@willglynn Regardless, I think that custom event passing is a more flexible solution than closure passing. My big reason for that is it allows users to implement a much more flexible closure-passing system than we could by just exposing a closure API, while the same couldn't be done the other way around. By letting users define their own closures (or enums of closures) as events, they can send closures with more return types or parameters than we could expose through the Fn trait. That also means we don't have to keep track of some user-defined type within winit, as any custom types that need to be used in a closure can just be passed by the main, event-loop closure.

[sodiumjoe]

Speaking from almost complete ignorance here, but I wanted to throw out a suggestion.

1. Abstracting over the various backends is a very thorny, if not intractable problem.
2. An inverted model exists, in which core logic of a given application is written in rust as a library, then pulled into separate application packages that use the native platforms.
3. My personal experience with winit in trying to contribute to alacritty has been bewildering. I've had to navigate the stack of alacritty -> glutin -> winit -> core-foundation-rs -> rust-objc, and try to translate cocoa documentation up through that stack.

Is it conceivable to try to invert winit's model, so it's more of a set of ui toolkits for the various backends, rather than an abstraction over the various windowing apis? I'm not sure that even makes sense as an idea, or whether there are more granular abstractions that make sense to make over parts of the different backends enough to warrant them all being in a single project. I also don't know how this would interact with glutin.

Please don't take this as criticism of your great work!

[sodiumjoe]

Btw, here's some discussion about inverting alacritty's model as well as a wip branch.

[francesca64]

I'm also in favor of windows being send+sync and the event loop being neither, since... that's what we do now, and it seems fine. However, there's something much more controversial I want to throw out there - should we rename EventsLoop to EventLoop?

As for messages vs. closures, I'm not very opinionated about that, and I'll be the first to admit I'm not very good at imagining APIs before using them. The snippet in #231 (comment) looks the most like what I'd expect, though that's partially because of aesthetics and familiarity.

I understand the ControlFlow of @Osspial's design much more now than I did initially, and I think it's a good amount of power to have. My only concern is making usage clear, though given the current weirdness of the poll_events/run_forever dichotomy, it could likely only be an improvement.

Whatever design we use, I want to move forward on this; I've been wanting to do more to attract contributors/maintainers, but it seems like a good idea to wait to do that until after we tear up the floorboards. Plus, I'm a bit tired of poll_events spinning my CPU...

@Xaeroxe @icefoxen it would also be great to hear from both of you (sorry that this situation is sorta hard to follow, but the gist is that we want to change from "your code calls into winit" to "winit calls into your code" for better platform coherence/etc).

@sodiumjoe

Speaking from almost complete ignorance here

You'll be disappointed to discover that none of us are actually experts on creating cross-platform window/event abstractions. It's winit's best kept secret.

Abstracting over the various backends is a very thorny, if not intractable problem.

For as fragmented as winit's development has been, it does a pretty good job at this IMO. Of course, I don't doubt there are problems (and big ones at that), but I have a much easier time understanding what you mean when provided with concrete/specific examples.

As for your overall concept, I don't have the time or energy to investigate the viability and merits of it. I'm having a hard time visualizing what it would be like in practice, but I will agree that there are holes in the stack that need to be addressed somewhere.

[jwilm]

I'm currently trying to digest the full implications of this proposal, and I have a question. Is the events loop thread expected to do draw synchronously to windows? This seems desirable for actions like resizing and undesirable for normal usage where draw time is high.

[Osspial]

@jwilm This redesign makes it easier for the event loop thread to draw synchronously to windows, but it's by no means required. I don't think it negatively impacts normal redraws, as the currently encouraged design has programmers blocking the event loop to redraw on a normal update, which would still be happening here.

@sodiumjoe Correct me if I'm reading this wrong, but it sounds like you're suggesting winit be able to handle native OS widgets, right? I'm not opposed to that in principal, but native widgets are... complicated, in ways that make it pretty difficult for winit to handle properly without heavily expanding its scope (see #446). This is mostly Linux's fault, as it doesn't really have a native widget API in the way OSX and Windows do.

[Osspial]

@francesca64 @sodiumjoe I'm pretty strongly opposed to a total inversion of winit's control flow, as is done in the snippet from #231, for a couple reasons:

• There's no real technical reason to do so, when a closure can do everything a completely inverted control flow can.
• Requiring that a user implements a trait to use winit, and having winit provide initialization/destruction hooks with that trait, increases the API's surface area, obfuscates the code's control flow, and doesn't provide any benefits I can see over a closure-based design.

I'd like to clarify that second point using examples, but it can be difficult to do that without losing the forest for the trees, so to speak. Regardless, I'm not going to focus on the specifics of the API I present here, and rather look at the general issues I think you'd see with a totally inverted design.

Compare a placeholder totally inverted design here:

struct MyApp {
    window: winit::Window
}

impl winit::App for MyApp {
    fn init(windows: &mut EventsLoopWindows) -> MyApp {
        MyApp {
            window: winit::WindowBuilder::new()
                .with_title("Hello, Winit!")
                .build(windows)
        }
    }

    fn on_event(&mut self, event: winit::Event) -> winit::ControlFlow {
        match event {
            winit::Event::Closed => winit::ControlFlow::Break,
            // Process whatever other events you want to process.
            _ => winit::ControlFlow::Wait
        }
    }
}

fn main() {
    winit::Main::<MyApp>();
}

With a callback-based design, here:

fn main() {
    let events_loop = winit::EventsLoop::new();
    let window = winit::WindowBuilder::new()
        .with_title("Hello, Winit!")
        .build(&events_loop);

    events_loop.run_forever(|event| {
        match event {
            winit::Event::Closed => winit::ControlFlow::Break,
            // Process whatever other events you want to process.
            _ => winit::ControlFlow::Wait
        }
    });
}

The totally inverted design requires a user to:

• Create a custom struct, in which to place the windows.
• Use and understand the API and guarantees of a new trait.
• Assume winit upholds guarantees described by the documentation (specifically, that init will only be called once), and keep those guarantees in mind while designing their code.

A closure-based avoids the increased API surface from the first two points, and doesn't require users juggle API the guarantees from the third in their head by making them implicit to the structure of the code. Users don't need to assume that winit will call init, or that init will only be called once. Instead, they can look at the code, even while ignoring function names, and understand that the initialization code has to happen before the event loop runs because it's written before the event loop function is called.

I don't see how a completely inverted design can be made that works around those problems in as intuitive a way as closures already do.

[willglynn]

@Osspial fn main() needs to return entirely – pop itself completely off the call stack – in order to receive events on the web platform. Relaxing winit's current requirement that the user can keep their code on the stack simplifies and fixes bugs on most other platforms.

The scheme I outlined in #231 was designed around the idea that fn main() is not allowed to block, since… sometimes it isn't allowed to block. Having the user create a data structure of their choosing, transfer ownership to winit, and allow winit to control the lifecycle seemed like the way to go.

Is there an API you like better which allows winit the ability to return out of fn main()?

[Osspial]

@mtak- We've slowly been merging backends on the eventloop-2.0 branch. Would you be able to submit a PR for the iOS backend?

[nvzqz]

Since the main discussion seems to be happening here regarding these changes, I'll address #638 (comment) here.

The extension traits have been renamed to include the operating system name, which feels very awkward in my opinion since this isn't standard convention. It's also not documented in CHANGELOG.md. It would be worth adding since it's a breaking change.

[goddessfreya]

@nvzqz That does sound awkward, I too wonder why this is the case.

[nvzqz]

Not sure if this has been proposed before, but since some platforms may have multiple backends, it may be worth making winit types generic over a Backend trait. See #908.

[ryanisaacg]

Potential source of difficulty again with wasm: Instant is required in the Winit public API, but as far as I know you cannot obtain an Instant on web.

[cheako]

I'm confused!

Hardly anything is explained for the behavior, what happens in explicit detail, for winit::event_loop::ControlFlow. The docs should read like a man page in that, after reading, one should have the feeling of being able to write the entire event loop. Every conditional branch(if and match statements) should be explained, as should every other code block.

winit::event::Event would logically have:

1. Something to emit when polling: Update
2. Something to emit when a timer expires and obviously this needs to contain a handle to the timer that's expiring: Timer(T)

Consider redoing winit::event_loop::ControlFlow as a trait instance that has callbacks for set_timer(T, Instant), set_sleeping(), set_polling() and set_exiting().

Edit:
set_timer() needs to return a handle to the timer so it can be altered/canceled.
Edit: #1018

[cheako]

Rational:
Given that this event loop is implemented multiple times it's vastly consequential to avoid undefined behavior. Undefined behavior isn't just a collection of unexplained consequence, instead there is an entire field of both theory and practice with as much depth as any paradox.

Users of winit should know, unequivocally, what to expect from the event loop. If instead applications are coded to the probed behavior of the system they are developing on, this will lead to poor support for the other platforms. When behavior is discovered that's unexpected users of winit should know to seek help prior to continuing as a determination of feature or bug must be made and the documentation should be further refined in effect to remove ambiguity.

[goddessfreya]

@cheako Make a separate issue please.

[expikr]

My apologies in advance for digging up this topic, but I am wondering if this implementation using custom events might allow for high-precision timestamping of the exact time that the OS places a message into the app's queue?

Make it so that the very first thing the handler does upon unblocking is a query to the platform's performance counter, to ensure the absolute minimal latency error in the timestamp. Then just include the timestamp information in the custom event being forwarded.

This level of timestamp accuracy is only achievable with wait-based message loops. A polling loop will invariably create a beat pattern/frequency aliasing in the timestamp deltas for high frequency events.

It is not really an option to have the timestamping implemented by individual app developers further down the pipeline, since all the extra plumbings in the backend introduces timing variance that defeats the whole purpose of using the high-precision nano/microsecond timers.

Making high-precision timestamp available across the board instantly give Winit the market corner for latency benchmarking. No other frameworks offer anything like it -- SDL2 doesn't even have a functioning wait_event, and most other frameworks' wait-based message loops are just nominal abstractions over their polling based backend.