Hot reloading workflows

[guybedford]

The previous version of SystemJS had a separate wrapper project (https://github.com/alexisvincent/systemjs-hot-reloader), that implemented full hot reloading support.

The primitives for hot reloading in SystemJS are the trace API, and the registry delete API, both of which are also available in SystemJS 4.x.

Fleshing out these workflows could be done through the extras model here to define the patterns for a hot reloading API / protocol, or it could be done in a separate project. But it would definitely be worthwhile and useful work.

For example, RollupJS currently has no simple hot-reloading workflows, where SystemJS + preserveModules in Rollup would be a nice standard hot reloading process there.

Please do join this discussion if you are interested in this topic further.

[guybedford]

//cc @lukastaegert @LarsDenBakker

[lewisl9029]

Hi, I'm currently experimenting with using SystemJS in an attempt to implement a hot reloading workflow that doesn't require any build steps.

I've played around a bit with the onload hook and the System.delete API but haven't really been able to figure out how to piece things together to go from the id of a changed module to a list of modules that depend on the changed module.

Wondering if you could point me in the right direction?

I'm guessing the idea is to use the onload hook to build a dependency graph somehow, but I couldn't figure out how to do that given the argument to the hook is just the url of the current module being loaded. In addition to that, wouldn't we at least need the url of the module that's importing it as well in order to build the graph? Maybe there's some SystemJS API I could use to get that from within the hook?

Thanks!

[guybedford]

@lewisl9029 thanks for posting your question here - yes we need dependency information :) Let me see if I can put together a PR here.

The other thing to handle carefully is the error cases, and hot reloading dev workflows in particular can hit up against all those edge cases too, so that may take some back and fourth as well.

[LarsDenBakker]

Sorry I missed the CC here. This is pretty interesting for us as well. It seems like implementing hot reloading in system js would be easier than implementing it with es modules.

[lukastaegert]

Definitely! Especially since you cannot just mess with the module loader in ESM. Also very interested in any progress here from Rollup side as there is a lot of interest in establishing HMR workflows.

[LarsDenBakker]

@lewisl9029 I didn't see you also started working on this, sorry about that. do you have this pushed anywhere? Interested to see the approach, maybe we can merge our work.

[lewisl9029]

@LarsDenBakker The approach I took was fairly use-case specific and probably very naive, but it's been working adequately for me so far. I've extracted the relevant parts out into a demo project here with some documentation: https://github.com/lewisl9029/buildless-hot-reload-demo

Feel free to give it a spin and take whatever you find is useful.

I took a look at your PR and our approach feels very similar. The main difference seems to be my approach involves deleting every dependent and dependents of those dependents until the root module, and then reloading and re-initializing the root module: https://github.com/lewisl9029/buildless-hot-reload-demo/blob/master/dev.js#L48, but your approach seems to be to simply change references of the immediate dependents somehow? https://github.com/systemjs/systemjs/pull/2014/files#diff-a2feb6cf203732e761ecbc83395dcbd8R41

I'm not familiar enough with the internals of SystemJS to fully grasp the practical implications here, but I'd be curious to find out for instance how a React app would know to re-render when a reference to a module is changed using this approach (I'm guessing there's probably some webpack HMR like api like module.hot.accept for users to act on modules getting replaced?). The approach itself does look a lot more elegant and probably much more performant than what I was doing though.

Also, I struggled a bit to figure out how to handle reloading for dynamically imported modules. I wrote about it in detail here: https://github.com/lewisl9029/buildless-hot-reload-demo#hot-reloading-for-dynamically-loaded-modules. Would love to hear if people here have any thoughts.

I'll try to take some time either tomorrow or next weekend to migrate my project over to this new reload-extra to get a better feel for how it works in practice and report back on my experience.

Thank you for working on this! Really love that this is finally being added as an officially supported extension.

[guybedford]

I've merged the update function adjustments for hot reloading in #2020, making it easier to manage reloading by pushing bindings updates without having to reload parents.

[LarsDenBakker]

Thanks. Still planning to wrap this one up, I was almost there. Sorry for the delay

[bhauman]

Just a note: providing an onreload() event that developers can register listeners to can help during development. This prevents needing to reload all the modules to the root as you can have the reload-listener do some bookeeping and then call React/render etc. This can lead to a very stable fast reloading workflow.

[guybedford]

@jkrems was mentioning that import.meta.hot is now a common pattern for hot reloading. It sounds like implementing whatever the emerging convention is along these lines would be ideal for SystemJS.

That hot reloading dev server project is still just sitting for someone to pick it for fame and glory or to sponsor work on!

If anyone has resources on the current import.meta.hot api details that would help to track here too.

[joeldenning]

I might have some time next week to work on this, but have never heard of import.meta.hot. I know about webpack's approach with module.hot.accept(), and used to use the old SystemJS hot reloader. However, I wouldn't know where to start with a new version.

My (probably quite naive) first attempt at it would be the following:

1. Provide an API for reloading a module, perhaps System.reload(System.resolve('module-name'))
2. The reload api would refetch and execute the the code at the same URL from before, replacing the namespace object in the registry with the new one.
3. All linked setters would be called with the updated namespace object.
4. Add a bound hot function to the module context, that automatically hot reloads the current module.

This approach leaves SystemJS as the thing fetching and instantiating hot reloaded modules. It also only supports refetching the entire module, instead of dynamically executing code that is sent over the web socket (which is how webpack does hot reloading).

[LarsDenBakker]

You can check out https://github.com/pikapkg/esm-hmr for the import.meta.hot.

We are currently working on implementing it in modernweb-dev/web#685

Sorry I never got around to finishing my work on the branch... meanwhile a lot of other projects demanded my attention.

[joeldenning]

Thanks, that link is helpful. The thing I'm struggling to wrap my mind around here is whether a systemjs hot reloader would require full integration into how the modules are hosted. Reading through pika pack's esm-hmr documentation, it seems like following that API would very much require full integration into how the modules are hosted.

If systemjs hot reloading requires using a special web server, I'm actually less excited about the feature, as that likely precludes using the vast majority of all local development tooling (in particular, webpack-dev-server and rollup's dev server). The reason I thought of the approach above is that it only requires an empty web socket push notification to trigger the hot reload, instead of requiring the web socket data to be the full module in a specific format.

Another way of putting this is that pika has to work around its lack of control over the browser module registry, but that with SystemJS we do not have that limitation. In SystemJS, we can modify everything about a module namespace object, including fully replacing its live bindings.

[rixo]

From what I understand, the approach you want is basically what rollup-plugin-hot is doing, so you might find some inspiration in there. It is using SystemJS at full capacity, with Lars' reload branch, to pilot HMR from the browser. Even if this branch is unfinished, I've never had a glitch with it, or something missing for my needs.

Over time I've added some serving capacity and stuff to the included dev server but those are comfort options, what's needed for HMR is really just notifications of what has changed, the reload logic takes place in the browser. The Node code in lib is mainly Rollup integration and fluff; the meat of the HMR server is broadcasting change messages.

On the client, the "hot API" is mainly implemented in this file. The crux of the HMR client to track the dependency graph (what Snowpack does on the server, but SystemJS allows to do client-side) and call reload on the right modules in the right order, according to the hot API spec.

Regarding the "hot" / import.meta.hot API, I strongly suggest that you align with something close to this. It is essentially the same surface API as esm-hmr that Snowpack and Vite implements. The main difference is that rollup-plugin-hot updates dispose / accept handlers with each updates while Snowpack reruns the first ones forever (I think Vite does the same as rollup-plugin-hot).

This limited API has served me very well to support Svelte HMR, including advanced "skip unneeded updates" scenarios thanks to the bubbled argument in the accept handler (easy to implement once you have the bigger thing; also supported by Snowpack now). The beforeUpdate / afterUpdate global hooks also enable tighter integration for HMR-aware tooling. For example, I'm working on a Storybook like project, and it uses those hooks to restore the scroll position after an update.

Hope this helps. Happy to answer any questions if you have some.