refactor(develop): expose APIs to suspend/resume webpack watching to gatsby internals

[pieh]

Description

This pull request just exposes ability to pause/resume webpack watching, but doesn't make any use of it yet. Main reason to split this to its own PR is the hairy TS situation:

• missing types (upstream) on webpack-dev-middleware - context is skipped there, but according to clarification if context is public or private API webpack/webpack-dev-middleware#656 (comment) context is public API and needs documentation
• missing types (upstream) on webpack.Watching - feat(watching): allow suspend/resume compilation webpack/webpack#9214 added suspend/resume to Watching but those are missing from @types/webpack

Related Issues

Tracking PR for data-driven code-splitting - #24903

[CH9864]

[blainekasten]

Code looks good assuming it works as you want. I had a question about needing the returned key to be a function to access the webpack watching value. But if it works as you want, then I'm good with this!

[ascorbic]

This is great. I'm really glad that you've unearthed this. It's going to be very useful.

[sidharthachatterjee]

Nice work!

[wardpeet]

@pieh code looks great! I don't see much value in abstracting this away from the larger PR as this doesn't show me usage of the functionality so I won't be able to see test failing, ... cause it's unused.

I know I can go to the larger PR but what's the point of abstracting it away?

[pieh]

@wardpeet the reason for exposing those methods is to be able to control when webpack recompiles (and emit hot update).

Current (master) method of running requires-writer in long running develop process is to listen to various actions and then call debounced write call. While not ideal, it's also not that problematic right now, because it just reacts to CREATE_PAGE (and similar) actions because it mostly just needs list of template components and templates remain mostly "fixed" and that's not changing over time that much (but especially in larger sites where createPages can take a while - this cause to rewrite (a)sync-requires multiple times during same createPages run.

Query modules to work need to add things to (a)sync-requires - this happen during query running. Because query running can also take a lot of time, this also mean rerunning it multiple times but this is worse than createPages because modules are "garbage collected" if not used, and because I do clean modules when query is marked as dirty, meaning that those extra runs might output broken code with missing modules that will be added eventually, but becasue of lot of asynchronity in all of this this can take a while and leave browser session broken with runtime errors overlays).

The follow up PR for this will look something like 2cc9227...df0c7fd (I'm still having some issues with e2e tests, so there likely be still some changes, but the idea of those changes is:
"don't run something if something we rely on is still running or we know that will run soon" and relationship between systems I defined as follow there:

• schema rebuild (doesn't depend on anything from this list, so it will seemingly run anytime still)
• createPages run (depend on schema rebuild)
• query running (depend on both schema rebuild and createPages)
• requires writer (depend on createPages) <- this is in master - with query modules this will depend on both createPages AND query running
• webpack (re)compilation (depend on requires writer)
• page data flush (depend on query running and webpack)

For this model to work we need to be able to control when things are running and webpack is only thing from the list that we don't control

Also just note that this follow up PR will be eventually made redundant with more robust solution in form of state machine (ideally I would wait for it, but it's still quite some time before it lands fully)

[wardpeet]

Looking great!