Normative: Top-level Await

[codehag]

This PR introduces the necessary changes to the spec for top-level await

I hit a couple of conflicts (you will notice while reviewing) and I am not sure if that is the new style of the spec. I tried to resolve most of them but I sometimes wasn't sure. If the original is intended, let me know and I will update the spec accordingly.

Since the last meeting, one editorial change was merged in, introducing prose around what happens when an error is thrown during async evaluation. We are discussing a further clarification with @guybedford , but it doesn't affect the normative spec text. It will hopefully make it more readable. What is the best way to do this? Should we introduce that after this proposal is merged in?

[jmdyck]

Rather than make a large number of 'suggested changes', I've submitted a PR against codehag:proposal-top-level-await.

[codehag]

We have a pr ready to help readability of the spec, should we just merge that in with this batch? tc39/proposal-top-level-await#179

[codehag]

@jmdyck I am unable to find your pr, against my fork or the main repo; do you have a link?

[nicolo-ribaudo]

@codehag I think it's codehag#4

[codehag]

@nicolo-ribaudo ahhhh thanks!

[jmdyck]

@codehag I think it's codehag#4

Yup!

[codehag]

I haven't gotten a response here about the editorial change, so I will layer it in.

[syg]

First pass review. I haven't fully worked through the example yet.

[jmdyck]

I'm leery of this, as I'm pretty sure there's nowhere in the current spec that behavior depends on the order in which fields were set to some value.

[bakkot]

We discussed this briefly in the editor call today. I also don't love it, but haven't yet thought of an alternative I like more, and there's nothing technically wrong with this approach. (A global counter would work, for example, but isn't exactly cleaner.)

The closest thing we currently have is the step in OrdinaryOwnPropertyKeys which reads

For each own property key P of O such that Type(P) is String and P is not an array index, in ascending chronological order of property creation, do

[guybedford]

It's a novel construct yes, but it might be ok. If there are further concerns here, an alternative to explicitly calling out field ordering might be to more generally reference "the original DFS visitor ordering via InnerModuleEvaluation that initiated the execution of this module". I've posted a possible adjustment to this wording in tc39/proposal-top-level-await#181. I can see either approach being fine here though. Further feedback welcome.

[codehag]

I prefer the current wording or something similar, as it gets the intention across more clearly.

I agree -- it is the first time we do this and I was a bit worried about it when we first introduced it. I am open to ways we might do this while still maintaining a clear model for readers.

[syg]

Yes, the main reason I'm okay with this current wording is because as @bakkot says: an explicit "implementation" of the intent via e.g. a global counter just isn't any clearer.

[syg]

Second pass. I think mostly fine, but the example is confused in some parts.

[syg]

These tables are pretty wide and I can't get a horizontal scrollbar in my local rendering on Chrome for some reason.

[syg]

The example is confused. The first paragraph talks about E transitioning to evaluated, suggesting it's not async. But here it's being async evaluated. I guess the first paragraph is wrong?

[codehag]

Yes, I think the previous paragraph was wrong and E should have been evaluating-async at that point. It only transitions here.

[bakkot]

Force-pushed to resolve conflicts, mostly with #545, and also added a commit adopting the format introduced by #545.

[jmdyck]

Now that #2439 has merged, should ExecuteAsyncModule and FinishDynamicImport use abstract closures (to pass to CreateBuiltinFunction)?

[bakkot]

Yeah, probably. I'll do that shortly after discussing my review comments, to avoid conflicts.

[bakkot]

I think this works. Just had some comments on the details.

[bakkot]

I'm not 100% convinced this is sensible according to the current definition of Contains, but I see that we do something similar with "Contains super" in the Early Errors for script and module bodies, so I guess there's precedent.

(Note to self: if we decide this is sensible, it can probably be adopted by class static blocks as well.)

[syg]

Seems like a worthwhile shorthand to define. I'm leaning towards allow this here and elsewhere.

[jmdyck]

I see that we do something similar with "Contains super" in the Early Errors for script and module bodies, so I guess there's precedent.

E.g. It is a Syntax Error if |ModuleItemList| Contains `super`. But that's an example of omitting the is *true*, so isn't a precedent for the usage here. I.e.
Let _async_ be _body_ Contains `await`.
isn't an abbreviation of:
Let _async_ be _body_ Contains `await` is *true*.

---

If you're talking about a Contains-invocation supplying the value for a Let, then the only precedent is in the default definition for Contains:
Let _contained_ be the result of _child_ Contains _symbol_.
I think the result of is there to prevent a garden-path parse. E.g., if it were
Let _contained_ be _child_ Contains _symbol_.
then someone might read
Let _contained_ be _child_
as a unit, and then be confused when they hit Contains. But if you think there's a low risk of confusion, then dropping the result of is okay. (Most Contains-invocations don't have it.)

[bakkot]

"async module" isn't defined, and it's not 100% clear what it means here. There's also a use of "asynchronous module" below.

We should maybe dfn the definition of the [[Async]] field which says something like A module for which this field is *true* is called an <dfn>async module</dfn>. (assuming that's an accurate definition, anyway).

[syg]

In this particular instance "async module" seems to mean individually async, i.e. a module whose [[Async]] field is true. Colloquially "async module" probably doesn't mean just that but also means if there's a module whose [[Async]] field is true in the transitive closure of dependencies.

For here I'd replace "async module" with either "individually async module" or "module whose [[Async]] field is true".

[guybedford]

There is definitely room for confusion between an individually async module, and a module with async dependencies.

Perhaps we can avoid the term entirely and use something like "or it is a module containing a top-level await that has been executed and is pending top-level completion".

[syg]

Concrete suggestion after chatting on the editor call. The root cause here is that [[Async]] is a poorly named field. Let's rename it to [[HasTLA]] (and expand the TLA acronym in the definition), and replace this use of "async module" with "module whose [[HasTLA]] field is true (i.e. containing a top-level await)".

There's another occurrence of "async module", not yet clear what to replace that with.

[bakkot]

Just to check my understand, are all of these statements true?

• All of the modules in a given strongly connected component have the same [[CycleRoot]]
• All of the modules in a given strongly connected component have the same [[DFSAncestorIndex]]
• Said [[DFSAncestorIndex]] is equal to [[CycleRoot]].[[DFSIndex]]
• m.[[TopLevelCapability]] being non-empty implies m.[[CycleRoot]] = m (though the reverse implication does not hold)

[guybedford]

@bakkot yes those statements are all correct.

[codehag]

Hey folks, I’m out sick at the moment — feel free to make changes here or I will get to it but it will be some time.

…

On Thursday, July 22, 2021, Shu-yu Guo ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In spec.html <#2408 (comment)>: > </td> <td> - Initially ~unlinked~. Transitions to ~linking~, ~linked~, ~evaluating~, ~evaluated~ (in that order) as the module progresses throughout its lifecycle. + Initially ~unlinked~. Transitions to ~linking~, ~linked~, ~evaluating~, possibly ~evaluating-async~, ~evaluated~ (in that order) as the module progresses throughout its lifecycle. ~evaluating-async~ indicates this module is queued to execute on completion of its async dependencies or it is an async module that has been executed and is pending top-level completion. In this particular instance "async module" seems to mean individually async, i.e. a module whose [[Async]] field is true. Colloquially "async module" probably doesn't mean just that but also means if there's a module whose [[Async]] field is true in the transitive closure of dependencies. For here I'd replace "async module" with either "individually async module" or "module whose [[Async]] field is true". ------------------------------ In spec.html <#2408 (comment)>: > + <td> + [[AsyncEvaluation]] + </td> + <td> + Boolean + </td> + <td> + Whether this module is either itself asynchronous or has an asynchronous dependency. Note: The order in which this field is set is used to order queued executions, see <emu-xref href="#sec-async-module-execution-fulfilled"></emu-xref>. + </td> + </tr> + <tr> + <td> + [[TopLevelCapability]] + </td> + <td> + Promise Capability | ~empty~ +1 ------------------------------ In spec.html <#2408 (comment)>: > </td> <td> - Evaluate the module's code within its execution context. + Evaluate the module's code within its execution context. If this module has *true* in [[Async]], then a Promise Capability is passed as an argument, and the method is expected to resolve or reject the given capability. In this case, the method must not throw an exception, but instead reject the Promise Capability if necessary. +1 ------------------------------ In spec.html <#2408 (comment)>: > + <td> + List of Cyclic Module Records + </td> + <td> + If this module or a dependency has [[Async]] *true*, and execution is in progress, this tracks the parent importers of this module for the top-level execution job. These parent modules will not start executing before this module has successfully completed execution. + </td> + </tr> + <tr> + <td> + [[PendingAsyncDependencies]] + </td> + <td> + Integer | ~empty~ + </td> + <td> + This tracks the number of asynchronous dependency modules remaining to execute for this module if it has any asynchronous dependencies. A module with async dependencies will be executed when this field reaches 0 and there are no execution errors. +1 ------------------------------ In spec.html <#2408 (comment)>: > 1. Return _index_. 1. Assert: _module_.[[Status]] is ~unlinked~. 1. Set _module_.[[Status]] to ~linking~. 1. Set _module_.[[DFSIndex]] to _index_. 1. Set _module_.[[DFSAncestorIndex]] to _index_. 1. Set _index_ to _index_ + 1. 1. Append _module_ to _stack_. - 1. For each String _required_ of _module_.[[RequestedModules]], do + 1. For each String _required_ that is an element of _module_.[[RequestedModules]], do +1 ------------------------------ In spec.html <#2408 (comment)>: > 1. Set _index_ to _index_ + 1. 1. Append _module_ to _stack_. - 1. For each String _required_ of _module_.[[RequestedModules]], do + 1. For each String _required_ that is an element of _module_.[[RequestedModules]], do +1 ------------------------------ In spec.html <#2408 (comment)>: > </dl> <emu-alg> 1. Assert: This call to Evaluate is not happening at the same time as another call to Evaluate within the surrounding agent. - 1. Assert: _module_.[[Status]] is ~linked~ or ~evaluated~. + 1. Assert: _module_.[[Status]] is ~linked~, ~evaluating-async~, or ~evaluated~. + 1. If _module_.[[Status]] is ~evaluating-async~ or ~evaluated~, set _module_ to _module_.[[CycleRoot]]. In light of the discussion, we're leaving this line as-is, yeah? ------------------------------ In spec.html <#2408 (comment)>: > @@ -25493,11 +25738,10 @@ <h1>Example Cyclic Module Record Graphs</h1> <img alt="A module graph in which module A depends on module B, and module B depends on module C" width="121" height="211" src="img/module-graph-simple.svg"> </emu-figure> - <p>Let's first assume that there are no error conditions. When a host first calls _A_.Link(), this will complete successfully by assumption, and recursively link modules _B_ and _C_ as well, such that _A_.[[Status]] = _B_.[[Status]] = _C_.[[Status]] = ~linked~. This preparatory step can be performed at any time. Later, when the host is ready to incur any possible side effects of the modules, it can call _A_.Evaluate(), which will complete successfully (again by assumption), recursively having evaluated first _C_ and then _B_. Each module's [[Status]] at this point will be ~evaluated~.</p> - + <p>Let's first assume that there are no error conditions. When a host first calls _A_.Link(), this will complete successfully by assumption, and recursively link modules _B_ and _C_ as well, such that _A_.[[Status]] = _B_.[[Status]] = _C_.[[Status]] = ~linked~. This preparatory step can be performed at any time. Later, when the host is ready to incur any possible side effects of the modules, it can call _A_.Evaluate(), which will complete successfully, returning a Promise resolving to undefined (again by assumption), recursively having evaluated first _C_ and then _B_. Each module's [[Status]] at this point will be ~evaluated~.</p> +1 ------------------------------ In spec.html <#2408 (comment)>: > @@ -25521,7 +25765,381 @@ <h1>Example Cyclic Module Record Graphs</h1> <p>Now consider a case where _A_ has an linking error; for example, it tries to import a binding from _C_ that does not exist. In that case, the above steps still occur, including the early return from the second call to InnerModuleLinking on _A_. However, once we unwind back to the original InnerModuleLinking on _A_, it fails during InitializeEnvironment, namely right after _C_.ResolveExport(). The thrown *SyntaxError* exception propagates up to _A_.Link, which resets all modules that are currently on its _stack_ (these are always exactly the modules that are still ~linking~). Hence both _A_ and _B_ become ~unlinked~. Note that _C_ is left as ~linked~.</p> - <p>Finally, consider a case where _A_ has an evaluation error; for example, its source code throws an exception. In that case, the evaluation-time analog of the above steps still occurs, including the early return from the second call to InnerModuleEvaluation on _A_. However, once we unwind back to the original InnerModuleEvaluation on _A_, it fails by assumption. The exception thrown propagates up to _A_.Evaluate(), which records the error in all modules that are currently on its _stack_ (i.e., the modules that are still ~evaluating~). Hence both _A_ and _B_ become ~evaluated~ and the exception is recorded in both _A_ and _B_'s [[EvaluationError]] fields, while _C_ is left as ~evaluated~ with no [[EvaluationError]].</p> + <p>Alternatively, consider a case where _A_ has an evaluation error; for example, its source code throws an exception. In that case, the evaluation-time analog of the above steps still occurs, including the early return from the second call to InnerModuleEvaluation on _A_. However, once we unwind back to the original InnerModuleEvaluation on _A_, it fails by assumption. The exception thrown propagates up to _A_.Evaluate(), which records the error in all modules that are currently on its _stack_ (i.e., the modules that are still ~evaluating~) as well as via [[AsyncParentModules]], which form a chain for modules which contain or depend on top-level `await` through the whole dependency graph through the AsyncModuleExecutionRejected algorithm. Hence both _A_ and _B_ become ~evaluated~ and the exception is recorded in both _A_ and _B_'s [[EvaluationError]] fields, while _C_ is left as ~evaluated~ with no [[EvaluationError]].</p> + + <p>Lastly, consider a module graph with a cycle, where all modules complete asynchronously:</p> + <emu-figure id="figure-module-graph-cycle-async" caption="An asynchronous cyclic module graph"> + <img alt="A module graph in which module A depends on module B and C, module B depends on module D, module C depends on module D and E, and module D depends on module A" width="241" height="211" src="img/module-graph-cycle-async.svg"> + </emu-figure> + <p>Linking happens as before, and all modules end up with [[Status]] set to ~linked~.</p> + + <p>Calling _A_.Evaluate() calls InnerModuleEvaluation on _A_, _B_, and _D_, which all transition to ~evaluating~. Then InnerModuleEvaluation is called on _A_ again, which is a no-op because it is already ~evaluating~. At this point, _D_.[[PendingAsyncDependencies]] is 0, so ExecuteAsyncModule(_D_) is called and we call Evaluate with a new PromiseCapability tracking the asynchronous execution of _D_. We unwind back to the InnerModuleEvaluation on _B_, setting _B_.[[PendingAsyncDependencies]] to 1 and _B_.[[AsyncEvaluation]] to *true*. We unwind back to the original InnerModuleEvaluation on _A_, setting _A_.[[PendingAsyncDependencies]] to 1. In the next iteration of the loop over _A_'s dependencies, we call InnerModuleEvaluation on _C_ and thus on _D_ (again a no-op) and _E_. As _E_ has no dependencies and is not part of a cycle, we call ExecuteAsyncModule(_E_) in the same manner as _D_ and _E_ is immediately removed from the stack. We unwind once more to the original InnerModuleEvaluation on _A_, setting _C_.[[AsyncEvaluation]] to *true*. Now we finish the loop over _A_'s dependencies, set _A_.[[AsyncEvaluation]] to *true*, and remove the entire strongly connected component from the stack, transitioning all of the modules to ~evaluating-async~ at once. at this point, the fields of the modules are as given in <emu-xref href="#table-module-graph-cycle-async-fields-1"></emu-xref>.</p> +1 ------------------------------ In spec.html <#2408 (comment)>: > + <emu-alg> + 1. If _module_.[[Status]] is ~evaluated~, then + 1. Assert: _module_.[[EvaluationError]] is not ~empty~. + 1. Return *undefined*. + 1. Assert: _module_.[[Status]] is ~evaluating-async~. + 1. Assert: _module_.[[AsyncEvaluation]] is *true*. + 1. Assert: _module_.[[EvaluationError]] is ~empty~. + 1. Set _module_.[[AsyncEvaluation]] to *false*. + 1. Set _module_.[[Status]] to ~evaluated~. + 1. If _module_.[[TopLevelCapability]] is not ~empty~, then + 1. Assert: _module_.[[CycleRoot]] is _module_. + 1. Perform ! Call(_module_.[[TopLevelCapability]].[[Resolve]], *undefined*, &laquo; *undefined* &raquo;). + 1. Let _execList_ be a new empty List. + 1. Perform ! GatherAsyncParentCompletions(_module_, _execList_). + 1. Let _sortedExecList_ be a List whose elements are the elements of _execList_, in the order in which they had their [[AsyncEvaluation]] fields set to *true* in InnerModuleEvaluation. + 1. Assert: All elements of _sortedExecList_ have their [[AsyncEvaluation]] field set to *true*, [[PendingAsyncDependencies]] field set to 0, and [[EvaluationError]] field set to *undefined*. +1 ------------------------------ In spec.html <#2408 (comment)>: > + 1. Perform ! AsyncModuleExecutionFulfilled(_module_). + 1. Return. + </emu-alg> + + <p>A CallAsyncModuleRejected function is an anonymous built-in function with a [[Module]] internal slot. When a CallAsyncModuleRejected function is called with argument _error_ it performs the following steps:</p> + <emu-alg> + 1. Let _f_ be the active function object. + 1. Let _module_ be _f_.[[Module]]. + 1. Perform ! AsyncModuleExecutionRejected(_module_, _error_). + 1. Return. + </emu-alg> + </emu-clause> + + <emu-clause id="sec-gather-async-parent-completions" type="abstract operation"> + <h1> + GatherAsyncParentCompletions ( GatherAvailableAncestors sgtm. ------------------------------ In spec.html <#2408 (comment)>: > + <td>1 (_D_)</td> + </tr> + <tr> + <th>_E_</th> + <td>4</td> + <td>4</td> + <td>~evaluated~</td> + <td>*true*</td> + <td>&laquo; _C_ &raquo;</td> + <td>0</td> + </tr> + </tbody> + </table> + </emu-table> + + <p>_D_ is next to finish (as it was the only module that was still executing). When that happens, AsyncModuleExecutionFulfilled is called again and _D_.[[Status]] is set to ~evaluated~. Then _B_.[[PendingAsyncDependencies]] is decremented to become 0, ExecuteAsyncModule is called on _B_, and it starts executing. _C_.[[PendingAsyncDependencies]] is also decremented to become 0, and _C_ starts executing. The fields of the updated modules are as given in <emu-xref href="#table-module-graph-cycle-async-fields-3"></emu-xref>.</p> Sure why not. ------------------------------ In spec.html <#2408 (comment)>: > + </thead> + <tbody> + <tr> + <th>_A_</th> + <td>0</td> + <td>0</td> + <td>~evaluated~</td> + <td>*true*</td> + <td>&laquo; &raquo;</td> + <td>0</td> + </tr> + </tbody> + </table> + </emu-table> + + <p>Alternatively, consider a failure case where _C_ fails execution and returns an error before _B_ has finished executing. When that happens, AsyncModuleExecutionRejected is called and _C_.[[Status]] is set to ~evaluated~. We then set the evaluation error of the module to the error returned by executing _C_. We then propagate this error to all of the AsyncParentModules by performing AsyncModuleExeutionRejected on each of them. The fields of the updated modules are as given in <emu-xref href="#table-module-graph-cycle-async-fields-7"></emu-xref>.</p> +1 ------------------------------ In spec.html <#2408 (comment)>: > 1. Assert: _requiredModule_.[[Status]] is ~evaluating~ if and only if _requiredModule_ is in _stack_. 1. If _requiredModule_.[[Status]] is ~evaluating~, then 1. Set _module_.[[DFSAncestorIndex]] to min(_module_.[[DFSAncestorIndex]], _requiredModule_.[[DFSAncestorIndex]]). - 1. Perform ? _module_.ExecuteModule(). + 1. Else, + 1. Set _requiredModule_ to _requiredModule_.[[CycleRoot]]. + 1. Assert: _requiredModule_.[[Status]] is ~evaluating-async~ or ~evaluated~. + 1. If _requiredModule_.[[EvaluationError]] is not ~empty~, return _requiredModule_.[[EvaluationError]]. + 1. If _requiredModule_.[[AsyncEvaluation]] is *true*, then + 1. Set _module_.[[PendingAsyncDependencies]] to _module_.[[PendingAsyncDependencies]] + 1. + 1. Append _module_ to _requiredModule_.[[AsyncParentModules]]. + 1. If _module_.[[PendingAsyncDependencies]] &gt; 0 or _module_.[[Async]] is *true*, then + 1. Assert: _module_.[[AsyncEvaluation]] is *false* and was never previously set to *true*. + 1. Set _module_.[[AsyncEvaluation]] to *true*. + 1. NOTE: The order in which [[AsyncEvaluation]] transitions to *true* is significant. (See <emu-xref href="#sec-async-module-execution-fulfilled"></emu-xref>). +1 — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#2408 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGNYEJ3DWTHMK3NJSM2YDZTTY5HSXANCNFSM4437SBQQ> .

-- r+

[bakkot]

Pushed a few commits resolving all of my comments except the "async module" one, because I didn't have a wording I liked to use there.

[codehag]

@bakkot looks good to me

[syg]

"async dependencies" -> "asynchronous dependencies"

changes addressed