Define sufficient in-the-field experience #15
Conversation
This patch defines requirements for reaching Stage 4 as being satisfied by a sufficiently used native implementation, such as a JavaScript engine in a web browser in a "preview" mode. This patch does not encode the committee's current policy but rather defines a new policy.
|
As stated elsewhere, we object to this change, under the belief that text should not be in the main spec without the in the field experience provided by two stable browsers. |
|
Thanks for your quick reviews @msaboff @ljharb and @domenic . I apologize if these proposals came off as abrupt; I was trying to follow advice from @domenic to make a specific proposal to change the process document. I'm really happy that @msaboff and others wants to contribute to pushing the web forward, and hope we can come to a process that make as many implementers as possible capable of contributing in this sort of way. I'd be interested in hearing from more implementers in what they would feel most comfortable with. Although it's a change in written policy, this patch tries to encode what we've asked for in bringing several existing proposals to Stage 4 (or we've required less in some cases). cc @bterlson @tschneidereit |
|
If the policy should be that we wait for two stable browsers, I think it'd be great to write that specifically in the process document as well. We don't currently seem to be in agreement in the committee that that's the requirement. |
|
@domenic Could you elaborate on stable versus shipping. I'd like to understand what I think is your stricter criteria and the reasons behind it. |
|
In the past, we've disambiguated between some proposals requiring "unflagged" shipping implementations, and some being sufficient with "flagged" (I'm avoiding the word "stable" here, intentionally). I think that broadly allowing "unflagged" implementations to grant stage 4 to proposals would be a wildly reckless policy (which I think means I agree with @domenic here), however, I think there has been and will continue to be case-by-case scenarios where a proposal having "flagged" implementations is sufficient (which is where I think @domenic and I disagree). |
|
@msaboff I mean the feature should be shipped to the web at large, i.e. in a browser intended for consumers, in order to get the significant in the field experience we need before moving it into the main spec |
|
@ljharb Could you clarify more what you mean about the distinction here? Is this about times when there is web compatibility data that we want to get prior to Stage 4, or are there more reasons why you think we should wait for unflagged implementations? |
|
@domenic Why wouldn't we consider that a Chrome Canary or Safari Tech Preview being shipping to the web at large. Certainly previews like that contain features intended for consumers made available to get significant in field experience. Maybe my question is better understood in terms of SW release process. i.e. Do you see that shipping implementations should be alpha, beta or released products? My view is that these are beta quality due to the possibility that something might change. |
|
@littledan Thanks for continued effort in clearing up the process document. @ljharb I’d be interested in hearing more about the distinction as well. Does the shape or size of the feature have any weight at all? Does the amount of changes the proposal underwent on its way from Stage 1 to 3, or even within Stage 3 itself, matter? |
I don't think the populations involved are significant compared to the non-preview releases. They consist largely of developers or beta testers, not the "web at large" which uses stable releases.
Released products. We should not put something in stage 4 that might change. Thus, we should make sure that it successfully shipped to multiple browsers' stable populations without changing, before we advance it to stage 4. |
|
@littledan @mathiasbynens to clarify: i'm saying that there are some cases where we don't need to wait for unflagged implementations; i think in the general case, we do need to wait for it, because stage 4 should be for things that are not likely to change. Specific cases include static API methods that did not pose a reasonable web compat risk (like |
|
I agree with most of @ljharb's points but take a bit of a stronger stance because I just don't see the value of allowing something like Object.values to advance to stage 4 earlier. |
|
I really appreciate how @msaboff and his team have put in significant effort to implement new language features; Apple is being a great citizen here in pushing the web forward. As an intermediate policy, what if allowed Stage 4 with two complete implementations, one of which is shipping to many users? That one may give us the web compatibility evidence we need. I'm not sure if there are tons of other browsers lying around who are eager to ship Stage 3 features. |
|
By the way, in a previous thread, some objected to requiring the implementation to be a browser. What possible non-browser could we allow which would get sufficiently wide distribution to reach Stage 4, presumably wider distribution than a preview version of a web browser? |
|
I'd much rather address this problem:
That's the purpose of stage 3 in the current process. Why are we contemplating changing stage 4, when it appears the real issue is that people are not clear on the purpose of stage 3? If we need to do a re-branding where there's a "stage 3.5" where some browsers feel comfortable shipping, then maybe that's one way out of it. Or we could just move the "put it in the spec" step to a new stage 5, under the assumption that browsers would be willing to ship stage 4 features even in a 5-stage process. But we shouldn't lower the bar for putting things in the spec. |
I think you and I have agreement that we don't want to put something in stage 4 that might change. As an implementor, I don't want to put something in a released engine that might change. Therefore the reluctance to ship a stage 3 feature in a released product. Seems like a catch 22 or in CS terms, a deadlock. If stage 4 requires two released product implementations, this probably rules out JavaScriptCore as one of those implementations as historically Safari has shipped at most two "feature" releases a year. I want to aggressively implement new ES features and make them available in "beta" forms like Safari Technology Preview, but it isn't too likely they'll appear in a stable release before they make it to stage 4. |
Can you help me understand this a bit more? Pretend we didn't have official labels/stages at all. What is the constraint? Is it that Safari doesn't want to ship features to stable until they are also shipping to stable in two other browsers? In my mind, that's what "it isn't too likely they'll appear in a stable release before they make it to stage 4" means. Is that what it means to you too? |
There are two constraints. The first is shipping schedule. In 2017, we shipped 10.1 in March and 11.0 in September. Certainly the March release would not be too useful for a feature that was trying to get to stage 4 for ES2017. My recollection for the state of proposals around September is that there weren't too many that were at stage 3 by then and less when we had finalized what was in 11.0 many weeks before. The second is that we don't want to ship something that might change, that is a stage 3 proposal. It isn't that we would want to wait for two other browsers. We implement pre-stage 4 features, but it would be rare for us to ship such a feature in a stable release before stage 4.
That comment is mostly related to the above reasons. If we continue with two releases a year and ignoring that we don't want to ship a pre-stage 4 feature, we have very limited opportunity to be one of the implementations with a stage 3 feature. As I said, I want to aggressively implement new ES features and make them available in "beta" forms like Safari Technology Preview. If that doesn't provide what the committee wants for a stage 3 implementation there is little else we can do. |
My claim is that stage 3 vs. stage 4 is completely irrelevant to whether something might change. Something stops changing once it's shipped to stable in 2 or more browsers; that's what locks it in. If we lowered the requirements to stage 4 to be only preview/beta releases, then I think things would change just as much during nu-stage 4 as they do during current-stage 4. Which would mean that Safari would then choose not to ship nu-stage 4 features, if I understand them correctly. So we're right back where we started. |
I don't think that's historically true. Features have tweaks applied to them after they have reached stage 4. I think it's more about the barrier to change, in current stage 3 there can be quite significant pushback on a feature. On more than one occasion I have seen implementors request significant changes to a feature during stage 3. If the current status of discounting non-shipping engines remains then it seems likely that engines will just leave features in stage 3 limbo until 3-4 engines have complete implementations. Perhaps there is a misunderstanding of the concern at hand. If the concern is backwards incompatibility then I somewhat agree that beta releases may not be sufficient. The issue I'm more concerned with is that an implementor, say JSC, ships a stage 3 feature and another implementor later vetoes it. At that point JSC is stuck with this non-standard feature forever and has to figure out how new features should interact with the non-standard one. To me, by moving a feature to stage 4, it means that can't be undone without consensus, which no browser shipping the feature would want to agree to. It seems much easier to get consensus to undo an incompatible change than get an implementor to unship a feature. So I'm not particularly swayed by the beta release concern. |
I agree with you about the stages. If two implementations implement a stage 3 proposal with the same functionality, I believe it that should lock it in. I would add that even if those implementations are beta. Additionally, I'm comfortable at that point to move to stage 4 and have those implementations ship the feature in a stable version. @kmiller68 Gives greater details into why JSC doesn't want to ship a stage 3 feature in a stable release. |
That's exactly the intent of stage 3, so that sounds great to me!
What do you mean by vetoes the feature? That's supposed to generally happen during the stage 1 -> 2 transition. Are you concerned about something softer than a full "we don't think this should be in the spec?" (An entry criterion for stage 2.)
The same is true for stage 2 though.
I don't understand this point. Are you talking about nu-stage 4, where everyone is shipping only in preview releases, and the feature still reaches stage 4? Then that seems wrong; if it's only in preview releases, browsers can easily agree to un-ship them. Or are you talking about current-stage 4, where it's already shipping to stable in two browsers? In that case, yes, it's not going to go backward---which is precisely why I think the current definition of stage 4 is valuable. But that seems to imply you're not wanting to ship until we have a guaranteed anti-moving-backwards backstop in place via another browser shipping to stable, whereas above @msaboff said "It isn't that we would want to wait for two other browsers". Is it that you're wanting to wait for one other browser? In that case though, you'd be the second, and thus the two browsers requirement ends up working great. In either case though, I don't think this is too relevant, as undoing the decision to include the feature in some form would require consensus (to move back from stage 2 to 1), and undoing the decision to settle on the proposed specific shape of the feature and spec text would also require consensus (to move back from stage 3 to stage 2). So I think consensus will keep things just as stable and give you just as many guarantees during stages 2 and 3 as it does during 4.
No, it isn't locked in when it's in a preview release. Preview releases are malleable and change frequently. Things get locked in only when it's shipped to stable, as that's when backward-incompatible changes are costly. That's why we need to wait for shipping to stable before going to stage 4, because only by going to stable do we actually have such guaranteed lock-in. |
|
I take @msaboff to mean "should lock it in" as a signal to the committee: given two interoperable implementations in engines, the committee ought to be reluctant to make unnecessary changes (e.g., cosmetic changes). Whereas @domenic means "shipping in stable locks it in" to be a statement of pragmatic reality. As an implementer, I agree with @msaboff that I'd like completed implementations to be a signal to the committee that a spec should stop changing, if at all possible. |
|
Is there anything better we can do to better incentivize early implementers? For contentious features, early implementers take on a lot of risk as @msaboff and @kmiller68 have pointed out, for little gain. But someone has to be first. |
I agree in general, preview releases are malleable and change frequently. For Safari Tech Preview that is at the macro level. In STP, individual features typical change due to filling out the feature or due to bug fixes. That is certainly the case for JSC development, as we intend that what goes in to an STP will make it to the shipping version.
Although I may agree that we might be talking about stability in differing terms, I don't think we want any distinction based on implementors practices ensconced in our process. Even with such distinctions, @domenic, as well as others, want a feature to be available to a wider audience. As I spelled out earlier, I doubt that new features in a released Safari would happen in time to fulfill that desire. |
|
Riffing on Dan's last question, I'm curious about the following: if we added a "stage 5" to the process, and left stage 4 completely unchanged except we moved the "merge the pull request into tc39/ecma262" into stage 5, would Safari still be comfortable shipping to stable during stage 4? Or would Safari now require the spec to advance to stage 5 before shipping to stable? |
|
@domenic The answer in #16 is an unequivocal "no". I don't really see a big difference between putting something in the spec slightly sooner or later if we have other good signals (implementation feedback, buy-in from multiple implementers). We've changed things in the final spec before in significant ways later, such as adding subclassable builtins in ES2015. |
|
Closing in favor of the current process document, with its deliberate ambiguity and lack of a requirement for two shipping implementations or qualification of which kinds of implementations count. |
This patch defines requirements for reaching Stage 4 as being satisfied
by a sufficiently used native implementation, such as a JavaScript engine
in a web browser in a "preview" mode.
This patch does not encode the committee's current policy but
rather defines a new policy.