Support full-duplex HTTP streaming

[louiscryan]

I’d like to ask that Fetch API explicitly supports full-duplex HTTP streaming. The API structure already logically allows for this, with the only missing component being request streams, which I understand are coming eventually. This was discussed informally before and has been proposed to the IETF in a form that is now explicitly supported by the HTTP2 specification (“A server can send a complete response prior to the client sending an entire request if the response does not depend on any portion of the request that has not been sent and received”).

If there are ecosystem concerns this behavior could be confined to HTTP2 transports only. I believe a number of proxies support this behavior today and when TLS is used would be oblivious to it anyway. I know a number of HTTP2 implementations already support this (nghttp2, Google) and there are probably others

This would give service implementers a clear roadmap for full duplex streaming in the browser over HTTP2 without waiting for draft-hirano-httpbis-websocket-over-http2 to come back from the dead.

[annevk]

This seems like a duplicate of #51 or #65 (one of which is probably a dupe of the other) unless I'm missing something.

[domenic]

No, full duplex streaming is not related to HTTP2 push. It can be implemented in HTTP 1.1 for example.

[annevk]

You mean through pipelining? I belive that's incompatible with the web.

[benjchristensen]

I too would like to see this explicitly supported to enable streams and bi-directional channels over HTTP/2 in a standardized way. I can provide more information on use cases if that would help.

[louiscryan]

Supporting for HTTP/1.1 might be an overreach though its worth discussing. For HTTP2 though we should be fine as it's already in spec.

For HTTP/1.1:
No pipelining is not what is meant here. Its literally a server that responds with payload before the client has sent the last bytes of the request. Servers (potentially out of spec ones) can do this today anyway and I'd be curious to know how much of the web that actually breaks . HTTP clients & proxies already have to tolerate full-duplex IO for Expect/Continue handling so it might not actually break all that much.

All that said I would be totally fine with HTTP2 only given that the web is rapidly moving in that direction anyway.

[domenic]

It's also important to note that this would be opt-in. fetch() is an explicit call where you talk to specific endpoints that you know the behavior of.

It's not clear to me what fetch() does with servers that today start sending the response body before the request is finished uploading, but it might already be supported. I don't remember anything in the response body stream parts of the spec that buffer until the request is finished.

[annevk]

So, given that HTTP/2 already requires support for full-duplex, what is there to do for Fetch?

[louiscryan]

I think there are a couple of things...

• Document that this is explicitly supported so folks know they can rely on it
• Have this case be included in the test suite (assuming there is one)

[domenic]

Also:

• Finish speccing request body streams
• Make sure there's nothing in the spec that breaks this (e.g. buffering until the request body is finished) or makes assumptions contrary to it (e.g. assumes that response body bytes are received after request body bytes are finished uploading).

[mnot]

Just some background / random thoughts FWIW --

Using "full duplex" here is at best confusing.

It's true that a HTTP/2 stream is fully duplexed, but that's completely separated from how HTTP uses the streaming layer.

I think what's being asked for is the ability to consume the response body before the request has been completely sent.

Both HTTP/1 and HTTP/2 allow that, although from what we saw at the time, some implementations -- including browsers -- of HTTP/1 have a harder time doing that in practice. HTTP/2's multiplexing makes it easier to write an implementation that allows this.

So, I think it's completely reasonable to make this explicit in Fetch, but you might want to check and see how well it's supported in browsers first.

[louiscryan]

Mark - thanks for the notes.

You are correct about what is being asked for, the ability to consume the response before the request has been completely produced by the caller (as opposed to fully produced but buffered in some IO layer)

With respect to browser implementations, they may or may not support this in their internals but there has never been a way to express this behavior in XHR because of the API shape. Addressing the API must come first, then implementers can support as desired. Hopefully they see value in doing so.

As for HTTP/1.1 support for this behavior: I've found it impossible to find anything which explicitly says this is allowed anywhere. If you have a canonical reference that would be helpful. The best I can say is that nothing says it's not allowed which leads to no shortage of confusion when I talk to people about this. If there isn't anything explicit could it be added?

[grmocg]

There are a couple of separable parts here:

1. Beginning to interpret a response before a request is complete
2. Interpreting/seeing parts of a response before the whole response has been received.

Browsers have supported no. 2 indirectly with various data formats (anything that can be interpreted before the whole entity has been received, e.g. progressive images, videos, etc.), but not for javascript.

The request here would be to make this functionality visible for use in the API if the browser supported it.
It'd be best if the browser enabled the API entry point if both 1,2 were true, but enabling it for 2 is already a large potential win.

[domenic]

2 is already in the spec. That doesn't seem related to the OP's request for bidirectional communication.

[grmocg]

You're right. Serves me right for being out of date with progress.

[annevk]

If we spot a redirect, would we stop sending the request to the original URL? Or continue sending it there while also sending it to the new URL? I'm guessing the latter. Should probably walk through the algorithms again to see if that all works, but I think it does (stream API support is ongoing).

I'm a little wary of requiring things already required upstream. Testing it seems perfectly fine, we often test upstream requirements, but that's a distinct project, see the README. I'm sure they'd appreciate your help.

[mnot]

In H1 you'd need to continue sending the request to the origin if you want to continue to use the connection for subsequent requests.

In H2, you can send a RST_STREAM and continue on your merry way.

[annevk]

The client can? That's an optional behavior for redirects or recommended? Redirects don't appear to be mentioned in https://tools.ietf.org/html/rfc7540.

[mnot]

re: H1 - see http://httpwg.org/specs/rfc7230.html#persistent.connections

re: H2 -- optional, but sensible. We don't spec out every little optimisation, although maybe we should start a "Cool HTTP/2 tricks" draft...

[annevk]

Well, you can say it's an optimization, but one could also argue that the server wants to see the full request even though it redirects. There's nothing about redirect semantics that implies the request is meaningless, is there?

[mnot]

301, 302, 307 and 308 literally mean "the thing you're looking for is over there." The request is not considered completed by those status codes, so this should be safe to do.

303 is a bit different; although its use is more rare (outside of Semantic Web circles). Although if the server is processing the request, they probably really won't want to respond until they have the whole thing (for this reason as well as many others). And of course the client can choose not to RST_STREAM it.

[annevk]

Okay, that optimization for 301/302/307/308 is worth mentioning.

[annevk]

So another problem here is that step 17 of main fetch assumes that by the time we receive a response (all its headers, not necessarily the body), the request has been fully transmitted. It's important that step 17 does not do that notification earlier (due to redirects), but we could maybe change it to do it later. We'd have to figure out what an acceptable signal would be for doing it later (that is, what fully transmitted means in this new world).

[wenbozhu]

An HTTP/1.1 client can presumably terminate a streamed request by producing a last chunk for the purpose of reusing the same connection. I can't see how HTTP/1.1 makes anything harder (in theory) except that many implementations probably don't have much interest to touch their HTTP/1.1 code.

For browsers, is it common to change the run-time behavior based on the HTTP protocol version? The algorithm would be simpler if early-response data is delivered to the app regardless of the negotiated protocol ... or not?

[mnot]

Many HTTP/1 clients don't use chunked encoding in requests, because so many servers require Content-Length.

[annevk]

Yeah, I think the web basically requires Content-Length for request bodies. We'll soon let developers opt out of that using streams with fetch(), but I think we'll continue to require that behavior for legacy paths, such as <form> or simply when not using streams.

[wenbozhu]

When streaming requests from browsers, unlikely the app will be able to provide the C-L. Sending a half-close to terminate the request body is probably not a safe thing to do either, as servers may accidentally abort the request.

[annevk]

Correct, we don't have a design for streaming that allows for setting Content-Length. Servers will have to update.

[annevk]

Does anyone here know the answer to #229 (comment)? What signal do browsers use to determine a request body being "fully" transmitted? I always thought it was a response coming in (and in terms of events with a simple server setup that appears to be what is happening), but obviously that does not work for the scenario where a response is transmitted early.

[annevk]

Has anyone created tests around this? It seems unlikely the server from web-platform-tests can handle this.

[annevk]

I guess in the case the response comes while the request has not been fully written to the socket (the stream is not yet closed or errored), we'll need to wait with the notification until it has (in parallel). Unfortunately we cannot use TCP ACK for the request in that scenario: https://bugzilla.mozilla.org/show_bug.cgi?id=637002#c17. That seems relatively straightforward to specify.

Allowing for sending RST_STREAM when a redirect arrives early seems simple too.

Testing either is probably not.

Let me know if anyone has concerns with this approach.

[louiscryan]

What language would you like to have the test server written in? I'm happy to do it in Java with Netty but if there's a preference for something else I can help track that down.

[annevk]

@louiscryan thank you for the offer. I think ideally this would be patched into https://github.com/w3c/wptserve, which is a Python library we use for all tests that require some server logic. (It exposes a fairly low-level of abstraction so you can control all kinds of details, but according to @jgraham it was unlikely that it supported sending the response early at the moment.)

[louiscryan]

@tbetbetbe
@Lukasa

Would hyper-h2 be a good solution for building out a server in python to help drive testing for this?

[Lukasa]

@louiscryan I believe that it would be, yes: certainly it should be, if it isn't now. From skim-reading this issue I'm about 90% certain that the current release of hyper-h2 should be capable of everything you want from it.

Give me a shout if you'd like assistance with this, or if you need something from hyper-h2 that it doesn't currently provide: I'd be happy to help.

[annevk]

Thanks, the standard is now fixed. Tests live in a different repository, I recommend following up at the pointers I provided.

[ioquatix]

I've been playing around with full duplex streaming for both HTTP/1 and HTTP2 clients and servers. It can and does work well in practice, but I haven't tested browser support. Are there some examples of how to do this bidirectional streaming?

[annevk]

@ioquatix no browser implements request streams as of yet, so no.

[ioquatix]

It's been about 2 years since this was initially proposed/implemented. Is request streaming on the cards?

[annevk]

Yes, it definitely still is. It's however not as much a priority for implementations as response streaming, apparently.

[ioquatix]

Would it help if I provided a working HTTP/2 client/server implementation in Ruby for testing?

[ioquatix]

Also, @annevk Is there an open issue tracking these implementations? Thanks so much for your continued help.

[annevk]

There's https://bugzilla.mozilla.org/show_bug.cgi?id=1387483 for Firefox.

Since any "official" testing would be part of https://github.com/w3c/web-platform-tests I suspect we'd want a Python implementation given that's the language of choice for testing thus far. That's tracked by web-platform-tests/wpt#7693 and web-platform-tests/wpt#8371.

[ioquatix]

Okay, I already implemented a client and server in Ruby. I'm not interested in implementing it in Python 2.7, sorry. I look forward to the results of this proposal.