H2C support

[StarpTech]

As titled. Already reported in fastify/fastify#3131

[szmarczak]

Can you try with the main branch?

[StarpTech]

Same error.

[szmarczak]

You're running an insecure HTTP/2 server. Got doesn't support insecure HTTP/2 servers.

[StarpTech]

Why it doesn't support it? I think the error could be more meaningful. There are some action items that can be done in this issue.

[szmarczak]

I think the error could be more meaningful.

It does not know if the protocol you're connecting to is h2c or h1. There's no protocol negotiation here.

Although it's possible to negotiate the protocol using HTTP/1.1, it's very complicated. Note that HTTP/3 does not support unencrypted traffic at all.

[szmarczak]

https://github.com/szmarczak/http2-wrapper/blob/117e112d68ce4a583b1578c69a65deea8b2ec224/test/request.js#L896

It's possible to h2c if you know it's h2c in advance. See the example above.

Since it's a quite popular issue, I'll reopen. This should be documented.

[StarpTech]

Some libs like https://github.com/grantila/fetch-h2 support http2:// to identify http2 over TCP. Would that be an option or is too wild?

[szmarczak]

The underlying HTTP/2 client here is http2-wrapper. It uses an Agent (similar to HTTP/1 but not the same) to manage HTTP/2 sessions. It's explicitly written for secure connections.

There is no origin set when using an insecure HTTP/2 connection. Therefore it's not possible to authenticate the server.

The best is to provide your own session - that way you have 100% confidence that the server you're connecting to is the one you want.

---

TL;DR

It would require many changes in http2-wrapper, because there is no origin set we can refer to.

One workaround is https://runkit.com/szmarczak/60c628126cf4590013d5e786 but I strongly discourage it.

[szmarczak]

Would that be an option or is too wild?

You can provide your own HTTP/2 session via the h2session option and it will work too. Note that Got v12 is not released yet. Got v11 uses an older version of http2-wrapper and I'm not sure if it supports h2session or not.

[StarpTech]

The underlying HTTP/2 client here is http2-wrapper. It uses an Agent (similar to HTTP/1 but not the same) to manage HTTP/2 sessions. It's explicitly written for secure connections.

Out of curiosity: What the exact reason to implement an agent for HTTP2? Do you have any benchmarks? Node.js doesn't provide one.

The best is to provide your own session - that way you have 100% confidence that the server you're connecting to is the one you want.

I don't get it. I always specify the origin explicitly. The use case is for internal communication.

You can provide your own HTTP/2 session via the h2session option and it will work too. Note that Got v12 is not released yet. Got v11 uses an older version of http2-wrapper and I'm not sure if it supports h2session or not.

That's great but I don't see the benefit to do it always on my own. I'd expect that got can do it for me. Everything is provided by Node.js.

[szmarczak]

What the exact reason to implement an agent for HTTP2?

Session management according to the HTTP/2 RFC. Node.js does not provide any interface for session management. How else would you connect to a HTTP/2 website?

Do you have any benchmarks?

I do. On my hardware http2-wrapper is almost 1:1 with https - keepalive (h2o as the HTTP/2 server).

If you really care about performance you might want to go with undici. It's even faster than the native HTTP/2 module.

I don't get it. I always specify the origin explicitly.

But it doesn't work like that under the hood with HTTP/2. It's different than HTTP/1.

The http2wrapper.Agent depends on session.originSet. Per Node.js docs, it's undefined when running insecure HTTP/2 so Agent does not know what authorities it can handle and there's no way we can get the original authority out of a session. It works like this:

Got tells http2-wrapper to send a request to X. But http2-wrapper has no session connected to X, so it adds an entry to the queue. Once it connects, it's removed from the queue but since we can't tell what is the origin of the session (Node.js does not expose anything) it's a dead end, therefore a workaround like the above is needed in order to run insecure HTTP/2 sessions. It's needed to manually remember the origin and artifically inject it into the origin set.

Manually providing the session is a much simpler approach. As I've mentioned before, the http2wrapper.Agent is designed exactly for secure connections. Extending support for h2c just makes things more complicated and adds unnecessary overhead.

The use case is for internal communication.

Well, then one session should be enough. You can easily attach h2session automatically depending on the URL in a beforeRequest hook.

I don't see the benefit to do it always on my own.

I don't see the benefit here too:

Extending support for h2c just makes things more complicated and adds unnecessary overhead.

I'd expect that got can do it for me.

What is the reason for using HTTP/2 and not HTTP/1? What is stopping you from using secure HTTP/2? Even browsers don't support unencrypted HTTP/2.

[StarpTech]

First of all thank you for the write-up, very informative. I'm not very familiar with the internals of HTTP2.

What is the reason for using HTTP/2 and not HTTP/1? What is stopping you from using secure HTTP/2? Even browsers don't support unencrypted HTTP/2.

I expected that using HTTP/2 for server-to-server communication is more efficient than HTTP/1 but I already benchmarked this on my own and found out that this is not the case (at least for Node.js). TLS would be already ensured by a sidecar like linkerd.

[szmarczak]

Note aside: I tried making the Agent support both secure and insecure connections. The code became hard to manage so I ended up exposing an option to pass unencrypted sessions. I've just checked and Got v11 supports h2session. I'll provide you an example shortly.

[szmarczak]

First of all thank you for the write-up, very informative. I'm not very familiar with the internals of HTTP2.

No problem. I'm happy to answer all the questions :)

I expected that using HTTP/2 for server-to-server communication is more efficient than HTTP/1 but I already benchmarked this on my own and found out that this is not the case (at least for Node.js).

HTTP/2 is much much faster than HTTP/1, that's right. But networking generally in Node.js is kinda slow when compared to other languages.

TLS would be already ensured by a sidecar like linkerd.

Hmm... That's actually a remarkable point! Does linkerd detect already encrypted sockets?

[StarpTech]

Does linkerd detect already encrypted sockets?

As far I know yes. In that case, it just forwards it.

[StarpTech]

HTTP/2 is much much faster than HTTP/1, that's right. But networking generally in Node.js is kinda slow when compared to other languages.

Very sad because this is one of the selling points of Node.js

[szmarczak]

As far I know yes. In that case, it just forwards it.

As you can see the following https://runkit.com/szmarczak/60c660d153638b001aae99af is much simpler than the UnsecureAgent approach :)

By using this the performance will be better because there's no stuff related to origin sets that is typical for secure HTTP/2 etc.

[szmarczak]

In Got it's not possible to add custom protocols (we support only unix: http: and https:), but if you don't like the hostname approach you can set an option in context e.g. {context: {h2c: true}} and check this instead of hostnames etc.

[StarpTech]

Thank you, I will check this out.

[szmarczak]

Also if you're transferring big amounts of data, then keep in mind that currently it's hard to do so. See nodejs/node#38426

[StarpTech]

Yeah, I saw that workaround already in http2-wrapper 😄