Support for Channel based APIs

The Kubernetes API contains a small number of special APIs which allow for bidirectional data flow:

• Pod Exec
• Pod Attach
• Pod Port-forward

These APIs all use a Kubernetes concept called "channels" to expose their streaming functionality.

/x/kubernetes_client includes experimental support for those channel-based APIs. However, there are multiple significant restrictions, because of various causes which we will get into below.

TIP: For local development, it's very likely that these APIs will not successfully connect.
The most reliable workaround is launching a local authenticating proxy:

kubectl proxy --reject-paths='^-$'

Note that this can have security implications depending on what else has access to your localhost.

Kubernetes Background

Traditionally, the channel-based APIs in Kubernetes have been served by apiserver over the SPDY networking protocol. Kubectl commands such as kubectl exec and kubectl port-forward continue to use SPDY (see #48633). Unfortunately, SPDY became an obsolete protocol, and new software does not support SPDY anymore.

SPDY's useful bits were borrowed when defining the modern HTTP/2 specification. There is no cross-compatibility between the two, though, and SPDY and HTTP/2 are two different protocols. Kubernetes may one day replace SPDY with HTTP/2 (#7452).

When Kubernetes was faced with the fading compatibility of SPDY, it added WebSocket alternatives for the relevant APIs (exec/attach and port-forward). These endpoints have been available in Kubernetes control planes for years.

The WebSocket endpoints offered by Kubernetes have limitations compared to their SPDY ancestors. They are also somewhat incompatible with browser-style contexts like Deno. Kubectl may still switch to WebSocket for exec/attach (see #89163), but the Kubernetes WebSocket limitations will have to be addressed first.

Limitations imposed by Deno

Requires running deno with --unstable

• Tracking issue: https://chromestatus.com/feature/5189728691290112

The current implementation is using WebSocketStream which is not yet accepted into the wider web platform. Deno therefore gates WebSocketStream behind --unstable.

TLS authentication is not supported

• Tracking issue: https://github.com/denoland/deno/issues/11846
• Workarounds:
  • Switch to a header-based authentication scheme such as token auth
  • Connect thru kubectl proxy --reject-paths='^-$' instead of connecting directly to the cluster

Cannot connect to bare IP addresses over HTTPS

• Tracking issue: https://github.com/denoland/deno/issues/7660
• Workarounds:
  • Add an entry to /etc/hosts (or to actual DNS) for your cluster's IP address
  • Connect thru kubectl proxy --reject-paths='^-$' instead of connecting directly to the cluster

Limitations imposed by Kubernetes

Cannot send EOF for exec/attach stdin

• Caused by: Kubernetes limitation
• Tracking issue: https://github.com/kubernetes/kubernetes/issues/89899
• Workarounds: None

The only way to EOF or terminate the remote process over Kubernetes WebSocket is by closing the socket. This means that any final output will likely not be returned to the client, and any final processing might get cancelled.

Cannot open more port forwards on existing connection

• Caused by: Kubernetes limitation
• Tracking issue: (none)
• Workarounds: None

Whenever a new tunneled connection is desired for port-forwarding, a whole new WebSocket (and thus TCP socket) must be set up. This is specific to the Kubernetes WebSocket protocol as the SPDY protocol can multiplex efficiently.