Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal(std.http): provide interface to plug in custom TLS implementation #18963

Closed
truemedian opened this issue Feb 16, 2024 · 1 comment
Closed
Labels
proposal This issue suggests modifications. If it also has the "accepted" label then it is planned. standard library This issue involves writing Zig code for the standard library.
Milestone

Comments

@truemedian
Copy link
Contributor

truemedian commented Feb 16, 2024

The Background

The current implementation of std.http uses std.crypto.tls for its TLS implementation, and that's the only choice. This causes a few problems, mostly around the deficiencies of std.crypto.tls:

  • It is not possible to use TLS 1.2, still the most common form of TLS on the internet. Many sites and services still do not support TLS 1.3.
  • It is not possible to use TLS in the server, because std.crypto.tls only provides a Client at the moment.
  • It is not possible to use TLS extensions like ALPN, which are required for certain protocols, and are generally useful for protocol negotiation (like starting HTTP/2 without bouncing through a HTTP/1.1 connection upgrade).

This implementation requires a user to duplicate std.http and replace the std.crypto.tls usage with their own TLS implementation, which is not ideal and leads to code duplication and maintenance burden. An example of this is zig-tls12.

The Proposal

The proposal is to replace the direct std.crypto.tls usage in std.http with a TlsProvider interface, which will allow users to plug in their own TLS implementation that may provide the above features.

A tentative example of what this interface could look like (the following is what std.http currently depends on):

pub const ClientProvider = struct {
    /// Whatever persistent state the TLS implementation needs to store, like a certificate bundle
    context: *anyopaque,

    /// Create a new TLS client and perform the handshake, returns the associated TLS implementation
    new_client: *const fn(context: *anyopaque, allocator: Allocator, stream: net.Stream, hostname: []const u8) NewClientError!ClientConnection,
};

pub const ClientConnection = struct {
    /// Whatever connection-local state the TLS implementation needs to store, like a session or read/write buffers.
    context: *anyopaque,

    /// Reads from the TLS connection
    readv: *const fn(context: *anyopaque, stream: net.Stream, buffers: []os.iovec) ReadError!usize,

    /// Writes to the TLS connection
    writev: *const fn(context: *anyopaque, stream: net.Stream, buffers: []os.iovec_const) WriteError!usize,

    /// Flushes any buffered data and close the TLS connection
    close: *const fn(context: *anyopaque, stream: net.Stream) WriteError!void,
};

Such an interface would be duplicated for the server side, and would allow std.http.Server to be used with a TLS implementation as it is right now.

Changes Introduced

The Connection struct will no longer store a crypto.tls.Client directly. Instead it will use the client/server's Tls(Client/Server)Provider interface, which will be used to create a TLS connection when needed.

Following this change, std.crypto.tls will provide this interface and integrate into std.http as it does now. However, users will have to initialize and pass this interface into std.http if they want to use TLS. This will be a breaking change for existing code.

Additionally, existing code will have at least one extra allocation (the context for a connection will have to be allocated, there is no place to store it) and all tls operations will be a virtual function call. This is a small performance hit, but it is necessary to allow for custom TLS implementations.

However, this change will allow users to provide their own TLS implementation as a library that other users can consume and plug into std.http.

As a side-effect of this change, the http_disable_tls std option can be removed, and the special cases for http_disable_tls in std.http can be removed. As users can simply not pass in the an empty TlsProvider if they do not want to use TLS.

Terminology Introduced

TLS Provider

A TLS Provider is an interface that provides the necessary persistent state and functions to create a TLS Connection.

An example TLS provider is std.crypto.tls. As of the current implementation:

  • The only persistent state is the certificate bundle.
  • The only function std.crypto.tls.Client.init (as new_client) which initializes a local TLS state and performs a handshake along the connection.

A TLS Provider is used to obtain a TLS Connection.

TLS Connection

A TLS Connection is an interface that provides the local state and functions to handle, read, write to and close a TLS connection. This local state often includes any session information, read and write buffers, and the underlying TLS state machine.

An example TLS Connection is std.crypto.tls.Client. As of the current implementation:

  • The local state is *std.crypto.tls.Client, it must be allocated because there is no place to store it in the TLS Provider.
  • The functions are read, write, and writeEnd (as close).

A TLS Connection is used to handle a the TLS state on top of a socket, therefore all reads and writes should be done through the TLS Connection.

@jacobly0 jacobly0 added standard library This issue involves writing Zig code for the standard library. proposal This issue suggests modifications. If it also has the "accepted" label then it is planned. labels Feb 17, 2024
@clickingbuttons
Copy link
Contributor

I think your stated deficiencies of the stdlib are weak, but I'm still in favor of it.

It is not possible to use TLS 1.2

I think the standard lib should support TLS 1.2 on the client, but not the server. I'll do it in #19308 , @andrewrk permitting.

It is not possible to use TLS in the server, because std.crypto.tls only provides a Client at the moment.

Fixed in #19308.

It is not possible to use TLS extensions like ALPN, which are required for certain protocols, and are generally useful for protocol negotiation (like starting HTTP/2 without bouncing through a HTTP/1.1 connection upgrade).

There are many TLS extensions. I think the more practical ones should be added to the standard lib (like new session tickets for connection pools). I think the base impl being solid (which it currently is not) is more important than supporting extensions or alternative implementations, though.

As a side-effect of this change, the http_disable_tls std option can be removed, and the special cases for http_disable_tls in std.http can be removed. As users can simply not pass in the an empty TlsProvider if they do not want to use TLS.

I am very much in favor of this change to decouple TLS from higher level protocols like HTTP. My only thought is that net.Stream should be generic.

@Vexu Vexu added this to the 0.13.0 milestone Mar 26, 2024
@andrewrk andrewrk closed this as not planned Won't fix, can't repro, duplicate, stale Feb 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
proposal This issue suggests modifications. If it also has the "accepted" label then it is planned. standard library This issue involves writing Zig code for the standard library.
Projects
None yet
Development

No branches or pull requests

5 participants