Server IP Address Migration

[eskimor]

I understand from the IETF draft that a server is not supposed to change IP address/port except for migrating to a preferred address at connection setup. Nevertheless in an p2p context, the server and client are pretty much identical once the connection is established. In addition "servers" are not necessarily servers, but can very well be a mobile device changing IP addresses....

So ... could we have a non-standard equivalent of migration in ClientConfig as well, defaulting to false obviously?

I understand that there might be good reasons for not allowing address migration for the server in the standard. Are those valid in a p2p context? Would it be sensible to provide such an option?

[djc]

I understand from the IETF draft that a server is not supposed to change IP address/port except for migrating to a preferred address at connection setup.

Can you link to the part of the spec where you got this from? Section 9.2 starts with "An endpoint can migrate a connection to a new local address by sending packets containing non-probing frames from that address." and does not seem to restrict this capability to a particular role. (If you're talking about the preferred address stuff, I think that's different from migration.)

[djc]

(For reference, this was introduced in #330 while updating to draft 20.)

[eskimor]

It is right in the last paragraph of the super section 9:

This document limits migration of connections to new client
addresses, except as described in Section 9.6.  Clients are
responsible for initiating all migrations.  Servers do not send non-
probing packets (see Section 9.1) toward a client address until they
see a non-probing packet from that address.  If a client receives
packets from an unknown server address, the client MUST discard these
packets.

Also in section 9.6 it says:

Migrating a connection to a new server address mid-connection is left
for future work.  If a client receives packets from a new server
address not indicated by the preferred_address transport parameter,
the client SHOULD discard these packets.

[eskimor]

But indeed when starting reading I was optimistic as well, because the document kept talking about endpoints .... well until I hit those paragraphs.

[djc]

I've asked some questions in the implementer's Slack, let's see what people come up with.

[eskimor]

Thanks a lot! :-)

[djc]

The feedback I got about why this is is basically that coordinating migration when both endpoints can move is a hard problem. How would you solve that in your application?

[eskimor]

I would not. If both endpoints move at the same time, the connection will die. Which is better than dying already when just the server moves. It simply makes the connection more robust, not unbreakable.

[eskimor]

Not sure if that really answers your question. I am assuming that coordination would only be a problem if both move at about the same time. But I am not that familiar with Quic protocol internals, so it might be a harder problem than I am imagining ;-)

[Ralith]

As a nonconformant feature, I'd be hesitant to merge such a feature unless the implementation is very low-impact and obviously correct. There's a significant danger of inadvertently opening up new denial-of-service attacks, and just complicating the maintenance/implementation of conformant functionality. If you're still interested in the functionality, I'd encourage you to experiment in a branch and see how it can be done; even if we can't merge it in the short term, such experimentation is the foundation on which eventual standardization of new functionality occurs.

[Demi-Marie]

@djc @Ralith if both endpoints move simultaneously, the connection can still be kept alive if one of the endpoints can find the other’s IP address, such as by Dynamic DNS.

[eskimor]

@Ralith I believe an implementation could be as simple as to not drop packages. But yeah, I will just do some experimentation and let you guys know about my findings :-)

[joshtriplett]

I'm interested in this as well. I have a multi-server application in which I control both ends of the connection, and I'm interested in allowing a client to connect to one server and then have that server hand off the connection to another server without closing it.

[Ralith]

Prior discussion is still current; this is a hard problem. Leaving aside the (major, unsolved) protocol design challenges, migrating an existing connection to a different physical servers would require sharing a significant amount of complex state between servers.

Some alternatives, relevance depending on exact requirements:

• Server preferred addresses: migration immediately after connection establishment. Mostly useful for switching away from an anycast IP address. Exposed via ServerConfig.
• Retry w/ packet-level load balancer: migration prior to connection establishment
• Address validation token w/ packet-level load balancer: active routing of a future connection attempt
• Application-layer reverse proxy: does whatever you like at the cost of being fairly complex to maintain. Lots of precedent in the HTTP space.

[joshtriplett]

I realize that it's an open issue. Having to hand off a bundle of state to the other server wouldn't be a problem, though.

That said, something similar to "server preferred addresses" seems potentially workable. What would that look like?

[Ralith]

I realize that it's an open issue

No worries, just thought I'd confirm the current state since it's been some years.

Having to hand off a bundle of state to the other server wouldn't be a problem, though.

Specifically it's a tricky problem to implement: we'd need to serialize/deserialize everything (including e.g. internal rustls state, some of which may be behind trait objects), it would fail if you weren't running exactly matching versions of Quinn, etc. Not an absolute obstacle, and perhaps useful independently for restart-in-place, but hard.

Conversely, with address validation tokens, session tickets, and/or 0-RTT, it might be very simple and efficient for the application layer to just establish a brand new connection.

That said, something similar to "server preferred addresses" seems potentially workable. What would that look like?

Set ServerConfig::preferred_address_{v4, v6} accordingly. The configured address must ultimately route to the same endpoint that you first received the connection on, and the address must be specified before the connection is accepted.

I don't think we ever actually implemented the client side of this, so current Quinn clients won't switch server addresses as-is, but it would be a relatively straightforward extension.

[flub]

The QUIC-NAT-traversal draft https://datatracker.ietf.org/doc/draft-seemann-quic-nat-traversal/ is allowing servers to migrate IP addresses. This is specifically in the setting of p2p. I think currently the proposal there looks fairly reasonable, though I'm not aware of any implementations yet.

We (iroh) plan to implement this in Quinn and hopefully upstream it. The picoquic author has also expressed interest in implementing this draft and has been involved in it already.