Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add content for hole punching #200

Merged
merged 8 commits into from
Oct 26, 2022
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
193 changes: 193 additions & 0 deletions content/concepts/hole-punching.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,193 @@
---
title: Hole Punching
weight: 3
---

## Types of nodes

Nodes on the public internet can be divided into two groups:
public and non-public. Most nodes are not publicly accessible as
routers act as firewalls and allow for NATs.
salmad3 marked this conversation as resolved.
Show resolved Hide resolved

To be clear, a non-public node from a standard LAN:
salmad3 marked this conversation as resolved.
Show resolved Hide resolved

- can dial other public nodes;
- cannot dial non-public nodes.

Likewise, a public node on the internet:

- can dial other public nodes;
- cannot dial non-public nodes.
salmad3 marked this conversation as resolved.
Show resolved Hide resolved

### How can a node dial a non-public node?
salmad3 marked this conversation as resolved.
Show resolved Hide resolved

Here are a few methods that nodes can use to dial a non-public node:

- UPnP (Universal Plug and Play): A protocol that enables nodes
to discover and connect by automatically opening
ports into a firewall.
salmad3 marked this conversation as resolved.
Show resolved Hide resolved
- Port forwarding: Manually configuring a port forward on a router.
salmad3 marked this conversation as resolved.
Show resolved Hide resolved
- Traversal Using Relay NAT (TURN): A protocol that can traverse
a NAT, allowing a client to obtain IP addresses and ports from
relaying.
salmad3 marked this conversation as resolved.
Show resolved Hide resolved

### Limitations

UPnP automates the process of node discovery and connectivity. Still,
salmad3 marked this conversation as resolved.
Show resolved Hide resolved
it may not be available everywhere, and there can posses the risk
salmad3 marked this conversation as resolved.
Show resolved Hide resolved
salmad3 marked this conversation as resolved.
Show resolved Hide resolved
of establishing connections with untrustworthy nodes as the protocol
does not enforce authentication or authorization. Manually opening a
port requires technical expertise and does not enforce
authentication or authorization. Using a relay node also requires
technical expertise. Relaying adds additional latency and is resource
intensive.

### Possible solution: hole punching
salmad3 marked this conversation as resolved.
Show resolved Hide resolved

In the case where the other options aren't sufficient, networks can
use a technique called hole punching to establish connections with
non-public nodes.

Each node connects to an unrestricted third-party server and
salmad3 marked this conversation as resolved.
Show resolved Hide resolved
shares its external and internal address and port information.
salmad3 marked this conversation as resolved.
Show resolved Hide resolved
The server temporarily stores the node's information and relays
each node's information to the other. Clients can use this
information to establish direct connections with each other.

Take two nodes, `A` and `B`, that would like the dial each other:

1. The first packet of both nodes (e.g., in the case of TCP, an SYN)
passes through their respective routers.
2. The routers add a 5-tuple to their router's state table.
salmad3 marked this conversation as resolved.
Show resolved Hide resolved
3. `Packet A` and `B` "punch holes" into their respective routers'
firewalls.
4. Both packets arrive at the opposite router.
5. Once `A`'s packet arrives at `router B`, `router B` checks its state
table and finds a 5-tuple previously added through the packet sent by
node B.
6. The routers forward the packets through the "punched holes". 6. It
then forwards the packet to `B`. The same occurs with `B`'s packet;
salmad3 marked this conversation as resolved.
Show resolved Hide resolved
upon arriving at `router A`, it matches a 5-tuple in `router A`'s state
table and thus forwards the packet to `A`.

The following use case diagram illustrates the above process.

![](https://i.imgur.com/0k2Zlj3.png)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's not use any of these Plantumls. Every time I see them, my eyes start to hurt.

Copy link
Member Author

@salmad3 salmad3 Sep 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, how about we transform them into simpler diagrams that are direct? Here's an idea I sketched out (only showing packetA here and can of course be improved/corrected).

20220927_230108.jpg

cc. @mxinden

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have an opinion here.

Another option would be to use mermaid. GitHub supports it. I am not sure our CMS does.

https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/creating-diagrams

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That could work too. As these diagrams already exist for hole punching, does it make sense to include them in this PR and then create an issue to plan out what we can substitute them for?

In general, I'd like to further enhance the concept docs with visuals and establish a guideline we can follow. As the docs seem to aim for high-level content and clean explainers, we may decide to have simple and fun graphics such as this, or perhaps more direct diagrams like the one above.

I also think it's good to have UML diagrams exist somewhere, as they can serve as a more formal representation.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would also add that we currently have similar sequence diagrams on some of the concept pages, circuit relay is an example. What do we think @marten-seemann @mxinden?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks folks. We'll be looking to get this off the ground asap. In the meantime, does it make sense to keep the uml diagrams in this PR and substitute them once we have new ones?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the meantime, does it make sense to keep the uml diagrams in this PR and substitute them once we have new ones?

Yup, I think that's reasonable.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DannyS03 I guess a minor nit from me is: I'd rather commit the images to this repo (in a media/ folder) and link to it's location there. Imo it's better than hosting it on imgur

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, likewise, I was just trying to limit the merge conflicts with the structure changes we make and planned to add them after

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#207 <- Tracking issue to update diagrams once we have them available

salmad3 marked this conversation as resolved.
Show resolved Hide resolved

{{% notice "note" %}}
This process assumes a mechanism to synchronize `A` and `B` simultaneously.
{{% /notice %}}

## Hole punching in libp2p

Inspired by the
[ICE protocol](https://datatracker.ietf.org/doc/html/rfc8445)
specified by the IETF, libp2p includes a decentralized hole punching
salmad3 marked this conversation as resolved.
Show resolved Hide resolved
feature that allows for firewall and NAT traversal without the need
for central coordination servers like STUN and TURN.

The following sequence diagram illustrates the whole process.

![](https://i.imgur.com/sMGMfGZ.png)
salmad3 marked this conversation as resolved.
Show resolved Hide resolved

libp2p hole punching can be derived in two phases, a preparation phase and
salmad3 marked this conversation as resolved.
Show resolved Hide resolved
a hole punching phase.

### Phase I: Preparation

1. [AutoNAT](/concepts/nat/#autonat): Determine whether a node is dialable,
as in, discover if a node is behind a NAT or firewall.

> This is equivalent to the
> [STUN protocol](https://www.rfc-editor.org/rfc/rfc3489) in ICE.

![](https://i.imgur.com/NeufUm7.png)

- `B` reaches out to `Other_Peers` (e.g., boot nodes) on the network it
is on and asks each node to dial it on a set of addresses it suspects
salmad3 marked this conversation as resolved.
Show resolved Hide resolved
could be reachable.
- `Other_Peers`attempt to dial each of `B`'s addresses and report the
outcome back to `B`.
- Based on the reports, `B` can gauge whether it is publicly dialable and
determine if hole punching is needed.

<!-- to add routing reference when available -->
1. Routing: Discover the k-closest public Relay nodes using a lookup method
(e.g. IPFS uses Kademlia DHT): `/<RELAY_ADDR>/p2p-circuit/<PEER_ID_B>`
salmad3 marked this conversation as resolved.
Show resolved Hide resolved

![](https://i.imgur.com/cdqmJCo.png)

- `Other_Peers` outside `B`'s network can dial `B` indirectly through
a public Relay node. In the case of [IPFS](https://ipfs.tech/), each public
node would serve as a `Relay`. `B` would either perform a lookup on the
[Kademlia DHT](https://github.com/libp2p/specs/blob/master/kad-dht/README.md)
for the closest peers to its Peer ID or choose a subset of the public nodes
it is already connected to.

2. [Circuit Relay](/concepts/circuit-relay): Connect to and request
reservations with the discovered Relay nodes. A node can advertise itself as being
reachable through a remote Relay node.

> This is equivalent to the
> [TURN protocol](https://datatracker.ietf.org/doc/html/rfc5766) in ICE.

![](https://i.imgur.com/kqRnQUV.png)

- `Relay` can limit the resources used to relay connections (e.g., by the number
of connections, the time, and bytes) via Circuit Relay v2. In the case of IPFS,
this allows every public node in the network to serve as a relay without high
resource consumption.
- For each discovered `Relay`, `B`:
- connects to the remote node and requests the Relay node to listen to
connections on its behalf, known as a reservation;
- if `Relay` accepts reservation requests, `B` can advertise itself as being
reachable through `Relay`.

### Phase II: Hole punching

1. [Circuit Relay](/concepts/circuit-relay): Establish a secure relay connection
through the public Relay node. The node establishes a direct connection with
the Relay node. It then requests a relayed connection to the other node through
salmad3 marked this conversation as resolved.
Show resolved Hide resolved
the Relay node, creating a bi-directional channel and using TLS to secure the
channel.

![](https://i.imgur.com/zoIWcwK.png)

- `A` establishes a relayed connection to `B` via the `Relay` using the
information contained in `B`'s advertised address.
- `A` first establishes a direct connection to `Relay` and then
requests a relayed connection to `B` from `Relay`.
- `Relay` forwards said request to `B` and accepts.
- `Relay` forwards the acceptance to `A`.
- `A` and `B` can use the bi-directional channel over `Relay` to
communicate.
- `A` and `B` upgrade the relayed connection with a security protocol
like TLS.
<!-- to add dcutr reference when available -->
2. [DCUtR](https://github.com/libp2p/specs/blob/master/relay/DCUtR.md): Use
DCUtR as a synchronization mechanism to coordinate hole punching.

![](https://i.imgur.com/wXehUlC.png)

- `A` sends a `Connect` message to `B` through `Relay`.
- `Connect` contains the addresses of A. libp2p and offers multiple
salmad3 marked this conversation as resolved.
Show resolved Hide resolved
mechanisms to discover one's addresses, e.g., via the libp2p identify
protocol.
- `B` receives the `Connect` message on the relayed connection and replies
with a `Connect` message containing its (non-relayed) addresses.
- `A` measures the time between sending its message and receiving `B`'s
message, thereby determining the round-trip time between `A` and `B` via `Relay`.
- Then, `A` sends a `Sync` message to `B` on the relayed connection.
- `A` waits for half the round-trip time, then directly dials `B` via the
addresses received in `B`'s `Connect`.
- As soon as `B` receives `A`'s `Sync` message, it directly dials `A` with the
addresses provided in `A`'s `Connect` message.
- Once `A` and `B` dial each other simultaneously, a hole punch occurs.

### Resources

- This guide is a byproduct of the
[Hole punching in libp2p - Overcoming Firewalls](https://blog.ipfs.tech/2022-01-20-libp2p-hole-punching/)
salmad3 marked this conversation as resolved.
Show resolved Hide resolved
blog post by Max Inden.
- Keep up with the [libp2p implementations page](https://libp2p.io/implementations/) for
the state on different hole punching implementations.