From cab291ed1f3ffcf2623bce2166f6eb11195f89f5 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Mon, 26 Sep 2022 07:11:19 -0400 Subject: [PATCH 1/8] add hole punching material --- content/concepts/hole-punching.md | 193 ++++++++++++++++++++++++++++++ 1 file changed, 193 insertions(+) create mode 100644 content/concepts/hole-punching.md diff --git a/content/concepts/hole-punching.md b/content/concepts/hole-punching.md new file mode 100644 index 00000000..11909c9b --- /dev/null +++ b/content/concepts/hole-punching.md @@ -0,0 +1,193 @@ +--- +title: Hole Punching +weight: 3 +--- + +## Types of nodes + +Nodes on the public internet can be divided into two groups: +public and non-public. Most nodes are not publicly accessible as +routers act as firewalls and allow for NATs. + +To be clear, a non-public node from a standard LAN: + +- can dial other public nodes; +- cannot dial non-public nodes. + +Likewise, a public node on the internet: + +- can dial other public nodes; +- cannot dial non-public nodes. + +### How can a node dial a non-public node? + +Here are a few methods that nodes can use to dial a non-public node: + +- UPnP (Universal Plug and Play): A protocol that enables nodes + to discover and connect by automatically opening + ports into a firewall. +- Port forwarding: Manually configuring a port forward on a router. +- Traversal Using Relay NAT (TURN): A protocol that can traverse + a NAT, allowing a client to obtain IP addresses and ports from + relaying. + +### Limitations + +UPnP automates the process of node discovery and connectivity. Still, +it may not be available everywhere, and there can posses the risk +of establishing connections with untrustworthy nodes as the protocol +does not enforce authentication or authorization. Manually opening a +port requires technical expertise and does not enforce +authentication or authorization. Using a relay node also requires +technical expertise. Relaying adds additional latency and is resource +intensive. + +### Possible solution: hole punching + +In the case where the other options aren't sufficient, networks can +use a technique called hole punching to establish connections with +non-public nodes. + +Each node connects to an unrestricted third-party server and +shares its external and internal address and port information. +The server temporarily stores the node's information and relays +each node's information to the other. Clients can use this +information to establish direct connections with each other. + +Take two nodes, `A` and `B`, that would like the dial each other: + +1. The first packet of both nodes (e.g., in the case of TCP, an SYN) + passes through their respective routers. +2. The routers add a 5-tuple to their router's state table. +3. `Packet A` and `B` "punch holes" into their respective routers' + firewalls. +4. Both packets arrive at the opposite router. +5. Once `A`'s packet arrives at `router B`, `router B` checks its state + table and finds a 5-tuple previously added through the packet sent by + node B. +6. The routers forward the packets through the "punched holes". 6. It + then forwards the packet to `B`. The same occurs with `B`'s packet; + upon arriving at `router A`, it matches a 5-tuple in `router A`'s state + table and thus forwards the packet to `A`. + +The following use case diagram illustrates the above process. + +![](https://i.imgur.com/0k2Zlj3.png) + +{{% notice "note" %}} +This process assumes a mechanism to synchronize `A` and `B` simultaneously. +{{% /notice %}} + +## Hole punching in libp2p + +Inspired by the +[ICE protocol](https://datatracker.ietf.org/doc/html/rfc8445) +specified by the IETF, libp2p includes a decentralized hole punching +feature that allows for firewall and NAT traversal without the need +for central coordination servers like STUN and TURN. + +The following sequence diagram illustrates the whole process. + +![](https://i.imgur.com/sMGMfGZ.png) + +libp2p hole punching can be derived in two phases, a preparation phase and +a hole punching phase. + +### Phase I: Preparation + +1. [AutoNAT](/concepts/nat/#autonat): Determine whether a node is dialable, + as in, discover if a node is behind a NAT or firewall. + + > This is equivalent to the + > [STUN protocol](https://www.rfc-editor.org/rfc/rfc3489) in ICE. + + ![](https://i.imgur.com/NeufUm7.png) + + - `B` reaches out to `Other_Peers` (e.g., boot nodes) on the network it + is on and asks each node to dial it on a set of addresses it suspects + could be reachable. + - `Other_Peers`attempt to dial each of `B`'s addresses and report the + outcome back to `B`. + - Based on the reports, `B` can gauge whether it is publicly dialable and + determine if hole punching is needed. + + +1. Routing: Discover the k-closest public Relay nodes using a lookup method + (e.g. IPFS uses Kademlia DHT): `//p2p-circuit/` + + ![](https://i.imgur.com/cdqmJCo.png) + + - `Other_Peers` outside `B`'s network can dial `B` indirectly through + a public Relay node. In the case of [IPFS](https://ipfs.tech/), each public + node would serve as a `Relay`. `B` would either perform a lookup on the + [Kademlia DHT](https://github.com/libp2p/specs/blob/master/kad-dht/README.md) + for the closest peers to its Peer ID or choose a subset of the public nodes + it is already connected to. + +2. [Circuit Relay](/concepts/circuit-relay): Connect to and request + reservations with the discovered Relay nodes. A node can advertise itself as being + reachable through a remote Relay node. + + > This is equivalent to the + > [TURN protocol](https://datatracker.ietf.org/doc/html/rfc5766) in ICE. + + ![](https://i.imgur.com/kqRnQUV.png) + + - `Relay` can limit the resources used to relay connections (e.g., by the number + of connections, the time, and bytes) via Circuit Relay v2. In the case of IPFS, + this allows every public node in the network to serve as a relay without high + resource consumption. + - For each discovered `Relay`, `B`: + - connects to the remote node and requests the Relay node to listen to + connections on its behalf, known as a reservation; + - if `Relay` accepts reservation requests, `B` can advertise itself as being + reachable through `Relay`. + +### Phase II: Hole punching + +1. [Circuit Relay](/concepts/circuit-relay): Establish a secure relay connection + through the public Relay node. The node establishes a direct connection with + the Relay node. It then requests a relayed connection to the other node through + the Relay node, creating a bi-directional channel and using TLS to secure the + channel. + + ![](https://i.imgur.com/zoIWcwK.png) + + - `A` establishes a relayed connection to `B` via the `Relay` using the + information contained in `B`'s advertised address. + - `A` first establishes a direct connection to `Relay` and then + requests a relayed connection to `B` from `Relay`. + - `Relay` forwards said request to `B` and accepts. + - `Relay` forwards the acceptance to `A`. + - `A` and `B` can use the bi-directional channel over `Relay` to + communicate. + - `A` and `B` upgrade the relayed connection with a security protocol + like TLS. + +2. [DCUtR](https://github.com/libp2p/specs/blob/master/relay/DCUtR.md): Use + DCUtR as a synchronization mechanism to coordinate hole punching. + + ![](https://i.imgur.com/wXehUlC.png) + + - `A` sends a `Connect` message to `B` through `Relay`. + - `Connect` contains the addresses of A. libp2p and offers multiple + mechanisms to discover one's addresses, e.g., via the libp2p identify + protocol. + - `B` receives the `Connect` message on the relayed connection and replies + with a `Connect` message containing its (non-relayed) addresses. + - `A` measures the time between sending its message and receiving `B`'s + message, thereby determining the round-trip time between `A` and `B` via `Relay`. + - Then, `A` sends a `Sync` message to `B` on the relayed connection. + - `A` waits for half the round-trip time, then directly dials `B` via the + addresses received in `B`'s `Connect`. + - As soon as `B` receives `A`'s `Sync` message, it directly dials `A` with the + addresses provided in `A`'s `Connect` message. + - Once `A` and `B` dial each other simultaneously, a hole punch occurs. + +### Resources + +- This guide is a byproduct of the + [Hole punching in libp2p - Overcoming Firewalls](https://blog.ipfs.tech/2022-01-20-libp2p-hole-punching/) + blog post by Max Inden. +- Keep up with the [libp2p implementations page](https://libp2p.io/implementations/) for + the state on different hole punching implementations. From fe771ffbbdc219bf6f1b77ce68cd219c0cee9f47 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Tue, 27 Sep 2022 19:40:07 -0400 Subject: [PATCH 2/8] Apply suggestions from code review Co-authored-by: Marten Seemann Co-authored-by: Max Inden --- content/concepts/hole-punching.md | 26 +++++++++++--------------- 1 file changed, 11 insertions(+), 15 deletions(-) diff --git a/content/concepts/hole-punching.md b/content/concepts/hole-punching.md index 11909c9b..23a8bffa 100644 --- a/content/concepts/hole-punching.md +++ b/content/concepts/hole-punching.md @@ -5,11 +5,10 @@ weight: 3 ## Types of nodes -Nodes on the public internet can be divided into two groups: -public and non-public. Most nodes are not publicly accessible as -routers act as firewalls and allow for NATs. +Nodes on a p2p network can be categorized into two groups: +public and non-public. Public nodes are those nodes that have unobstructed access to the internet, whereas non-public nodes are located behind some kind of firewall. This applies to most nodes in home and in corporate network, as well as mobile phones. -To be clear, a non-public node from a standard LAN: +To be clear, a non-public node behind a firewall and/or NAT: - can dial other public nodes; - cannot dial non-public nodes. @@ -19,14 +18,12 @@ Likewise, a public node on the internet: - can dial other public nodes; - cannot dial non-public nodes. -### How can a node dial a non-public node? +### How can a node become dialable despite being behind a firewall and/or NAT? Here are a few methods that nodes can use to dial a non-public node: -- UPnP (Universal Plug and Play): A protocol that enables nodes - to discover and connect by automatically opening - ports into a firewall. -- Port forwarding: Manually configuring a port forward on a router. +- UPnP (Universal Plug and Play): A protocol spoken between routers and computers inside the network. It allows the computer to request that certain ports be opened and forward to that computer. +- Port forwarding: Manually configuring a port forwarding on a router. - Traversal Using Relay NAT (TURN): A protocol that can traverse a NAT, allowing a client to obtain IP addresses and ports from relaying. @@ -34,7 +31,7 @@ Here are a few methods that nodes can use to dial a non-public node: ### Limitations UPnP automates the process of node discovery and connectivity. Still, -it may not be available everywhere, and there can posses the risk +it may not be available everywhere, and can posses the risk of establishing connections with untrustworthy nodes as the protocol does not enforce authentication or authorization. Manually opening a port requires technical expertise and does not enforce @@ -65,8 +62,7 @@ Take two nodes, `A` and `B`, that would like the dial each other: 5. Once `A`'s packet arrives at `router B`, `router B` checks its state table and finds a 5-tuple previously added through the packet sent by node B. -6. The routers forward the packets through the "punched holes". 6. It - then forwards the packet to `B`. The same occurs with `B`'s packet; +6. The routers forward the packets through the "punched holes" to `B`. The same occurs with `B`'s packet; upon arriving at `router A`, it matches a 5-tuple in `router A`'s state table and thus forwards the packet to `A`. @@ -82,7 +78,7 @@ This process assumes a mechanism to synchronize `A` and `B` simultaneously. Inspired by the [ICE protocol](https://datatracker.ietf.org/doc/html/rfc8445) -specified by the IETF, libp2p includes a decentralized hole punching +libp2p includes a decentralized hole punching feature that allows for firewall and NAT traversal without the need for central coordination servers like STUN and TURN. @@ -90,7 +86,7 @@ The following sequence diagram illustrates the whole process. ![](https://i.imgur.com/sMGMfGZ.png) -libp2p hole punching can be derived in two phases, a preparation phase and +libp2p hole punching can be divided into two phases, a preparation phase and a hole punching phase. ### Phase I: Preparation @@ -170,7 +166,7 @@ a hole punching phase. ![](https://i.imgur.com/wXehUlC.png) - `A` sends a `Connect` message to `B` through `Relay`. - - `Connect` contains the addresses of A. libp2p and offers multiple + - `Connect` contains the addresses of A. libp2p offers multiple mechanisms to discover one's addresses, e.g., via the libp2p identify protocol. - `B` receives the `Connect` message on the relayed connection and replies From f0b01726180745bfcdebaad20b7dce4f43b60da1 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Wed, 28 Sep 2022 04:50:17 -0400 Subject: [PATCH 3/8] edits --- content/concepts/hole-punching.md | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/content/concepts/hole-punching.md b/content/concepts/hole-punching.md index 23a8bffa..5ffc677f 100644 --- a/content/concepts/hole-punching.md +++ b/content/concepts/hole-punching.md @@ -3,7 +3,9 @@ title: Hole Punching weight: 3 --- -## Types of nodes +## Background + +### Types of nodes Nodes on a p2p network can be categorized into two groups: public and non-public. Public nodes are those nodes that have unobstructed access to the internet, whereas non-public nodes are located behind some kind of firewall. This applies to most nodes in home and in corporate network, as well as mobile phones. @@ -45,11 +47,10 @@ In the case where the other options aren't sufficient, networks can use a technique called hole punching to establish connections with non-public nodes. -Each node connects to an unrestricted third-party server and -shares its external and internal address and port information. -The server temporarily stores the node's information and relays -each node's information to the other. Clients can use this -information to establish direct connections with each other. +Each node connects to a Relay server and shares its external and internal +address and port information. The server temporarily stores the node's +information and relays each node's information to the other. Clients can +use this information to establish direct connections with each other. Take two nodes, `A` and `B`, that would like the dial each other: @@ -62,9 +63,9 @@ Take two nodes, `A` and `B`, that would like the dial each other: 5. Once `A`'s packet arrives at `router B`, `router B` checks its state table and finds a 5-tuple previously added through the packet sent by node B. -6. The routers forward the packets through the "punched holes" to `B`. The same occurs with `B`'s packet; - upon arriving at `router A`, it matches a 5-tuple in `router A`'s state - table and thus forwards the packet to `A`. +6. The routers forward the packets through the "punched holes" to `B`. + The same occurs with `B`'s packet; upon arriving at `router A`, it matches + a 5-tuple in `router A`'s state table and thus forwards the packet to `A`. The following use case diagram illustrates the above process. @@ -77,7 +78,7 @@ This process assumes a mechanism to synchronize `A` and `B` simultaneously. ## Hole punching in libp2p Inspired by the -[ICE protocol](https://datatracker.ietf.org/doc/html/rfc8445) +[ICE protocol](https://datatracker.ietf.org/doc/html/rfc8445), libp2p includes a decentralized hole punching feature that allows for firewall and NAT traversal without the need for central coordination servers like STUN and TURN. From 39b3ab0f1f9b36c0573052e714a2733168d0beda Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Wed, 28 Sep 2022 05:43:12 -0400 Subject: [PATCH 4/8] address PR feedback --- content/concepts/hole-punching.md | 78 ++++++++++++++++++++----------- 1 file changed, 52 insertions(+), 26 deletions(-) diff --git a/content/concepts/hole-punching.md b/content/concepts/hole-punching.md index 5ffc677f..bc1e0471 100644 --- a/content/concepts/hole-punching.md +++ b/content/concepts/hole-punching.md @@ -8,7 +8,10 @@ weight: 3 ### Types of nodes Nodes on a p2p network can be categorized into two groups: -public and non-public. Public nodes are those nodes that have unobstructed access to the internet, whereas non-public nodes are located behind some kind of firewall. This applies to most nodes in home and in corporate network, as well as mobile phones. +public and non-public. Public nodes are those nodes that have unobstructed +access to the internet, whereas non-public nodes are located behind some kind +of firewall. This applies to most nodes in home and in corporate network, as +well as mobile phones. To be clear, a non-public node behind a firewall and/or NAT: @@ -26,29 +29,36 @@ Here are a few methods that nodes can use to dial a non-public node: - UPnP (Universal Plug and Play): A protocol spoken between routers and computers inside the network. It allows the computer to request that certain ports be opened and forward to that computer. - Port forwarding: Manually configuring a port forwarding on a router. -- Traversal Using Relay NAT (TURN): A protocol that can traverse - a NAT, allowing a client to obtain IP addresses and ports from - relaying. ### Limitations -UPnP automates the process of node discovery and connectivity. Still, -it may not be available everywhere, and can posses the risk -of establishing connections with untrustworthy nodes as the protocol -does not enforce authentication or authorization. Manually opening a -port requires technical expertise and does not enforce -authentication or authorization. Using a relay node also requires -technical expertise. Relaying adds additional latency and is resource -intensive. +In many settings, UPnP is disabled by the router or a firewall. +UPnP may also not work depending on the router's firmware. + +Manually opening a port requires technical expertise and does not +enforce authentication or authorization. ### Possible solution: hole punching +#### Relaying overview + +Relaying is a mechanism used to send information between two ends. +In the case of non-public nodes: + +Node A maintains a permanent connection to a relay node, R, and when node B +wants to connect to node A, it first establishes a connection to node R, +where R forwards all the packets on the connection. Relaying adds additional +latency and is resource intensive as node R needs to handle a lot of traffic. +Using a relay node also requires technical expertise. + +#### What if we could use node R to help facilitate a **direct connection** between node A and node B? + In the case where the other options aren't sufficient, networks can use a technique called hole punching to establish connections with non-public nodes. -Each node connects to a Relay server and shares its external and internal -address and port information. The server temporarily stores the node's +Each node connects to a relay node and shares its external address and port +information. The server temporarily stores the node's information and relays each node's information to the other. Clients can use this information to establish direct connections with each other. @@ -56,7 +66,16 @@ Take two nodes, `A` and `B`, that would like the dial each other: 1. The first packet of both nodes (e.g., in the case of TCP, an SYN) passes through their respective routers. -2. The routers add a 5-tuple to their router's state table. +2. The routers add a 5-tuple to their router's state table. + + {{% notice "info" %}} + A router state table (routing table) is data store within a router that lists + the routes to particular network destinations. + + The 5-tuple structure includes the source IP address, source port, + destination IP address, destination port, and transport protocol. + {{% /notice %}} + 3. `Packet A` and `B` "punch holes" into their respective routers' firewalls. 4. Both packets arrive at the opposite router. @@ -102,28 +121,33 @@ a hole punching phase. - `B` reaches out to `Other_Peers` (e.g., boot nodes) on the network it is on and asks each node to dial it on a set of addresses it suspects - could be reachable. - - `Other_Peers`attempt to dial each of `B`'s addresses and report the + could be reachable. A libp2p node has multiple ways of discovering its + addresses, but the most prominent is using the + [libp2p Identify protocol](https://github.com/libp2p/specs/blob/master/identify/README.md). + - `Other_Peers` attempt to dial each of `B`'s addresses and report the outcome back to `B`. - Based on the reports, `B` can gauge whether it is publicly dialable and determine if hole punching is needed. -1. Routing: Discover the k-closest public Relay nodes using a lookup method - (e.g. IPFS uses Kademlia DHT): `//p2p-circuit/` + + +1. AutoRelay: Dynamically discover and bind to relay nodes on the network. + > IPFS discovers the k-closest public relay nodes using a lookup method + > via Kademlia DHT): `//p2p-circuit/` ![](https://i.imgur.com/cdqmJCo.png) - `Other_Peers` outside `B`'s network can dial `B` indirectly through - a public Relay node. In the case of [IPFS](https://ipfs.tech/), each public + a public relay node. In the case of [IPFS](https://ipfs.tech/), each public node would serve as a `Relay`. `B` would either perform a lookup on the [Kademlia DHT](https://github.com/libp2p/specs/blob/master/kad-dht/README.md) for the closest peers to its Peer ID or choose a subset of the public nodes it is already connected to. 2. [Circuit Relay](/concepts/circuit-relay): Connect to and request - reservations with the discovered Relay nodes. A node can advertise itself as being - reachable through a remote Relay node. + reservations with the discovered relay nodes. A node can advertise itself as + being reachable through a remote relay node. > This is equivalent to the > [TURN protocol](https://datatracker.ietf.org/doc/html/rfc5766) in ICE. @@ -143,9 +167,9 @@ a hole punching phase. ### Phase II: Hole punching 1. [Circuit Relay](/concepts/circuit-relay): Establish a secure relay connection - through the public Relay node. The node establishes a direct connection with - the Relay node. It then requests a relayed connection to the other node through - the Relay node, creating a bi-directional channel and using TLS to secure the + through the public relay node. Node `A` establishes a direct connection with + the relay node. Node `B` then requests a relayed connection to node `A` through + the relay node, creating a bi-directional channel and uses TLS to secure the channel. ![](https://i.imgur.com/zoIWcwK.png) @@ -160,7 +184,9 @@ a hole punching phase. communicate. - `A` and `B` upgrade the relayed connection with a security protocol like TLS. + + 2. [DCUtR](https://github.com/libp2p/specs/blob/master/relay/DCUtR.md): Use DCUtR as a synchronization mechanism to coordinate hole punching. @@ -168,7 +194,7 @@ a hole punching phase. - `A` sends a `Connect` message to `B` through `Relay`. - `Connect` contains the addresses of A. libp2p offers multiple - mechanisms to discover one's addresses, e.g., via the libp2p identify + mechanisms to discover one's addresses, e.g., via the libp2p Identify protocol. - `B` receives the `Connect` message on the relayed connection and replies with a `Connect` message containing its (non-relayed) addresses. From b967be226824c02f08b075a4d2f1384b64e3b546 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Wed, 28 Sep 2022 05:47:22 -0400 Subject: [PATCH 5/8] address PR feedback --- content/concepts/hole-punching.md | 18 +++++------------- 1 file changed, 5 insertions(+), 13 deletions(-) diff --git a/content/concepts/hole-punching.md b/content/concepts/hole-punching.md index bc1e0471..d46e6c71 100644 --- a/content/concepts/hole-punching.md +++ b/content/concepts/hole-punching.md @@ -7,21 +7,13 @@ weight: 3 ### Types of nodes -Nodes on a p2p network can be categorized into two groups: +Nodes on a peer-to-peer network can be categorized into two groups: public and non-public. Public nodes are those nodes that have unobstructed access to the internet, whereas non-public nodes are located behind some kind -of firewall. This applies to most nodes in home and in corporate network, as -well as mobile phones. - -To be clear, a non-public node behind a firewall and/or NAT: - -- can dial other public nodes; -- cannot dial non-public nodes. - -Likewise, a public node on the internet: - -- can dial other public nodes; -- cannot dial non-public nodes. +of firewall. This applies to most nodes in home and in corporate network, +as well as mobile phones. In most configurations, both public and non-public +nodes can dial connections to other public nodes. However, it's not possible +to establish a connection from the public internet to a non-public node. ### How can a node become dialable despite being behind a firewall and/or NAT? From 356b5132c8bd1fd88d9335fa7128ea92fa628340 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Wed, 28 Sep 2022 05:50:53 -0400 Subject: [PATCH 6/8] edits --- content/concepts/hole-punching.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/content/concepts/hole-punching.md b/content/concepts/hole-punching.md index d46e6c71..273610f5 100644 --- a/content/concepts/hole-punching.md +++ b/content/concepts/hole-punching.md @@ -19,7 +19,9 @@ to establish a connection from the public internet to a non-public node. Here are a few methods that nodes can use to dial a non-public node: -- UPnP (Universal Plug and Play): A protocol spoken between routers and computers inside the network. It allows the computer to request that certain ports be opened and forward to that computer. +- UPnP (Universal Plug and Play): A protocol spoken between routers and computers + inside the network. It allows the computer to request that certain ports be + opened and forward to that computer. - Port forwarding: Manually configuring a port forwarding on a router. ### Limitations @@ -68,15 +70,15 @@ Take two nodes, `A` and `B`, that would like the dial each other: destination IP address, destination port, and transport protocol. {{% /notice %}} -3. `Packet A` and `B` "punch holes" into their respective routers' +3. `PacketA` and `PacketB` "punch holes" into their respective routers' firewalls. 4. Both packets arrive at the opposite router. -5. Once `A`'s packet arrives at `router B`, `router B` checks its state +5. Once `A`'s packet arrives at `Router_B`, `Router_B` checks its state table and finds a 5-tuple previously added through the packet sent by node B. 6. The routers forward the packets through the "punched holes" to `B`. - The same occurs with `B`'s packet; upon arriving at `router A`, it matches - a 5-tuple in `router A`'s state table and thus forwards the packet to `A`. + The same occurs with `B`'s packet; upon arriving at `Router_A`, it matches + a 5-tuple in `Router_A`'s state table and thus forwards the packet to `A`. The following use case diagram illustrates the above process. From 7b01c29df20212b37c3e9d0fbfbdafbada22020d Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Fri, 14 Oct 2022 05:44:51 -0400 Subject: [PATCH 7/8] add research paper --- content/concepts/hole-punching.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/content/concepts/hole-punching.md b/content/concepts/hole-punching.md index 273610f5..a0d36421 100644 --- a/content/concepts/hole-punching.md +++ b/content/concepts/hole-punching.md @@ -206,5 +206,7 @@ a hole punching phase. - This guide is a byproduct of the [Hole punching in libp2p - Overcoming Firewalls](https://blog.ipfs.tech/2022-01-20-libp2p-hole-punching/) blog post by Max Inden. +- Research paper on + [decentralized hole punching by Protocol Labs Research](https://research.protocol.ai/publications/decentralized-hole-punching/) - Keep up with the [libp2p implementations page](https://libp2p.io/implementations/) for the state on different hole punching implementations. From cb9b07438d730b7a7073306cf01575b137f38952 Mon Sep 17 00:00:00 2001 From: Danny Salman Date: Tue, 18 Oct 2022 09:02:00 -0400 Subject: [PATCH 8/8] import uml diagrams as svg assets --- content/concepts/assets/.DS_Store | Bin 0 -> 6148 bytes .../hole-punching/libp2p-hole-punching-1.svg | 45 ++++++++++ .../hole-punching/libp2p-hole-punching-10.svg | 30 +++++++ .../hole-punching/libp2p-hole-punching-2.svg | 31 +++++++ .../hole-punching/libp2p-hole-punching-3.svg | 30 +++++++ .../hole-punching/libp2p-hole-punching-4.svg | 78 ++++++++++++++++++ .../hole-punching/libp2p-hole-punching-5.svg | 28 +++++++ .../hole-punching/libp2p-hole-punching-6.svg | 23 ++++++ .../hole-punching/libp2p-hole-punching-7.svg | 28 +++++++ .../hole-punching/libp2p-hole-punching-8.svg | 33 ++++++++ .../hole-punching/libp2p-hole-punching-9.svg | 33 ++++++++ content/concepts/hole-punching.md | 16 ++-- 12 files changed, 367 insertions(+), 8 deletions(-) create mode 100644 content/concepts/assets/.DS_Store create mode 100644 content/concepts/assets/hole-punching/libp2p-hole-punching-1.svg create mode 100644 content/concepts/assets/hole-punching/libp2p-hole-punching-10.svg create mode 100644 content/concepts/assets/hole-punching/libp2p-hole-punching-2.svg create mode 100644 content/concepts/assets/hole-punching/libp2p-hole-punching-3.svg create mode 100644 content/concepts/assets/hole-punching/libp2p-hole-punching-4.svg create mode 100644 content/concepts/assets/hole-punching/libp2p-hole-punching-5.svg create mode 100644 content/concepts/assets/hole-punching/libp2p-hole-punching-6.svg create mode 100644 content/concepts/assets/hole-punching/libp2p-hole-punching-7.svg create mode 100644 content/concepts/assets/hole-punching/libp2p-hole-punching-8.svg create mode 100644 content/concepts/assets/hole-punching/libp2p-hole-punching-9.svg diff --git a/content/concepts/assets/.DS_Store b/content/concepts/assets/.DS_Store new file mode 100644 index 0000000000000000000000000000000000000000..2c99682a5a5df54a7256944ff040a0ab3fd0f20d GIT binary patch literal 6148 zcmeHK!D_=W43)MX0%OqA*wH_*L;t`n*%$N&Mw`(Nk~xhK^mv z7!ue|wk1V=5ZfXmULBeR(VU1XG(i?+LS%Y!>B@{}KrS&JsHX$H(w6oUf&Su@ck6pW4thcG_2dQ7M?owTE+O&an_5Rk}b;r~4a{MkeKlJGLXBiEWbJZDe z2AlzBz!~_K0o>Ul)s3R}&VV!E416&l=R-gfjD}gU93ALN2>_I5bP?!MONdV}jD}ee zD-c#!pt`b^7_9D?4`vq)v!c2aTk*m6<Network_ANetwork_BInternetARouter_ABRouter_B \ No newline at end of file diff --git a/content/concepts/assets/hole-punching/libp2p-hole-punching-10.svg b/content/concepts/assets/hole-punching/libp2p-hole-punching-10.svg new file mode 100644 index 00000000..d98b2e83 --- /dev/null +++ b/content/concepts/assets/hole-punching/libp2p-hole-punching-10.svg @@ -0,0 +1,30 @@ +AARouter_ARouter_ARouter_BRouter_BBBPacketAAdd 5-tupleto local state tablePacketBAdd 5-tupleto local state tablePacketAPacketB5-tuple existsin local state tablePacketB5-tuple existsin local state tablePacketA \ No newline at end of file diff --git a/content/concepts/assets/hole-punching/libp2p-hole-punching-2.svg b/content/concepts/assets/hole-punching/libp2p-hole-punching-2.svg new file mode 100644 index 00000000..630d3c53 --- /dev/null +++ b/content/concepts/assets/hole-punching/libp2p-hole-punching-2.svg @@ -0,0 +1,31 @@ +AARouter_ARouter_ARouter_BRouter_BBBA tries to connect to BPacketAPacketAChecking state tableNo 5-tuple.Dropping PacketAB tries to connect to APacketBPacketBChecking state tableNo 5-tuple.Dropping PacketB \ No newline at end of file diff --git a/content/concepts/assets/hole-punching/libp2p-hole-punching-3.svg b/content/concepts/assets/hole-punching/libp2p-hole-punching-3.svg new file mode 100644 index 00000000..d98b2e83 --- /dev/null +++ b/content/concepts/assets/hole-punching/libp2p-hole-punching-3.svg @@ -0,0 +1,30 @@ +AARouter_ARouter_ARouter_BRouter_BBBPacketAAdd 5-tupleto local state tablePacketBAdd 5-tupleto local state tablePacketAPacketB5-tuple existsin local state tablePacketB5-tuple existsin local state tablePacketA \ No newline at end of file diff --git a/content/concepts/assets/hole-punching/libp2p-hole-punching-4.svg b/content/concepts/assets/hole-punching/libp2p-hole-punching-4.svg new file mode 100644 index 00000000..ce099611 --- /dev/null +++ b/content/concepts/assets/hole-punching/libp2p-hole-punching-4.svg @@ -0,0 +1,78 @@ +Goal: Peer A establishes a direct connection to non-dialable peer B.AARelayRelayBBOther_PeersOther_PeersPhase 1 - PreparationB determining whether it is dialable via AutoNAT protocolDialRequest {supposedly_public_addresses}For each provided supposedly public addressDialalt[One of the dials succeeded.Thus B is dialable. No need for Hole Punching.Don't continue.]DialOutcome {"Ok"successful_address}[None of the dials succeeded.Thus not dialable. Hole Punching needed. Continue.]DialOutcome {"Error"}B finding closest public Relay nodes via Kademlia ProtocolFindNodes(PEER_ID_B)[ Public nodes closest to PEER_ID_B ]B listening for incoming connections via closest RelayFor each closest Relay via Circuit Relay v2 ProtocolEstablish connectionRequest reservationAccept reservationAdvertise oneself as/<RELAY_ADDR>/p2p-circuit/<PEER_ID_B>Phase 2 - Hole PunchingA establish relayed connection to BEstablish ConnectionCircuit Relay v2 ProtocolRequest connection to BRequest connection from AAccept connection requestAccept connection requestRelayed Connection establishedA and B coordinate dial via Direct Connection Upgrade through Relay (DCUtR) Protocolloop[Until direct connection established]Via relayed connectionConnect { addresses }Measure round-trip timeConnect { addresses }SyncSimultaneously establish connection- A after 1/2 RTT- B when receiving SyncAttempt direct connection / Hole Punch \ No newline at end of file diff --git a/content/concepts/assets/hole-punching/libp2p-hole-punching-5.svg b/content/concepts/assets/hole-punching/libp2p-hole-punching-5.svg new file mode 100644 index 00000000..292e6a14 --- /dev/null +++ b/content/concepts/assets/hole-punching/libp2p-hole-punching-5.svg @@ -0,0 +1,28 @@ +BBOther_PeersOther_PeersB determining whether it is dialable via AutoNAT protocolDialRequest {supposedly_public_addresses}For each provided supposedly public addressDialalt[One of the dials succeeded.Thus B is dialable. No need for Hole Punching.Don't continue.]DialOutcome {"Ok"successful_address}[None of the dials succeeded.Thus not dialable. Hole Punching needed. Continue.]DialOutcome {"Error"} \ No newline at end of file diff --git a/content/concepts/assets/hole-punching/libp2p-hole-punching-6.svg b/content/concepts/assets/hole-punching/libp2p-hole-punching-6.svg new file mode 100644 index 00000000..dbf95aa9 --- /dev/null +++ b/content/concepts/assets/hole-punching/libp2p-hole-punching-6.svg @@ -0,0 +1,23 @@ +BBOther_PeersOther_PeersB finding closest public Relay nodes via Kademlia ProtocolFindNodes(PEER_ID_B)[ Public nodes closest to PEER_ID_B ] \ No newline at end of file diff --git a/content/concepts/assets/hole-punching/libp2p-hole-punching-7.svg b/content/concepts/assets/hole-punching/libp2p-hole-punching-7.svg new file mode 100644 index 00000000..ba2f3891 --- /dev/null +++ b/content/concepts/assets/hole-punching/libp2p-hole-punching-7.svg @@ -0,0 +1,28 @@ +RelayRelayBBOther_PeersOther_PeersB listening for incoming connections via closest RelayFor each closest Relay via Circuit Relay v2 ProtocolEstablish connectionRequest reservationAccept reservationAdvertise oneself as/<RELAY_ADDR>/p2p-circuit/<PEER_ID_B> \ No newline at end of file diff --git a/content/concepts/assets/hole-punching/libp2p-hole-punching-8.svg b/content/concepts/assets/hole-punching/libp2p-hole-punching-8.svg new file mode 100644 index 00000000..4f2520e6 --- /dev/null +++ b/content/concepts/assets/hole-punching/libp2p-hole-punching-8.svg @@ -0,0 +1,33 @@ +AARelayRelayBBA establish relayed connection to BEstablish ConnectionCircuit Relay v2 ProtocolRequest connection to BRequest connection from AAccept connection requestAccept connection requestRelayed Connection established \ No newline at end of file diff --git a/content/concepts/assets/hole-punching/libp2p-hole-punching-9.svg b/content/concepts/assets/hole-punching/libp2p-hole-punching-9.svg new file mode 100644 index 00000000..71e22dca --- /dev/null +++ b/content/concepts/assets/hole-punching/libp2p-hole-punching-9.svg @@ -0,0 +1,33 @@ +AARelayRelayBBA and B coordinate dial via Direct Connection Upgrade through Relay (DCUtR) Protocolloop[Until direct connection established]Via relayed connectionConnect { addresses }Measure round-trip timeConnect { addresses }SyncSimultaneously establish connection- A after 1/2 RTT- B when receiving SyncAttempt direct connection / Hole Punch \ No newline at end of file diff --git a/content/concepts/hole-punching.md b/content/concepts/hole-punching.md index a0d36421..39dfb54d 100644 --- a/content/concepts/hole-punching.md +++ b/content/concepts/hole-punching.md @@ -82,7 +82,7 @@ Take two nodes, `A` and `B`, that would like the dial each other: The following use case diagram illustrates the above process. -![](https://i.imgur.com/0k2Zlj3.png) +![](../assets/hole-punching/libp2p-hole-punching-2.svg) {{% notice "note" %}} This process assumes a mechanism to synchronize `A` and `B` simultaneously. @@ -98,7 +98,7 @@ for central coordination servers like STUN and TURN. The following sequence diagram illustrates the whole process. -![](https://i.imgur.com/sMGMfGZ.png) +![](../assets/hole-punching/libp2p-hole-punching-4.svg) libp2p hole punching can be divided into two phases, a preparation phase and a hole punching phase. @@ -111,7 +111,7 @@ a hole punching phase. > This is equivalent to the > [STUN protocol](https://www.rfc-editor.org/rfc/rfc3489) in ICE. - ![](https://i.imgur.com/NeufUm7.png) + ![](../assets/hole-punching/libp2p-hole-punching-5.svg) - `B` reaches out to `Other_Peers` (e.g., boot nodes) on the network it is on and asks each node to dial it on a set of addresses it suspects @@ -130,7 +130,7 @@ a hole punching phase. > IPFS discovers the k-closest public relay nodes using a lookup method > via Kademlia DHT): `//p2p-circuit/` - ![](https://i.imgur.com/cdqmJCo.png) + ![](../assets/hole-punching/libp2p-hole-punching-6.svg) - `Other_Peers` outside `B`'s network can dial `B` indirectly through a public relay node. In the case of [IPFS](https://ipfs.tech/), each public @@ -146,7 +146,7 @@ a hole punching phase. > This is equivalent to the > [TURN protocol](https://datatracker.ietf.org/doc/html/rfc5766) in ICE. - ![](https://i.imgur.com/kqRnQUV.png) + ![](../assets/hole-punching/libp2p-hole-punching-7.svg) - `Relay` can limit the resources used to relay connections (e.g., by the number of connections, the time, and bytes) via Circuit Relay v2. In the case of IPFS, @@ -166,7 +166,7 @@ a hole punching phase. the relay node, creating a bi-directional channel and uses TLS to secure the channel. - ![](https://i.imgur.com/zoIWcwK.png) + ![](../assets/hole-punching/libp2p-hole-punching-8.svg) - `A` establishes a relayed connection to `B` via the `Relay` using the information contained in `B`'s advertised address. @@ -184,8 +184,8 @@ a hole punching phase. 2. [DCUtR](https://github.com/libp2p/specs/blob/master/relay/DCUtR.md): Use DCUtR as a synchronization mechanism to coordinate hole punching. - ![](https://i.imgur.com/wXehUlC.png) - + ![](../assets/hole-punching/libp2p-hole-punching-9.svg) + - `A` sends a `Connect` message to `B` through `Relay`. - `Connect` contains the addresses of A. libp2p offers multiple mechanisms to discover one's addresses, e.g., via the libp2p Identify