Internal IP address is used on EC2 hosts instead of Elastic IP Address

[threejeez]

• Version: 0.23.0
• Platform: Linux ip-10-0-1-123 4.14.55-62.37.amzn1.x86_64 We are renaming the prefix of our JavaScript modules from 'node-' to 'js-' #1 SMP Fri Jul 20 00:44:08 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
• Subsystem: TCP, SECIO, KadDHT

Type: Question/Enhancement

Severity: High

Description:

I'm running a bootstrap node (B) and a node that connects to it that provides some content (C). Both nodes are running on an EC2 host and each instance has a NIC with a private IP address as well as an associated public elastic IP address. I then launch a node on my local PC (P) using the Bootstrap node and call findProviders to find the content on node C. While the content can be found through node B, my local node cannot connect to C because the IP address in the PeerInfo object for C is the internal one instead of the public Elastic IP.

To track this down I turned on debug logging and saw the following lines:

libp2p:dht:query:� ������كO�X��_��Ь`���O��geIK�G:Qmf4BRdg queue:work +0ms
libp2p:dht:net:Qmf4BRdg sending to: QmYqacKLM4ST232aSRuTCVzvYxz1rA1mMhN3fAuRBBzzAN +206ms
libp2p:switch:dial dialing QmYqacKLM4ST232aSRuTCVzvYxz1rA1mMhN3fAuRBBzzAN +205ms
libp2p:switch:dial dialing transport TCP +0ms
libp2p:switch:transport dialing TCP [ '/ip4/10.0.0.246/tcp/3000/ipfs/QmYqacKLM4ST232aSRuTCVzvYxz1rA1mMhN3fAuRBBzzAN' ] +18s
libp2p:swarm:dialer dialMany:start +18s
libp2p:swarm:dialer dialSingle: QmYqacKLM4ST232aSRuTCVzvYxz1rA1mMhN3fAuRBBzzAN:/ip4/10.0.0.246/tcp/3000/ipfs/QmYqacKLM4ST232aSRuTCVzvYxz1rA1mMhN3fAuRBBzzAN +1ms```

While the content does reside on node `QmYqacKLM4ST232aSRuTCVzvYxz1rA1mMhN3fAuRBBzzAN`, it will obviously fail to dial the node because `10.0.0.246` is the private IP Address for the machine.

We obviously can't bind libp2p to the public Elastic IP since it's not an IP that's associated with an actual NIC. I've looked into `multiaddress`'s `encapsulate` but haven't success with that.

So, the question is: what can be done to get the IP address that's broadcast to the network to be the public Elastic IP address instead for the private one? Does libp2p support this functionality or does it need to be enhanced?

#### Steps to reproduce the error:

1. Run a bootstrap node on an EC2 host
2. Run a node that provides content on another EC2 host and use the bootrap node from step 1
3. Start another node outside of AWS (local PC, for example) and connect to the network via the bootstrap node from step 1
4. Try to `findProviders` from the node in step 3

[threejeez]

Correct me if I'm wrong, but I think this has to do with NAT traversal and is related to issue #104

[jacobheun]

This is something that we should be able to address with NAT Traversal. Ideally the Nat Manager will be able to allow node C, via the relay through node B, to have node P dial it on its public address so the direct connection can be made.

[jeshocarmel]

Hi @threejeez any solution you found? I'm having the same issue as well

[threejeez]

Yes. The machine you're running in has to have an public facing IP and you need to pass that IP into your application on the command line and tell libp2p to use it. Here's the go code we wrote to accomplish that:

    multiaddrString := fmt.Sprintf("/ip4/%s/tcp/%d", bindIP, libp2pPort)

    // 0.0.0.0 will listen on any interface device.
    sourceMultiAddr, _ := maddr.NewMultiaddr(multiaddrString)

    var extMultiAddr maddr.Multiaddr
    if broadcastIP == "" {
        a8util.Log.Warning("External IP not defined, Peers might not be able to resolve this node if behind NAT\n")
    } else {
        // here we're creating the multiaddr that others should use to connect to me
        extMultiAddr, err = maddr.NewMultiaddr(fmt.Sprintf("/ip4/%s/tcp/%d", broadcastIP, libp2pPort))
        if err != nil {
            a8util.Log.Errorf("Error creating multiaddress: %v\n", err)
            // return nil, err
        }
    }
    addressFactory := func(addrs []maddr.Multiaddr) []maddr.Multiaddr {
        if extMultiAddr != nil {
            // here we're appending the external facing multiaddr we created above to the addressFactory so it will be broadcast out when I connect to a bootstrap node.
            addrs = append(addrs, extMultiAddr)
        }
        return addrs
    }

    host, err := libp2p.New(
        ctx,
        libp2p.ListenAddrs(sourceMultiAddr),
        libp2p.Identity(prvKey),
        libp2p.AddrsFactory(addressFactory),
    )

bindIP is basically always 0.0.0.0.
broadcastIP is the public IP of the EC2 host passed from the CLI. This is what the node will tell to peers about how to connect to it. This is the bit you need.